2024-09-19T07:17:06.157 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T07:17:06.189 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:17:06.280 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911803' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAxw2JEYZhE9zbrCAcHvSjpM6MtTbptF59+mnToyGUrpYyIzkm/vx+U3Lhgsrcp11kUSCcGkWbx+0nlOgaIMnTc= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtMrLnbCg/CcoD1WcTURtWKZdcqU3ZVHKC3oTDh2E9QeIj7gsFDD4vJG4IYUtKlW/WOeJOTmb6BhNkNo9oWM6A= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGwmGBonaIzOAnc6XZmUmK/grNUsxg7pAryupWfEwuRxx2pD4t1qs7cDDjrf+vMRc0cI+jc/Soc00zmVeUs5RsQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T07:17:06.280 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T07:17:06.281 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T07:17:06.281 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T07:17:06.282 INFO:teuthology.task.internal:Checking packages... 2024-09-19T07:17:06.305 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T07:17:06.306 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T07:17:06.306 INFO:teuthology.packaging:ref: None 2024-09-19T07:17:06.306 INFO:teuthology.packaging:tag: None 2024-09-19T07:17:06.306 INFO:teuthology.packaging:branch: squid 2024-09-19T07:17:06.306 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:17:06.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T07:17:06.530 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T07:17:06.532 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T07:17:06.538 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T07:17:06.538 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T07:17:06.565 INFO:teuthology.task.internal:Saving configuration 2024-09-19T07:17:06.589 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T07:17:06.596 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T07:17:06.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:11:43.648254', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAxw2JEYZhE9zbrCAcHvSjpM6MtTbptF59+mnToyGUrpYyIzkm/vx+U3Lhgsrcp11kUSCcGkWbx+0nlOgaIMnTc='} 2024-09-19T07:17:06.640 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:11:43.651280', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMtMrLnbCg/CcoD1WcTURtWKZdcqU3ZVHKC3oTDh2E9QeIj7gsFDD4vJG4IYUtKlW/WOeJOTmb6BhNkNo9oWM6A='} 2024-09-19T07:17:06.660 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 07:11:43.650132', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGwmGBonaIzOAnc6XZmUmK/grNUsxg7pAryupWfEwuRxx2pD4t1qs7cDDjrf+vMRc0cI+jc/Soc00zmVeUs5RsQ='} 2024-09-19T07:17:06.660 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T07:17:06.666 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T07:17:06.667 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T07:17:06.667 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T07:17:06.667 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T07:17:06.818 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4b41ec3be0>, signals=[15]) 2024-09-19T07:17:06.818 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T07:17:06.825 INFO:teuthology.task.internal:Opening connections... 2024-09-19T07:17:06.825 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:17:06.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:17:06.906 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-19T07:17:06.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:17:06.982 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-09-19T07:17:06.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:17:07.058 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T07:17:07.065 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-19T07:17:07.084 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-19T07:17:07.084 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-19T07:17:07.142 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-19T07:17:07.142 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-19T07:17:07.142 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-19T07:17:07.142 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:17:07.142 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-19T07:17:07.142 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:17:07.143 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:17:07.144 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-19T07:17:07.169 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-19T07:17:07.188 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-19T07:17:07.188 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-19T07:17:07.246 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-19T07:17:07.246 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-19T07:17:07.246 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:17:07.247 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:17:07.248 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-19T07:17:07.269 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-09-19T07:17:07.288 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-09-19T07:17:07.288 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T07:17:07.346 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-09-19T07:17:07.347 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-09-19T07:17:07.347 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T07:17:07.347 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-09-19T07:17:07.347 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T07:17:07.347 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T07:17:07.347 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T07:17:07.347 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-09-19T07:17:07.369 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T07:17:07.379 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T07:17:07.398 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T07:17:07.399 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:17:07.401 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:17:07.406 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-09-19T07:17:07.423 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T07:17:07.429 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T07:17:07.429 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:17:07.458 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:17:07.463 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-09-19T07:17:07.690 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T07:17:07.771 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T07:17:07.785 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T07:17:07.786 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T07:17:07.786 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T07:17:07.786 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:17:07.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:17:07.804 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-19T07:17:07.862 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-19T07:17:07.862 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-503.el9.x86_64 2024-09-19T07:17:07.862 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-09-19T07:17:10.356 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-19T07:17:13.200 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-19T07:17:18.326 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.5 MB 00:01 2024-09-19T07:17:20.035 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 104 kB/s | 19 kB 00:00 2024-09-19T07:17:21.397 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-19T07:17:26.794 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-19T07:17:28.717 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T07:17:28.718 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T07:17:28.799 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T07:17:28.811 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T07:17:28.812 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T07:17:28.812 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:17:28.812 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-19T07:17:28.812 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-19T07:17:28.812 INFO:teuthology.orchestra.run.smithi103.stdout:Install 4 Packages 2024-09-19T07:17:28.812 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:17:28.812 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 79 M 2024-09-19T07:17:28.813 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 126 M 2024-09-19T07:17:28.813 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-19T07:17:29.212 INFO:teuthology.orchestra.run.smithi103.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 120 kB/s | 36 kB 00:00 2024-09-19T07:17:46.780 INFO:teuthology.orchestra.run.smithi103.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 897 kB/s | 16 MB 00:17 2024-09-19T07:18:04.646 INFO:teuthology.orchestra.run.smithi103.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 826 kB/s | 29 MB 00:35 2024-09-19T07:18:11.003 INFO:teuthology.orchestra.run.smithi103.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 840 kB/s | 35 MB 00:42 2024-09-19T07:18:11.004 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:18:11.004 INFO:teuthology.orchestra.run.smithi103.stdout:Total 1.9 MB/s | 79 MB 00:42 2024-09-19T07:18:11.684 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-19T07:18:12.047 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-19T07:18:12.047 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-19T07:18:13.024 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-19T07:18:13.025 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-19T07:18:15.484 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-19T07:18:16.179 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:18:16.262 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:18:19.028 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:18:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:18:24.300 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:18:24.339 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:18:29.231 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:19:07.080 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:19:07.119 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:19:07.437 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:19:07.437 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T07:19:07.437 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T07:19:07.437 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:19:08.304 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-19T07:19:08.679 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:19:08.766 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:19:08.767 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:19:08.767 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T07:19:08.767 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T07:19:08.767 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T07:19:08.767 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T07:19:08.767 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T07:19:08.768 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T07:19:08.825 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T07:19:08.826 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T07:19:08.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 82.el9 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8274670 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T07:19:08.858 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T07:19:08.859 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T07:19:08.860 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T07:19:08.860 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-09-19T07:19:08.860 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:19:08.860 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T07:19:08.861 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T07:19:08.861 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 5.14.0-508.el9.x86_64 2024-09-19T07:19:08.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T07:19:09.530 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-09-19T07:19:10.887 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T07:19:10.914 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-09-19T07:19:10.920 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-19T07:19:10.936 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.1NtV38ev9q 2024-09-19T07:19:10.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1NtV38ev9q 2024-09-19T07:19:11.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.1NtV38ev9q 2024-09-19T07:19:11.142 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.1NtV38ev9q is 6KB 2024-09-19T07:19:11.191 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.1NtV38ev9q 2024-09-19T07:19:11.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T07:19:11.270 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T07:19:11.270 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T07:19:11.270 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T07:19:11.270 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T07:19:11.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T07:19:11.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-09-19T07:19:41.428 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T07:19:41.428 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T07:19:41.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:20:15.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-19T07:20:24.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T07:20:24.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:20:27.997 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-19T07:20:39.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T07:20:39.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:21:12.258 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-19T07:21:12.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T07:21:12.781 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T07:21:12.782 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-19T07:21:12.796 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-508.el9.x86_64 2024-09-19T07:21:12.796 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T07:21:12.796 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T07:21:12.797 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T07:21:13.797 DEBUG:teuthology.parallel:result is None 2024-09-19T07:21:13.798 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T07:21:13.805 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T07:21:13.805 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:21:13.808 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:21:13.811 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T07:21:13.830 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T07:21:13.840 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T07:21:13.874 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T07:21:13.874 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:21:13.877 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:21:13.879 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T07:21:13.934 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T07:21:13.940 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T07:21:13.940 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:21:13.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:21:13.965 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T07:21:13.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:21:13.983 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:21:13.986 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:21:14.009 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T07:21:14.052 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:21:14.055 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:21:14.057 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:21:14.070 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:21:14.073 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:21:14.090 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T07:21:14.093 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T07:21:14.100 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T07:21:14.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:21:14.104 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:21:14.107 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T07:21:14.142 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T07:21:14.151 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T07:21:14.151 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:21:14.172 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:21:14.175 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T07:21:14.198 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:21:14.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:21:14.329 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:21:14.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:21:14.412 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:21:14.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:21:14.477 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:21:14.519 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:21:14.581 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:21:14.623 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:21:14.668 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:21:14.668 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:21:14.734 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:21:14.780 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T07:21:14.841 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:21:14.882 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T07:21:14.929 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:21:14.929 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T07:21:14.995 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-19T07:21:14.998 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-19T07:21:15.000 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-09-19T07:21:15.033 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:21:15.044 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:21:15.083 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:21:15.512 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T07:21:15.519 INFO:teuthology.task.internal:Starting timer... 2024-09-19T07:21:15.519 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T07:21:15.570 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T07:21:15.602 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T07:21:15.602 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-19T07:21:15.660 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-19T07:21:15.662 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-19T07:21:15.712 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-19T07:21:15.714 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-09-19T07:21:15.753 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-09-19T07:21:15.755 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T07:21:15.755 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-19T07:21:15.794 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-19T07:21:15.794 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-19T07:21:15.823 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-19T07:21:15.824 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-09-19T07:21:15.848 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-09-19T07:21:15.848 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-09-19T07:21:15.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:21:15.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:21:15.874 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:21:15.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:21:15.903 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:21:15.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:21:15.928 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T07:21:15.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:21:15.961 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:21:15.993 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-09-19T07:21:16.022 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T07:21:16.032 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T07:21:16.045 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T07:21:16.046 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi134.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T07:25:21.256 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-09-19T07:25:21.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T07:25:21.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:25:21.384 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-19T07:25:21.461 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-19T07:25:21.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-19T07:25:21.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:25:21.534 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-19T07:25:21.621 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-19T07:25:21.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T07:25:21.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T07:25:21.699 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-19T07:25:21.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-19T07:25:21.782 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T07:25:21.792 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T07:25:21.792 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:25:21.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:25:21.795 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:25:21.795 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:25:21.797 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T07:25:21.798 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:25:21.825 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:25:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:25:21.840 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:25:21.842 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:25:21.851 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T07:25:21.866 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-19T07:25:21.867 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T07:25:21.867 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-19T07:25:21.877 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-19T07:25:21.879 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-19T07:25:21.891 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:25:21.892 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-09-19T07:25:21.894 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:25:21.903 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-09-19T07:25:21.905 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:25:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:25:21.917 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T07:25:21.931 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T07:25:21.958 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:25:21.962 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:25:21.962 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-19T07:25:21.962 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.962 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.962 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.963 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.972 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:25:21.975 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:25:21.975 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-19T07:25:21.975 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.975 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.975 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.975 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.987 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:25:21.990 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:25:21.990 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-19T07:25:21.990 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.990 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.990 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.990 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:25:21.991 INFO:teuthology.run_tasks:Running task install... 2024-09-19T07:25:21.999 DEBUG:teuthology.task.install:project ceph 2024-09-19T07:25:21.999 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T07:25:21.999 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T07:25:22.000 INFO:teuthology.task.install:Using flavor: default 2024-09-19T07:25:22.005 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T07:25:22.005 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T07:25:22.005 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:25:22.006 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:25:22.008 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:25:22.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:25:22.009 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T07:25:22.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:25:22.148 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:25:22.148 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:25:22.153 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:25:22.153 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:25:22.164 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T07:25:22.164 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T07:25:22.260 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:25:22.260 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:25:22.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:25:22.267 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:25:22.268 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:25:22.268 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:25:22.272 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T07:25:22.272 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:25:22.272 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T07:25:22.287 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:25:22.287 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:25:22.293 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:25:22.293 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:25:22.299 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T07:25:22.299 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T07:25:22.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:25:22.359 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:25:22.368 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T07:25:22.430 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:25:22.434 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:25:22.443 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T07:25:22.494 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-19T07:25:22.495 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-19T07:25:22.498 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-09-19T07:25:22.500 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-09-19T07:25:22.507 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-19T07:25:22.509 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-19T07:25:22.724 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-19T07:25:22.725 INFO:teuthology.orchestra.run.smithi174.stdout:45 files removed 2024-09-19T07:25:22.733 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-19T07:25:22.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:25:22.753 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:25:22.761 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T07:25:23.287 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-19T07:25:23.305 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-19T07:25:23.314 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T07:25:23.554 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-19T07:25:23.598 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-19T07:25:23.598 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T07:25:23.830 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T07:25:23.865 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-19T07:25:23.873 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T07:25:24.380 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-19T07:25:25.132 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.3 MB 00:01 2024-09-19T07:25:25.207 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.3 MB 00:01 2024-09-19T07:25:27.042 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-19T07:25:27.759 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-19T07:25:29.872 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-19T07:25:32.260 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.5 MB 00:00 2024-09-19T07:25:32.768 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-19T07:25:33.886 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 19 kB 00:00 2024-09-19T07:25:34.151 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 111 kB/s | 19 kB 00:00 2024-09-19T07:25:34.414 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.5 MB 00:00 2024-09-19T07:25:34.924 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-19T07:25:35.164 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-19T07:25:36.107 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-19T07:25:36.969 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-19T07:25:40.329 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T07:25:40.661 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-19T07:25:42.425 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-19T07:25:42.542 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:25:42.543 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:25:42.557 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:25:42.558 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:25:42.559 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:25:42.560 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:25:42.562 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:25:42.564 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:25:42.565 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:25:42.628 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-19T07:25:42.650 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T07:25:42.650 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-19T07:25:42.650 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T07:25:42.650 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-19T07:25:42.650 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:25:42.650 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:25:42.650 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:25:42.651 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:25:42.652 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:25:42.653 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:25:42.654 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:25:42.655 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:25:42.656 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:25:42.657 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:25:42.658 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:25:42.659 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:25:42.660 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:25:42.661 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:25:42.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:25:42.663 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-19T07:25:42.664 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-19T07:25:42.664 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-09-19T07:25:42.664 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 8 Packages 2024-09-19T07:25:42.664 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:25:42.664 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 226 M 2024-09-19T07:25:42.664 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-19T07:25:42.812 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:25:42.813 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:25:42.824 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:25:42.825 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:25:42.826 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:25:42.827 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:25:42.828 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:25:42.831 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:25:42.831 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:25:42.893 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-19T07:25:42.917 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-19T07:25:42.917 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-19T07:25:42.917 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-19T07:25:42.917 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-19T07:25:42.917 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:25:42.917 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:25:42.917 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:25:42.918 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:25:42.919 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:25:42.920 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:25:42.921 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:25:42.922 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:25:42.923 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:25:42.924 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:25:42.925 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:25:42.926 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:25:42.927 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:25:42.928 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:25:42.929 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:25:42.930 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 8 Packages 2024-09-19T07:25:42.931 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:25:42.932 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 226 M 2024-09-19T07:25:42.932 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-19T07:25:43.240 INFO:teuthology.orchestra.run.smithi103.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 47 kB/s | 6.6 kB 00:00 2024-09-19T07:25:43.546 INFO:teuthology.orchestra.run.smithi134.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 41 kB/s | 6.6 kB 00:00 2024-09-19T07:25:43.565 INFO:teuthology.orchestra.run.smithi103.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.7 MB/s | 896 kB 00:00 2024-09-19T07:25:43.632 INFO:teuthology.orchestra.run.smithi103.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T07:25:43.796 INFO:teuthology.orchestra.run.smithi134.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 3.5 MB/s | 896 kB 00:00 2024-09-19T07:25:43.854 INFO:teuthology.orchestra.run.smithi134.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T07:25:44.057 INFO:teuthology.orchestra.run.smithi103.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.6 MB/s | 5.4 MB 00:00 2024-09-19T07:25:44.107 INFO:teuthology.orchestra.run.smithi103.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 5.0 MB/s | 2.4 MB 00:00 2024-09-19T07:25:44.258 INFO:teuthology.orchestra.run.smithi103.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-19T07:25:44.330 INFO:teuthology.orchestra.run.smithi134.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 5.0 MB/s | 2.4 MB 00:00 2024-09-19T07:25:44.530 INFO:teuthology.orchestra.run.smithi134.stdout:(5/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 4.7 MB/s | 5.4 MB 00:01 2024-09-19T07:25:44.563 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:25:44.564 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T07:25:44.577 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T07:25:44.578 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T07:25:44.579 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T07:25:44.580 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T07:25:44.582 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T07:25:44.585 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T07:25:44.586 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T07:25:44.649 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-19T07:25:44.659 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T07:25:44.659 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-09-19T07:25:44.659 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T07:25:44.659 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-19T07:25:44.659 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T07:25:44.660 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T07:25:44.661 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T07:25:44.662 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T07:25:44.663 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T07:25:44.664 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T07:25:44.665 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T07:25:44.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T07:25:44.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T07:25:44.668 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T07:25:44.669 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T07:25:44.670 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T07:25:44.671 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 8 Packages 2024-09-19T07:25:44.672 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:25:44.673 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 226 M 2024-09-19T07:25:44.673 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-19T07:25:45.042 INFO:teuthology.orchestra.run.smithi103.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 5.0 MB/s | 4.7 MB 00:00 2024-09-19T07:25:45.239 INFO:teuthology.orchestra.run.smithi134.stdout:(6/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 6.6 MB/s | 4.7 MB 00:00 2024-09-19T07:25:45.314 INFO:teuthology.orchestra.run.smithi134.stdout:(7/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 1.5 MB/s | 1.5 MB 00:00 2024-09-19T07:25:45.360 INFO:teuthology.orchestra.run.smithi174.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 32 kB/s | 6.6 kB 00:00 2024-09-19T07:25:45.583 INFO:teuthology.orchestra.run.smithi103.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 8.5 MB/s | 21 MB 00:02 2024-09-19T07:25:45.659 INFO:teuthology.orchestra.run.smithi103.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 337 kB/s | 25 kB 00:00 2024-09-19T07:25:45.752 INFO:teuthology.orchestra.run.smithi174.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.2 MB/s | 896 kB 00:00 2024-09-19T07:25:45.835 INFO:teuthology.orchestra.run.smithi174.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T07:25:46.242 INFO:teuthology.orchestra.run.smithi103.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 8.5 MB/s | 17 MB 00:01 2024-09-19T07:25:46.351 INFO:teuthology.orchestra.run.smithi103.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 2.1 MB/s | 228 kB 00:00 2024-09-19T07:25:46.451 INFO:teuthology.orchestra.run.smithi103.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 270 kB/s | 27 kB 00:00 2024-09-19T07:25:46.551 INFO:teuthology.orchestra.run.smithi103.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 316 kB/s | 32 kB 00:00 2024-09-19T07:25:46.577 INFO:teuthology.orchestra.run.smithi174.stdout:(4/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.2 MB/s | 2.4 MB 00:00 2024-09-19T07:25:46.673 INFO:teuthology.orchestra.run.smithi134.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 6.5 MB/s | 21 MB 00:03 2024-09-19T07:25:46.710 INFO:teuthology.orchestra.run.smithi103.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 4.6 MB/s | 751 kB 00:00 2024-09-19T07:25:46.756 INFO:teuthology.orchestra.run.smithi134.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 302 kB/s | 25 kB 00:00 2024-09-19T07:25:46.802 INFO:teuthology.orchestra.run.smithi103.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.7 MB/s | 163 kB 00:00 2024-09-19T07:25:46.927 INFO:teuthology.orchestra.run.smithi103.stdout:(16/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 5.6 MB/s | 11 MB 00:01 2024-09-19T07:25:46.953 INFO:teuthology.orchestra.run.smithi103.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 842 kB/s | 127 kB 00:00 2024-09-19T07:25:47.044 INFO:teuthology.orchestra.run.smithi103.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 4.2 MB/s | 506 kB 00:00 2024-09-19T07:25:47.069 INFO:teuthology.orchestra.run.smithi174.stdout:(5/156): ceph-common-19.1.1-361.g2f599a46.el9.x 11 MB/s | 21 MB 00:01 2024-09-19T07:25:47.112 INFO:teuthology.orchestra.run.smithi103.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 676 kB/s | 45 kB 00:00 2024-09-19T07:25:47.144 INFO:teuthology.orchestra.run.smithi174.stdout:(6/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 2.7 MB/s | 5.4 MB 00:01 2024-09-19T07:25:47.204 INFO:teuthology.orchestra.run.smithi103.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.5 MB/s | 136 kB 00:00 2024-09-19T07:25:47.296 INFO:teuthology.orchestra.run.smithi103.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.7 MB/s | 161 kB 00:00 2024-09-19T07:25:47.303 INFO:teuthology.orchestra.run.smithi174.stdout:(7/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 2.1 MB/s | 1.5 MB 00:00 2024-09-19T07:25:47.371 INFO:teuthology.orchestra.run.smithi103.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.2 MB/s | 320 kB 00:00 2024-09-19T07:25:47.480 INFO:teuthology.orchestra.run.smithi103.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.7 MB/s | 300 kB 00:00 2024-09-19T07:25:47.503 INFO:teuthology.orchestra.run.smithi174.stdout:(8/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-19T07:25:47.563 INFO:teuthology.orchestra.run.smithi103.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.2 MB/s | 99 kB 00:00 2024-09-19T07:25:47.586 INFO:teuthology.orchestra.run.smithi174.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 302 kB/s | 25 kB 00:00 2024-09-19T07:25:47.655 INFO:teuthology.orchestra.run.smithi103.stdout:(25/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 7.6 MB/s | 5.3 MB 00:00 2024-09-19T07:25:47.681 INFO:teuthology.orchestra.run.smithi103.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 730 kB/s | 85 kB 00:00 2024-09-19T07:25:47.764 INFO:teuthology.orchestra.run.smithi103.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-19T07:25:47.831 INFO:teuthology.orchestra.run.smithi134.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 6.5 MB/s | 17 MB 00:02 2024-09-19T07:25:47.856 INFO:teuthology.orchestra.run.smithi103.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 346 kB/s | 32 kB 00:00 2024-09-19T07:25:47.923 INFO:teuthology.orchestra.run.smithi134.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 2.4 MB/s | 228 kB 00:00 2024-09-19T07:25:47.956 INFO:teuthology.orchestra.run.smithi103.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.4 MB/s | 146 kB 00:00 2024-09-19T07:25:48.015 INFO:teuthology.orchestra.run.smithi134.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 295 kB/s | 27 kB 00:00 2024-09-19T07:25:48.107 INFO:teuthology.orchestra.run.smithi134.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 346 kB/s | 32 kB 00:00 2024-09-19T07:25:48.123 INFO:teuthology.orchestra.run.smithi103.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.6 MB/s | 3.1 MB 00:00 2024-09-19T07:25:48.285 INFO:teuthology.orchestra.run.smithi134.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 4.2 MB/s | 751 kB 00:00 2024-09-19T07:25:48.383 INFO:teuthology.orchestra.run.smithi134.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.6 MB/s | 163 kB 00:00 2024-09-19T07:25:48.474 INFO:teuthology.orchestra.run.smithi134.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.4 MB/s | 127 kB 00:00 2024-09-19T07:25:48.641 INFO:teuthology.orchestra.run.smithi134.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 3.0 MB/s | 506 kB 00:00 2024-09-19T07:25:48.716 INFO:teuthology.orchestra.run.smithi103.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 4.7 MB/s | 3.6 MB 00:00 2024-09-19T07:25:48.808 INFO:teuthology.orchestra.run.smithi103.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.7 MB/s | 249 kB 00:00 2024-09-19T07:25:48.825 INFO:teuthology.orchestra.run.smithi134.stdout:(18/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 3.0 MB/s | 11 MB 00:03 2024-09-19T07:25:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 494 kB/s | 49 kB 00:00 2024-09-19T07:25:48.917 INFO:teuthology.orchestra.run.smithi134.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 494 kB/s | 45 kB 00:00 2024-09-19T07:25:48.983 INFO:teuthology.orchestra.run.smithi103.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 219 kB/s | 16 kB 00:00 2024-09-19T07:25:49.017 INFO:teuthology.orchestra.run.smithi134.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T07:25:49.075 INFO:teuthology.orchestra.run.smithi103.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.0 MB/s | 285 kB 00:00 2024-09-19T07:25:49.109 INFO:teuthology.orchestra.run.smithi134.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.7 MB/s | 161 kB 00:00 2024-09-19T07:25:49.259 INFO:teuthology.orchestra.run.smithi103.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 4.1 MB/s | 765 kB 00:00 2024-09-19T07:25:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 1.9 MB/s | 320 kB 00:00 2024-09-19T07:25:49.351 INFO:teuthology.orchestra.run.smithi103.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 199 kB/s | 18 kB 00:00 2024-09-19T07:25:49.426 INFO:teuthology.orchestra.run.smithi134.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.0 MB/s | 300 kB 00:00 2024-09-19T07:25:49.468 INFO:teuthology.orchestra.run.smithi103.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 5.5 MB/s | 7.4 MB 00:01 2024-09-19T07:25:49.543 INFO:teuthology.orchestra.run.smithi134.stdout:(24/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 5.9 MB/s | 5.3 MB 00:00 2024-09-19T07:25:49.569 INFO:teuthology.orchestra.run.smithi134.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 701 kB/s | 99 kB 00:00 2024-09-19T07:25:49.585 INFO:teuthology.orchestra.run.smithi103.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 341 kB/s | 80 kB 00:00 2024-09-19T07:25:49.610 INFO:teuthology.orchestra.run.smithi134.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-19T07:25:49.620 INFO:teuthology.orchestra.run.smithi174.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 6.8 MB/s | 17 MB 00:02 2024-09-19T07:25:49.669 INFO:teuthology.orchestra.run.smithi134.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T07:25:49.677 INFO:teuthology.orchestra.run.smithi103.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-19T07:25:49.678 INFO:teuthology.orchestra.run.smithi174.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.8 MB/s | 228 kB 00:00 2024-09-19T07:25:49.719 INFO:teuthology.orchestra.run.smithi103.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-19T07:25:49.719 INFO:teuthology.orchestra.run.smithi134.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 635 kB/s | 32 kB 00:00 2024-09-19T07:25:49.744 INFO:teuthology.orchestra.run.smithi103.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-19T07:25:49.745 INFO:teuthology.orchestra.run.smithi174.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 406 kB/s | 27 kB 00:00 2024-09-19T07:25:49.794 INFO:teuthology.orchestra.run.smithi103.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T07:25:49.795 INFO:teuthology.orchestra.run.smithi134.stdout:(29/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.9 MB/s | 146 kB 00:00 2024-09-19T07:25:49.812 INFO:teuthology.orchestra.run.smithi174.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 476 kB/s | 32 kB 00:00 2024-09-19T07:25:49.836 INFO:teuthology.orchestra.run.smithi103.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-09-19T07:25:49.862 INFO:teuthology.orchestra.run.smithi103.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-19T07:25:49.887 INFO:teuthology.orchestra.run.smithi103.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T07:25:49.921 INFO:teuthology.orchestra.run.smithi103.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.5 MB/s | 215 kB 00:00 2024-09-19T07:25:49.954 INFO:teuthology.orchestra.run.smithi174.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-19T07:25:50.038 INFO:teuthology.orchestra.run.smithi174.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.9 MB/s | 163 kB 00:00 2024-09-19T07:25:50.138 INFO:teuthology.orchestra.run.smithi174.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 1.2 MB/s | 127 kB 00:00 2024-09-19T07:25:50.219 INFO:teuthology.orchestra.run.smithi134.stdout:(30/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 4.8 MB/s | 3.1 MB 00:00 2024-09-19T07:25:50.336 INFO:teuthology.orchestra.run.smithi134.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 6.6 MB/s | 3.6 MB 00:00 2024-09-19T07:25:50.394 INFO:teuthology.orchestra.run.smithi134.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.2 MB/s | 249 kB 00:00 2024-09-19T07:25:50.470 INFO:teuthology.orchestra.run.smithi134.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 659 kB/s | 49 kB 00:00 2024-09-19T07:25:50.520 INFO:teuthology.orchestra.run.smithi134.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 329 kB/s | 16 kB 00:00 2024-09-19T07:25:50.570 INFO:teuthology.orchestra.run.smithi134.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T07:25:50.647 INFO:teuthology.orchestra.run.smithi174.stdout:(17/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 16 MB/s | 49 MB 00:03 2024-09-19T07:25:50.662 INFO:teuthology.orchestra.run.smithi134.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 8.2 MB/s | 765 kB 00:00 2024-09-19T07:25:50.680 INFO:teuthology.orchestra.run.smithi174.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 934 kB/s | 506 kB 00:00 2024-09-19T07:25:50.712 INFO:teuthology.orchestra.run.smithi134.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 365 kB/s | 18 kB 00:00 2024-09-19T07:25:50.789 INFO:teuthology.orchestra.run.smithi174.stdout:(19/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 3.0 MB/s | 11 MB 00:03 2024-09-19T07:25:50.814 INFO:teuthology.orchestra.run.smithi174.stdout:(20/156): python3-ceph-argparse-19.1.1-361.g2f5 338 kB/s | 45 kB 00:00 2024-09-19T07:25:50.829 INFO:teuthology.orchestra.run.smithi134.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 685 kB/s | 80 kB 00:00 2024-09-19T07:25:50.864 INFO:teuthology.orchestra.run.smithi174.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T07:25:50.889 INFO:teuthology.orchestra.run.smithi174.stdout:(22/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T07:25:50.915 INFO:teuthology.orchestra.run.smithi174.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.3 MB/s | 320 kB 00:00 2024-09-19T07:25:50.938 INFO:teuthology.orchestra.run.smithi134.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 7.2 MB/s | 801 kB 00:00 2024-09-19T07:25:50.955 INFO:teuthology.orchestra.run.smithi103.stdout:(48/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 9.2 MB/s | 49 MB 00:05 2024-09-19T07:25:50.980 INFO:teuthology.orchestra.run.smithi103.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 630 kB/s | 689 kB 00:01 2024-09-19T07:25:50.980 INFO:teuthology.orchestra.run.smithi134.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-19T07:25:50.981 INFO:teuthology.orchestra.run.smithi174.stdout:(24/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-19T07:25:51.007 INFO:teuthology.orchestra.run.smithi174.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.1 MB/s | 99 kB 00:00 2024-09-19T07:25:51.022 INFO:teuthology.orchestra.run.smithi134.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-09-19T07:25:51.032 INFO:teuthology.orchestra.run.smithi174.stdout:(26/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.1 MB/s | 300 kB 00:00 2024-09-19T07:25:51.057 INFO:teuthology.orchestra.run.smithi174.stdout:(27/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-19T07:25:51.082 INFO:teuthology.orchestra.run.smithi174.stdout:(28/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-19T07:25:51.089 INFO:teuthology.orchestra.run.smithi134.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-09-19T07:25:51.097 INFO:teuthology.orchestra.run.smithi103.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-19T07:25:51.108 INFO:teuthology.orchestra.run.smithi174.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 635 kB/s | 32 kB 00:00 2024-09-19T07:25:51.133 INFO:teuthology.orchestra.run.smithi174.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T07:25:51.164 INFO:teuthology.orchestra.run.smithi103.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 759 kB/s | 943 kB 00:01 2024-09-19T07:25:51.181 INFO:teuthology.orchestra.run.smithi134.stdout:(43/156): ceph-mgr-diskprediction-local-19.1.1- 7.7 MB/s | 7.4 MB 00:00 2024-09-19T07:25:51.214 INFO:teuthology.orchestra.run.smithi103.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-19T07:25:51.231 INFO:teuthology.orchestra.run.smithi134.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 751 kB/s | 106 kB 00:00 2024-09-19T07:25:51.273 INFO:teuthology.orchestra.run.smithi103.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-19T07:25:51.273 INFO:teuthology.orchestra.run.smithi134.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-19T07:25:51.292 INFO:teuthology.orchestra.run.smithi174.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 19 MB/s | 3.6 MB 00:00 2024-09-19T07:25:51.315 INFO:teuthology.orchestra.run.smithi134.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-09-19T07:25:51.342 INFO:teuthology.orchestra.run.smithi174.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.9 MB/s | 249 kB 00:00 2024-09-19T07:25:51.349 INFO:teuthology.orchestra.run.smithi134.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 807 kB/s | 135 kB 00:00 2024-09-19T07:25:51.381 INFO:teuthology.orchestra.run.smithi103.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-19T07:25:51.392 INFO:teuthology.orchestra.run.smithi174.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 990 kB/s | 49 kB 00:00 2024-09-19T07:25:51.415 INFO:teuthology.orchestra.run.smithi103.stdout:(55/156): boost-program-options-1.75.0-8.el9.x8 232 kB/s | 107 kB 00:00 2024-09-19T07:25:51.442 INFO:teuthology.orchestra.run.smithi174.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 329 kB/s | 16 kB 00:00 2024-09-19T07:25:51.474 INFO:teuthology.orchestra.run.smithi134.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-09-19T07:25:51.493 INFO:teuthology.orchestra.run.smithi174.stdout:(35/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 6.4 MB/s | 3.1 MB 00:00 2024-09-19T07:25:51.499 INFO:teuthology.orchestra.run.smithi103.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-19T07:25:51.508 INFO:teuthology.orchestra.run.smithi134.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-09-19T07:25:51.518 INFO:teuthology.orchestra.run.smithi174.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 3.7 MB/s | 285 kB 00:00 2024-09-19T07:25:51.558 INFO:teuthology.orchestra.run.smithi134.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-19T07:25:51.568 INFO:teuthology.orchestra.run.smithi174.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 365 kB/s | 18 kB 00:00 2024-09-19T07:25:51.592 INFO:teuthology.orchestra.run.smithi134.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 913 kB/s | 107 kB 00:00 2024-09-19T07:25:51.616 INFO:teuthology.orchestra.run.smithi103.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-19T07:25:51.627 INFO:teuthology.orchestra.run.smithi174.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 5.6 MB/s | 765 kB 00:00 2024-09-19T07:25:51.650 INFO:teuthology.orchestra.run.smithi103.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 684 kB/s | 160 kB 00:00 2024-09-19T07:25:51.650 INFO:teuthology.orchestra.run.smithi134.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-19T07:25:51.676 INFO:teuthology.orchestra.run.smithi134.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 392 kB/s | 33 kB 00:00 2024-09-19T07:25:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-09-19T07:25:51.734 INFO:teuthology.orchestra.run.smithi134.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-09-19T07:25:51.752 INFO:teuthology.orchestra.run.smithi174.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-19T07:25:51.777 INFO:teuthology.orchestra.run.smithi174.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 382 kB/s | 80 kB 00:00 2024-09-19T07:25:51.784 INFO:teuthology.orchestra.run.smithi134.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-19T07:25:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 210 kB/s | 24 kB 00:00 2024-09-19T07:25:51.868 INFO:teuthology.orchestra.run.smithi134.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-19T07:25:51.869 INFO:teuthology.orchestra.run.smithi174.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-19T07:25:51.894 INFO:teuthology.orchestra.run.smithi174.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-19T07:25:51.925 INFO:teuthology.orchestra.run.smithi103.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-19T07:25:51.935 INFO:teuthology.orchestra.run.smithi134.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 680 kB/s | 45 kB 00:00 2024-09-19T07:25:51.943 INFO:teuthology.orchestra.run.smithi174.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-19T07:25:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 1.9 MB/s | 581 kB 00:00 2024-09-19T07:25:51.968 INFO:teuthology.orchestra.run.smithi174.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-19T07:25:51.976 INFO:teuthology.orchestra.run.smithi103.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 315 kB/s | 47 kB 00:00 2024-09-19T07:25:52.002 INFO:teuthology.orchestra.run.smithi174.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.2 MB 00:00 2024-09-19T07:25:52.035 INFO:teuthology.orchestra.run.smithi174.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-19T07:25:52.043 INFO:teuthology.orchestra.run.smithi103.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-19T07:25:52.052 INFO:teuthology.orchestra.run.smithi134.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 662 kB 00:00 2024-09-19T07:25:52.061 INFO:teuthology.orchestra.run.smithi174.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-19T07:25:52.068 INFO:teuthology.orchestra.run.smithi103.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-19T07:25:52.093 INFO:teuthology.orchestra.run.smithi103.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 328 kB/s | 38 kB 00:00 2024-09-19T07:25:52.094 INFO:teuthology.orchestra.run.smithi174.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-19T07:25:52.119 INFO:teuthology.orchestra.run.smithi134.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.4 MB/s | 3.0 MB 00:00 2024-09-19T07:25:52.186 INFO:teuthology.orchestra.run.smithi134.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-19T07:25:52.211 INFO:teuthology.orchestra.run.smithi134.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 3.6 MB/s | 581 kB 00:00 2024-09-19T07:25:52.212 INFO:teuthology.orchestra.run.smithi174.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 917 kB/s | 107 kB 00:00 2024-09-19T07:25:52.236 INFO:teuthology.orchestra.run.smithi103.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T07:25:52.253 INFO:teuthology.orchestra.run.smithi174.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-19T07:25:52.261 INFO:teuthology.orchestra.run.smithi134.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-09-19T07:25:52.279 INFO:teuthology.orchestra.run.smithi174.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-09-19T07:25:52.312 INFO:teuthology.orchestra.run.smithi134.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-09-19T07:25:52.312 INFO:teuthology.orchestra.run.smithi174.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-09-19T07:25:52.329 INFO:teuthology.orchestra.run.smithi174.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-19T07:25:52.354 INFO:teuthology.orchestra.run.smithi174.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-19T07:25:52.369 INFO:teuthology.orchestra.run.smithi103.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 6.7 MB/s | 910 kB 00:00 2024-09-19T07:25:52.370 INFO:teuthology.orchestra.run.smithi134.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-19T07:25:52.371 INFO:teuthology.orchestra.run.smithi174.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 434 kB/s | 18 kB 00:00 2024-09-19T07:25:52.388 INFO:teuthology.orchestra.run.smithi174.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 330 kB/s | 11 kB 00:00 2024-09-19T07:25:52.395 INFO:teuthology.orchestra.run.smithi134.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 462 kB/s | 38 kB 00:00 2024-09-19T07:25:52.422 INFO:teuthology.orchestra.run.smithi174.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T07:25:52.447 INFO:teuthology.orchestra.run.smithi174.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-19T07:25:52.489 INFO:teuthology.orchestra.run.smithi174.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-19T07:25:52.494 INFO:teuthology.orchestra.run.smithi103.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-19T07:25:52.539 INFO:teuthology.orchestra.run.smithi174.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-19T07:25:52.554 INFO:teuthology.orchestra.run.smithi134.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-19T07:25:52.564 INFO:teuthology.orchestra.run.smithi174.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-19T07:25:52.570 INFO:teuthology.orchestra.run.smithi103.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-19T07:25:52.590 INFO:teuthology.orchestra.run.smithi174.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T07:25:52.615 INFO:teuthology.orchestra.run.smithi174.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-19T07:25:52.640 INFO:teuthology.orchestra.run.smithi174.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-19T07:25:52.666 INFO:teuthology.orchestra.run.smithi174.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-19T07:25:52.691 INFO:teuthology.orchestra.run.smithi174.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-19T07:25:52.696 INFO:teuthology.orchestra.run.smithi134.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T07:25:52.758 INFO:teuthology.orchestra.run.smithi174.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-19T07:25:52.813 INFO:teuthology.orchestra.run.smithi134.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 7.6 MB/s | 910 kB 00:00 2024-09-19T07:25:52.825 INFO:teuthology.orchestra.run.smithi174.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-19T07:25:52.891 INFO:teuthology.orchestra.run.smithi174.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-19T07:25:52.913 INFO:teuthology.orchestra.run.smithi134.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 9.1 MB/s | 4.9 MB 00:00 2024-09-19T07:25:52.917 INFO:teuthology.orchestra.run.smithi174.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-19T07:25:52.955 INFO:teuthology.orchestra.run.smithi134.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 969 kB/s | 137 kB 00:00 2024-09-19T07:25:52.959 INFO:teuthology.orchestra.run.smithi174.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-19T07:25:53.017 INFO:teuthology.orchestra.run.smithi174.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-19T07:25:53.042 INFO:teuthology.orchestra.run.smithi174.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-19T07:25:53.081 INFO:teuthology.orchestra.run.smithi134.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-19T07:25:53.084 INFO:teuthology.orchestra.run.smithi174.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T07:25:53.147 INFO:teuthology.orchestra.run.smithi134.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-19T07:25:53.151 INFO:teuthology.orchestra.run.smithi174.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-19T07:25:53.177 INFO:teuthology.orchestra.run.smithi174.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-19T07:25:53.202 INFO:teuthology.orchestra.run.smithi174.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-19T07:25:53.231 INFO:teuthology.orchestra.run.smithi134.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-09-19T07:25:53.235 INFO:teuthology.orchestra.run.smithi174.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-09-19T07:25:53.260 INFO:teuthology.orchestra.run.smithi174.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-09-19T07:25:53.277 INFO:teuthology.orchestra.run.smithi174.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-19T07:25:53.290 INFO:teuthology.orchestra.run.smithi134.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-19T07:25:53.303 INFO:teuthology.orchestra.run.smithi174.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 831 kB/s | 35 kB 00:00 2024-09-19T07:25:53.382 INFO:teuthology.orchestra.run.smithi134.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-19T07:25:53.386 INFO:teuthology.orchestra.run.smithi174.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-19T07:25:53.412 INFO:teuthology.orchestra.run.smithi174.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-19T07:25:53.437 INFO:teuthology.orchestra.run.smithi174.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-19T07:25:53.462 INFO:teuthology.orchestra.run.smithi174.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T07:25:53.487 INFO:teuthology.orchestra.run.smithi174.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-19T07:25:53.521 INFO:teuthology.orchestra.run.smithi174.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T07:25:53.546 INFO:teuthology.orchestra.run.smithi174.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-19T07:25:53.572 INFO:teuthology.orchestra.run.smithi174.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-19T07:25:53.597 INFO:teuthology.orchestra.run.smithi174.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T07:25:53.622 INFO:teuthology.orchestra.run.smithi174.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T07:25:53.698 INFO:teuthology.orchestra.run.smithi174.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T07:25:53.722 INFO:teuthology.orchestra.run.smithi174.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-19T07:25:53.748 INFO:teuthology.orchestra.run.smithi174.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 887 kB/s | 44 kB 00:00 2024-09-19T07:25:53.773 INFO:teuthology.orchestra.run.smithi174.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T07:25:53.798 INFO:teuthology.orchestra.run.smithi174.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-19T07:25:53.824 INFO:teuthology.orchestra.run.smithi174.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-19T07:25:53.849 INFO:teuthology.orchestra.run.smithi174.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T07:25:53.874 INFO:teuthology.orchestra.run.smithi174.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T07:25:53.882 INFO:teuthology.orchestra.run.smithi134.stdout:(76/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 6.8 MB/s | 49 MB 00:07 2024-09-19T07:25:53.899 INFO:teuthology.orchestra.run.smithi174.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T07:25:53.925 INFO:teuthology.orchestra.run.smithi174.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T07:25:53.950 INFO:teuthology.orchestra.run.smithi174.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-09-19T07:25:53.966 INFO:teuthology.orchestra.run.smithi134.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 7.3 MB/s | 6.0 MB 00:00 2024-09-19T07:25:54.016 INFO:teuthology.orchestra.run.smithi134.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 391 kB/s | 248 kB 00:00 2024-09-19T07:25:54.041 INFO:teuthology.orchestra.run.smithi134.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 162 kB/s | 26 kB 00:00 2024-09-19T07:25:54.067 INFO:teuthology.orchestra.run.smithi134.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-19T07:25:54.070 INFO:teuthology.orchestra.run.smithi103.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 689 kB/s | 1.0 MB 00:01 2024-09-19T07:25:54.117 INFO:teuthology.orchestra.run.smithi134.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-19T07:25:54.133 INFO:teuthology.orchestra.run.smithi134.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 377 kB/s | 35 kB 00:00 2024-09-19T07:25:54.158 INFO:teuthology.orchestra.run.smithi174.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-19T07:25:54.209 INFO:teuthology.orchestra.run.smithi174.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T07:25:54.217 INFO:teuthology.orchestra.run.smithi134.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-19T07:25:54.242 INFO:teuthology.orchestra.run.smithi174.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T07:25:54.243 INFO:teuthology.orchestra.run.smithi134.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-19T07:25:54.268 INFO:teuthology.orchestra.run.smithi174.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-19T07:25:54.278 INFO:teuthology.orchestra.run.smithi103.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 420 kB/s | 87 kB 00:00 2024-09-19T07:25:54.309 INFO:teuthology.orchestra.run.smithi134.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-09-19T07:25:54.326 INFO:teuthology.orchestra.run.smithi134.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 506 kB/s | 42 kB 00:00 2024-09-19T07:25:54.385 INFO:teuthology.orchestra.run.smithi174.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-19T07:25:54.410 INFO:teuthology.orchestra.run.smithi174.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 46 kB/s | 22 kB 00:00 2024-09-19T07:25:54.435 INFO:teuthology.orchestra.run.smithi134.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-19T07:25:54.443 INFO:teuthology.orchestra.run.smithi174.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-09-19T07:25:54.460 INFO:teuthology.orchestra.run.smithi134.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-09-19T07:25:54.468 INFO:teuthology.orchestra.run.smithi174.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-19T07:25:54.485 INFO:teuthology.orchestra.run.smithi134.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-19T07:25:54.494 INFO:teuthology.orchestra.run.smithi174.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2024-09-19T07:25:54.519 INFO:teuthology.orchestra.run.smithi174.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-19T07:25:54.544 INFO:teuthology.orchestra.run.smithi134.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-09-19T07:25:54.561 INFO:teuthology.orchestra.run.smithi174.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-19T07:25:54.569 INFO:teuthology.orchestra.run.smithi134.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 644 kB/s | 54 kB 00:00 2024-09-19T07:25:54.586 INFO:teuthology.orchestra.run.smithi174.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T07:25:54.603 INFO:teuthology.orchestra.run.smithi134.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-09-19T07:25:54.611 INFO:teuthology.orchestra.run.smithi174.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-19T07:25:54.637 INFO:teuthology.orchestra.run.smithi174.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-19T07:25:54.653 INFO:teuthology.orchestra.run.smithi134.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-09-19T07:25:54.662 INFO:teuthology.orchestra.run.smithi174.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-19T07:25:54.679 INFO:teuthology.orchestra.run.smithi134.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-09-19T07:25:54.679 INFO:teuthology.orchestra.run.smithi174.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-19T07:25:54.704 INFO:teuthology.orchestra.run.smithi174.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-19T07:25:54.729 INFO:teuthology.orchestra.run.smithi174.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-19T07:25:54.737 INFO:teuthology.orchestra.run.smithi134.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-09-19T07:25:54.755 INFO:teuthology.orchestra.run.smithi174.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T07:25:54.780 INFO:teuthology.orchestra.run.smithi174.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T07:25:54.796 INFO:teuthology.orchestra.run.smithi134.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-09-19T07:25:54.805 INFO:teuthology.orchestra.run.smithi174.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-19T07:25:54.821 INFO:teuthology.orchestra.run.smithi134.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-09-19T07:25:54.830 INFO:teuthology.orchestra.run.smithi174.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-19T07:25:54.856 INFO:teuthology.orchestra.run.smithi174.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-19T07:25:54.881 INFO:teuthology.orchestra.run.smithi174.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-19T07:25:54.904 INFO:teuthology.orchestra.run.smithi134.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 696 kB/s | 58 kB 00:00 2024-09-19T07:25:54.906 INFO:teuthology.orchestra.run.smithi174.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-19T07:25:54.938 INFO:teuthology.orchestra.run.smithi134.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-19T07:25:54.957 INFO:teuthology.orchestra.run.smithi174.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-09-19T07:25:54.982 INFO:teuthology.orchestra.run.smithi174.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T07:25:54.997 INFO:teuthology.orchestra.run.smithi134.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-19T07:25:55.015 INFO:teuthology.orchestra.run.smithi174.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-09-19T07:25:55.041 INFO:teuthology.orchestra.run.smithi174.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-19T07:25:55.047 INFO:teuthology.orchestra.run.smithi134.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-09-19T07:25:55.072 INFO:teuthology.orchestra.run.smithi134.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 572 kB/s | 96 kB 00:00 2024-09-19T07:25:55.074 INFO:teuthology.orchestra.run.smithi174.stdout:(132/156): protobuf-compiler-3.14.0-13.el9.x86_ 768 kB/s | 863 kB 00:01 2024-09-19T07:25:55.097 INFO:teuthology.orchestra.run.smithi134.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-19T07:25:55.108 INFO:teuthology.orchestra.run.smithi174.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-09-19T07:25:55.133 INFO:teuthology.orchestra.run.smithi174.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-19T07:25:55.156 INFO:teuthology.orchestra.run.smithi134.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T07:25:55.158 INFO:teuthology.orchestra.run.smithi174.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-09-19T07:25:55.192 INFO:teuthology.orchestra.run.smithi174.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 485 kB/s | 28 kB 00:00 2024-09-19T07:25:55.198 INFO:teuthology.orchestra.run.smithi134.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.3 MB/s | 308 kB 00:00 2024-09-19T07:25:55.217 INFO:teuthology.orchestra.run.smithi174.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-09-19T07:25:55.224 INFO:teuthology.orchestra.run.smithi134.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 771 kB/s | 19 kB 00:00 2024-09-19T07:25:55.242 INFO:teuthology.orchestra.run.smithi174.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 679 kB/s | 90 kB 00:00 2024-09-19T07:25:55.265 INFO:teuthology.orchestra.run.smithi134.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-09-19T07:25:55.268 INFO:teuthology.orchestra.run.smithi174.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-19T07:25:55.293 INFO:teuthology.orchestra.run.smithi174.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-19T07:25:55.316 INFO:teuthology.orchestra.run.smithi134.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-19T07:25:55.318 INFO:teuthology.orchestra.run.smithi174.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-19T07:25:55.343 INFO:teuthology.orchestra.run.smithi174.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-19T07:25:55.349 INFO:teuthology.orchestra.run.smithi134.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T07:25:55.369 INFO:teuthology.orchestra.run.smithi174.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-19T07:25:55.375 INFO:teuthology.orchestra.run.smithi134.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T07:25:55.385 INFO:teuthology.orchestra.run.smithi174.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-19T07:25:55.411 INFO:teuthology.orchestra.run.smithi174.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-09-19T07:25:55.436 INFO:teuthology.orchestra.run.smithi174.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T07:25:55.495 INFO:teuthology.orchestra.run.smithi174.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-19T07:25:55.533 INFO:teuthology.orchestra.run.smithi134.stdout:(111/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-09-19T07:25:55.562 INFO:teuthology.orchestra.run.smithi174.stdout:(148/156): python3-tempora-5.0.0-2.el9.noarch.r 112 kB/s | 36 kB 00:00 2024-09-19T07:25:55.567 INFO:teuthology.orchestra.run.smithi134.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 789 kB/s | 151 kB 00:00 2024-09-19T07:25:55.617 INFO:teuthology.orchestra.run.smithi134.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-19T07:25:55.643 INFO:teuthology.orchestra.run.smithi134.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T07:25:55.668 INFO:teuthology.orchestra.run.smithi134.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T07:25:55.670 INFO:teuthology.orchestra.run.smithi174.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 22 MB/s | 2.3 MB 00:00 2024-09-19T07:25:55.693 INFO:teuthology.orchestra.run.smithi134.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 569 kB/s | 14 kB 00:00 2024-09-19T07:25:55.718 INFO:teuthology.orchestra.run.smithi134.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-19T07:25:55.760 INFO:teuthology.orchestra.run.smithi134.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-19T07:25:55.779 INFO:teuthology.orchestra.run.smithi174.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-09-19T07:25:55.827 INFO:teuthology.orchestra.run.smithi134.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-19T07:25:55.838 INFO:teuthology.orchestra.run.smithi174.stdout:(151/156): librados2-19.1.1-361.g2f599a46.el9.x 8.5 MB/s | 3.4 MB 00:00 2024-09-19T07:25:55.838 INFO:teuthology.orchestra.run.smithi103.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 1.3 MB/s | 4.9 MB 00:03 2024-09-19T07:25:55.852 INFO:teuthology.orchestra.run.smithi134.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-09-19T07:25:55.864 INFO:teuthology.orchestra.run.smithi174.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 491 kB/s | 41 kB 00:00 2024-09-19T07:25:55.878 INFO:teuthology.orchestra.run.smithi134.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T07:25:55.911 INFO:teuthology.orchestra.run.smithi134.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-09-19T07:25:55.921 INFO:teuthology.orchestra.run.smithi174.stdout:(153/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.4 MB/s | 3.2 MB 00:00 2024-09-19T07:25:55.936 INFO:teuthology.orchestra.run.smithi134.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-19T07:25:55.946 INFO:teuthology.orchestra.run.smithi174.stdout:(154/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.2 MB/s | 249 kB 00:00 2024-09-19T07:25:55.954 INFO:teuthology.orchestra.run.smithi103.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-09-19T07:25:55.962 INFO:teuthology.orchestra.run.smithi134.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-19T07:25:55.987 INFO:teuthology.orchestra.run.smithi134.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T07:25:55.988 INFO:teuthology.orchestra.run.smithi174.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-19T07:25:56.012 INFO:teuthology.orchestra.run.smithi134.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-19T07:25:56.037 INFO:teuthology.orchestra.run.smithi134.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-19T07:25:56.063 INFO:teuthology.orchestra.run.smithi134.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-19T07:25:56.071 INFO:teuthology.orchestra.run.smithi103.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-19T07:25:56.096 INFO:teuthology.orchestra.run.smithi134.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-19T07:25:56.121 INFO:teuthology.orchestra.run.smithi134.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T07:25:56.147 INFO:teuthology.orchestra.run.smithi134.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T07:25:56.181 INFO:teuthology.orchestra.run.smithi134.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-19T07:25:56.189 INFO:teuthology.orchestra.run.smithi174.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-09-19T07:25:56.194 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:25:56.194 INFO:teuthology.orchestra.run.smithi174.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-19T07:25:56.206 INFO:teuthology.orchestra.run.smithi134.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-19T07:25:56.221 INFO:teuthology.orchestra.run.smithi103.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-19T07:25:56.231 INFO:teuthology.orchestra.run.smithi134.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T07:25:56.256 INFO:teuthology.orchestra.run.smithi134.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-19T07:25:56.281 INFO:teuthology.orchestra.run.smithi134.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T07:25:56.299 INFO:teuthology.orchestra.run.smithi134.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-09-19T07:25:56.338 INFO:teuthology.orchestra.run.smithi103.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-09-19T07:25:56.341 INFO:teuthology.orchestra.run.smithi134.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-09-19T07:25:56.357 INFO:teuthology.orchestra.run.smithi134.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-19T07:25:56.383 INFO:teuthology.orchestra.run.smithi134.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 859 kB/s | 36 kB 00:00 2024-09-19T07:25:56.408 INFO:teuthology.orchestra.run.smithi134.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-19T07:25:56.433 INFO:teuthology.orchestra.run.smithi134.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-19T07:25:56.447 INFO:teuthology.orchestra.run.smithi103.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-19T07:25:56.458 INFO:teuthology.orchestra.run.smithi134.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T07:25:56.484 INFO:teuthology.orchestra.run.smithi134.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-19T07:25:56.509 INFO:teuthology.orchestra.run.smithi134.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-19T07:25:56.534 INFO:teuthology.orchestra.run.smithi134.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T07:25:56.556 INFO:teuthology.orchestra.run.smithi103.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-19T07:25:56.560 INFO:teuthology.orchestra.run.smithi134.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-19T07:25:56.610 INFO:teuthology.orchestra.run.smithi134.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T07:25:56.664 INFO:teuthology.orchestra.run.smithi103.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-19T07:25:56.794 INFO:teuthology.orchestra.run.smithi134.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-19T07:25:56.861 INFO:teuthology.orchestra.run.smithi134.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-09-19T07:25:56.902 INFO:teuthology.orchestra.run.smithi134.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 983 kB/s | 41 kB 00:00 2024-09-19T07:25:56.986 INFO:teuthology.orchestra.run.smithi134.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-19T07:25:57.036 INFO:teuthology.orchestra.run.smithi134.stdout:(153/156): librbd1-19.1.1-361.g2f599a46.el9.x86 6.6 MB/s | 3.2 MB 00:00 2024-09-19T07:25:57.095 INFO:teuthology.orchestra.run.smithi134.stdout:(154/156): librados2-19.1.1-361.g2f599a46.el9.x 6.1 MB/s | 3.4 MB 00:00 2024-09-19T07:25:57.137 INFO:teuthology.orchestra.run.smithi134.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-19T07:25:57.220 INFO:teuthology.orchestra.run.smithi134.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-09-19T07:25:57.227 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:25:57.227 INFO:teuthology.orchestra.run.smithi134.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-19T07:25:57.429 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-19T07:25:57.578 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-19T07:25:57.578 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-19T07:25:58.458 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-19T07:25:58.608 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-19T07:25:58.608 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-19T07:25:59.373 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-19T07:25:59.373 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-19T07:26:00.410 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-19T07:26:00.410 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-19T07:26:00.981 INFO:teuthology.orchestra.run.smithi103.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 697 kB/s | 5.8 MB 00:08 2024-09-19T07:26:01.357 INFO:teuthology.orchestra.run.smithi103.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 668 kB/s | 250 kB 00:00 2024-09-19T07:26:01.540 INFO:teuthology.orchestra.run.smithi103.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.0 MB/s | 5.1 MB 00:04 2024-09-19T07:26:01.765 INFO:teuthology.orchestra.run.smithi103.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 544 kB/s | 222 kB 00:00 2024-09-19T07:26:01.791 INFO:teuthology.orchestra.run.smithi103.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 308 kB/s | 77 kB 00:00 2024-09-19T07:26:01.866 INFO:teuthology.orchestra.run.smithi103.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 805 kB/s | 6.0 MB 00:07 2024-09-19T07:26:01.891 INFO:teuthology.orchestra.run.smithi103.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 336 kB/s | 42 kB 00:00 2024-09-19T07:26:02.066 INFO:teuthology.orchestra.run.smithi103.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 792 kB/s | 159 kB 00:00 2024-09-19T07:26:02.183 INFO:teuthology.orchestra.run.smithi103.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-09-19T07:26:02.267 INFO:teuthology.orchestra.run.smithi103.stdout:(90/156): python3-protobuf-3.14.0-13.el9.noarch 566 kB/s | 269 kB 00:00 2024-09-19T07:26:02.292 INFO:teuthology.orchestra.run.smithi103.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-09-19T07:26:02.376 INFO:teuthology.orchestra.run.smithi103.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 575 kB/s | 279 kB 00:00 2024-09-19T07:26:02.404 INFO:teuthology.orchestra.run.smithi103.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-19T07:26:02.485 INFO:teuthology.orchestra.run.smithi103.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-19T07:26:02.519 INFO:teuthology.orchestra.run.smithi103.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-19T07:26:02.594 INFO:teuthology.orchestra.run.smithi103.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 330 kB/s | 36 kB 00:00 2024-09-19T07:26:02.627 INFO:teuthology.orchestra.run.smithi103.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-19T07:26:02.736 INFO:teuthology.orchestra.run.smithi103.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-19T07:26:02.845 INFO:teuthology.orchestra.run.smithi103.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 882 kB/s | 96 kB 00:00 2024-09-19T07:26:02.954 INFO:teuthology.orchestra.run.smithi103.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-09-19T07:26:03.062 INFO:teuthology.orchestra.run.smithi103.stdout:(101/156): socat-1.7.4.1-6.el9.x86_64.rpm 650 kB/s | 304 kB 00:00 2024-09-19T07:26:03.087 INFO:teuthology.orchestra.run.smithi103.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 335 kB/s | 45 kB 00:00 2024-09-19T07:26:03.171 INFO:teuthology.orchestra.run.smithi103.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-19T07:26:03.230 INFO:teuthology.orchestra.run.smithi103.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T07:26:03.263 INFO:teuthology.orchestra.run.smithi103.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T07:26:03.297 INFO:teuthology.orchestra.run.smithi103.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-19T07:26:03.322 INFO:teuthology.orchestra.run.smithi103.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 331 kB/s | 19 kB 00:00 2024-09-19T07:26:03.356 INFO:teuthology.orchestra.run.smithi103.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T07:26:03.381 INFO:teuthology.orchestra.run.smithi103.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-19T07:26:03.605 INFO:teuthology.orchestra.run.smithi103.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 301 kB/s | 67 kB 00:00 2024-09-19T07:26:03.690 INFO:teuthology.orchestra.run.smithi103.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 1.7 MB/s | 151 kB 00:00 2024-09-19T07:26:03.741 INFO:teuthology.orchestra.run.smithi103.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-19T07:26:03.783 INFO:teuthology.orchestra.run.smithi103.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T07:26:03.799 INFO:teuthology.orchestra.run.smithi103.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T07:26:03.817 INFO:teuthology.orchestra.run.smithi103.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T07:26:03.834 INFO:teuthology.orchestra.run.smithi103.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 852 kB/s | 14 kB 00:00 2024-09-19T07:26:03.859 INFO:teuthology.orchestra.run.smithi103.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-19T07:26:03.893 INFO:teuthology.orchestra.run.smithi103.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-19T07:26:03.959 INFO:teuthology.orchestra.run.smithi103.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-09-19T07:26:03.993 INFO:teuthology.orchestra.run.smithi103.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-19T07:26:04.018 INFO:teuthology.orchestra.run.smithi103.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T07:26:04.052 INFO:teuthology.orchestra.run.smithi103.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-19T07:26:04.077 INFO:teuthology.orchestra.run.smithi103.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-19T07:26:04.128 INFO:teuthology.orchestra.run.smithi103.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-19T07:26:04.170 INFO:teuthology.orchestra.run.smithi103.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-09-19T07:26:04.195 INFO:teuthology.orchestra.run.smithi103.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 775 kB/s | 19 kB 00:00 2024-09-19T07:26:04.221 INFO:teuthology.orchestra.run.smithi103.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-19T07:26:04.271 INFO:teuthology.orchestra.run.smithi103.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-09-19T07:26:04.296 INFO:teuthology.orchestra.run.smithi103.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T07:26:04.322 INFO:teuthology.orchestra.run.smithi103.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T07:26:04.347 INFO:teuthology.orchestra.run.smithi103.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T07:26:04.389 INFO:teuthology.orchestra.run.smithi103.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-19T07:26:04.414 INFO:teuthology.orchestra.run.smithi103.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-19T07:26:04.439 INFO:teuthology.orchestra.run.smithi103.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-19T07:26:04.465 INFO:teuthology.orchestra.run.smithi103.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-19T07:26:04.490 INFO:teuthology.orchestra.run.smithi103.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-19T07:26:04.515 INFO:teuthology.orchestra.run.smithi103.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-19T07:26:04.540 INFO:teuthology.orchestra.run.smithi103.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T07:26:04.566 INFO:teuthology.orchestra.run.smithi103.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T07:26:04.597 INFO:teuthology.orchestra.run.smithi103.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 640 kB/s | 36 kB 00:00 2024-09-19T07:26:04.622 INFO:teuthology.orchestra.run.smithi103.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.3 MB/s | 74 kB 00:00 2024-09-19T07:26:04.648 INFO:teuthology.orchestra.run.smithi103.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T07:26:04.673 INFO:teuthology.orchestra.run.smithi103.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T07:26:04.698 INFO:teuthology.orchestra.run.smithi103.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-19T07:26:04.723 INFO:teuthology.orchestra.run.smithi103.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T07:26:04.749 INFO:teuthology.orchestra.run.smithi103.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-19T07:26:04.807 INFO:teuthology.orchestra.run.smithi103.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-19T07:26:05.108 INFO:teuthology.orchestra.run.smithi103.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 9.5 MB/s | 3.4 MB 00:00 2024-09-19T07:26:05.158 INFO:teuthology.orchestra.run.smithi103.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 9.0 MB/s | 3.2 MB 00:00 2024-09-19T07:26:05.558 INFO:teuthology.orchestra.run.smithi103.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.5 MB/s | 1.8 MB 00:00 2024-09-19T07:26:05.608 INFO:teuthology.orchestra.run.smithi103.stdout:(151/156): NetworkManager-1.51.0-1.el9.x86_64.r 4.6 MB/s | 2.3 MB 00:00 2024-09-19T07:26:05.634 INFO:teuthology.orchestra.run.smithi103.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 545 kB/s | 41 kB 00:00 2024-09-19T07:26:05.637 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-19T07:26:05.667 INFO:teuthology.orchestra.run.smithi103.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 4.1 MB/s | 249 kB 00:00 2024-09-19T07:26:05.759 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:26:05.801 INFO:teuthology.orchestra.run.smithi103.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-19T07:26:05.881 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:26:05.885 INFO:teuthology.orchestra.run.smithi103.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-19T07:26:06.043 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:26:06.076 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:26:06.102 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:26:06.145 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:26:06.474 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:26:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:06.647 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-19T07:26:06.699 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:06.732 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:26:06.736 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:06.771 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:26:06.827 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:06.831 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:26:06.857 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:26:06.866 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:26:06.891 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:26:06.942 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:26:07.006 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:26:07.054 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:26:07.082 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:07.169 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:07.192 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:26:07.246 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:26:07.267 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:26:07.294 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:26:07.368 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:26:07.484 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:07.485 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:26:07.607 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:07.615 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:26:07.635 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:07.727 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:07.740 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:26:07.756 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:26:07.840 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:07.906 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:26:07.908 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:07.953 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:26:07.981 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:08.076 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:08.098 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:26:08.128 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:08.144 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:26:08.175 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:26:08.197 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:26:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:26:08.316 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:26:08.346 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:26:08.389 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:26:08.393 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:26:08.462 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:26:08.507 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:26:08.545 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:26:08.555 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:26:08.599 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:26:08.631 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:26:08.659 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:26:08.659 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:26:08.727 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:26:08.752 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:08.781 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:26:08.815 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:26:08.832 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:08.862 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:26:08.935 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:26:08.970 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:26:09.012 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:26:09.042 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:26:09.061 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:26:09.173 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:26:09.190 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:26:09.248 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:26:09.288 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:26:09.332 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:26:09.376 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:26:09.388 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:26:09.442 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:26:09.457 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:26:09.501 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:26:09.533 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:26:09.578 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:26:09.609 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:26:09.637 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:26:09.707 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:26:09.769 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:26:09.806 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:26:09.850 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:26:09.926 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:26:09.961 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:26:09.963 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:26:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:26:10.033 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:26:10.037 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:26:10.075 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:26:10.103 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:26:10.117 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:26:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:26:10.155 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:26:10.197 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:26:10.230 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:26:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:26:10.372 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:26:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:26:10.426 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:26:10.488 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:26:10.586 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:26:10.626 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:26:10.662 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:26:10.708 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:26:10.744 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:26:10.794 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:26:10.836 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:26:10.955 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:26:10.972 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:26:11.024 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:26:11.037 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:26:11.064 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:26:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:26:11.101 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:26:11.146 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:26:11.166 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:26:11.187 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:26:11.207 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:26:11.246 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:26:11.272 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:26:11.296 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:26:11.339 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:26:11.371 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:26:11.400 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:26:11.453 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:26:11.526 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:26:11.556 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:26:11.566 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:26:11.605 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:26:11.629 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:26:11.649 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:26:11.656 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:26:11.685 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:26:11.691 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:26:11.727 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:26:11.730 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:26:11.761 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:26:11.770 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:26:11.791 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:26:11.817 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:26:11.851 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:26:11.892 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:26:11.905 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:26:11.928 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:26:11.955 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:26:12.005 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:26:12.014 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:26:12.065 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:26:12.096 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:26:12.107 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:26:12.149 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:26:12.188 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:26:12.199 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:26:12.217 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:26:12.239 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:26:12.256 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:26:12.313 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:26:12.602 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:26:12.646 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:26:12.673 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:26:12.708 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:26:12.747 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:26:12.778 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:26:12.808 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:26:12.843 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:26:12.877 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:26:12.918 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:26:12.953 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:26:12.977 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:26:13.019 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:26:13.038 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:26:13.052 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:26:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:26:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:26:13.153 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:26:13.224 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:26:13.232 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:26:13.273 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:26:13.278 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:26:13.366 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:26:13.413 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:26:13.596 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:26:13.627 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:26:13.704 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:26:14.020 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:26:14.070 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:26:14.103 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:26:14.201 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:26:14.260 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:26:14.303 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:26:14.340 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:26:14.382 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:26:14.414 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:26:14.459 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:26:14.606 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:26:14.637 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:26:14.714 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:26:15.357 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:26:15.476 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:26:15.689 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:26:15.763 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:26:15.791 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:26:15.824 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:26:16.132 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:26:16.178 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:26:16.269 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:26:16.300 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:26:16.336 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:26:16.688 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:26:16.773 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:26:16.801 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:26:16.815 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:26:16.834 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:26:16.839 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:26:16.911 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:26:16.941 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:26:17.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:26:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:26:17.279 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:26:17.310 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:26:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:26:17.830 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:26:17.866 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:26:17.962 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:26:18.001 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:26:19.253 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:19.287 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:19.832 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:19.873 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:26:19.966 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:26:19.966 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:26:19.966 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:26:19.966 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:20.000 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:26:20.271 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:20.305 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:20.850 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:20.900 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:26:20.993 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:26:20.993 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:26:20.993 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:26:20.996 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:21.027 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:26:25.909 INFO:teuthology.orchestra.run.smithi103.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 704 kB/s | 16 MB 00:23 2024-09-19T07:26:25.913 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:26:25.913 INFO:teuthology.orchestra.run.smithi103.stdout:Total 5.2 MB/s | 226 MB 00:43 2024-09-19T07:26:27.095 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-19T07:26:27.245 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-19T07:26:27.245 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-19T07:26:29.533 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-19T07:26:29.534 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-19T07:26:31.959 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-09-19T07:26:31.960 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:32.134 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:26:32.151 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:26:32.151 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:32.152 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:26:32.152 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-19T07:26:32.641 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:32.811 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:26:32.831 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:26:32.831 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:32.831 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:26:32.831 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:33.646 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:26:33.673 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:26:33.673 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:33.673 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:26:33.673 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:26:33.673 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:26:33.673 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:33.896 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:26:33.920 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:26:33.920 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:33.920 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:26:33.920 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:26:33.921 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:26:33.921 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:34.314 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:26:34.341 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:26:34.341 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:34.341 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:26:34.341 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:26:34.341 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:26:34.342 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:34.347 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:26:34.371 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:26:34.371 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:34.371 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:26:34.371 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:26:34.372 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:26:34.372 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:34.485 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:26:34.542 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:26:34.573 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:26:34.596 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:26:34.596 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:34.596 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:26:34.597 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:26:34.597 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:26:34.597 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:34.598 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:26:34.784 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:26:34.831 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:26:34.909 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:26:34.996 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:26:35.032 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:26:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:26:35.056 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:35.056 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:26:35.056 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:26:35.056 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:26:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:35.126 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:26:35.162 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:26:35.172 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:26:35.204 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:26:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:26:35.231 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:26:35.274 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:26:35.468 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:26:35.515 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:26:35.585 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:26:35.619 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:26:35.646 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:26:35.673 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:26:35.678 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:26:35.744 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:26:35.769 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:26:35.813 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:26:35.848 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-19T07:26:35.865 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:26:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:26:35.925 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:26:35.980 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T07:26:36.019 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T07:26:36.080 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T07:26:36.114 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T07:26:36.139 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T07:26:36.216 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T07:26:36.421 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:26:36.449 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:26:36.480 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:26:36.543 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:26:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:26:36.580 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T07:26:36.665 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:36.787 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:36.816 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:36.819 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:26:36.854 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:26:36.908 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:36.942 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:26:37.107 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T07:26:37.160 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T07:26:37.197 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:37.291 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:37.313 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:26:37.376 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T07:26:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:26:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T07:26:37.609 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:26:37.624 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:26:37.657 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:26:37.779 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T07:26:37.816 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:26:37.838 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:26:37.913 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T07:26:37.942 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:26:38.019 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:38.107 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:38.108 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:26:38.140 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:26:38.164 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:26:38.164 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:38.164 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:26:38.164 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:26:38.164 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:26:38.164 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:38.224 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:26:38.236 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:26:38.333 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:26:38.335 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T07:26:38.381 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:26:38.445 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T07:26:38.541 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T07:26:38.587 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T07:26:38.621 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T07:26:38.622 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:26:38.657 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:26:38.685 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T07:26:38.731 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T07:26:38.758 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:26:38.764 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T07:26:38.807 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T07:26:38.840 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T07:26:38.867 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T07:26:38.921 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:26:38.935 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T07:26:38.968 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:26:38.989 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T07:26:38.992 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:26:38.992 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:38.992 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:26:38.992 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:26:38.992 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:26:38.992 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:39.021 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T07:26:39.052 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:26:39.063 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:26:39.064 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T07:26:39.140 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T07:26:39.161 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:26:39.175 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T07:26:39.214 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T07:26:39.216 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:26:39.255 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T07:26:39.301 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:26:39.324 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T07:26:39.326 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:26:39.350 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:26:39.351 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:39.351 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:26:39.351 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:26:39.351 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:26:39.351 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:39.370 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T07:26:39.394 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:26:39.417 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:26:39.417 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:39.417 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:26:39.418 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:39.460 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T07:26:39.469 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:26:39.493 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:26:39.493 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:39.493 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:26:39.493 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:26:39.493 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:26:39.493 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:39.601 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T07:26:39.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T07:26:39.795 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:26:39.818 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:26:39.818 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:39.819 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:26:39.819 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:26:39.819 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:26:39.819 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:26:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:26:40.179 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T07:26:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:26:40.195 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:40.195 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:26:40.195 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:26:40.195 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:26:40.195 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:40.241 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:26:40.261 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:26:40.261 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:40.261 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:26:40.261 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:40.267 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T07:26:40.297 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T07:26:40.305 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:26:40.329 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:26:40.329 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:40.329 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:26:40.329 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:26:40.329 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:26:40.329 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:40.341 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T07:26:40.378 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T07:26:40.421 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T07:26:40.581 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T07:26:40.631 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:26:40.631 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T07:26:40.654 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:26:40.655 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:26:40.655 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:26:40.655 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:26:40.655 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:26:40.655 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:40.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T07:26:40.758 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T07:26:40.791 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T07:26:40.827 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T07:26:40.873 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T07:26:40.918 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T07:26:40.961 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T07:26:41.003 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T07:26:41.139 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T07:26:41.189 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T07:26:41.230 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T07:26:41.333 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T07:26:41.372 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T07:26:41.413 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T07:26:41.450 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T07:26:41.474 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T07:26:41.566 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T07:26:41.692 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T07:26:41.724 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T07:26:41.762 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T07:26:41.794 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T07:26:41.833 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T07:26:41.865 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T07:26:41.895 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T07:26:41.929 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T07:26:41.963 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T07:26:42.004 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T07:26:42.039 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T07:26:42.125 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T07:26:42.169 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T07:26:42.201 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T07:26:42.303 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T07:26:42.359 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T07:26:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T07:26:43.116 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T07:26:43.149 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T07:26:43.257 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T07:26:43.312 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T07:26:43.365 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T07:26:43.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T07:26:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T07:26:43.660 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T07:26:43.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T07:26:43.778 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T07:26:44.348 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:26:44.389 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:26:44.425 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T07:26:44.430 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:26:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:26:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:26:44.547 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T07:26:44.681 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:26:44.736 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:26:44.766 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:26:44.796 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:26:44.842 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:26:45.201 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:26:45.250 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:26:45.291 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:26:45.323 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:26:45.352 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:26:45.455 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:26:45.497 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:26:45.517 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:26:45.531 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:26:45.590 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:26:45.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:26:45.648 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:26:45.695 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:26:45.800 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T07:26:45.888 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T07:26:45.916 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T07:26:45.949 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T07:26:46.258 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T07:26:46.295 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:26:46.319 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:26:46.336 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:26:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:26:46.377 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T07:26:46.380 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:26:46.408 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T07:26:46.418 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:26:46.452 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T07:26:46.458 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:26:46.539 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:26:46.596 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:26:46.621 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:26:46.647 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:26:46.647 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:26:46.731 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:26:46.731 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:26:46.882 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:26:46.898 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:26:46.904 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:26:46.904 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:26:46.926 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T07:26:47.084 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:26:47.084 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:26:47.148 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:26:47.167 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:26:47.217 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:26:47.231 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:26:47.313 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:26:47.354 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:26:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T07:26:47.434 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:26:47.442 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T07:26:47.474 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:26:47.499 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:26:47.534 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:26:47.534 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:26:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:26:47.618 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:26:47.727 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:26:47.752 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:26:47.757 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:26:47.758 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:26:47.937 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:26:47.937 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:26:48.021 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:26:48.092 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:26:49.763 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:49.797 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T07:26:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:26:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T07:26:50.475 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T07:26:50.475 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T07:26:50.475 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:26:50.519 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:26:52.587 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:52.588 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:26:52.589 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:26:52.590 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:26:52.591 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:26:52.592 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:26:52.593 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:26:52.594 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:26:52.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:26:52.596 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:26:52.597 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:26:52.598 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:26:52.599 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:26:52.600 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:26:52.601 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:26:53.775 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:26:53.776 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:26:53.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:26:53.789 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:26:53.789 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:26:53.789 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:26:53.790 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:26:53.791 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:26:53.792 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:26:53.792 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:26:53.792 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:26:53.792 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:26:53.792 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:26:53.792 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:26:53.792 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:26:53.793 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:26:53.794 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:26:53.795 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:26:53.796 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:26:53.797 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:26:53.798 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:26:53.799 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:26:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:26:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:26:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.508 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.510 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:26:55.511 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:26:55.512 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:26:55.513 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:26:55.514 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.515 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:26:55.516 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:26:55.517 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:26:55.517 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:26:55.517 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:26:55.517 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:26:55.517 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:26:55.517 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:26:55.517 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:26:55.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:26:55.519 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:26:55.520 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:26:55.521 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:26:55.522 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:26:55.523 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:26:55.523 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-19T07:26:55.942 DEBUG:teuthology.parallel:result is None 2024-09-19T07:26:56.185 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:26:56.185 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:56.185 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-19T07:26:56.185 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:56.185 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.186 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.187 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:26:56.188 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:26:56.189 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:26:56.190 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:26:56.191 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:26:56.192 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:26:56.193 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:26:56.194 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.195 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:26:56.196 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:26:56.197 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:26:56.198 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:26:56.198 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:26:56.198 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-19T07:26:56.612 DEBUG:teuthology.parallel:result is None 2024-09-19T07:27:02.364 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T07:27:02.364 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-19T07:27:02.365 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-19T07:27:02.365 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-19T07:27:02.365 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-19T07:27:02.365 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-19T07:27:02.365 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-19T07:27:02.365 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-19T07:27:02.365 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:02.535 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:27:02.551 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T07:27:02.551 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:02.551 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T07:27:02.551 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:27:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T07:27:04.064 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:04.064 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T07:27:04.064 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:27:04.065 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T07:27:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:27:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T07:27:04.312 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:04.312 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T07:27:04.312 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:27:04.312 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T07:27:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:27:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T07:27:04.780 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:04.780 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T07:27:04.780 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:27:04.780 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T07:27:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T07:27:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T07:27:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T07:27:05.183 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T07:27:05.241 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T07:27:05.310 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T07:27:05.407 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T07:27:05.547 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T07:27:05.602 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T07:27:05.640 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T07:27:05.668 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T07:27:06.121 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:27:06.150 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T07:27:06.181 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T07:27:06.244 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T07:27:06.273 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:27:07.322 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T07:27:07.358 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:27:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T07:27:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:27:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T07:27:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T07:27:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:27:08.693 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T07:27:08.694 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:08.694 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T07:27:08.694 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:27:08.694 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T07:27:08.694 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:08.745 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:27:08.757 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T07:27:08.855 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T07:27:08.920 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T07:27:09.843 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T07:27:09.882 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:27:09.905 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T07:27:09.905 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:09.906 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T07:27:09.906 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:27:09.906 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T07:27:09.906 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:09.949 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:27:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T07:27:09.972 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:09.972 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T07:27:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:10.017 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:27:10.039 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T07:27:10.040 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:10.040 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T07:27:10.040 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:27:10.040 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T07:27:10.040 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:10.342 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:27:10.366 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T07:27:10.366 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T07:27:10.366 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T07:27:10.366 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:27:10.366 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T07:27:10.366 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:14.972 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T07:27:15.013 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T07:27:15.054 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T07:27:15.086 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T07:27:15.114 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T07:27:15.313 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T07:27:15.361 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T07:27:15.400 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T07:27:15.430 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T07:27:15.475 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T07:27:16.065 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:27:16.107 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:27:16.127 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T07:27:16.962 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T07:27:17.055 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T07:27:17.102 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T07:27:17.196 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T07:27:17.225 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T07:27:17.259 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T07:27:17.285 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T07:27:17.285 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:27:17.361 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T07:27:17.361 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:27:17.436 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T07:27:17.462 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T07:27:17.467 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:27:17.467 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:27:17.639 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T07:27:17.639 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:27:17.723 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T07:27:17.788 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:27:23.578 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:27:23.578 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T07:27:23.578 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T07:27:23.579 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T07:27:23.580 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T07:27:23.581 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T07:27:23.582 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T07:27:23.583 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T07:27:23.584 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T07:27:23.585 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T07:27:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T07:27:23.587 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T07:27:23.588 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T07:27:23.589 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T07:27:23.590 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T07:27:23.591 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T07:27:23.592 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T07:27:23.592 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T07:27:23.592 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T07:27:23.592 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T07:27:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T07:27:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:26.304 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T07:27:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T07:27:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T07:27:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T07:27:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T07:27:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T07:27:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T07:27:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T07:27:26.313 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T07:27:26.314 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T07:27:26.315 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T07:27:26.316 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T07:27:26.317 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T07:27:26.317 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T07:27:26.317 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T07:27:26.317 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:27:26.317 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-19T07:27:26.721 DEBUG:teuthology.parallel:result is None 2024-09-19T07:27:26.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:27:26.863 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:27:26.887 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:27:26.888 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:27:26.888 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:27:26.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:27:27.031 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:27:27.058 INFO:teuthology.orchestra.run.smithi134.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:27:27.059 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:27:27.059 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:27:27.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:27:27.197 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T07:27:27.224 INFO:teuthology.orchestra.run.smithi174.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T07:27:27.224 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T07:27:27.224 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T07:27:27.226 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T07:27:27.226 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:27:27.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:27:27.255 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:27:27.255 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:27:27.290 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:27:27.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T07:27:27.321 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T07:27:27.321 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:27:27.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:27:27.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:27:27.412 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:27:27.412 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:27:27.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:27:27.512 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:27:27.512 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T07:27:27.543 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T07:27:27.612 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T07:27:27.612 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:27:27.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:27:27.637 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:27:27.702 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:27:27.702 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:27:27.729 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:27:27.794 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:27:27.794 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T07:27:27.820 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T07:27:27.887 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T07:27:27.888 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:27:27.888 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:27:27.912 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:27:27.976 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:27:27.976 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:27:28.003 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:27:28.068 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:27:28.069 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T07:27:28.094 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T07:27:28.161 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T07:27:28.263 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2}} 2024-09-19T07:27:28.264 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:27:28.264 INFO:tasks.cephadm:Cluster fsid is 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:27:28.264 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T07:27:28.265 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.103', 'mon.b': '172.21.15.134', 'mon.c': '172.21.15.174'} 2024-09-19T07:27:28.265 INFO:tasks.cephadm:First mon is mon.a on smithi103 2024-09-19T07:27:28.265 INFO:tasks.cephadm:First mgr is x 2024-09-19T07:27:28.265 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T07:27:28.265 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-19T07:27:28.291 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-09-19T07:27:28.316 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-09-19T07:27:28.342 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T07:27:28.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:27:28.478 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T07:27:28.614 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:27:28.615 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T07:27:28.615 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T07:27:28.616 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:28.991 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 07:27 /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:28.991 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:29.403 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 07:27 /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:29.403 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:29.796 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 07:27 /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:29.797 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:29.816 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:29.833 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T07:27:29.867 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T07:27:29.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T07:27:29.870 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T07:27:29.876 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T07:27:31.309 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:27:31.496 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:27:31.542 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:27:58.249 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:27:58.249 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:27:58.249 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T07:27:58.250 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-19T07:27:58.250 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T07:27:58.250 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:27:58.250 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:28:00.835 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:28:00.835 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:28:00.835 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T07:28:00.835 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-09-19T07:28:00.835 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T07:28:00.835 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-09-19T07:28:00.835 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:28:07.796 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:28:07.796 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T07:28:07.796 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T07:28:07.797 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-09-19T07:28:07.797 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T07:28:07.797 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-19T07:28:07.797 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:28:07.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-19T07:28:07.852 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-09-19T07:28:07.879 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-09-19T07:28:07.913 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-19T07:28:07.941 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-09-19T07:28:07.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-09-19T07:28:07.995 INFO:tasks.cephadm:Writing seed config... 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T07:28:08.000 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T07:28:08.001 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T07:28:08.002 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T07:28:08.003 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T07:28:08.004 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:28:08.004 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T07:28:08.022 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9fd714d2-7658-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T07:28:08.023 DEBUG:teuthology.orchestra.run.smithi103:mon.a> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T07:28:08.065 DEBUG:teuthology.orchestra.run.smithi103:mgr.x> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T07:28:08.108 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T07:28:08.109 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.103 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:28:08.315 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-19T07:28:08.315 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', '9fd714d2-7658-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.103', '--skip-admin-label'] 2024-09-19T07:28:08.315 INFO:teuthology.orchestra.run.smithi103.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T07:28:08.315 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying podman|docker is present... 2024-09-19T07:28:08.339 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-19T07:28:08.340 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying lvm2 is present... 2024-09-19T07:28:08.340 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying time synchronization is in place... 2024-09-19T07:28:08.347 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T07:28:08.347 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T07:28:08.353 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T07:28:08.353 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T07:28:08.360 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-19T07:28:08.366 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-19T07:28:08.367 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-19T07:28:08.367 INFO:teuthology.orchestra.run.smithi103.stdout:Repeating the final host check... 2024-09-19T07:28:08.389 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-19T07:28:08.389 INFO:teuthology.orchestra.run.smithi103.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T07:28:08.390 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl is present 2024-09-19T07:28:08.390 INFO:teuthology.orchestra.run.smithi103.stdout:lvcreate is present 2024-09-19T07:28:08.396 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T07:28:08.396 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T07:28:08.402 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T07:28:08.402 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T07:28:08.409 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-19T07:28:08.415 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-19T07:28:08.415 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-19T07:28:08.415 INFO:teuthology.orchestra.run.smithi103.stdout:Host looks OK 2024-09-19T07:28:08.416 INFO:teuthology.orchestra.run.smithi103.stdout:Cluster fsid: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:08.416 INFO:teuthology.orchestra.run.smithi103.stdout:Acquiring lock 140514923271168 on /run/cephadm/9fd714d2-7658-11ef-bceb-c7b262605968.lock 2024-09-19T07:28:08.416 INFO:teuthology.orchestra.run.smithi103.stdout:Lock 140514923271168 acquired on /run/cephadm/9fd714d2-7658-11ef-bceb-c7b262605968.lock 2024-09-19T07:28:08.416 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 3300 ... 2024-09-19T07:28:08.416 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 6789 ... 2024-09-19T07:28:08.417 INFO:teuthology.orchestra.run.smithi103.stdout:Base mon IP(s) is [172.21.15.103:3300, 172.21.15.103:6789], mon addrv is [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-19T07:28:08.420 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.103 metric 100 2024-09-19T07:28:08.420 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.103 metric 100 2024-09-19T07:28:08.422 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T07:28:08.422 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-19T07:28:08.422 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-19T07:28:08.425 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T07:28:08.425 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T07:28:08.425 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T07:28:08.425 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-19T07:28:08.425 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-19T07:28:08.426 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T07:28:08.426 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-19T07:28:08.426 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-19T07:28:08.426 INFO:teuthology.orchestra.run.smithi103.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T07:28:08.426 INFO:teuthology.orchestra.run.smithi103.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T07:28:08.427 INFO:teuthology.orchestra.run.smithi103.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:28:08.909 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T07:28:08.909 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T07:28:08.909 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T07:28:08.909 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T07:28:08.909 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T07:28:08.909 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T07:28:08.909 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T07:28:14.491 INFO:teuthology.orchestra.run.smithi103.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T07:28:14.491 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T07:28:14.491 INFO:teuthology.orchestra.run.smithi103.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T07:28:23.830 INFO:teuthology.orchestra.run.smithi103.stdout:stat: stdout 167 167 2024-09-19T07:28:23.830 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial keys... 2024-09-19T07:28:24.734 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQCY0utmEWW/BxAA/T2D+B4ZICbYi+JUZEw5Uw== 2024-09-19T07:28:25.675 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQCZ0utm0cWxBBAAUSGnKvmxzva5Tayh7IO1cw== 2024-09-19T07:28:26.647 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQCa0utmkFuKARAAlwfNUidz1c+4BT0qJb0cGA== 2024-09-19T07:28:26.648 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial monmap... 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool for a [v2:172.21.15.103:3300,v1:172.21.15.103:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: set fsid to 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T07:28:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:28:27.571 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mon... 2024-09-19T07:28:28.493 INFO:teuthology.orchestra.run.smithi103.stdout:create mon.a on 2024-09-19T07:28:28.701 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T07:28:28.864 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T07:28:29.436 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9fd714d2-7658-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9fd714d2-7658-11ef-bceb-c7b262605968.target. 2024-09-19T07:28:29.436 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9fd714d2-7658-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-9fd714d2-7658-11ef-bceb-c7b262605968.target. 2024-09-19T07:28:30.072 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a 2024-09-19T07:28:30.073 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T07:28:30.246 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9fd714d2-7658-11ef-bceb-c7b262605968.target.wants/ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9fd714d2-7658-11ef-bceb-c7b262605968@.service. 2024-09-19T07:28:30.774 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:30 smithi103 ceph-mon[30831]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T07:28:30.774 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:30 smithi103 bash[30813]: b7aed0eda3c7f75a9dbb7f2a1ae8182f8086d09888e56b23e3e4e82ea6274192 2024-09-19T07:28:30.787 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:28:30.788 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T07:28:30.795 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:28:30.795 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T07:28:30.795 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T07:28:30.795 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T07:28:30.795 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon to start... 2024-09-19T07:28:30.796 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon... 2024-09-19T07:28:31.034 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:30 smithi103 systemd[1]: Started Ceph mon.a for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: monmap epoch 1 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: last_changed 2024-09-19T07:28:26.970477+0000 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: min_mon_release 19 (squid) 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: election_strategy: 1 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: fsmap 2024-09-19T07:28:31.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T07:28:31.915 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: mgrmap e1: no daemons active 2024-09-19T07:28:31.915 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:31 smithi103 ceph-mon[30831]: from='client.? 172.21.15.103:0/3688300690' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout id: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout services: 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.558057s) 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T07:28:31.922 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout data: 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.211+0000 7fe0c71d8640 1 Processor -- start 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.211+0000 7fe0c71d8640 1 -- start start 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.212+0000 7fe0c71d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c0107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.212+0000 7fe0c71d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0108000 con 0x7fe0c0107630 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.212+0000 7fe0c4f74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c0107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:31.923 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.212+0000 7fe0c4f74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c0107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36832/0 (socket says 172.21.15.103:36832) 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.212+0000 7fe0c4f74640 1 -- 172.21.15.103:0/3761717757 learned_addr learned my addr 172.21.15.103:0/3761717757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.212+0000 7fe0c4f74640 1 -- 172.21.15.103:0/3761717757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c01087e0 con 0x7fe0c0107630 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.214+0000 7fe0c4f74640 1 --2- 172.21.15.103:0/3761717757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c0107a30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009820 tx=0x7fe0b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c18a8e41d8ed28f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0bf7fe640 1 -- 172.21.15.103:0/3761717757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0b4037070 con 0x7fe0c0107630 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0bf7fe640 1 -- 172.21.15.103:0/3761717757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b4032440 con 0x7fe0c0107630 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0bf7fe640 1 -- 172.21.15.103:0/3761717757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0b4032740 con 0x7fe0c0107630 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3761717757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 msgr2=0x7fe0c0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0c71d8640 1 --2- 172.21.15.103:0/3761717757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c0107a30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009820 tx=0x7fe0b402a270 comp rx=0 tx=0).stop 2024-09-19T07:28:31.924 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3761717757 shutdown_connections 2024-09-19T07:28:31.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0c71d8640 1 --2- 172.21.15.103:0/3761717757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c0107a30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:31.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3761717757 >> 172.21.15.103:0/3761717757 conn(0x7fe0c0102da0 msgr2=0x7fe0c0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:31.925 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3761717757 shutdown_connections 2024-09-19T07:28:31.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.215+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3761717757 wait complete. 2024-09-19T07:28:31.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c71d8640 1 Processor -- start 2024-09-19T07:28:31.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c71d8640 1 -- start start 2024-09-19T07:28:31.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c71d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:31.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c71d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0108d80 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c4f74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c4f74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36840/0 (socket says 172.21.15.103:36840) 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c4f74640 1 -- 172.21.15.103:0/3688300690 learned_addr learned my addr 172.21.15.103:0/3688300690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.216+0000 7fe0c4f74640 1 -- 172.21.15.103:0/3688300690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c019b2e0 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0c4f74640 1 --2- 172.21.15.103:0/3688300690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c019ada0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40090b0 tx=0x7fe0b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0bdffb640 1 -- 172.21.15.103:0/3688300690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0b4041070 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0bdffb640 1 -- 172.21.15.103:0/3688300690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b403c7a0 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c019b570 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0bdffb640 1 -- 172.21.15.103:0/3688300690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0b4037040 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c019e220 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0bdffb640 1 -- 172.21.15.103:0/3688300690 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe0b4044070 con 0x7fe0c0107630 2024-09-19T07:28:31.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.217+0000 7fe0bdffb640 1 -- 172.21.15.103:0/3688300690 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe0b403d3f0 con 0x7fe0c0107630 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.218+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c01093d0 con 0x7fe0c0107630 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.220+0000 7fe0bdffb640 1 -- 172.21.15.103:0/3688300690 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe0b4046450 con 0x7fe0c0107630 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.279+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe0c019e810 con 0x7fe0c0107630 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.279+0000 7fe0bdffb640 1 -- 172.21.15.103:0/3688300690 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe0b4046630 con 0x7fe0c0107630 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.280+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 msgr2=0x7fe0c019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.280+0000 7fe0c71d8640 1 --2- 172.21.15.103:0/3688300690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c019ada0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40090b0 tx=0x7fe0b40046c0 comp rx=0 tx=0).stop 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.280+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 shutdown_connections 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.280+0000 7fe0c71d8640 1 --2- 172.21.15.103:0/3688300690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0c0107630 0x7fe0c019ada0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.280+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 >> 172.21.15.103:0/3688300690 conn(0x7fe0c0102da0 msgr2=0x7fe0c018f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.280+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 shutdown_connections 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:31.280+0000 7fe0c71d8640 1 -- 172.21.15.103:0/3688300690 wait complete. 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:mon is available 2024-09-19T07:28:31.930 INFO:teuthology.orchestra.run.smithi103.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T07:28:32.954 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:32 smithi103 ceph-mon[30831]: from='client.? 172.21.15.103:0/1010860108' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T07:28:32.954 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:32 smithi103 ceph-mon[30831]: from='client.? 172.21.15.103:0/1010860108' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T07:28:32.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:32.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T07:28:32.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T07:28:32.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.335+0000 7efcbbb50640 1 Processor -- start 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.335+0000 7efcbbb50640 1 -- start start 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.336+0000 7efcbbb50640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb4107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.336+0000 7efcbbb50640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4107fd0 con 0x7efcb4107600 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.336+0000 7efcb98ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb4107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.336+0000 7efcb98ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb4107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36846/0 (socket says 172.21.15.103:36846) 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.336+0000 7efcb98ec640 1 -- 172.21.15.103:0/2532946326 learned_addr learned my addr 172.21.15.103:0/2532946326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.336+0000 7efcb98ec640 1 -- 172.21.15.103:0/2532946326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb4108800 con 0x7efcb4107600 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.337+0000 7efcb98ec640 1 --2- 172.21.15.103:0/2532946326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb4107a00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efca8009820 tx=0x7efca802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cf825db1b54771 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:32.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.337+0000 7efcb88ea640 1 -- 172.21.15.103:0/2532946326 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca8037070 con 0x7efcb4107600 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.337+0000 7efcb88ea640 1 -- 172.21.15.103:0/2532946326 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca8032440 con 0x7efcb4107600 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.337+0000 7efcb88ea640 1 -- 172.21.15.103:0/2532946326 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca8032740 con 0x7efcb4107600 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 -- 172.21.15.103:0/2532946326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 msgr2=0x7efcb4107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 --2- 172.21.15.103:0/2532946326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb4107a00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7efca8009820 tx=0x7efca802a270 comp rx=0 tx=0).stop 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 -- 172.21.15.103:0/2532946326 shutdown_connections 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 --2- 172.21.15.103:0/2532946326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb4107a00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 -- 172.21.15.103:0/2532946326 >> 172.21.15.103:0/2532946326 conn(0x7efcb4102dd0 msgr2=0x7efcb41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 -- 172.21.15.103:0/2532946326 shutdown_connections 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 -- 172.21.15.103:0/2532946326 wait complete. 2024-09-19T07:28:32.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 Processor -- start 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 -- start start 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.338+0000 7efcbbb50640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcbbb50640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4108da0 con 0x7efcb4107600 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcb98ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcb98ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36852/0 (socket says 172.21.15.103:36852) 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcb98ec640 1 -- 172.21.15.103:0/1010860108 learned_addr learned my addr 172.21.15.103:0/1010860108 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcb98ec640 1 -- 172.21.15.103:0/1010860108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb419b300 con 0x7efcb4107600 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcb98ec640 1 --2- 172.21.15.103:0/1010860108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb419adc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7efca80090b0 tx=0x7efca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:32.966 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca8041070 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca803c7a0 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb419b590 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca8037040 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb419e280 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efca8044070 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.339+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efca803d3f0 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.340+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb41093f0 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.342+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efca8046450 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.399+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7efcb402d940 con 0x7efcb4107600 2024-09-19T07:28:32.967 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.403+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7efca8046630 con 0x7efcb4107600 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efca6ffd640 1 -- 172.21.15.103:0/1010860108 <== mon.0 v2:172.21.15.103:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efca8056070 con 0x7efcb4107600 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 msgr2=0x7efcb419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efcbbb50640 1 --2- 172.21.15.103:0/1010860108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb419adc0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7efca80090b0 tx=0x7efca80046c0 comp rx=0 tx=0).stop 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 shutdown_connections 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efcbbb50640 1 --2- 172.21.15.103:0/1010860108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcb4107600 0x7efcb419adc0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 >> 172.21.15.103:0/1010860108 conn(0x7efcb4102dd0 msgr2=0x7efcb418efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 shutdown_connections 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:32.404+0000 7efcbbb50640 1 -- 172.21.15.103:0/1010860108 wait complete. 2024-09-19T07:28:32.968 INFO:teuthology.orchestra.run.smithi103.stdout:Generating new minimal ceph.conf... 2024-09-19T07:28:33.944 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:33 smithi103 ceph-mon[30831]: from='client.? 172.21.15.103:0/1382166716' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:28:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48c6f6640 1 Processor -- start 2024-09-19T07:28:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48c6f6640 1 -- start start 2024-09-19T07:28:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48c6f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff484107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48c6f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff484107ff0 con 0x7ff484107620 2024-09-19T07:28:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48a492640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff484107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48a492640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff484107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36862/0 (socket says 172.21.15.103:36862) 2024-09-19T07:28:33.952 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48a492640 1 -- 172.21.15.103:0/1038893164 learned_addr learned my addr 172.21.15.103:0/1038893164 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.333+0000 7ff48a492640 1 -- 172.21.15.103:0/1038893164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff484108820 con 0x7ff484107620 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.334+0000 7ff48a492640 1 --2- 172.21.15.103:0/1038893164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff484107a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff480009820 tx=0x7ff48002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475bc7501786dceb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.334+0000 7ff489490640 1 -- 172.21.15.103:0/1038893164 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480037070 con 0x7ff484107620 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.334+0000 7ff489490640 1 -- 172.21.15.103:0/1038893164 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff480032900 con 0x7ff484107620 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.334+0000 7ff489490640 1 -- 172.21.15.103:0/1038893164 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480030850 con 0x7ff484107620 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 -- 172.21.15.103:0/1038893164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 msgr2=0x7ff484107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 --2- 172.21.15.103:0/1038893164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff484107a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff480009820 tx=0x7ff48002a270 comp rx=0 tx=0).stop 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 -- 172.21.15.103:0/1038893164 shutdown_connections 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 --2- 172.21.15.103:0/1038893164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff484107a20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:33.953 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 -- 172.21.15.103:0/1038893164 >> 172.21.15.103:0/1038893164 conn(0x7ff484102db0 msgr2=0x7ff4841051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 -- 172.21.15.103:0/1038893164 shutdown_connections 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 -- 172.21.15.103:0/1038893164 wait complete. 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 Processor -- start 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.335+0000 7ff48c6f6640 1 -- start start 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48c6f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff48419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48c6f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff48410bbd0 con 0x7ff484107620 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48a492640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff48419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48a492640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff48419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36872/0 (socket says 172.21.15.103:36872) 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48a492640 1 -- 172.21.15.103:0/1382166716 learned_addr learned my addr 172.21.15.103:0/1382166716 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:33.954 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48a492640 1 -- 172.21.15.103:0/1382166716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48419bb90 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48a492640 1 --2- 172.21.15.103:0/1382166716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff48419b650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff480009940 tx=0x7ff4800326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff47b7fe640 1 -- 172.21.15.103:0/1382166716 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480042070 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff47b7fe640 1 -- 172.21.15.103:0/1382166716 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff480032be0 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48c6f6640 1 -- 172.21.15.103:0/1382166716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48419be20 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff47b7fe640 1 -- 172.21.15.103:0/1382166716 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480037040 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff48c6f6640 1 -- 172.21.15.103:0/1382166716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48419eb10 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff47b7fe640 1 -- 172.21.15.103:0/1382166716 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff480045070 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.336+0000 7ff47b7fe640 1 -- 172.21.15.103:0/1382166716 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4800463b0 con 0x7ff484107620 2024-09-19T07:28:33.955 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.337+0000 7ff4797fa640 1 -- 172.21.15.103:0/1382166716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c005290 con 0x7ff484107620 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.339+0000 7ff47b7fe640 1 -- 172.21.15.103:0/1382166716 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff480032d90 con 0x7ff484107620 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.397+0000 7ff4797fa640 1 -- 172.21.15.103:0/1382166716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff46c005850 con 0x7ff484107620 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.397+0000 7ff47b7fe640 1 -- 172.21.15.103:0/1382166716 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7ff480047440 con 0x7ff484107620 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.398+0000 7ff4797fa640 1 -- 172.21.15.103:0/1382166716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 msgr2=0x7ff48419b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.398+0000 7ff4797fa640 1 --2- 172.21.15.103:0/1382166716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff48419b650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff480009940 tx=0x7ff4800326e0 comp rx=0 tx=0).stop 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.398+0000 7ff4797fa640 1 -- 172.21.15.103:0/1382166716 shutdown_connections 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.398+0000 7ff4797fa640 1 --2- 172.21.15.103:0/1382166716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff484107620 0x7ff48419b650 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.398+0000 7ff4797fa640 1 -- 172.21.15.103:0/1382166716 >> 172.21.15.103:0/1382166716 conn(0x7ff484102db0 msgr2=0x7ff484103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.398+0000 7ff4797fa640 1 -- 172.21.15.103:0/1382166716 shutdown_connections 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:33.398+0000 7ff4797fa640 1 -- 172.21.15.103:0/1382166716 wait complete. 2024-09-19T07:28:33.956 INFO:teuthology.orchestra.run.smithi103.stdout:Restarting the monitor... 2024-09-19T07:28:34.214 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:33 smithi103 systemd[1]: Stopping Ceph mon.a for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:28:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:34 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a[30827]: 2024-09-19T07:28:34.213+0000 7f16c6812640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:28:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:34 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a[30827]: 2024-09-19T07:28:34.213+0000 7f16c6812640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T07:28:34.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:34 smithi103 podman[31073]: 2024-09-19 07:28:34.418198451 +0000 UTC m=+0.307305184 container died b7aed0eda3c7f75a9dbb7f2a1ae8182f8086d09888e56b23e3e4e82ea6274192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T07:28:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:34 smithi103 podman[31073]: 2024-09-19 07:28:34.755153685 +0000 UTC m=+0.644260408 container cleanup b7aed0eda3c7f75a9dbb7f2a1ae8182f8086d09888e56b23e3e4e82ea6274192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4) 2024-09-19T07:28:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:34 smithi103 bash[31073]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a 2024-09-19T07:28:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:34 smithi103 podman[31085]: 2024-09-19 07:28:34.922692649 +0000 UTC m=+0.505014226 container remove b7aed0eda3c7f75a9dbb7f2a1ae8182f8086d09888e56b23e3e4e82ea6274192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:28:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T07:28:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 systemd[1]: Stopped Ceph mon.a for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:28:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 systemd[1]: Starting Ceph mon.a for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:28:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 podman[31181]: 2024-09-19 07:28:35.373681231 +0000 UTC m=+0.103678274 container create 4589aa5fd5c9e6403cc484e63e5942af58178eb9eed84a33c89c35e671fc416a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T07:28:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 podman[31181]: 2024-09-19 07:28:35.286144912 +0000 UTC m=+0.016141966 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T07:28:35.714 INFO:teuthology.orchestra.run.smithi103.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T07:28:35.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 podman[31181]: 2024-09-19 07:28:35.621361939 +0000 UTC m=+0.351358992 container init 4589aa5fd5c9e6403cc484e63e5942af58178eb9eed84a33c89c35e671fc416a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T07:28:35.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 podman[31181]: 2024-09-19 07:28:35.624609592 +0000 UTC m=+0.354606635 container start 4589aa5fd5c9e6403cc484e63e5942af58178eb9eed84a33c89c35e671fc416a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T07:28:35.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T07:28:35.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: pidfile_write: ignore empty --pid-file 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: load: jerasure load: lrc 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: RocksDB version: 7.9.2 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Git sha 0 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: DB SUMMARY 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: DB Session ID: YQCCFPYZAPZY7N89HY8D 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: CURRENT file: CURRENT 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105300 ; 2024-09-19T07:28:35.828 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.error_if_exists: 0 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.create_if_missing: 0 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.paranoid_checks: 1 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.env: 0x5557051e1940 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.info_log: 0x5557069016e0 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T07:28:35.829 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.statistics: (nil) 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.use_fsync: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_log_file_size: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.allow_fallocate: 1 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.use_direct_reads: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.db_log_dir: 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.wal_dir: 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T07:28:35.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.write_buffer_manager: 0x555706910dc0 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.unordered_write: 0 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T07:28:35.833 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.row_cache: None 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.wal_filter: None 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.two_write_queues: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.wal_compression: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.atomic_flush: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.log_readahead_size: 0 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T07:28:35.834 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_background_jobs: 2 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_background_compactions: -1 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_subcompactions: 1 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T07:28:35.835 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_open_files: -1 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_background_flushes: -1 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Compression algorithms supported: 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kZSTD supported: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kXpressCompression supported: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kBZip2Compression supported: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kLZ4Compression supported: 1 2024-09-19T07:28:35.836 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kZlibCompression supported: 1 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: kSnappyCompression supported: 1 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.merge_operator: 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_filter: None 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T07:28:35.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555706901320) 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_type: 0 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_index_type: 0 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_shortening: 1 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: checksum: 4 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: no_block_cache: 0 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache: 0x5557068f51f0 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_options: 2024-09-19T07:28:35.838 INFO:journalctl@ceph.mon.a.smithi103.stdout: capacity : 536870912 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_shard_bits : 4 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: strict_capacity_limit : 0 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_compressed: (nil) 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: persistent_cache: (nil) 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size: 4096 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size_deviation: 10 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_restart_interval: 16 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_block_restart_interval: 1 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: metadata_block_size: 4096 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: partition_filters: 0 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: use_delta_encoding: 1 2024-09-19T07:28:35.839 INFO:journalctl@ceph.mon.a.smithi103.stdout: filter_policy: bloomfilter 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: whole_key_filtering: 1 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: verify_compression: 0 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: format_version: 5 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_index_compression: 1 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_align: 0 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: max_auto_readahead_size: 262144 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: prepopulate_block_cache: 0 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: initial_auto_readahead_size: 8192 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression: NoCompression 2024-09-19T07:28:35.840 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.num_levels: 7 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T07:28:35.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T07:28:35.842 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T07:28:35.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T07:28:35.844 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.inplace_update_support: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.bloom_locality: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.max_successive_merges: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T07:28:35.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.ttl: 2592000 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.enable_blob_files: false 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.min_blob_size: 0 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T07:28:35.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ddb012b8-970a-4432-9b07-97b40e5815b9 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730915651003, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730915651971, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100367, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98252, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14290, "raw_average_key_size": 49, "raw_value_size": 90525, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726730915, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ddb012b8-970a-4432-9b07-97b40e5815b9", "db_session_id": "YQCCFPYZAPZY7N89HY8D", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730915652067, "job": 1, "event": "recovery_finished"} 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555706922e00 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: DB pointer 0x555706a1e000 2024-09-19T07:28:35.847 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** DB Stats ** 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:28:35.848 INFO:journalctl@ceph.mon.a.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: L0 2/0 99.88 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 162.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: Sum 2/0 99.88 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 162.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 162.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 162.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T07:28:35.849 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative compaction: 0.00 GB write, 30.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval compaction: 0.00 GB write, 30.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache BinnedLRUCache@0x5557068f51f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: starting mon.a rank 0 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???) e1 preinit fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:35.850 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???).mds e1 new map 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???).mds e1 print_map 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout: e1 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout: btime 2024-09-19T07:28:30:723424+0000 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout: legacy client fscid: -1 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout: No filesystems configured 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T07:28:35.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 bash[31181]: 4589aa5fd5c9e6403cc484e63e5942af58178eb9eed84a33c89c35e671fc416a 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 systemd[1]: Started Ceph mon.a for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: monmap epoch 1 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: last_changed 2024-09-19T07:28:26.970477+0000 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: min_mon_release 19 (squid) 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: election_strategy: 1 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: fsmap 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T07:28:35.852 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:35 smithi103 ceph-mon[31201]: mgrmap e1: no daemons active 2024-09-19T07:28:36.747 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.201+0000 7ff39c63b640 1 Processor -- start 2024-09-19T07:28:36.747 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.201+0000 7ff39c63b640 1 -- start start 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.201+0000 7ff39c63b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff394107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.201+0000 7ff39c63b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff394107ff0 con 0x7ff394107620 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.202+0000 7ff39a3d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff394107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.202+0000 7ff39a3d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff394107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36884/0 (socket says 172.21.15.103:36884) 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.202+0000 7ff39a3d7640 1 -- 172.21.15.103:0/3307244934 learned_addr learned my addr 172.21.15.103:0/3307244934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.202+0000 7ff39a3d7640 1 -- 172.21.15.103:0/3307244934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff394108820 con 0x7ff394107620 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.204+0000 7ff39a3d7640 1 --2- 172.21.15.103:0/3307244934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff394107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff388009820 tx=0x7ff38802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee20d31da1c3e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.204+0000 7ff3993d5640 1 -- 172.21.15.103:0/3307244934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff388037070 con 0x7ff394107620 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.204+0000 7ff3993d5640 1 -- 172.21.15.103:0/3307244934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff388032900 con 0x7ff394107620 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.204+0000 7ff3993d5640 1 -- 172.21.15.103:0/3307244934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff388030850 con 0x7ff394107620 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.205+0000 7ff39c63b640 1 -- 172.21.15.103:0/3307244934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 msgr2=0x7ff394107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.205+0000 7ff39c63b640 1 --2- 172.21.15.103:0/3307244934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff394107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff388009820 tx=0x7ff38802a270 comp rx=0 tx=0).stop 2024-09-19T07:28:36.748 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.205+0000 7ff39c63b640 1 -- 172.21.15.103:0/3307244934 shutdown_connections 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.205+0000 7ff39c63b640 1 --2- 172.21.15.103:0/3307244934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff394107a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.205+0000 7ff39c63b640 1 -- 172.21.15.103:0/3307244934 >> 172.21.15.103:0/3307244934 conn(0x7ff394102db0 msgr2=0x7ff3941051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.205+0000 7ff39c63b640 1 -- 172.21.15.103:0/3307244934 shutdown_connections 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.205+0000 7ff39c63b640 1 -- 172.21.15.103:0/3307244934 wait complete. 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39c63b640 1 Processor -- start 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39c63b640 1 -- start start 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39c63b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff39419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39c63b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410bbd0 con 0x7ff394107620 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39a3d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff39419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39a3d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff39419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36892/0 (socket says 172.21.15.103:36892) 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39a3d7640 1 -- 172.21.15.103:0/3053186502 learned_addr learned my addr 172.21.15.103:0/3053186502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39a3d7640 1 -- 172.21.15.103:0/3053186502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39419bb90 con 0x7ff394107620 2024-09-19T07:28:36.749 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39a3d7640 1 --2- 172.21.15.103:0/3053186502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff39419b650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff388009940 tx=0x7ff3880326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff3877fe640 1 -- 172.21.15.103:0/3053186502 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff388042070 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39419be20 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.206+0000 7ff3877fe640 1 -- 172.21.15.103:0/3053186502 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff388032be0 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.207+0000 7ff3877fe640 1 -- 172.21.15.103:0/3053186502 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff388037040 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.207+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39419eb10 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.207+0000 7ff3877fe640 1 -- 172.21.15.103:0/3053186502 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff388045070 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.207+0000 7ff3877fe640 1 -- 172.21.15.103:0/3053186502 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff388047a80 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.207+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff39410bd50 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.210+0000 7ff3877fe640 1 -- 172.21.15.103:0/3053186502 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff388032d90 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.266+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff394107aa0 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.268+0000 7ff3877fe640 1 -- 172.21.15.103:0/3053186502 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff38803d430 con 0x7ff394107620 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.269+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 msgr2=0x7ff39419b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:36.750 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.269+0000 7ff39c63b640 1 --2- 172.21.15.103:0/3053186502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff39419b650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff388009940 tx=0x7ff3880326e0 comp rx=0 tx=0).stop 2024-09-19T07:28:36.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.269+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 shutdown_connections 2024-09-19T07:28:36.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.269+0000 7ff39c63b640 1 --2- 172.21.15.103:0/3053186502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff394107620 0x7ff39419b650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:36.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.269+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 >> 172.21.15.103:0/3053186502 conn(0x7ff394102db0 msgr2=0x7ff394103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:36.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.269+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 shutdown_connections 2024-09-19T07:28:36.751 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:36.269+0000 7ff39c63b640 1 -- 172.21.15.103:0/3053186502 wait complete. 2024-09-19T07:28:36.751 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T07:28:36.765 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:28:36.766 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mgr... 2024-09-19T07:28:36.766 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T07:28:36.766 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T07:28:37.017 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.x 2024-09-19T07:28:37.017 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T07:28:37.185 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9fd714d2-7658-11ef-bceb-c7b262605968.target.wants/ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9fd714d2-7658-11ef-bceb-c7b262605968@.service. 2024-09-19T07:28:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:37 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3053186502' entity='client.admin' 2024-09-19T07:28:37.470 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:37 smithi103 podman[31413]: 2024-09-19 07:28:37.427464273 +0000 UTC m=+0.123826351 container create a2584dd22f2bb1809b381c9e80076660755bff422046ad552b056bddb44d8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:28:37.777 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:28:37.777 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T07:28:37.784 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:28:37.785 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T07:28:37.785 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T07:28:37.785 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T07:28:37.793 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:28:37.793 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T07:28:37.801 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:28:37.801 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T07:28:37.801 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T07:28:37.801 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T07:28:37.801 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr to start... 2024-09-19T07:28:37.801 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr... 2024-09-19T07:28:37.814 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:37 smithi103 podman[31413]: 2024-09-19 07:28:37.687925943 +0000 UTC m=+0.384287980 container init a2584dd22f2bb1809b381c9e80076660755bff422046ad552b056bddb44d8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T07:28:37.815 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:37 smithi103 podman[31413]: 2024-09-19 07:28:37.690741091 +0000 UTC m=+0.387103134 container start a2584dd22f2bb1809b381c9e80076660755bff422046ad552b056bddb44d8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T07:28:37.815 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:37 smithi103 bash[31413]: a2584dd22f2bb1809b381c9e80076660755bff422046ad552b056bddb44d8e72 2024-09-19T07:28:37.815 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:37 smithi103 systemd[1]: Started Ceph mgr.x for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:28:38.095 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:37.813+0000 7f39ddf9e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:28:38.095 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:37.872+0000 7f39ddf9e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:28:38.095 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:38.026+0000 7f39ddf9e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:28:38.095 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:38.093+0000 7f39ddf9e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:28:38.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1786747157' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T07:28:38.393 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:38.247+0000 7f39ddf9e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:28:38.393 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:38.392+0000 7f39ddf9e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:28:38.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:38.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:28:38.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:28:38.926 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T07:28:38.927 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:38.928 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T07:28:30:723424+0000", 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T07:28:38.929 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T07:28:38.930 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T07:28:30.724082+0000", 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.210+0000 7fc8eeff6640 1 Processor -- start 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.210+0000 7fc8eeff6640 1 -- start start 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.210+0000 7fc8eeff6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e810a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.210+0000 7fc8eeff6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e807be30 con 0x7fc8e8108080 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.211+0000 7fc8ecd92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e810a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.211+0000 7fc8ecd92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e810a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59700/0 (socket says 172.21.15.103:59700) 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.211+0000 7fc8ecd92640 1 -- 172.21.15.103:0/2098737236 learned_addr learned my addr 172.21.15.103:0/2098737236 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.211+0000 7fc8ecd92640 1 -- 172.21.15.103:0/2098737236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e810a9b0 con 0x7fc8e8108080 2024-09-19T07:28:38.931 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.212+0000 7fc8ecd92640 1 --2- 172.21.15.103:0/2098737236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e810a470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009820 tx=0x7fc8dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fedb9d4e0f2b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.212+0000 7fc8e77fe640 1 -- 172.21.15.103:0/2098737236 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc037070 con 0x7fc8e8108080 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.212+0000 7fc8e77fe640 1 -- 172.21.15.103:0/2098737236 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc8dc032900 con 0x7fc8e8108080 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.212+0000 7fc8e77fe640 1 -- 172.21.15.103:0/2098737236 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc030850 con 0x7fc8e8108080 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.212+0000 7fc8eeff6640 1 -- 172.21.15.103:0/2098737236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 msgr2=0x7fc8e810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.212+0000 7fc8eeff6640 1 --2- 172.21.15.103:0/2098737236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e810a470 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009820 tx=0x7fc8dc02a270 comp rx=0 tx=0).stop 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 -- 172.21.15.103:0/2098737236 shutdown_connections 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 --2- 172.21.15.103:0/2098737236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e810a470 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 -- 172.21.15.103:0/2098737236 >> 172.21.15.103:0/2098737236 conn(0x7fc8e8101ca0 msgr2=0x7fc8e81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 -- 172.21.15.103:0/2098737236 shutdown_connections 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 -- 172.21.15.103:0/2098737236 wait complete. 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 Processor -- start 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 -- start start 2024-09-19T07:28:38.932 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e8107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8eeff6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810b9f0 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.213+0000 7fc8ecd92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e8107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8ecd92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e8107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59716/0 (socket says 172.21.15.103:59716) 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8ecd92640 1 -- 172.21.15.103:0/1786747157 learned_addr learned my addr 172.21.15.103:0/1786747157 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8ecd92640 1 -- 172.21.15.103:0/1786747157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e8107e40 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8ecd92640 1 --2- 172.21.15.103:0/1786747157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e8107900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097f0 tx=0x7fc8dc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8e5ffb640 1 -- 172.21.15.103:0/1786747157 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc042070 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8e5ffb640 1 -- 172.21.15.103:0/1786747157 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc8dc032ca0 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e8106030 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8e5ffb640 1 -- 172.21.15.103:0/1786747157 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8dc037040 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e8106570 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.214+0000 7fc8e5ffb640 1 -- 172.21.15.103:0/1786747157 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8dc045070 con 0x7fc8e8108080 2024-09-19T07:28:38.933 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.215+0000 7fc8e5ffb640 1 -- 172.21.15.103:0/1786747157 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8dc0463b0 con 0x7fc8e8108080 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.215+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e81084e0 con 0x7fc8e8108080 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.217+0000 7fc8e5ffb640 1 -- 172.21.15.103:0/1786747157 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc8dc03db10 con 0x7fc8e8108080 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.277+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc8e8106870 con 0x7fc8e8108080 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.278+0000 7fc8e5ffb640 1 -- 172.21.15.103:0/1786747157 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fc8dc047440 con 0x7fc8e8108080 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.279+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 msgr2=0x7fc8e8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.279+0000 7fc8eeff6640 1 --2- 172.21.15.103:0/1786747157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e8107900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0097f0 tx=0x7fc8dc030ec0 comp rx=0 tx=0).stop 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.279+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 shutdown_connections 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.279+0000 7fc8eeff6640 1 --2- 172.21.15.103:0/1786747157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e8108080 0x7fc8e8107900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.279+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 >> 172.21.15.103:0/1786747157 conn(0x7fc8e8101ca0 msgr2=0x7fc8e8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.279+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 shutdown_connections 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:38.279+0000 7fc8eeff6640 1 -- 172.21.15.103:0/1786747157 wait complete. 2024-09-19T07:28:38.934 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (1/15)... 2024-09-19T07:28:38.962 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:38.714+0000 7f39ddf9e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:28:38.963 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:38.775+0000 7f39ddf9e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:28:39.220 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:38.961+0000 7f39ddf9e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:28:39.970 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:39 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:39.844+0000 7f39ddf9e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:28:39.970 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:39 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:39.911+0000 7f39ddf9e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:28:40.923 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:40.651+0000 7f39ddf9e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:28:40.923 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:40.774+0000 7f39ddf9e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:28:40.923 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:40.923+0000 7f39ddf9e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:28:41.178 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:40.990+0000 7f39ddf9e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:28:41.179 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.148+0000 7f39ddf9e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:28:41.459 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.221+0000 7f39ddf9e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:28:41.460 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.285+0000 7f39ddf9e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:28:41.460 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.460+0000 7f39ddf9e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:28:41.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/472740989' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T07:28:41.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.527+0000 7f39ddf9e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:28:41.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.598+0000 7f39ddf9e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:28:41.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.681+0000 7f39ddf9e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:28:42.063 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.744+0000 7f39ddf9e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:28:42.063 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:41.803+0000 7f39ddf9e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:42.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:42.076 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T07:28:42.077 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T07:28:30:723424+0000", 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T07:28:42.078 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T07:28:30.724082+0000", 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T07:28:42.079 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe203165640 1 Processor -- start 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe203165640 1 -- start start 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe203165640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc10a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe203165640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc07be30 con 0x7fe1fc108080 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe200f01640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc10a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe200f01640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc10a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59724/0 (socket says 172.21.15.103:59724) 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe200f01640 1 -- 172.21.15.103:0/3710762426 learned_addr learned my addr 172.21.15.103:0/3710762426 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.344+0000 7fe200f01640 1 -- 172.21.15.103:0/3710762426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc10a9b0 con 0x7fe1fc108080 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.345+0000 7fe200f01640 1 --2- 172.21.15.103:0/3710762426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc10a470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009820 tx=0x7fe1f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f0a0b5f1cf0edb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.346+0000 7fe1fb7fe640 1 -- 172.21.15.103:0/3710762426 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f0037070 con 0x7fe1fc108080 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.346+0000 7fe1fb7fe640 1 -- 172.21.15.103:0/3710762426 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1f0032900 con 0x7fe1fc108080 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.346+0000 7fe1fb7fe640 1 -- 172.21.15.103:0/3710762426 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f0030850 con 0x7fe1fc108080 2024-09-19T07:28:42.080 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 -- 172.21.15.103:0/3710762426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 msgr2=0x7fe1fc10a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 --2- 172.21.15.103:0/3710762426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc10a470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009820 tx=0x7fe1f002a270 comp rx=0 tx=0).stop 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 -- 172.21.15.103:0/3710762426 shutdown_connections 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 --2- 172.21.15.103:0/3710762426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc10a470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 -- 172.21.15.103:0/3710762426 >> 172.21.15.103:0/3710762426 conn(0x7fe1fc101ca0 msgr2=0x7fe1fc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 -- 172.21.15.103:0/3710762426 shutdown_connections 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 -- 172.21.15.103:0/3710762426 wait complete. 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 Processor -- start 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 -- start start 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.347+0000 7fe203165640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc10b9f0 con 0x7fe1fc108080 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe200f01640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe200f01640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59732/0 (socket says 172.21.15.103:59732) 2024-09-19T07:28:42.081 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe200f01640 1 -- 172.21.15.103:0/472740989 learned_addr learned my addr 172.21.15.103:0/472740989 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe200f01640 1 -- 172.21.15.103:0/472740989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc107e40 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe200f01640 1 --2- 172.21.15.103:0/472740989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc107900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00097f0 tx=0x7fe1f0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe1f9ffb640 1 -- 172.21.15.103:0/472740989 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f0042070 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe1f9ffb640 1 -- 172.21.15.103:0/472740989 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1f0032ca0 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe1f9ffb640 1 -- 172.21.15.103:0/472740989 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1f0037040 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1fc106030 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe1f9ffb640 1 -- 172.21.15.103:0/472740989 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1f0045070 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.348+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1fc106570 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.349+0000 7fe1f9ffb640 1 -- 172.21.15.103:0/472740989 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1f00463b0 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.349+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1fc1084e0 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.352+0000 7fe1f9ffb640 1 -- 172.21.15.103:0/472740989 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe1f004d030 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.414+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe1fc1068b0 con 0x7fe1fc108080 2024-09-19T07:28:42.082 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.414+0000 7fe1f9ffb640 1 -- 172.21.15.103:0/472740989 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe1f00465b0 con 0x7fe1fc108080 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.415+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 msgr2=0x7fe1fc107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.415+0000 7fe203165640 1 --2- 172.21.15.103:0/472740989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc107900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00097f0 tx=0x7fe1f0030ec0 comp rx=0 tx=0).stop 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.415+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 shutdown_connections 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.415+0000 7fe203165640 1 --2- 172.21.15.103:0/472740989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1fc108080 0x7fe1fc107900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.415+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 >> 172.21.15.103:0/472740989 conn(0x7fe1fc101ca0 msgr2=0x7fe1fc102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.416+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 shutdown_connections 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:41.416+0000 7fe203165640 1 -- 172.21.15.103:0/472740989 wait complete. 2024-09-19T07:28:42.083 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (2/15)... 2024-09-19T07:28:42.970 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:42.608+0000 7f39ddf9e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:28:42.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: Activating manager daemon x 2024-09-19T07:28:42.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: mgrmap e2: x(active, starting, since 0.00328691s) 2024-09-19T07:28:42.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:28:42.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:28:42.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:28:42.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:28:42.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:28:42.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: Manager daemon x is now available 2024-09-19T07:28:42.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:28:42.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:28:42.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' 2024-09-19T07:28:42.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' 2024-09-19T07:28:42.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:42 smithi103 ceph-mon[31201]: from='mgr.14100 172.21.15.103:0/1113792231' entity='mgr.x' 2024-09-19T07:28:44.739 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:44 smithi103 ceph-mon[31201]: mgrmap e3: x(active, since 1.00709s) 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T07:28:45.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T07:28:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:45.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T07:28:45.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T07:28:45.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T07:28:45.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T07:28:45.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T07:28:45.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T07:28:45.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T07:28:30:723424+0000", 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T07:28:45.294 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:45.295 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T07:28:30.724082+0000", 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.482+0000 7fdfe4c42640 1 Processor -- start 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.482+0000 7fdfe4c42640 1 -- start start 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.483+0000 7fdfe4c42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe0106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.483+0000 7fdfe4c42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0106ee0 con 0x7fdfe0106510 2024-09-19T07:28:45.296 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.483+0000 7fdfde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe0106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.483+0000 7fdfde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe0106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59782/0 (socket says 172.21.15.103:59782) 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.483+0000 7fdfde59c640 1 -- 172.21.15.103:0/2137051612 learned_addr learned my addr 172.21.15.103:0/2137051612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.483+0000 7fdfde59c640 1 -- 172.21.15.103:0/2137051612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe01076c0 con 0x7fdfe0106510 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.484+0000 7fdfde59c640 1 --2- 172.21.15.103:0/2137051612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe0106910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009820 tx=0x7fdfd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c59a65b920d9684 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.484+0000 7fdfddd9b640 1 -- 172.21.15.103:0/2137051612 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd0037070 con 0x7fdfe0106510 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.484+0000 7fdfddd9b640 1 -- 172.21.15.103:0/2137051612 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdfd0032900 con 0x7fdfe0106510 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.484+0000 7fdfddd9b640 1 -- 172.21.15.103:0/2137051612 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd0030850 con 0x7fdfe0106510 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 -- 172.21.15.103:0/2137051612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 msgr2=0x7fdfe0106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:45.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 --2- 172.21.15.103:0/2137051612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe0106910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009820 tx=0x7fdfd002a270 comp rx=0 tx=0).stop 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 -- 172.21.15.103:0/2137051612 shutdown_connections 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 --2- 172.21.15.103:0/2137051612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe0106910 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 -- 172.21.15.103:0/2137051612 >> 172.21.15.103:0/2137051612 conn(0x7fdfe0101ca0 msgr2=0x7fdfe01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 -- 172.21.15.103:0/2137051612 shutdown_connections 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 -- 172.21.15.103:0/2137051612 wait complete. 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.485+0000 7fdfe4c42640 1 Processor -- start 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfe4c42640 1 -- start start 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfe4c42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:45.298 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfe4c42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe010aa70 con 0x7fdfe0106510 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe019b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59790/0 (socket says 172.21.15.103:59790) 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfde59c640 1 -- 172.21.15.103:0/1118874618 learned_addr learned my addr 172.21.15.103:0/1118874618 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfde59c640 1 -- 172.21.15.103:0/1118874618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe019bb70 con 0x7fdfe0106510 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.486+0000 7fdfde59c640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe019b630 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00097f0 tx=0x7fdfd0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.487+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd0042070 con 0x7fdfe0106510 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.487+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdfd0032ca0 con 0x7fdfe0106510 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.487+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe019be00 con 0x7fdfe0106510 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.487+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfd0037040 con 0x7fdfe0106510 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.487+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe019eaf0 con 0x7fdfe0106510 2024-09-19T07:28:45.299 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.488+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fdfd0045070 con 0x7fdfe0106510 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.488+0000 7fdfd7fff640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fdfcc0390a0 0x7fdfcc03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.488+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdfd006bf50 con 0x7fdfe0106510 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.488+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe010abf0 con 0x7fdfe0106510 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.488+0000 7fdfd5d9b640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fdfcc0390a0 0x7fdfcc03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.488+0000 7fdfd5d9b640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fdfcc0390a0 0x7fdfcc03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80098c0 tx=0x7fdfc8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.493+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdfd0045360 con 0x7fdfe0106510 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.620+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fdfd003c6a0 con 0x7fdfe0106510 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.728+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdfe002da10 con 0x7fdfe0106510 2024-09-19T07:28:45.300 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.728+0000 7fdfd7fff640 1 -- 172.21.15.103:0/1118874618 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fdfd0070250 con 0x7fdfe0106510 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fdfcc0390a0 msgr2=0x7fdfcc03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fdfcc0390a0 0x7fdfcc03b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80098c0 tx=0x7fdfc8006db0 comp rx=0 tx=0).stop 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 msgr2=0x7fdfe019b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe019b630 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00097f0 tx=0x7fdfd0030ec0 comp rx=0 tx=0).stop 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 shutdown_connections 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fdfcc0390a0 0x7fdfcc03b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 --2- 172.21.15.103:0/1118874618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe0106510 0x7fdfe019b630 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.730+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 >> 172.21.15.103:0/1118874618 conn(0x7fdfe0101ca0 msgr2=0x7fdfe0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.731+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 shutdown_connections 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:44.731+0000 7fdfe4c42640 1 -- 172.21.15.103:0/1118874618 wait complete. 2024-09-19T07:28:45.301 INFO:teuthology.orchestra.run.smithi103.stdout:mgr is available 2024-09-19T07:28:45.892 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:45 smithi103 ceph-mon[31201]: mgrmap e4: x(active, since 2s) 2024-09-19T07:28:45.892 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:45 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1118874618' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T07:28:46.468 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:46.468 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T07:28:46.468 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:46.468 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T07:28:46.469 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.715+0000 7f4ef1ce5640 1 Processor -- start 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.716+0000 7f4ef1ce5640 1 -- start start 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.716+0000 7f4ef1ce5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.716+0000 7f4ef1ce5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec106ee0 con 0x7f4eec106510 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.716+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.716+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59798/0 (socket says 172.21.15.103:59798) 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.716+0000 7f4eeb7fe640 1 -- 172.21.15.103:0/1951056939 learned_addr learned my addr 172.21.15.103:0/1951056939 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.716+0000 7f4eeb7fe640 1 -- 172.21.15.103:0/1951056939 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec1076c0 con 0x7f4eec106510 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.717+0000 7f4eeb7fe640 1 --2- 172.21.15.103:0/1951056939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec106910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009820 tx=0x7f4edc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0923729dcb99d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:46.470 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.717+0000 7f4eeaffd640 1 -- 172.21.15.103:0/1951056939 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4edc037070 con 0x7f4eec106510 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.717+0000 7f4eeaffd640 1 -- 172.21.15.103:0/1951056939 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4edc032900 con 0x7f4eec106510 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.717+0000 7f4eeaffd640 1 -- 172.21.15.103:0/1951056939 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4edc030850 con 0x7f4eec106510 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/1951056939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 msgr2=0x7f4eec106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 --2- 172.21.15.103:0/1951056939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec106910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009820 tx=0x7f4edc02a270 comp rx=0 tx=0).stop 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/1951056939 shutdown_connections 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 --2- 172.21.15.103:0/1951056939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec106910 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/1951056939 >> 172.21.15.103:0/1951056939 conn(0x7f4eec101ca0 msgr2=0x7f4eec1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/1951056939 shutdown_connections 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/1951056939 wait complete. 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 Processor -- start 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.718+0000 7f4ef1ce5640 1 -- start start 2024-09-19T07:28:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4ef1ce5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4ef1ce5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec10aa70 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4eeb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59808/0 (socket says 172.21.15.103:59808) 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4eeb7fe640 1 -- 172.21.15.103:0/3607494345 learned_addr learned my addr 172.21.15.103:0/3607494345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4eeb7fe640 1 -- 172.21.15.103:0/3607494345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec19aaa0 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4eeb7fe640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec19a560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009940 tx=0x7f4edc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4ee97fa640 1 -- 172.21.15.103:0/3607494345 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4edc042070 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4ee97fa640 1 -- 172.21.15.103:0/3607494345 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4edc032be0 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec19ad30 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4ee97fa640 1 -- 172.21.15.103:0/3607494345 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4edc037040 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.719+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec19da20 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.720+0000 7f4ee97fa640 1 -- 172.21.15.103:0/3607494345 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f4edc045070 con 0x7f4eec106510 2024-09-19T07:28:46.472 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.720+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec10abf0 con 0x7f4eec106510 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.720+0000 7f4ee97fa640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f4ed80391b0 0x7f4ed803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.720+0000 7f4ee97fa640 1 -- 172.21.15.103:0/3607494345 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4edc06c0b0 con 0x7f4eec106510 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.720+0000 7f4ee2ffd640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f4ed80391b0 0x7f4ed803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.721+0000 7f4ee2ffd640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f4ed80391b0 0x7f4ed803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40098c0 tx=0x7f4ed4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.725+0000 7f4ee97fa640 1 -- 172.21.15.103:0/3607494345 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4edc045360 con 0x7f4eec106510 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.892+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4eec106990 con 0x7f4eec106510 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.893+0000 7f4ee97fa640 1 -- 172.21.15.103:0/3607494345 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f4edc032d90 con 0x7f4eec106510 2024-09-19T07:28:46.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f4ed80391b0 msgr2=0x7f4ed803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:46.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f4ed80391b0 0x7f4ed803b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40098c0 tx=0x7f4ed4006db0 comp rx=0 tx=0).stop 2024-09-19T07:28:46.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 msgr2=0x7f4eec19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:46.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec19a560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009940 tx=0x7f4edc0326e0 comp rx=0 tx=0).stop 2024-09-19T07:28:46.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 shutdown_connections 2024-09-19T07:28:46.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f4ed80391b0 0x7f4ed803b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:46.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 --2- 172.21.15.103:0/3607494345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4eec106510 0x7f4eec19a560 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:46.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 >> 172.21.15.103:0/3607494345 conn(0x7f4eec101ca0 msgr2=0x7f4eec102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:46.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 shutdown_connections 2024-09-19T07:28:46.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:45.895+0000 7f4ef1ce5640 1 -- 172.21.15.103:0/3607494345 wait complete. 2024-09-19T07:28:46.475 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling cephadm module... 2024-09-19T07:28:46.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:46 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3607494345' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T07:28:47.941 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:47 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1247989153' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T07:28:47.941 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ignoring --setuser ceph since I am not root 2024-09-19T07:28:47.941 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ignoring --setgroup ceph since I am not root 2024-09-19T07:28:47.942 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:47.763+0000 7fb573f17040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:28:47.942 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:47.820+0000 7fb573f17040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:28:48.220 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:47.970+0000 7fb573f17040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:28:48.220 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:48.037+0000 7fb573f17040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:28:48.220 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:48.186+0000 7fb573f17040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:28:48.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.858+0000 7f0a9259e640 1 Processor -- start 2024-09-19T07:28:48.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.858+0000 7f0a9259e640 1 -- start start 2024-09-19T07:28:48.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.858+0000 7f0a9259e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c10a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:48.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.859+0000 7f0a9259e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c07bd70 con 0x7f0a8c108140 2024-09-19T07:28:48.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.859+0000 7f0a8bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c10a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:48.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.859+0000 7f0a8bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c10a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59818/0 (socket says 172.21.15.103:59818) 2024-09-19T07:28:48.258 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.859+0000 7f0a8bfff640 1 -- 172.21.15.103:0/419216200 learned_addr learned my addr 172.21.15.103:0/419216200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.859+0000 7f0a8bfff640 1 -- 172.21.15.103:0/419216200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c10aa70 con 0x7f0a8c108140 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.860+0000 7f0a8bfff640 1 --2- 172.21.15.103:0/419216200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c10a530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009820 tx=0x7f0a7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ab62bdf51336f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.860+0000 7f0a8b7fe640 1 -- 172.21.15.103:0/419216200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c037070 con 0x7f0a8c108140 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.860+0000 7f0a8b7fe640 1 -- 172.21.15.103:0/419216200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a7c032900 con 0x7f0a8c108140 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.860+0000 7f0a8b7fe640 1 -- 172.21.15.103:0/419216200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c030850 con 0x7f0a8c108140 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.860+0000 7f0a9259e640 1 -- 172.21.15.103:0/419216200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 msgr2=0x7f0a8c10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.860+0000 7f0a9259e640 1 --2- 172.21.15.103:0/419216200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c10a530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009820 tx=0x7f0a7c02a270 comp rx=0 tx=0).stop 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 -- 172.21.15.103:0/419216200 shutdown_connections 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 --2- 172.21.15.103:0/419216200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c10a530 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 -- 172.21.15.103:0/419216200 >> 172.21.15.103:0/419216200 conn(0x7f0a8c101ca0 msgr2=0x7f0a8c1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 -- 172.21.15.103:0/419216200 shutdown_connections 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 -- 172.21.15.103:0/419216200 wait complete. 2024-09-19T07:28:48.259 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 Processor -- start 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 -- start start 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.861+0000 7f0a9259e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c10b9f0 con 0x7f0a8c108140 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a8bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a8bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c19b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59832/0 (socket says 172.21.15.103:59832) 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a8bfff640 1 -- 172.21.15.103:0/1247989153 learned_addr learned my addr 172.21.15.103:0/1247989153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a8bfff640 1 -- 172.21.15.103:0/1247989153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c19bb70 con 0x7f0a8c108140 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a8bfff640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c19b630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0097f0 tx=0x7f0a7c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c042070 con 0x7f0a8c108140 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a7c032ca0 con 0x7f0a8c108140 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c19be00 con 0x7f0a8c108140 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a7c037040 con 0x7f0a8c108140 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.862+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c19eaf0 con 0x7f0a8c108140 2024-09-19T07:28:48.260 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.863+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f0a7c045070 con 0x7f0a8c108140 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.863+0000 7f0a89ffb640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f0a780391b0 0x7f0a7803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.863+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c1085a0 con 0x7f0a8c108140 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.863+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a7c06c260 con 0x7f0a8c108140 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.863+0000 7f0a83fff640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f0a780391b0 0x7f0a7803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.864+0000 7f0a83fff640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f0a780391b0 0x7f0a7803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a740098c0 tx=0x7f0a74006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:46.868+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0a7c03d770 con 0x7f0a8c108140 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.076+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f0a8c19f0d0 con 0x7f0a8c108140 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.623+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0a7c03c720 con 0x7f0a8c108140 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.625+0000 7f0a89ffb640 1 -- 172.21.15.103:0/1247989153 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f0a7c03db10 con 0x7f0a8c108140 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f0a780391b0 msgr2=0x7f0a7803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f0a780391b0 0x7f0a7803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a740098c0 tx=0x7f0a74006db0 comp rx=0 tx=0).stop 2024-09-19T07:28:48.261 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 msgr2=0x7f0a8c19b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:48.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c19b630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0097f0 tx=0x7f0a7c030ec0 comp rx=0 tx=0).stop 2024-09-19T07:28:48.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 shutdown_connections 2024-09-19T07:28:48.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f0a780391b0 0x7f0a7803b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:48.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 --2- 172.21.15.103:0/1247989153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8c108140 0x7f0a8c19b630 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:48.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.626+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 >> 172.21.15.103:0/1247989153 conn(0x7f0a8c101ca0 msgr2=0x7f0a8c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:48.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.627+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 shutdown_connections 2024-09-19T07:28:48.262 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:47.627+0000 7f0a9259e640 1 -- 172.21.15.103:0/1247989153 wait complete. 2024-09-19T07:28:48.503 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:48.338+0000 7fb573f17040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:28:48.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1247989153' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T07:28:48.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-mon[31201]: mgrmap e5: x(active, since 5s) 2024-09-19T07:28:48.887 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:48.661+0000 7fb573f17040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:28:48.887 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:48.726+0000 7fb573f17040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:28:49.215 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:48.917+0000 7fb573f17040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:28:49.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedf5bf3640 1 Processor -- start 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedf5bf3640 1 -- start start 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedf5bf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf0106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedf5bf3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0106ee0 con 0x7fedf0106510 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf0106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:49.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf0106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35496/0 (socket says 172.21.15.103:35496) 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedef7fe640 1 -- 172.21.15.103:0/996408849 learned_addr learned my addr 172.21.15.103:0/996408849 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.681+0000 7fedef7fe640 1 -- 172.21.15.103:0/996408849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf01076c0 con 0x7fedf0106510 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.682+0000 7fedef7fe640 1 --2- 172.21.15.103:0/996408849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf0106910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fede0009820 tx=0x7fede002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58ab1fdd32ee2c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.682+0000 7fedee7fc640 1 -- 172.21.15.103:0/996408849 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fede0037070 con 0x7fedf0106510 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.682+0000 7fedee7fc640 1 -- 172.21.15.103:0/996408849 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fede0032900 con 0x7fedf0106510 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.682+0000 7fedee7fc640 1 -- 172.21.15.103:0/996408849 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fede0030850 con 0x7fedf0106510 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.683+0000 7fedf5bf3640 1 -- 172.21.15.103:0/996408849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 msgr2=0x7fedf0106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.683+0000 7fedf5bf3640 1 --2- 172.21.15.103:0/996408849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf0106910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fede0009820 tx=0x7fede002a270 comp rx=0 tx=0).stop 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.683+0000 7fedf5bf3640 1 -- 172.21.15.103:0/996408849 shutdown_connections 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.683+0000 7fedf5bf3640 1 --2- 172.21.15.103:0/996408849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf0106910 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.683+0000 7fedf5bf3640 1 -- 172.21.15.103:0/996408849 >> 172.21.15.103:0/996408849 conn(0x7fedf0101ca0 msgr2=0x7fedf01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:49.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.683+0000 7fedf5bf3640 1 -- 172.21.15.103:0/996408849 shutdown_connections 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.683+0000 7fedf5bf3640 1 -- 172.21.15.103:0/996408849 wait complete. 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedf5bf3640 1 Processor -- start 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedf5bf3640 1 -- start start 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedf5bf3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedf5bf3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010aa70 con 0x7fedf0106510 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf019b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35502/0 (socket says 172.21.15.103:35502) 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedef7fe640 1 -- 172.21.15.103:0/1735494716 learned_addr learned my addr 172.21.15.103:0/1735494716 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.684+0000 7fedef7fe640 1 -- 172.21.15.103:0/1735494716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf019bb70 con 0x7fedf0106510 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.685+0000 7fedef7fe640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf019b630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fede0009940 tx=0x7fede00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.685+0000 7fedecff9640 1 -- 172.21.15.103:0/1735494716 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fede0042070 con 0x7fedf0106510 2024-09-19T07:28:49.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.685+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf019be00 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.685+0000 7fedecff9640 1 -- 172.21.15.103:0/1735494716 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fede0032be0 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.685+0000 7fedecff9640 1 -- 172.21.15.103:0/1735494716 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fede0037040 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.685+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf019eaf0 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.686+0000 7fedecff9640 1 -- 172.21.15.103:0/1735494716 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fede0045070 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.686+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf010abf0 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.686+0000 7fedecff9640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 0x7fedd803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.686+0000 7fedecff9640 1 -- 172.21.15.103:0/1735494716 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fede006c160 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.686+0000 7fedeeffd640 1 -- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 msgr2=0x7fedd803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1746275589 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.686+0000 7fedeeffd640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 0x7fedd803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.691+0000 7fedecff9640 1 -- 172.21.15.103:0/1735494716 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fede0045e10 con 0x7fedf0106510 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.887+0000 7fedeeffd640 1 -- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 msgr2=0x7fedd803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1746275589 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.887+0000 7fedeeffd640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 0x7fedd803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:28:49.479 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.909+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fedf002da30 con 0x7fedf0106510 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.910+0000 7fedecff9640 1 -- 172.21.15.103:0/1735494716 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fede0032d90 con 0x7fedf0106510 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 msgr2=0x7fedd803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 0x7fedd803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 msgr2=0x7fedf019b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf019b630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fede0009940 tx=0x7fede00326e0 comp rx=0 tx=0).stop 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 shutdown_connections 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7fedd80391b0 0x7fedd803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 --2- 172.21.15.103:0/1735494716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedf0106510 0x7fedf019b630 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 >> 172.21.15.103:0/1735494716 conn(0x7fedf0101ca0 msgr2=0x7fedf0102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 shutdown_connections 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:48.912+0000 7fedf5bf3640 1 -- 172.21.15.103:0/1735494716 wait complete. 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-19T07:28:49.480 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 5... 2024-09-19T07:28:49.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:49 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1735494716' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T07:28:50.103 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:49.827+0000 7fb573f17040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:28:50.104 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:49.898+0000 7fb573f17040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:28:50.869 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:50.610+0000 7fb573f17040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:28:50.869 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:50.727+0000 7fb573f17040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:28:51.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:50.869+0000 7fb573f17040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:28:51.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:50.930+0000 7fb573f17040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:28:51.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.081+0000 7fb573f17040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:28:51.149 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.149+0000 7fb573f17040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:28:51.431 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.208+0000 7fb573f17040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:28:51.431 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.366+0000 7fb573f17040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:28:51.431 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.431+0000 7fb573f17040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:28:51.705 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.498+0000 7fb573f17040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:28:51.706 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.581+0000 7fb573f17040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:28:51.706 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.645+0000 7fb573f17040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:28:51.706 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:51.706+0000 7fb573f17040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:28:52.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: Active manager daemon x restarted 2024-09-19T07:28:52.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: Activating manager daemon x 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: mgrmap e6: x(active, starting, since 0.00350024s) 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: Manager daemon x is now available 2024-09-19T07:28:52.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:28:52.972 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:28:52.972 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:52.972 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:52.972 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:28:52.972 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:28:52.972 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:52.491+0000 7fb573f17040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:28:53.790 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-mon[31201]: Found migration_current of "None". Setting to last migration. 2024-09-19T07:28:53.790 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:28:53.791 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:28:53.791 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-mon[31201]: mgrmap e7: x(active, since 1.00542s) 2024-09-19T07:28:54.073 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:53.789+0000 7fb4db271640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:07:28:53] ENGINE Error in HTTPServer.serve 2024-09-19T07:28:54.073 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: Traceback (most recent call last): 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._connections.run(self.expiration_interval) 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._run(expiration_interval) 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: s = self.context.wrap_socket( 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: return self.sslsocket_class._create( 2024-09-19T07:28:54.074 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:28:54.075 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self.do_handshake() 2024-09-19T07:28:54.075 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:28:54.075 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._sslobj.do_handshake() 2024-09-19T07:28:54.075 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:28:54.075 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.897+0000 7f7ff11da640 1 Processor -- start 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.898+0000 7f7ff11da640 1 -- start start 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.898+0000 7f7ff11da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec10a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.898+0000 7f7ff11da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec07bdc0 con 0x7f7fec108140 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.898+0000 7f7fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec10a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.898+0000 7f7fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec10a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35506/0 (socket says 172.21.15.103:35506) 2024-09-19T07:28:54.083 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.898+0000 7f7fead9d640 1 -- 172.21.15.103:0/2720108654 learned_addr learned my addr 172.21.15.103:0/2720108654 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.898+0000 7f7fead9d640 1 -- 172.21.15.103:0/2720108654 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec10aa70 con 0x7f7fec108140 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.899+0000 7f7fead9d640 1 --2- 172.21.15.103:0/2720108654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec10a530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009820 tx=0x7f7fdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec3461c71559a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.899+0000 7f7fe9d9b640 1 -- 172.21.15.103:0/2720108654 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fdc037070 con 0x7f7fec108140 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.899+0000 7f7fe9d9b640 1 -- 172.21.15.103:0/2720108654 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fdc032900 con 0x7f7fec108140 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.899+0000 7f7fe9d9b640 1 -- 172.21.15.103:0/2720108654 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fdc030850 con 0x7f7fec108140 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 -- 172.21.15.103:0/2720108654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 msgr2=0x7f7fec10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 --2- 172.21.15.103:0/2720108654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec10a530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009820 tx=0x7f7fdc02a270 comp rx=0 tx=0).stop 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 -- 172.21.15.103:0/2720108654 shutdown_connections 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 --2- 172.21.15.103:0/2720108654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec10a530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 -- 172.21.15.103:0/2720108654 >> 172.21.15.103:0/2720108654 conn(0x7f7fec101d00 msgr2=0x7f7fec104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 -- 172.21.15.103:0/2720108654 shutdown_connections 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 -- 172.21.15.103:0/2720108654 wait complete. 2024-09-19T07:28:54.084 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.900+0000 7f7ff11da640 1 Processor -- start 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7ff11da640 1 -- start start 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7ff11da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7ff11da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10ba50 con 0x7f7fec108140 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fead9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35518/0 (socket says 172.21.15.103:35518) 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fead9d640 1 -- 172.21.15.103:0/1287437100 learned_addr learned my addr 172.21.15.103:0/1287437100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fead9d640 1 -- 172.21.15.103:0/1287437100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec19dd70 con 0x7f7fec108140 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fead9d640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec19d830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc032ce0 tx=0x7f7fdc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fdc042070 con 0x7f7fec108140 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fdc0308c0 con 0x7f7fec108140 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec19e000 con 0x7f7fec108140 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.901+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fdc037040 con 0x7f7fec108140 2024-09-19T07:28:54.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.902+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec1a0cf0 con 0x7f7fec108140 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.902+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f7fdc045070 con 0x7f7fec108140 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.903+0000 7f7fd3fff640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 0x7f7fd403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.903+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 --> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7fd403bdd0 con 0x7f7fd4039200 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.903+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fdc06c280 con 0x7f7fec108140 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.903+0000 7f7fea59c640 1 -- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 msgr2=0x7f7fd403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1746275589 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:49.903+0000 7f7fea59c640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 0x7f7fd403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:50.104+0000 7f7fea59c640 1 -- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 msgr2=0x7f7fd403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1746275589 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:50.104+0000 7f7fea59c640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 0x7f7fd403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:50.505+0000 7f7fea59c640 1 -- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 msgr2=0x7f7fd403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1746275589 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:50.505+0000 7f7fea59c640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 0x7f7fd403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:51.306+0000 7f7fea59c640 1 -- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 msgr2=0x7f7fd403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/1746275589 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:51.306+0000 7f7fea59c640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 0x7f7fd403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T07:28:54.086 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:52.495+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f7fdc06ae00 con 0x7f7fec108140 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:52.495+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 msgr2=0x7f7fd403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:52.495+0000 7f7fd3fff640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 0x7f7fd403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.495+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f7fdc03c760 con 0x7f7fec108140 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.495+0000 7f7fd3fff640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f7fd403cb30 0x7f7fd403ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.495+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7fdc05a940 con 0x7f7fd403cb30 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.496+0000 7f7fea59c640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f7fd403cb30 0x7f7fd403ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.496+0000 7f7fea59c640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f7fd403cb30 0x7f7fd403ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0003980 tx=0x7f7fe00092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.497+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7fdc05a940 con 0x7f7fd403cb30 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.503+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7fec1085a0 con 0x7f7fd403cb30 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7fd3fff640 1 -- 172.21.15.103:0/1287437100 <== mgr.14118 v2:172.21.15.103:6800/2002728097 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7fec1085a0 con 0x7f7fd403cb30 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f7fd403cb30 msgr2=0x7f7fd403ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f7fd403cb30 0x7f7fd403ef20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0003980 tx=0x7f7fe00092b0 comp rx=0 tx=0).stop 2024-09-19T07:28:54.087 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 msgr2=0x7f7fec19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec19d830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc032ce0 tx=0x7f7fdc032d10 comp rx=0 tx=0).stop 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 shutdown_connections 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f7fd403cb30 0x7f7fd403ef20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:6800/1746275589,v1:172.21.15.103:6801/1746275589] conn(0x7f7fd4039200 0x7f7fd403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 --2- 172.21.15.103:0/1287437100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fec108140 0x7f7fec19d830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 >> 172.21.15.103:0/1287437100 conn(0x7f7fec101d00 msgr2=0x7f7fec102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 shutdown_connections 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:53.504+0000 7f7ff11da640 1 -- 172.21.15.103:0/1287437100 wait complete. 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 5 is available 2024-09-19T07:28:54.088 INFO:teuthology.orchestra.run.smithi103.stdout:Setting orchestrator backend to cephadm... 2024-09-19T07:28:54.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:54 smithi103 ceph-mon[31201]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T07:28:54.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:54 smithi103 ceph-mon[31201]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T07:28:54.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:54 smithi103 ceph-mon[31201]: [19/Sep/2024:07:28:53] ENGINE Bus STARTING 2024-09-19T07:28:54.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:54 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7363fe2640 1 Processor -- start 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7363fe2640 1 -- start start 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7363fe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7363fe2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f735c07c880 con 0x7f735c07de50 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7361d7e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7361d7e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35570/0 (socket says 172.21.15.103:35570) 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7361d7e640 1 -- 172.21.15.103:0/102884671 learned_addr learned my addr 172.21.15.103:0/102884671 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.463+0000 7f7361d7e640 1 -- 172.21.15.103:0/102884671 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c07ca00 con 0x7f735c07de50 2024-09-19T07:28:55.304 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.464+0000 7f7361d7e640 1 --2- 172.21.15.103:0/102884671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c07c2b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7358009820 tx=0x7f735802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ebba7aca93a267 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.464+0000 7f7360d7c640 1 -- 172.21.15.103:0/102884671 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7358037070 con 0x7f735c07de50 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.464+0000 7f7360d7c640 1 -- 172.21.15.103:0/102884671 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7358032900 con 0x7f735c07de50 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.464+0000 7f7360d7c640 1 -- 172.21.15.103:0/102884671 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7358030850 con 0x7f735c07de50 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 -- 172.21.15.103:0/102884671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 msgr2=0x7f735c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 --2- 172.21.15.103:0/102884671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c07c2b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7358009820 tx=0x7f735802a270 comp rx=0 tx=0).stop 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 -- 172.21.15.103:0/102884671 shutdown_connections 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 --2- 172.21.15.103:0/102884671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c07c2b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 -- 172.21.15.103:0/102884671 >> 172.21.15.103:0/102884671 conn(0x7f735c102db0 msgr2=0x7f735c1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 -- 172.21.15.103:0/102884671 shutdown_connections 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 -- 172.21.15.103:0/102884671 wait complete. 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 Processor -- start 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.465+0000 7f7363fe2640 1 -- start start 2024-09-19T07:28:55.305 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7363fe2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7363fe2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f735c10dc50 con 0x7f735c07de50 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7361d7e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7361d7e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c19d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35584/0 (socket says 172.21.15.103:35584) 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7361d7e640 1 -- 172.21.15.103:0/862022457 learned_addr learned my addr 172.21.15.103:0/862022457 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7361d7e640 1 -- 172.21.15.103:0/862022457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c19dda0 con 0x7f735c07de50 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7361d7e640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c19d860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7358031000 tx=0x7f73580326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7358042070 con 0x7f735c07de50 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7358032be0 con 0x7f735c07de50 2024-09-19T07:28:55.306 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735c19e030 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7358037040 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.466+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735c1a0d20 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.467+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f7358045070 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.467+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f735c07c330 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.467+0000 7f7352ffd640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f73480390a0 0x7f734803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.467+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f735806c110 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.468+0000 7f736157d640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f73480390a0 0x7f734803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.468+0000 7f736157d640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f73480390a0 0x7f734803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f734c0098c0 tx=0x7f734c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.472+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7358042210 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.547+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7358047440 con 0x7f735c07de50 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.658+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f735c1040c0 con 0x7f73480390a0 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.662+0000 7f7352ffd640 1 -- 172.21.15.103:0/862022457 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f735c1040c0 con 0x7f73480390a0 2024-09-19T07:28:55.307 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.664+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f73480390a0 msgr2=0x7f734803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.664+0000 7f7363fe2640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f73480390a0 0x7f734803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f734c0098c0 tx=0x7f734c006db0 comp rx=0 tx=0).stop 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 msgr2=0x7f735c19d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c19d860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7358031000 tx=0x7f73580326e0 comp rx=0 tx=0).stop 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 shutdown_connections 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f73480390a0 0x7f734803b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 --2- 172.21.15.103:0/862022457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f735c07de50 0x7f735c19d860 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 >> 172.21.15.103:0/862022457 conn(0x7f735c102db0 msgr2=0x7f735c1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 shutdown_connections 2024-09-19T07:28:55.308 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:54.665+0000 7f7363fe2640 1 -- 172.21.15.103:0/862022457 wait complete. 2024-09-19T07:28:55.893 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: [19/Sep/2024:07:28:53] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T07:28:55.893 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: [19/Sep/2024:07:28:53] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T07:28:55.893 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: [19/Sep/2024:07:28:53] ENGINE Bus STARTED 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: [19/Sep/2024:07:28:53] ENGINE Error in HTTPServer.serve 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:28:55.894 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: mgrmap e8: x(active, since 2s) 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:28:55.895 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:55 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:28:56.525 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T07:28:56.525 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f43f34640 1 Processor -- start 2024-09-19T07:28:56.525 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f43f34640 1 -- start start 2024-09-19T07:28:56.525 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f43f34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c10a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:56.525 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f43f34640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c07be80 con 0x7f8f3c108070 2024-09-19T07:28:56.526 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f41cd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c10a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:56.526 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f41cd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c10a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35588/0 (socket says 172.21.15.103:35588) 2024-09-19T07:28:56.526 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f41cd0640 1 -- 172.21.15.103:0/950283686 learned_addr learned my addr 172.21.15.103:0/950283686 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:56.526 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.717+0000 7f8f41cd0640 1 -- 172.21.15.103:0/950283686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c10a9a0 con 0x7f8f3c108070 2024-09-19T07:28:56.526 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.718+0000 7f8f41cd0640 1 --2- 172.21.15.103:0/950283686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c10a460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009820 tx=0x7f8f3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41be72d37609620 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:56.526 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.718+0000 7f8f40cce640 1 -- 172.21.15.103:0/950283686 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f38037070 con 0x7f8f3c108070 2024-09-19T07:28:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.718+0000 7f8f40cce640 1 -- 172.21.15.103:0/950283686 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f38032900 con 0x7f8f3c108070 2024-09-19T07:28:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.718+0000 7f8f40cce640 1 -- 172.21.15.103:0/950283686 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f38030850 con 0x7f8f3c108070 2024-09-19T07:28:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 -- 172.21.15.103:0/950283686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 msgr2=0x7f8f3c10a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 --2- 172.21.15.103:0/950283686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c10a460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009820 tx=0x7f8f3802a270 comp rx=0 tx=0).stop 2024-09-19T07:28:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 -- 172.21.15.103:0/950283686 shutdown_connections 2024-09-19T07:28:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 --2- 172.21.15.103:0/950283686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c10a460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:56.528 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 -- 172.21.15.103:0/950283686 >> 172.21.15.103:0/950283686 conn(0x7f8f3c101d10 msgr2=0x7f8f3c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:56.528 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 -- 172.21.15.103:0/950283686 shutdown_connections 2024-09-19T07:28:56.528 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 -- 172.21.15.103:0/950283686 wait complete. 2024-09-19T07:28:56.528 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.719+0000 7f8f43f34640 1 Processor -- start 2024-09-19T07:28:56.528 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f43f34640 1 -- start start 2024-09-19T07:28:56.528 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f43f34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c1078f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:56.529 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f43f34640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c10ba40 con 0x7f8f3c108070 2024-09-19T07:28:56.529 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f41cd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c1078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:56.529 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f41cd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c1078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35600/0 (socket says 172.21.15.103:35600) 2024-09-19T07:28:56.529 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f41cd0640 1 -- 172.21.15.103:0/1484291812 learned_addr learned my addr 172.21.15.103:0/1484291812 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:56.529 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f41cd0640 1 -- 172.21.15.103:0/1484291812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c107e30 con 0x7f8f3c108070 2024-09-19T07:28:56.529 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f41cd0640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c1078f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f38032ce0 tx=0x7f8f38032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:56.529 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f32ffd640 1 -- 172.21.15.103:0/1484291812 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f38042070 con 0x7f8f3c108070 2024-09-19T07:28:56.530 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c106020 con 0x7f8f3c108070 2024-09-19T07:28:56.530 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f32ffd640 1 -- 172.21.15.103:0/1484291812 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f380308c0 con 0x7f8f3c108070 2024-09-19T07:28:56.530 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f32ffd640 1 -- 172.21.15.103:0/1484291812 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f38037040 con 0x7f8f3c108070 2024-09-19T07:28:56.530 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.720+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c106560 con 0x7f8f3c108070 2024-09-19T07:28:56.530 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.721+0000 7f8f32ffd640 1 -- 172.21.15.103:0/1484291812 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8f38045070 con 0x7f8f3c108070 2024-09-19T07:28:56.530 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.721+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c1084d0 con 0x7f8f3c108070 2024-09-19T07:28:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.721+0000 7f8f32ffd640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8f280391b0 0x7f8f2803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.721+0000 7f8f32ffd640 1 -- 172.21.15.103:0/1484291812 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f3806c340 con 0x7f8f3c108070 2024-09-19T07:28:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.722+0000 7f8f414cf640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8f280391b0 0x7f8f2803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.722+0000 7f8f414cf640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8f280391b0 0x7f8f2803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0098c0 tx=0x7f8f2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.726+0000 7f8f32ffd640 1 -- 172.21.15.103:0/1484291812 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f3804b170 con 0x7f8f3c108070 2024-09-19T07:28:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.892+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f8f3c106e10 con 0x7f8f280391b0 2024-09-19T07:28:56.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.893+0000 7f8f32ffd640 1 -- 172.21.15.103:0/1484291812 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f8f3c106e10 con 0x7f8f280391b0 2024-09-19T07:28:56.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8f280391b0 msgr2=0x7f8f2803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:56.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8f280391b0 0x7f8f2803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0098c0 tx=0x7f8f2c006db0 comp rx=0 tx=0).stop 2024-09-19T07:28:56.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 msgr2=0x7f8f3c1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:56.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c1078f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f38032ce0 tx=0x7f8f38032d10 comp rx=0 tx=0).stop 2024-09-19T07:28:56.532 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 shutdown_connections 2024-09-19T07:28:56.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8f280391b0 0x7f8f2803b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:56.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 --2- 172.21.15.103:0/1484291812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f3c108070 0x7f8f3c1078f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:56.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 >> 172.21.15.103:0/1484291812 conn(0x7f8f3c101d10 msgr2=0x7f8f3c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:56.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 shutdown_connections 2024-09-19T07:28:56.533 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:55.896+0000 7f8f43f34640 1 -- 172.21.15.103:0/1484291812 wait complete. 2024-09-19T07:28:56.533 INFO:teuthology.orchestra.run.smithi103.stdout:Generating ssh key... 2024-09-19T07:28:56.865 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:56 smithi103 ceph-mon[31201]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:57.676 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: Generating public/private rsa key pair. 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: Your identification has been saved in /tmp/tmpcvfv3u89/key 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: Your public key has been saved in /tmp/tmpcvfv3u89/key.pub 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: The key fingerprint is: 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: SHA256:wxJk3V4OA+/gtYc6mo0IT/Gt7Ko1KeZJWrBvO+Ckk78 ceph-9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: The key's randomart image is: 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: +---[RSA 3072]----+ 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: | o..o | 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: | o ..+ . | 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: | . ..o= | 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: | + +.o. | 2024-09-19T07:28:57.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: |. . . S + . | 2024-09-19T07:28:57.678 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: |.+ + o o . | 2024-09-19T07:28:57.678 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: |=o* = . + | 2024-09-19T07:28:57.678 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: |+OoB + * . | 2024-09-19T07:28:57.678 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: |.+E=+oB . | 2024-09-19T07:28:57.678 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:28:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: +----[SHA256]-----+ 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.947+0000 7fdedcfc7640 1 Processor -- start 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.947+0000 7fdedcfc7640 1 -- start start 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.947+0000 7fdedcfc7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded8107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.947+0000 7fdedcfc7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fded8107ff0 con 0x7fded8107620 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.948+0000 7fded659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded8107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.948+0000 7fded659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded8107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35612/0 (socket says 172.21.15.103:35612) 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.948+0000 7fded659c640 1 -- 172.21.15.103:0/506757137 learned_addr learned my addr 172.21.15.103:0/506757137 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.948+0000 7fded659c640 1 -- 172.21.15.103:0/506757137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded8108820 con 0x7fded8107620 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.949+0000 7fded659c640 1 --2- 172.21.15.103:0/506757137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded8107a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009820 tx=0x7fdec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83efc6cdd60cf2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.949+0000 7fded559a640 1 -- 172.21.15.103:0/506757137 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec8037070 con 0x7fded8107620 2024-09-19T07:28:57.961 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.949+0000 7fded559a640 1 -- 172.21.15.103:0/506757137 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdec8032900 con 0x7fded8107620 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.949+0000 7fded559a640 1 -- 172.21.15.103:0/506757137 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec8030850 con 0x7fded8107620 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.949+0000 7fdedcfc7640 1 -- 172.21.15.103:0/506757137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 msgr2=0x7fded8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.949+0000 7fdedcfc7640 1 --2- 172.21.15.103:0/506757137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded8107a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009820 tx=0x7fdec802a270 comp rx=0 tx=0).stop 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.950+0000 7fdedcfc7640 1 -- 172.21.15.103:0/506757137 shutdown_connections 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.950+0000 7fdedcfc7640 1 --2- 172.21.15.103:0/506757137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded8107a20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.950+0000 7fdedcfc7640 1 -- 172.21.15.103:0/506757137 >> 172.21.15.103:0/506757137 conn(0x7fded8102db0 msgr2=0x7fded81051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.950+0000 7fdedcfc7640 1 -- 172.21.15.103:0/506757137 shutdown_connections 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.950+0000 7fdedcfc7640 1 -- 172.21.15.103:0/506757137 wait complete. 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.950+0000 7fdedcfc7640 1 Processor -- start 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.950+0000 7fdedcfc7640 1 -- start start 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fdedcfc7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded807db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fdedcfc7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fded810bbd0 con 0x7fded8107620 2024-09-19T07:28:57.962 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fded659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded807db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fded659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded807db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35624/0 (socket says 172.21.15.103:35624) 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fded659c640 1 -- 172.21.15.103:0/523740469 learned_addr learned my addr 172.21.15.103:0/523740469 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fded659c640 1 -- 172.21.15.103:0/523740469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded807e080 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fded659c640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded807db40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009940 tx=0x7fdec80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fdec77fe640 1 -- 172.21.15.103:0/523740469 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec8042070 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded807c270 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fdec77fe640 1 -- 172.21.15.103:0/523740469 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdec8032be0 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fdec77fe640 1 -- 172.21.15.103:0/523740469 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec8037040 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.951+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded807c7b0 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.952+0000 7fdec77fe640 1 -- 172.21.15.103:0/523740469 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fdec8045070 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.952+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded8107aa0 con 0x7fded8107620 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.952+0000 7fdec77fe640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fdec00391b0 0x7fdec003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:57.963 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.953+0000 7fdec77fe640 1 -- 172.21.15.103:0/523740469 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdec806c230 con 0x7fded8107620 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.953+0000 7fded5d9b640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fdec00391b0 0x7fdec003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.953+0000 7fded5d9b640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fdec00391b0 0x7fdec003b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0098c0 tx=0x7fdecc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:56.957+0000 7fdec77fe640 1 -- 172.21.15.103:0/523740469 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdec8045e10 con 0x7fded8107620 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.123+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fded8104070 con 0x7fdec00391b0 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.383+0000 7fdec77fe640 1 -- 172.21.15.103:0/523740469 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fded8104070 con 0x7fdec00391b0 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fdec00391b0 msgr2=0x7fdec003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fdec00391b0 0x7fdec003b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0098c0 tx=0x7fdecc006db0 comp rx=0 tx=0).stop 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 msgr2=0x7fded807db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded807db40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009940 tx=0x7fdec80326e0 comp rx=0 tx=0).stop 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 shutdown_connections 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fdec00391b0 0x7fdec003b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 --2- 172.21.15.103:0/523740469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fded8107620 0x7fded807db40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:57.964 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 >> 172.21.15.103:0/523740469 conn(0x7fded8102db0 msgr2=0x7fded8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:57.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 shutdown_connections 2024-09-19T07:28:57.965 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:57.385+0000 7fdedcfc7640 1 -- 172.21.15.103:0/523740469 wait complete. 2024-09-19T07:28:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:58 smithi103 ceph-mon[31201]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:28:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:58 smithi103 ceph-mon[31201]: Generating ssh key... 2024-09-19T07:28:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:58 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:28:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:58 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:28:59.160 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1V0vwYiHnpiNYKb+0SUA1sMizGNTUMVU6itJ4+WEcEXh0YJx7yLcJAXV7B6ii3VkPtmfC/RqeJJovENmcucCYqrTOIyJ5aco7gzEvpQ910AvQRsAji68pYzwpCRQNVCcVV30oW5MuTc37D4Dr+pqbE45mRQbTiV44bJrGqIBvw5F7wytWmrDdw9XdRRNMMHQmRHuk3ngsCo3nVePeRUq0wib/mkHCwnnvdBBT/n1sgpqX7C2B+r3j9ZiW7dz9nwFPc8BCVuYEc+FuMBYojASZtAayCW7yJ+9Pa2Wn2AN8kktEsq7FqXYlbMaXwIU9YmpMHrmLb0zywxX6TKJoW4wURDmsAo8fHcaKUDDKczDZ6TFdByGEKwSpDhdYHVYU702GA5pn8P2CHeXwkM+lrTB//czPh1uoyTp+1b4s3i1qVtf4OeJl5vCh6SuA4ppKH/qTyblugAWQpwX/JkHxQqq+a7czkxqTIhofMfWRACaC+p7W48kS2gMYitRR6XnDWr8= ceph-9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:28:59.160 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8aba891640 1 Processor -- start 2024-09-19T07:28:59.160 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8aba891640 1 -- start start 2024-09-19T07:28:59.160 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8aba891640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab410a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:59.160 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8aba891640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab407be20 con 0x7f8ab41080f0 2024-09-19T07:28:59.160 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab410a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab410a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57916/0 (socket says 172.21.15.103:57916) 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8ab3fff640 1 -- 172.21.15.103:0/3992840795 learned_addr learned my addr 172.21.15.103:0/3992840795 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.386+0000 7f8ab3fff640 1 -- 172.21.15.103:0/3992840795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab410aa20 con 0x7f8ab41080f0 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.387+0000 7f8ab3fff640 1 --2- 172.21.15.103:0/3992840795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab410a4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009820 tx=0x7f8aa402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6dfbde5db11f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.387+0000 7f8ab2ffd640 1 -- 172.21.15.103:0/3992840795 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aa4037070 con 0x7f8ab41080f0 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.387+0000 7f8ab2ffd640 1 -- 172.21.15.103:0/3992840795 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8aa4032900 con 0x7f8ab41080f0 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.387+0000 7f8ab2ffd640 1 -- 172.21.15.103:0/3992840795 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aa4030850 con 0x7f8ab41080f0 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 -- 172.21.15.103:0/3992840795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 msgr2=0x7f8ab410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 --2- 172.21.15.103:0/3992840795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab410a4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009820 tx=0x7f8aa402a270 comp rx=0 tx=0).stop 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 -- 172.21.15.103:0/3992840795 shutdown_connections 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 --2- 172.21.15.103:0/3992840795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab410a4e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:59.161 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 -- 172.21.15.103:0/3992840795 >> 172.21.15.103:0/3992840795 conn(0x7f8ab4101cb0 msgr2=0x7f8ab41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 -- 172.21.15.103:0/3992840795 shutdown_connections 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 -- 172.21.15.103:0/3992840795 wait complete. 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 Processor -- start 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.388+0000 7f8aba891640 1 -- start start 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8aba891640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8aba891640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab410ba60 con 0x7f8ab41080f0 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57928/0 (socket says 172.21.15.103:57928) 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab3fff640 1 -- 172.21.15.103:0/241874631 learned_addr learned my addr 172.21.15.103:0/241874631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab3fff640 1 -- 172.21.15.103:0/241874631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab419bba0 con 0x7f8ab41080f0 2024-09-19T07:28:59.162 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab3fff640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab419b660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40097f0 tx=0x7f8aa4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab17fa640 1 -- 172.21.15.103:0/241874631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aa4042070 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab17fa640 1 -- 172.21.15.103:0/241874631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8aa4032ca0 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab419be30 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.389+0000 7f8ab17fa640 1 -- 172.21.15.103:0/241874631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8aa4037040 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.390+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab419eb20 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.391+0000 7f8ab17fa640 1 -- 172.21.15.103:0/241874631 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8aa4045070 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.391+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab4108550 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.391+0000 7f8ab17fa640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8a9c0391b0 0x7f8a9c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.391+0000 7f8ab17fa640 1 -- 172.21.15.103:0/241874631 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8aa406c3e0 con 0x7f8ab41080f0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.391+0000 7f8ab37fe640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8a9c0391b0 0x7f8a9c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:28:59.163 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.391+0000 7f8ab37fe640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8a9c0391b0 0x7f8a9c03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80098c0 tx=0x7f8aa8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.395+0000 7f8ab17fa640 1 -- 172.21.15.103:0/241874631 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8aa4045520 con 0x7f8ab41080f0 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.570+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8ab4102ff0 con 0x7f8a9c0391b0 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.571+0000 7f8ab17fa640 1 -- 172.21.15.103:0/241874631 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8ab4102ff0 con 0x7f8a9c0391b0 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.573+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8a9c0391b0 msgr2=0x7f8a9c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.573+0000 7f8aba891640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8a9c0391b0 0x7f8a9c03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80098c0 tx=0x7f8aa8006db0 comp rx=0 tx=0).stop 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.573+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 msgr2=0x7f8ab419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.573+0000 7f8aba891640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab419b660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40097f0 tx=0x7f8aa4030ec0 comp rx=0 tx=0).stop 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.574+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 shutdown_connections 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.574+0000 7f8aba891640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f8a9c0391b0 0x7f8a9c03b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.574+0000 7f8aba891640 1 --2- 172.21.15.103:0/241874631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab41080f0 0x7f8ab419b660 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:28:59.164 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.574+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 >> 172.21.15.103:0/241874631 conn(0x7f8ab4101cb0 msgr2=0x7f8ab41028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:28:59.165 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.574+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 shutdown_connections 2024-09-19T07:28:59.165 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:58.574+0000 7f8aba891640 1 -- 172.21.15.103:0/241874631 wait complete. 2024-09-19T07:28:59.165 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:28:59.165 INFO:teuthology.orchestra.run.smithi103.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T07:28:59.165 INFO:teuthology.orchestra.run.smithi103.stdout:Adding host smithi103... 2024-09-19T07:28:59.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:28:59 smithi103 ceph-mon[31201]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:00.798 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:00 smithi103 ceph-mon[31201]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:00.798 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:00 smithi103 ceph-mon[31201]: Deploying cephadm binary to smithi103 2024-09-19T07:29:03.492 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Added host 'smithi103' with addr '172.21.15.103' 2024-09-19T07:29:03.492 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.571+0000 7f1c4f865640 1 Processor -- start 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.571+0000 7f1c4f865640 1 -- start start 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.571+0000 7f1c4f865640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4807c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.571+0000 7f1c4f865640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4807c8a0 con 0x7f1c4807de70 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.572+0000 7f1c4d601640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4807c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.572+0000 7f1c4d601640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4807c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57944/0 (socket says 172.21.15.103:57944) 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.572+0000 7f1c4d601640 1 -- 172.21.15.103:0/159991348 learned_addr learned my addr 172.21.15.103:0/159991348 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.572+0000 7f1c4d601640 1 -- 172.21.15.103:0/159991348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4807ca20 con 0x7f1c4807de70 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c4d601640 1 --2- 172.21.15.103:0/159991348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4807c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009820 tx=0x7f1c4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec57670c0b02b208 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c3ffff640 1 -- 172.21.15.103:0/159991348 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44037070 con 0x7f1c4807de70 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c3ffff640 1 -- 172.21.15.103:0/159991348 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c44032900 con 0x7f1c4807de70 2024-09-19T07:29:03.493 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c3ffff640 1 -- 172.21.15.103:0/159991348 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44030850 con 0x7f1c4807de70 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c4f865640 1 -- 172.21.15.103:0/159991348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 msgr2=0x7f1c4807c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c4f865640 1 --2- 172.21.15.103:0/159991348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4807c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009820 tx=0x7f1c4402a270 comp rx=0 tx=0).stop 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c4f865640 1 -- 172.21.15.103:0/159991348 shutdown_connections 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c4f865640 1 --2- 172.21.15.103:0/159991348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4807c2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c4f865640 1 -- 172.21.15.103:0/159991348 >> 172.21.15.103:0/159991348 conn(0x7f1c48102db0 msgr2=0x7f1c481051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.573+0000 7f1c4f865640 1 -- 172.21.15.103:0/159991348 shutdown_connections 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4f865640 1 -- 172.21.15.103:0/159991348 wait complete. 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4f865640 1 Processor -- start 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4f865640 1 -- start start 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4f865640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4f865640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4810dc90 con 0x7f1c4807de70 2024-09-19T07:29:03.494 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4d601640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4d601640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4819d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57948/0 (socket says 172.21.15.103:57948) 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.574+0000 7f1c4d601640 1 -- 172.21.15.103:0/1021092347 learned_addr learned my addr 172.21.15.103:0/1021092347 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.575+0000 7f1c4d601640 1 -- 172.21.15.103:0/1021092347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4819dd90 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.575+0000 7f1c4d601640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4819d850 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c44031000 tx=0x7f1c440326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.575+0000 7f1c3e7fc640 1 -- 172.21.15.103:0/1021092347 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44042070 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.575+0000 7f1c3e7fc640 1 -- 172.21.15.103:0/1021092347 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c44032be0 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.575+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4819e020 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.575+0000 7f1c3e7fc640 1 -- 172.21.15.103:0/1021092347 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44037040 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.575+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c481a0d10 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.576+0000 7f1c3e7fc640 1 -- 172.21.15.103:0/1021092347 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f1c44045070 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.576+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4807c350 con 0x7f1c4807de70 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.576+0000 7f1c3e7fc640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f1c340391b0 0x7f1c3403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:03.495 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.576+0000 7f1c3e7fc640 1 -- 172.21.15.103:0/1021092347 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1c4406c2a0 con 0x7f1c4807de70 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.577+0000 7f1c4ce00640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f1c340391b0 0x7f1c3403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.577+0000 7f1c4ce00640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f1c340391b0 0x7f1c3403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c380098c0 tx=0x7f1c38006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.581+0000 7f1c3e7fc640 1 -- 172.21.15.103:0/1021092347 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1c4402f2a0 con 0x7f1c4807de70 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:28:59.754+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}) -- 0x7f1c481040a0 con 0x7f1c340391b0 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.936+0000 7f1c3e7fc640 1 -- 172.21.15.103:0/1021092347 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1c481040a0 con 0x7f1c340391b0 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f1c340391b0 msgr2=0x7f1c3403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f1c340391b0 0x7f1c3403b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1c380098c0 tx=0x7f1c38006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 msgr2=0x7f1c4819d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4819d850 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c44031000 tx=0x7f1c440326e0 comp rx=0 tx=0).stop 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 shutdown_connections 2024-09-19T07:29:03.496 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f1c340391b0 0x7f1c3403b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:03.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 --2- 172.21.15.103:0/1021092347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4807de70 0x7f1c4819d850 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:03.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 >> 172.21.15.103:0/1021092347 conn(0x7f1c48102db0 msgr2=0x7f1c48103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:03.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 shutdown_connections 2024-09-19T07:29:03.497 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:02.940+0000 7f1c4f865640 1 -- 172.21.15.103:0/1021092347 wait complete. 2024-09-19T07:29:03.497 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mon service... 2024-09-19T07:29:04.204 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:03 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:29:04.204 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:03 smithi103 ceph-mon[31201]: Added host smithi103 2024-09-19T07:29:04.204 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:03 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:05.029 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.993+0000 7fcd685d0640 1 Processor -- start 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.993+0000 7fcd685d0640 1 -- start start 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.994+0000 7fcd685d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6007bcc0 0x7fcd6010a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.994+0000 7fcd685d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010a740 con 0x7fcd6007bcc0 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.994+0000 7fcd6636c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6007bcc0 0x7fcd6010a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.994+0000 7fcd6636c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6007bcc0 0x7fcd6010a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57962/0 (socket says 172.21.15.103:57962) 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.994+0000 7fcd6636c640 1 -- 172.21.15.103:0/706861316 learned_addr learned my addr 172.21.15.103:0/706861316 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.994+0000 7fcd6636c640 1 -- 172.21.15.103:0/706861316 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6010a8c0 con 0x7fcd6007bcc0 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.995+0000 7fcd6636c640 1 --2- 172.21.15.103:0/706861316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6007bcc0 0x7fcd6010a170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009820 tx=0x7fcd5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc79cc6ccf0a8dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.995+0000 7fcd6536a640 1 -- 172.21.15.103:0/706861316 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c037070 con 0x7fcd6007bcc0 2024-09-19T07:29:05.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.995+0000 7fcd6536a640 1 -- 172.21.15.103:0/706861316 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcd5c032900 con 0x7fcd6007bcc0 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.995+0000 7fcd6536a640 1 -- 172.21.15.103:0/706861316 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c030850 con 0x7fcd6007bcc0 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.995+0000 7fcd685d0640 1 -- 172.21.15.103:0/706861316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6007bcc0 msgr2=0x7fcd6010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.995+0000 7fcd685d0640 1 --2- 172.21.15.103:0/706861316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6007bcc0 0x7fcd6010a170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009820 tx=0x7fcd5c02a270 comp rx=0 tx=0).stop 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 -- 172.21.15.103:0/706861316 shutdown_connections 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 --2- 172.21.15.103:0/706861316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6007bcc0 0x7fcd6010a170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 -- 172.21.15.103:0/706861316 >> 172.21.15.103:0/706861316 conn(0x7fcd60101ca0 msgr2=0x7fcd601040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 -- 172.21.15.103:0/706861316 shutdown_connections 2024-09-19T07:29:05.031 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 -- 172.21.15.103:0/706861316 wait complete. 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 Processor -- start 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 -- start start 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6019b3b0 0x7fcd6019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.996+0000 7fcd685d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010b9c0 con 0x7fcd6019b3b0 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd6636c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6019b3b0 0x7fcd6019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd6636c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6019b3b0 0x7fcd6019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57974/0 (socket says 172.21.15.103:57974) 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd6636c640 1 -- 172.21.15.103:0/1814431881 learned_addr learned my addr 172.21.15.103:0/1814431881 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd6636c640 1 -- 172.21.15.103:0/1814431881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019bd10 con 0x7fcd6019b3b0 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd6636c640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6019b3b0 0x7fcd6019b7d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c032ce0 tx=0x7fcd5c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd577fe640 1 -- 172.21.15.103:0/1814431881 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c037040 con 0x7fcd6019b3b0 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6019bfa0 con 0x7fcd6019b3b0 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd577fe640 1 -- 172.21.15.103:0/1814431881 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcd5c040070 con 0x7fcd6019b3b0 2024-09-19T07:29:05.032 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd577fe640 1 -- 172.21.15.103:0/1814431881 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c03a630 con 0x7fcd6019b3b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.997+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6019ea80 con 0x7fcd6019b3b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.998+0000 7fcd577fe640 1 -- 172.21.15.103:0/1814431881 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fcd5c03a870 con 0x7fcd6019b3b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.998+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60074ca0 con 0x7fcd6019b3b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.998+0000 7fcd577fe640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcd4c0391b0 0x7fcd4c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.998+0000 7fcd577fe640 1 -- 172.21.15.103:0/1814431881 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd5c06c960 con 0x7fcd6019b3b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.998+0000 7fcd65b6b640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcd4c0391b0 0x7fcd4c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:03.999+0000 7fcd65b6b640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcd4c0391b0 0x7fcd4c03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcd500098c0 tx=0x7fcd50006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.003+0000 7fcd577fe640 1 -- 172.21.15.103:0/1814431881 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcd5c043360 con 0x7fcd6019b3b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.203+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcd60102d40 con 0x7fcd4c0391b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.208+0000 7fcd577fe640 1 -- 172.21.15.103:0/1814431881 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcd60102d40 con 0x7fcd4c0391b0 2024-09-19T07:29:05.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcd4c0391b0 msgr2=0x7fcd4c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcd4c0391b0 0x7fcd4c03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcd500098c0 tx=0x7fcd50006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6019b3b0 msgr2=0x7fcd6019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6019b3b0 0x7fcd6019b7d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c032ce0 tx=0x7fcd5c032d10 comp rx=0 tx=0).stop 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 shutdown_connections 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcd4c0391b0 0x7fcd4c03b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 --2- 172.21.15.103:0/1814431881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd6019b3b0 0x7fcd6019b7d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 >> 172.21.15.103:0/1814431881 conn(0x7fcd60101ca0 msgr2=0x7fcd60102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 shutdown_connections 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:04.211+0000 7fcd685d0640 1 -- 172.21.15.103:0/1814431881 wait complete. 2024-09-19T07:29:05.034 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mgr service... 2024-09-19T07:29:05.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:05 smithi103 ceph-mon[31201]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:05.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:05 smithi103 ceph-mon[31201]: Saving service mon spec with placement count:5 2024-09-19T07:29:05.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:05 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:29:06.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T07:29:06.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.448+0000 7fcee4ec5640 1 Processor -- start 2024-09-19T07:29:06.630 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.448+0000 7fcee4ec5640 1 -- start start 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.448+0000 7fcee4ec5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee010a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.448+0000 7fcee4ec5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcee007be30 con 0x7fcee0108080 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.448+0000 7fcede59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee010a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.448+0000 7fcede59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee010a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57990/0 (socket says 172.21.15.103:57990) 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.448+0000 7fcede59c640 1 -- 172.21.15.103:0/14185710 learned_addr learned my addr 172.21.15.103:0/14185710 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.449+0000 7fcede59c640 1 -- 172.21.15.103:0/14185710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee010a9b0 con 0x7fcee0108080 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.449+0000 7fcede59c640 1 --2- 172.21.15.103:0/14185710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee010a470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fced0009820 tx=0x7fced002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a0f272f29aaae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.449+0000 7fcedd59a640 1 -- 172.21.15.103:0/14185710 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced0037070 con 0x7fcee0108080 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.449+0000 7fcedd59a640 1 -- 172.21.15.103:0/14185710 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fced0032900 con 0x7fcee0108080 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.449+0000 7fcedd59a640 1 -- 172.21.15.103:0/14185710 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced0030850 con 0x7fcee0108080 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.450+0000 7fcee4ec5640 1 -- 172.21.15.103:0/14185710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 msgr2=0x7fcee010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:06.631 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.450+0000 7fcee4ec5640 1 --2- 172.21.15.103:0/14185710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee010a470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fced0009820 tx=0x7fced002a270 comp rx=0 tx=0).stop 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.450+0000 7fcee4ec5640 1 -- 172.21.15.103:0/14185710 shutdown_connections 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.450+0000 7fcee4ec5640 1 --2- 172.21.15.103:0/14185710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee010a470 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.450+0000 7fcee4ec5640 1 -- 172.21.15.103:0/14185710 >> 172.21.15.103:0/14185710 conn(0x7fcee0101ca0 msgr2=0x7fcee01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.450+0000 7fcee4ec5640 1 -- 172.21.15.103:0/14185710 shutdown_connections 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.450+0000 7fcee4ec5640 1 -- 172.21.15.103:0/14185710 wait complete. 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcee4ec5640 1 Processor -- start 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcee4ec5640 1 -- start start 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcee4ec5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcee4ec5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcee010b9f0 con 0x7fcee0108080 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcede59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcede59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee019b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58002/0 (socket says 172.21.15.103:58002) 2024-09-19T07:29:06.632 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcede59c640 1 -- 172.21.15.103:0/326946051 learned_addr learned my addr 172.21.15.103:0/326946051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcede59c640 1 -- 172.21.15.103:0/326946051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee019bb40 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.451+0000 7fcede59c640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee019b600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fced00097f0 tx=0x7fced0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.452+0000 7fcecf7fe640 1 -- 172.21.15.103:0/326946051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced0042070 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.452+0000 7fcecf7fe640 1 -- 172.21.15.103:0/326946051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fced0032ca0 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.452+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee019bdd0 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.452+0000 7fcecf7fe640 1 -- 172.21.15.103:0/326946051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fced0037040 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.452+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee019eac0 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.452+0000 7fcecf7fe640 1 -- 172.21.15.103:0/326946051 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fced0045070 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.452+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee01084e0 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.453+0000 7fcecf7fe640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcec8039200 0x7fcec803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.453+0000 7fcecf7fe640 1 -- 172.21.15.103:0/326946051 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fced006c4a0 con 0x7fcee0108080 2024-09-19T07:29:06.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.453+0000 7fceddd9b640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcec8039200 0x7fcec803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.453+0000 7fceddd9b640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcec8039200 0x7fcec803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fced40098c0 tx=0x7fced4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.457+0000 7fcecf7fe640 1 -- 172.21.15.103:0/326946051 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fced003add0 con 0x7fcee0108080 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.636+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcee0102f80 con 0x7fcec8039200 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.640+0000 7fcecf7fe640 1 -- 172.21.15.103:0/326946051 <== mgr.14118 v2:172.21.15.103:6800/2002728097 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcee0102f80 con 0x7fcec8039200 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.642+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcec8039200 msgr2=0x7fcec803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.642+0000 7fcee4ec5640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcec8039200 0x7fcec803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fced40098c0 tx=0x7fced4006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.642+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 msgr2=0x7fcee019b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.642+0000 7fcee4ec5640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee019b600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fced00097f0 tx=0x7fced0030ec0 comp rx=0 tx=0).stop 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.643+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 shutdown_connections 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.643+0000 7fcee4ec5640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcec8039200 0x7fcec803b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.643+0000 7fcee4ec5640 1 --2- 172.21.15.103:0/326946051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee0108080 0x7fcee019b600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.643+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 >> 172.21.15.103:0/326946051 conn(0x7fcee0101ca0 msgr2=0x7fcee0102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:06.634 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.643+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 shutdown_connections 2024-09-19T07:29:06.635 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:05.643+0000 7fcee4ec5640 1 -- 172.21.15.103:0/326946051 wait complete. 2024-09-19T07:29:06.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:06 smithi103 ceph-mon[31201]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:06.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:06 smithi103 ceph-mon[31201]: Saving service mgr spec with placement count:2 2024-09-19T07:29:06.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:06 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:29:08.281 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.535+0000 7fde48199640 1 Processor -- start 2024-09-19T07:29:08.281 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.535+0000 7fde48199640 1 -- start start 2024-09-19T07:29:08.281 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.536+0000 7fde48199640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde401068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.536+0000 7fde48199640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde40106ea0 con 0x7fde401064d0 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.536+0000 7fde45f35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde401068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.536+0000 7fde45f35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde401068d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58004/0 (socket says 172.21.15.103:58004) 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.536+0000 7fde45f35640 1 -- 172.21.15.103:0/3348082436 learned_addr learned my addr 172.21.15.103:0/3348082436 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.536+0000 7fde45f35640 1 -- 172.21.15.103:0/3348082436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde401076d0 con 0x7fde401064d0 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.537+0000 7fde45f35640 1 --2- 172.21.15.103:0/3348082436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde401068d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fde30009820 tx=0x7fde3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc32536c274f990 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.537+0000 7fde44f33640 1 -- 172.21.15.103:0/3348082436 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde30037070 con 0x7fde401064d0 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.537+0000 7fde44f33640 1 -- 172.21.15.103:0/3348082436 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde30032920 con 0x7fde401064d0 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.537+0000 7fde44f33640 1 -- 172.21.15.103:0/3348082436 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde30030810 con 0x7fde401064d0 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 -- 172.21.15.103:0/3348082436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 msgr2=0x7fde401068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 --2- 172.21.15.103:0/3348082436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde401068d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fde30009820 tx=0x7fde3002a270 comp rx=0 tx=0).stop 2024-09-19T07:29:08.282 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 -- 172.21.15.103:0/3348082436 shutdown_connections 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 --2- 172.21.15.103:0/3348082436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde401068d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 -- 172.21.15.103:0/3348082436 >> 172.21.15.103:0/3348082436 conn(0x7fde40101d00 msgr2=0x7fde40104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 -- 172.21.15.103:0/3348082436 shutdown_connections 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 -- 172.21.15.103:0/3348082436 wait complete. 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 Processor -- start 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.538+0000 7fde48199640 1 -- start start 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde48199640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde4007ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde48199640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde4010aa80 con 0x7fde401064d0 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde45f35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde4007ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde45f35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde4007ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58016/0 (socket says 172.21.15.103:58016) 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde45f35640 1 -- 172.21.15.103:0/2648402455 learned_addr learned my addr 172.21.15.103:0/2648402455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde45f35640 1 -- 172.21.15.103:0/2648402455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4019c1b0 con 0x7fde401064d0 2024-09-19T07:29:08.283 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde45f35640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde4007ba20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde30030560 tx=0x7fde30032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde36ffd640 1 -- 172.21.15.103:0/2648402455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde30037040 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.539+0000 7fde36ffd640 1 -- 172.21.15.103:0/2648402455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde30040070 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.540+0000 7fde36ffd640 1 -- 172.21.15.103:0/2648402455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde3003a740 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.540+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde401a07c0 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.540+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde401a0b60 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.540+0000 7fde36ffd640 1 -- 172.21.15.103:0/2648402455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fde3003a9b0 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.541+0000 7fde36ffd640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fde2c0391b0 0x7fde2c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.541+0000 7fde36ffd640 1 -- 172.21.15.103:0/2648402455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fde3006c890 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.541+0000 7fde45734640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fde2c0391b0 0x7fde2c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.541+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4010ac00 con 0x7fde401064d0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.541+0000 7fde45734640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fde2c0391b0 0x7fde2c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098c0 tx=0x7fde3c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:08.284 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.546+0000 7fde36ffd640 1 -- 172.21.15.103:0/2648402455 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fde30045e20 con 0x7fde401064d0 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.725+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fde40106950 con 0x7fde401064d0 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.727+0000 7fde36ffd640 1 -- 172.21.15.103:0/2648402455 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fde30031e80 con 0x7fde401064d0 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fde2c0391b0 msgr2=0x7fde2c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fde2c0391b0 0x7fde2c03b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098c0 tx=0x7fde3c006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 msgr2=0x7fde4007ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde4007ba20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde30030560 tx=0x7fde30032670 comp rx=0 tx=0).stop 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 shutdown_connections 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fde2c0391b0 0x7fde2c03b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 --2- 172.21.15.103:0/2648402455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde401064d0 0x7fde4007ba20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 >> 172.21.15.103:0/2648402455 conn(0x7fde40101d00 msgr2=0x7fde401028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 shutdown_connections 2024-09-19T07:29:08.285 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:07.730+0000 7fde48199640 1 -- 172.21.15.103:0/2648402455 wait complete. 2024-09-19T07:29:08.452 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:08 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:29:08.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:08 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:29:08.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:08 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2648402455' entity='client.admin' 2024-09-19T07:29:10.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.318+0000 7fda29b16640 1 Processor -- start 2024-09-19T07:29:10.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.319+0000 7fda29b16640 1 -- start start 2024-09-19T07:29:10.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.319+0000 7fda29b16640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2410a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:10.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.319+0000 7fda29b16640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda2407be40 con 0x7fda241080d0 2024-09-19T07:29:10.126 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.319+0000 7fda237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2410a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.319+0000 7fda237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2410a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45594/0 (socket says 172.21.15.103:45594) 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.319+0000 7fda237fe640 1 -- 172.21.15.103:0/3188366279 learned_addr learned my addr 172.21.15.103:0/3188366279 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.319+0000 7fda237fe640 1 -- 172.21.15.103:0/3188366279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2410aa00 con 0x7fda241080d0 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.320+0000 7fda237fe640 1 --2- 172.21.15.103:0/3188366279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2410a4c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009820 tx=0x7fda0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc338793454f244 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.320+0000 7fda227fc640 1 -- 172.21.15.103:0/3188366279 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda0c037070 con 0x7fda241080d0 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda227fc640 1 -- 172.21.15.103:0/3188366279 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0c032920 con 0x7fda241080d0 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda227fc640 1 -- 172.21.15.103:0/3188366279 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda0c030810 con 0x7fda241080d0 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda29b16640 1 -- 172.21.15.103:0/3188366279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 msgr2=0x7fda2410a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda29b16640 1 --2- 172.21.15.103:0/3188366279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2410a4c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009820 tx=0x7fda0c02a270 comp rx=0 tx=0).stop 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda29b16640 1 -- 172.21.15.103:0/3188366279 shutdown_connections 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda29b16640 1 --2- 172.21.15.103:0/3188366279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2410a4c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda29b16640 1 -- 172.21.15.103:0/3188366279 >> 172.21.15.103:0/3188366279 conn(0x7fda24101cb0 msgr2=0x7fda241040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:10.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda29b16640 1 -- 172.21.15.103:0/3188366279 shutdown_connections 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.321+0000 7fda29b16640 1 -- 172.21.15.103:0/3188366279 wait complete. 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.322+0000 7fda29b16640 1 Processor -- start 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.322+0000 7fda29b16640 1 -- start start 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.322+0000 7fda29b16640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.322+0000 7fda29b16640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda2410ba80 con 0x7fda241080d0 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.322+0000 7fda237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda237fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45610/0 (socket says 172.21.15.103:45610) 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda237fe640 1 -- 172.21.15.103:0/494644842 learned_addr learned my addr 172.21.15.103:0/494644842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda237fe640 1 -- 172.21.15.103:0/494644842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2419bbc0 con 0x7fda241080d0 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda237fe640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2419b680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda0c030420 tx=0x7fda0c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda20ff9640 1 -- 172.21.15.103:0/494644842 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda0c037040 con 0x7fda241080d0 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda20ff9640 1 -- 172.21.15.103:0/494644842 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0c040070 con 0x7fda241080d0 2024-09-19T07:29:10.128 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2419be50 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda20ff9640 1 -- 172.21.15.103:0/494644842 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda0c03a6e0 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.323+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2419eb40 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.324+0000 7fda20ff9640 1 -- 172.21.15.103:0/494644842 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fda0c03a9e0 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.324+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda24108530 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.324+0000 7fda20ff9640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fda100391b0 0x7fda1003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.324+0000 7fda20ff9640 1 -- 172.21.15.103:0/494644842 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fda0c06c8b0 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.324+0000 7fda22ffd640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fda100391b0 0x7fda1003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.325+0000 7fda22ffd640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fda100391b0 0x7fda1003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fda140098c0 tx=0x7fda14006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.329+0000 7fda20ff9640 1 -- 172.21.15.103:0/494644842 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fda0c043940 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.504+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fda2419ed10 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.507+0000 7fda20ff9640 1 -- 172.21.15.103:0/494644842 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fda0c043ba0 con 0x7fda241080d0 2024-09-19T07:29:10.129 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fda100391b0 msgr2=0x7fda1003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fda100391b0 0x7fda1003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fda140098c0 tx=0x7fda14006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 msgr2=0x7fda2419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2419b680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda0c030420 tx=0x7fda0c032670 comp rx=0 tx=0).stop 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 shutdown_connections 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fda100391b0 0x7fda1003b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 --2- 172.21.15.103:0/494644842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda241080d0 0x7fda2419b680 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 >> 172.21.15.103:0/494644842 conn(0x7fda24101cb0 msgr2=0x7fda241028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 shutdown_connections 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:09.510+0000 7fda29b16640 1 -- 172.21.15.103:0/494644842 wait complete. 2024-09-19T07:29:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling the dashboard module... 2024-09-19T07:29:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:09 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:29:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:09 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/494644842' entity='client.admin' 2024-09-19T07:29:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-mon[31201]: from='mgr.14118 172.21.15.103:0/467302273' entity='mgr.x' 2024-09-19T07:29:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2643221021' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T07:29:11.470 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ignoring --setuser ceph since I am not root 2024-09-19T07:29:11.470 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ignoring --setgroup ceph since I am not root 2024-09-19T07:29:11.470 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:11.321+0000 7f6343a2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:29:11.471 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:11.379+0000 7f6343a2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:29:11.855 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:11.528+0000 7f6343a2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:29:11.855 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:11.594+0000 7f6343a2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:29:11.855 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:11.739+0000 7f6343a2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:29:11.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.568+0000 7fcf9f6da640 1 Processor -- start 2024-09-19T07:29:11.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.568+0000 7fcf9f6da640 1 -- start start 2024-09-19T07:29:11.862 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.568+0000 7fcf9f6da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf9810a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.568+0000 7fcf9f6da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9807bdd0 con 0x7fcf981080e0 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.569+0000 7fcf9d476640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf9810a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.569+0000 7fcf9d476640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf9810a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45614/0 (socket says 172.21.15.103:45614) 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.569+0000 7fcf9d476640 1 -- 172.21.15.103:0/831376386 learned_addr learned my addr 172.21.15.103:0/831376386 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.569+0000 7fcf9d476640 1 -- 172.21.15.103:0/831376386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9810aa10 con 0x7fcf981080e0 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.570+0000 7fcf9d476640 1 --2- 172.21.15.103:0/831376386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf9810a4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009820 tx=0x7fcf9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43f25106a3cb9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.570+0000 7fcf8ffff640 1 -- 172.21.15.103:0/831376386 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf94037070 con 0x7fcf981080e0 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.570+0000 7fcf8ffff640 1 -- 172.21.15.103:0/831376386 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf94032920 con 0x7fcf981080e0 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.570+0000 7fcf8ffff640 1 -- 172.21.15.103:0/831376386 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf94030810 con 0x7fcf981080e0 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 -- 172.21.15.103:0/831376386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 msgr2=0x7fcf9810a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 --2- 172.21.15.103:0/831376386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf9810a4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009820 tx=0x7fcf9402a270 comp rx=0 tx=0).stop 2024-09-19T07:29:11.863 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 -- 172.21.15.103:0/831376386 shutdown_connections 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 --2- 172.21.15.103:0/831376386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf9810a4d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 -- 172.21.15.103:0/831376386 >> 172.21.15.103:0/831376386 conn(0x7fcf98101ca0 msgr2=0x7fcf981040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 -- 172.21.15.103:0/831376386 shutdown_connections 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 -- 172.21.15.103:0/831376386 wait complete. 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 Processor -- start 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.571+0000 7fcf9f6da640 1 -- start start 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9f6da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf98107960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9f6da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9810ba30 con 0x7fcf981080e0 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9d476640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf98107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9d476640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf98107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45622/0 (socket says 172.21.15.103:45622) 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9d476640 1 -- 172.21.15.103:0/2643221021 learned_addr learned my addr 172.21.15.103:0/2643221021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9d476640 1 -- 172.21.15.103:0/2643221021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf98107ea0 con 0x7fcf981080e0 2024-09-19T07:29:11.864 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9d476640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf98107960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf94030420 tx=0x7fcf94032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf94037040 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf94040070 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf98106090 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf9403a6e0 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.572+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981065d0 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.573+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fcf9403a990 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.574+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf98108540 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.574+0000 7fcf8e7fc640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcf84039200 0x7fcf8403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.574+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf9406c9e0 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.574+0000 7fcf9cc75640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcf84039200 0x7fcf8403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.574+0000 7fcf9cc75640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcf84039200 0x7fcf8403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098c0 tx=0x7fcf88006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.578+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf940328c0 con 0x7fcf981080e0 2024-09-19T07:29:11.865 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:10.805+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcf98106d70 con 0x7fcf981080e0 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.174+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcf9403ad80 con 0x7fcf981080e0 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.175+0000 7fcf8e7fc640 1 -- 172.21.15.103:0/2643221021 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fcf94045980 con 0x7fcf981080e0 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcf84039200 msgr2=0x7fcf8403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcf84039200 0x7fcf8403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098c0 tx=0x7fcf88006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 msgr2=0x7fcf98107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf98107960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf94030420 tx=0x7fcf94032670 comp rx=0 tx=0).stop 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 shutdown_connections 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fcf84039200 0x7fcf8403b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 --2- 172.21.15.103:0/2643221021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf981080e0 0x7fcf98107960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 >> 172.21.15.103:0/2643221021 conn(0x7fcf98101ca0 msgr2=0x7fcf98102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 shutdown_connections 2024-09-19T07:29:11.866 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:11.177+0000 7fcf9f6da640 1 -- 172.21.15.103:0/2643221021 wait complete. 2024-09-19T07:29:12.166 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:11.885+0000 7f6343a2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:29:12.451 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:12 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:12.205+0000 7f6343a2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:29:12.451 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:12 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:12.270+0000 7f6343a2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:29:12.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:12 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2643221021' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T07:29:12.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:12 smithi103 ceph-mon[31201]: mgrmap e9: x(active, since 18s) 2024-09-19T07:29:12.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:12 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:12.461+0000 7f6343a2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:29:13.426 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:13 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/29072861' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T07:29:13.426 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:13.358+0000 7f6343a2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:29:13.426 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:13.425+0000 7f6343a2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:29:13.473 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f2805391640 1 Processor -- start 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f2805391640 1 -- start start 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f2805391640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f2800106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f2805391640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f280007bde0 con 0x7f2800105f10 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f27feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f2800106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:13.474 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f27feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f2800106310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45656/0 (socket says 172.21.15.103:45656) 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f27feffd640 1 -- 172.21.15.103:0/3737991158 learned_addr learned my addr 172.21.15.103:0/3737991158 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.244+0000 7f27feffd640 1 -- 172.21.15.103:0/3737991158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2800106850 con 0x7f2800105f10 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.245+0000 7f27feffd640 1 --2- 172.21.15.103:0/3737991158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f2800106310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009820 tx=0x7f27f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0097c071d38205f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.245+0000 7f27fdffb640 1 -- 172.21.15.103:0/3737991158 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f0037070 con 0x7f2800105f10 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.245+0000 7f27fdffb640 1 -- 172.21.15.103:0/3737991158 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f0032920 con 0x7f2800105f10 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.245+0000 7f27fdffb640 1 -- 172.21.15.103:0/3737991158 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f0030810 con 0x7f2800105f10 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.246+0000 7f2805391640 1 -- 172.21.15.103:0/3737991158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 msgr2=0x7f2800106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.246+0000 7f2805391640 1 --2- 172.21.15.103:0/3737991158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f2800106310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009820 tx=0x7f27f002a270 comp rx=0 tx=0).stop 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.246+0000 7f2805391640 1 -- 172.21.15.103:0/3737991158 shutdown_connections 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.246+0000 7f2805391640 1 --2- 172.21.15.103:0/3737991158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f2800106310 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.246+0000 7f2805391640 1 -- 172.21.15.103:0/3737991158 >> 172.21.15.103:0/3737991158 conn(0x7f2800101c80 msgr2=0x7f28001040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:13.475 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.246+0000 7f2805391640 1 -- 172.21.15.103:0/3737991158 shutdown_connections 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.246+0000 7f2805391640 1 -- 172.21.15.103:0/3737991158 wait complete. 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f2805391640 1 Processor -- start 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f2805391640 1 -- start start 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f2805391640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f280019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f2805391640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2800107990 con 0x7f2800105f10 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f27feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f280019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f27feffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f280019d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45662/0 (socket says 172.21.15.103:45662) 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f27feffd640 1 -- 172.21.15.103:0/29072861 learned_addr learned my addr 172.21.15.103:0/29072861 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f27feffd640 1 -- 172.21.15.103:0/29072861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280019dd80 con 0x7f2800105f10 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.247+0000 7f27feffd640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f280019d840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f27f0030420 tx=0x7f27f0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.248+0000 7f27e7fff640 1 -- 172.21.15.103:0/29072861 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f0037040 con 0x7f2800105f10 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.248+0000 7f27e7fff640 1 -- 172.21.15.103:0/29072861 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f0040070 con 0x7f2800105f10 2024-09-19T07:29:13.476 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.248+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280019e010 con 0x7f2800105f10 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.248+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28001a0d00 con 0x7f2800105f10 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.248+0000 7f27e7fff640 1 -- 172.21.15.103:0/29072861 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f003a6e0 con 0x7f2800105f10 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.248+0000 7f27e7fff640 1 -- 172.21.15.103:0/29072861 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f27f003a9e0 con 0x7f2800105f10 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.249+0000 7f27e7fff640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 0x7f27e803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.249+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec0052c0 con 0x7f2800105f10 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.249+0000 7f27fe7fc640 1 -- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 msgr2=0x7f27e803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2002728097 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.249+0000 7f27e7fff640 1 -- 172.21.15.103:0/29072861 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27f006d900 con 0x7f2800105f10 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.249+0000 7f27fe7fc640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 0x7f27e803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.254+0000 7f27e7fff640 1 -- 172.21.15.103:0/29072861 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27f00328c0 con 0x7f2800105f10 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.450+0000 7f27fe7fc640 1 -- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 msgr2=0x7f27e803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2002728097 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.450+0000 7f27fe7fc640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 0x7f27e803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:29:13.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.473+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f27ec005dc0 con 0x7f2800105f10 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.473+0000 7f27e7fff640 1 -- 172.21.15.103:0/29072861 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f27f0045980 con 0x7f2800105f10 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 msgr2=0x7f27e803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 0x7f27e803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 msgr2=0x7f280019d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f280019d840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f27f0030420 tx=0x7f27f0032670 comp rx=0 tx=0).stop 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 shutdown_connections 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7f27e8039200 0x7f27e803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 --2- 172.21.15.103:0/29072861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2800105f10 0x7f280019d840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 >> 172.21.15.103:0/29072861 conn(0x7f2800101c80 msgr2=0x7f2800191b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 shutdown_connections 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:12.477+0000 7f2805391640 1 -- 172.21.15.103:0/29072861 wait complete. 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-19T07:29:13.478 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 9... 2024-09-19T07:29:14.403 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.148+0000 7f6343a2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:29:14.403 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.262+0000 7f6343a2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:29:14.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.402+0000 7f6343a2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:29:14.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.464+0000 7f6343a2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:29:14.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.610+0000 7f6343a2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:29:14.677 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.676+0000 7f6343a2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:29:14.956 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.735+0000 7f6343a2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:29:14.956 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.891+0000 7f6343a2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:29:14.956 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:14.955+0000 7f6343a2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:29:15.219 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:15.020+0000 7f6343a2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:29:15.219 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:15.102+0000 7f6343a2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:29:15.219 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:15.160+0000 7f6343a2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:29:15.219 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:15.218+0000 7f6343a2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:29:16.470 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:15.997+0000 7f6343a2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:29:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: Active manager daemon x restarted 2024-09-19T07:29:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: Activating manager daemon x 2024-09-19T07:29:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T07:29:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: mgrmap e10: x(active, starting, since 0.00327412s) 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: Manager daemon x is now available 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:29:16.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:29:17.510 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:17.448+0000 7f62a94f5640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:07:29:17] ENGINE Error in HTTPServer.serve 2024-09-19T07:29:17.510 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: Traceback (most recent call last): 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._connections.run(self.expiration_interval) 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._run(expiration_interval) 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: s = self.context.wrap_socket( 2024-09-19T07:29:17.511 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:29:17.512 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: return self.sslsocket_class._create( 2024-09-19T07:29:17.512 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:29:17.512 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self.do_handshake() 2024-09-19T07:29:17.512 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:29:17.512 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._sslobj.do_handshake() 2024-09-19T07:29:17.512 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:29:17.512 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:29:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:29:17.810 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-19T07:29:17.810 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T07:29:17.810 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T07:29:17.810 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-19T07:29:17.810 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd43e3ac640 1 Processor -- start 2024-09-19T07:29:17.810 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd43e3ac640 1 -- start start 2024-09-19T07:29:17.810 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd43e3ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43810a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd43e3ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd43807be80 con 0x7fd438108070 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43810a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43810a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45664/0 (socket says 172.21.15.103:45664) 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd437fff640 1 -- 172.21.15.103:0/3204990472 learned_addr learned my addr 172.21.15.103:0/3204990472 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.846+0000 7fd437fff640 1 -- 172.21.15.103:0/3204990472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43810a9a0 con 0x7fd438108070 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.847+0000 7fd437fff640 1 --2- 172.21.15.103:0/3204990472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43810a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd428009820 tx=0x7fd42802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec196e4942d3ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.847+0000 7fd436ffd640 1 -- 172.21.15.103:0/3204990472 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd428037070 con 0x7fd438108070 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.847+0000 7fd436ffd640 1 -- 172.21.15.103:0/3204990472 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd428032920 con 0x7fd438108070 2024-09-19T07:29:17.811 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.847+0000 7fd436ffd640 1 -- 172.21.15.103:0/3204990472 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd428030810 con 0x7fd438108070 2024-09-19T07:29:17.812 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.848+0000 7fd43e3ac640 1 -- 172.21.15.103:0/3204990472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 msgr2=0x7fd43810a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:17.812 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.848+0000 7fd43e3ac640 1 --2- 172.21.15.103:0/3204990472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43810a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd428009820 tx=0x7fd42802a270 comp rx=0 tx=0).stop 2024-09-19T07:29:17.812 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.848+0000 7fd43e3ac640 1 -- 172.21.15.103:0/3204990472 shutdown_connections 2024-09-19T07:29:17.812 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.848+0000 7fd43e3ac640 1 --2- 172.21.15.103:0/3204990472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43810a460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:17.812 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.848+0000 7fd43e3ac640 1 -- 172.21.15.103:0/3204990472 >> 172.21.15.103:0/3204990472 conn(0x7fd438101d10 msgr2=0x7fd438104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:17.812 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.848+0000 7fd43e3ac640 1 -- 172.21.15.103:0/3204990472 shutdown_connections 2024-09-19T07:29:17.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.848+0000 7fd43e3ac640 1 -- 172.21.15.103:0/3204990472 wait complete. 2024-09-19T07:29:17.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd43e3ac640 1 Processor -- start 2024-09-19T07:29:17.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd43e3ac640 1 -- start start 2024-09-19T07:29:17.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd43e3ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:17.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd43e3ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810ba80 con 0x7fd438108070 2024-09-19T07:29:17.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:17.813 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45680/0 (socket says 172.21.15.103:45680) 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd437fff640 1 -- 172.21.15.103:0/2365372881 learned_addr learned my addr 172.21.15.103:0/2365372881 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.849+0000 7fd437fff640 1 -- 172.21.15.103:0/2365372881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43819bb00 con 0x7fd438108070 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.850+0000 7fd437fff640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43819b5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd428030380 tx=0x7fd428032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.850+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd428037040 con 0x7fd438108070 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.850+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43819bd90 con 0x7fd438108070 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.850+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd428040070 con 0x7fd438108070 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.850+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd42803a7b0 con 0x7fd438108070 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.850+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43819ea80 con 0x7fd438108070 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.851+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fd42803aa30 con 0x7fd438108070 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.851+0000 7fd4357fa640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 0x7fd42003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:17.814 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.851+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 --> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd42003bdd0 con 0x7fd420039200 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.851+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd42806c970 con 0x7fd438108070 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.851+0000 7fd4377fe640 1 -- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 msgr2=0x7fd42003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2002728097 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:13.851+0000 7fd4377fe640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 0x7fd42003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:14.052+0000 7fd4377fe640 1 -- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 msgr2=0x7fd42003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2002728097 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:14.052+0000 7fd4377fe640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 0x7fd42003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:14.453+0000 7fd4377fe640 1 -- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 msgr2=0x7fd42003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2002728097 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:14.453+0000 7fd4377fe640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 0x7fd42003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:15.254+0000 7fd4377fe640 1 -- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 msgr2=0x7fd42003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2002728097 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:15.254+0000 7fd4377fe640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 0x7fd42003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T07:29:17.815 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:16.001+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7fd42806b420 con 0x7fd438108070 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:16.001+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 msgr2=0x7fd42003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:16.001+0000 7fd4357fa640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 0x7fd42003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.002+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fd428045460 con 0x7fd438108070 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.002+0000 7fd4357fa640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd42003ccc0 0x7fd42003f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.002+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd4280067f0 con 0x7fd42003ccc0 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.003+0000 7fd4377fe640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd42003ccc0 0x7fd42003f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.004+0000 7fd4377fe640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd42003ccc0 0x7fd42003f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd42c003980 tx=0x7fd42c006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.004+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd4280067f0 con 0x7fd42003ccc0 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.011+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd4381084d0 con 0x7fd42003ccc0 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.011+0000 7fd4357fa640 1 -- 172.21.15.103:0/2365372881 <== mgr.14150 v2:172.21.15.103:6800/2164654927 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd4381084d0 con 0x7fd42003ccc0 2024-09-19T07:29:17.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd42003ccc0 msgr2=0x7fd42003f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:17.817 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd42003ccc0 0x7fd42003f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd42c003980 tx=0x7fd42c006e10 comp rx=0 tx=0).stop 2024-09-19T07:29:17.817 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 msgr2=0x7fd43819b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:17.817 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43819b5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd428030380 tx=0x7fd428032670 comp rx=0 tx=0).stop 2024-09-19T07:29:17.817 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 shutdown_connections 2024-09-19T07:29:17.817 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd42003ccc0 0x7fd42003f0b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:17.817 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:6800/2002728097,v1:172.21.15.103:6801/2002728097] conn(0x7fd420039200 0x7fd42003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:17.817 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 --2- 172.21.15.103:0/2365372881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd438108070 0x7fd43819b5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:17.818 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 >> 172.21.15.103:0/2365372881 conn(0x7fd438101d10 msgr2=0x7fd438102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:17.818 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 shutdown_connections 2024-09-19T07:29:17.818 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:17.012+0000 7fd43e3ac640 1 -- 172.21.15.103:0/2365372881 wait complete. 2024-09-19T07:29:17.818 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 9 is available 2024-09-19T07:29:17.818 INFO:teuthology.orchestra.run.smithi103.stdout:Generating a dashboard self-signed certificate... 2024-09-19T07:29:18.120 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:18 smithi103 ceph-mon[31201]: mgrmap e11: x(active, since 1.00564s) 2024-09-19T07:29:18.121 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:18 smithi103 ceph-mon[31201]: [19/Sep/2024:07:29:17] ENGINE Bus STARTING 2024-09-19T07:29:19.083 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: [19/Sep/2024:07:29:17] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T07:29:19.083 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: [19/Sep/2024:07:29:17] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: [19/Sep/2024:07:29:17] ENGINE Bus STARTED 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: [19/Sep/2024:07:29:17] ENGINE Error in HTTPServer.serve 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:29:19.084 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: mgrmap e12: x(active, since 2s) 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:19.085 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:19.091 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T07:29:19.091 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.197+0000 7faab48b9640 1 Processor -- start 2024-09-19T07:29:19.091 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.197+0000 7faab48b9640 1 -- start start 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.198+0000 7faab48b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab0107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.198+0000 7faab48b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faab01084a0 con 0x7faab0107ad0 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.198+0000 7faaae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab0107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.198+0000 7faaae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab0107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40600/0 (socket says 172.21.15.103:40600) 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.198+0000 7faaae59c640 1 -- 172.21.15.103:0/1915693609 learned_addr learned my addr 172.21.15.103:0/1915693609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.198+0000 7faaae59c640 1 -- 172.21.15.103:0/1915693609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab01066e0 con 0x7faab0107ad0 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faaae59c640 1 --2- 172.21.15.103:0/1915693609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab0107ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009820 tx=0x7faaa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07627fa98a013d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faaad59a640 1 -- 172.21.15.103:0/1915693609 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaa0037070 con 0x7faab0107ad0 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faaad59a640 1 -- 172.21.15.103:0/1915693609 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0032920 con 0x7faab0107ad0 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faaad59a640 1 -- 172.21.15.103:0/1915693609 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaa0030810 con 0x7faab0107ad0 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faab48b9640 1 -- 172.21.15.103:0/1915693609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 msgr2=0x7faab0107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faab48b9640 1 --2- 172.21.15.103:0/1915693609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab0107ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009820 tx=0x7faaa002a270 comp rx=0 tx=0).stop 2024-09-19T07:29:19.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faab48b9640 1 -- 172.21.15.103:0/1915693609 shutdown_connections 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faab48b9640 1 --2- 172.21.15.103:0/1915693609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab0107ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faab48b9640 1 -- 172.21.15.103:0/1915693609 >> 172.21.15.103:0/1915693609 conn(0x7faab0101440 msgr2=0x7faab0103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.199+0000 7faab48b9640 1 -- 172.21.15.103:0/1915693609 shutdown_connections 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faab48b9640 1 -- 172.21.15.103:0/1915693609 wait complete. 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faab48b9640 1 Processor -- start 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faab48b9640 1 -- start start 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faab48b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faab48b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faab010aaa0 con 0x7faab0107ad0 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faaae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faaae59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab019b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40616/0 (socket says 172.21.15.103:40616) 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.200+0000 7faaae59c640 1 -- 172.21.15.103:0/2981985149 learned_addr learned my addr 172.21.15.103:0/2981985149 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.201+0000 7faaae59c640 1 -- 172.21.15.103:0/2981985149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab019bb80 con 0x7faab0107ad0 2024-09-19T07:29:19.093 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.201+0000 7faaae59c640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab019b640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faaa0030560 tx=0x7faaa0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.201+0000 7faaa77fe640 1 -- 172.21.15.103:0/2981985149 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaa0037040 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.201+0000 7faaa77fe640 1 -- 172.21.15.103:0/2981985149 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0040070 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.201+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab019be10 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.201+0000 7faaa77fe640 1 -- 172.21.15.103:0/2981985149 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaa003a740 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.201+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab019eb00 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.202+0000 7faaa77fe640 1 -- 172.21.15.103:0/2981985149 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7faaa003a8e0 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.202+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab010ac20 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.202+0000 7faaa77fe640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa9c039260 0x7faa9c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.202+0000 7faaa77fe640 1 -- 172.21.15.103:0/2981985149 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faaa006ca80 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.202+0000 7faaadd9b640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa9c039260 0x7faa9c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.203+0000 7faaadd9b640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa9c039260 0x7faa9c03b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa980098c0 tx=0x7faa98006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.207+0000 7faaa77fe640 1 -- 172.21.15.103:0/2981985149 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faaa0044c70 con 0x7faab0107ad0 2024-09-19T07:29:19.094 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.405+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7faab002da60 con 0x7faa9c039260 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.468+0000 7faaa77fe640 1 -- 172.21.15.103:0/2981985149 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7faab002da60 con 0x7faa9c039260 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa9c039260 msgr2=0x7faa9c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa9c039260 0x7faa9c03b720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa980098c0 tx=0x7faa98006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 msgr2=0x7faab019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab019b640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faaa0030560 tx=0x7faaa0032670 comp rx=0 tx=0).stop 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 shutdown_connections 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa9c039260 0x7faa9c03b720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 --2- 172.21.15.103:0/2981985149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab0107ad0 0x7faab019b640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:19.095 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.470+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 >> 172.21.15.103:0/2981985149 conn(0x7faab0101440 msgr2=0x7faab0102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:19.096 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.471+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 shutdown_connections 2024-09-19T07:29:19.096 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:18.471+0000 7faab48b9640 1 -- 172.21.15.103:0/2981985149 wait complete. 2024-09-19T07:29:19.096 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial admin user... 2024-09-19T07:29:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:20 smithi103 ceph-mon[31201]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$tgxM2wRttwwoGPA.O/6YSuHOS/yZAoRW3FbOzcs26HMavz2ixQ9Ve", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726730959, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.492+0000 7f7970ff3640 1 Processor -- start 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.492+0000 7f7970ff3640 1 -- start start 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.492+0000 7f7970ff3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.492+0000 7f7970ff3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f796c107ff0 con 0x7f796c107620 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.492+0000 7f796a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.492+0000 7f796a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40626/0 (socket says 172.21.15.103:40626) 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.492+0000 7f796a59c640 1 -- 172.21.15.103:0/3001794371 learned_addr learned my addr 172.21.15.103:0/3001794371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.493+0000 7f796a59c640 1 -- 172.21.15.103:0/3001794371 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c108820 con 0x7f796c107620 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.493+0000 7f796a59c640 1 --2- 172.21.15.103:0/3001794371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c107a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f795c009820 tx=0x7f795c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9884310dbb52c009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:21.289 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.493+0000 7f796959a640 1 -- 172.21.15.103:0/3001794371 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f795c037070 con 0x7f796c107620 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.493+0000 7f796959a640 1 -- 172.21.15.103:0/3001794371 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f795c032920 con 0x7f796c107620 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.493+0000 7f796959a640 1 -- 172.21.15.103:0/3001794371 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f795c030810 con 0x7f796c107620 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.494+0000 7f7970ff3640 1 -- 172.21.15.103:0/3001794371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 msgr2=0x7f796c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.494+0000 7f7970ff3640 1 --2- 172.21.15.103:0/3001794371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c107a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f795c009820 tx=0x7f795c02a270 comp rx=0 tx=0).stop 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.494+0000 7f7970ff3640 1 -- 172.21.15.103:0/3001794371 shutdown_connections 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.494+0000 7f7970ff3640 1 --2- 172.21.15.103:0/3001794371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c107a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.494+0000 7f7970ff3640 1 -- 172.21.15.103:0/3001794371 >> 172.21.15.103:0/3001794371 conn(0x7f796c102db0 msgr2=0x7f796c1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.494+0000 7f7970ff3640 1 -- 172.21.15.103:0/3001794371 shutdown_connections 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.494+0000 7f7970ff3640 1 -- 172.21.15.103:0/3001794371 wait complete. 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f7970ff3640 1 Processor -- start 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f7970ff3640 1 -- start start 2024-09-19T07:29:21.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f7970ff3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f7970ff3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f796c10bbd0 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f796a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f796a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40638/0 (socket says 172.21.15.103:40638) 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f796a59c640 1 -- 172.21.15.103:0/3849996605 learned_addr learned my addr 172.21.15.103:0/3849996605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f796a59c640 1 -- 172.21.15.103:0/3849996605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c19de00 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.495+0000 7f796a59c640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c19d8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f795c030560 tx=0x7f795c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.496+0000 7f795b7fe640 1 -- 172.21.15.103:0/3849996605 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f795c037040 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.496+0000 7f795b7fe640 1 -- 172.21.15.103:0/3849996605 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f795c040070 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.496+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c19e090 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.496+0000 7f795b7fe640 1 -- 172.21.15.103:0/3849996605 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f795c03a6d0 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.496+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c1a0d80 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.496+0000 7f795b7fe640 1 -- 172.21.15.103:0/3849996605 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f795c03a870 con 0x7f796c107620 2024-09-19T07:29:21.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.497+0000 7f795b7fe640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7954039260 0x7f795403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.497+0000 7f795b7fe640 1 -- 172.21.15.103:0/3849996605 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f795c06c8f0 con 0x7f796c107620 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.497+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c10bd50 con 0x7f796c107620 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.497+0000 7f7969d9b640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7954039260 0x7f795403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.497+0000 7f7969d9b640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7954039260 0x7f795403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79600098c0 tx=0x7f7960006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.501+0000 7f795b7fe640 1 -- 172.21.15.103:0/3849996605 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f795c044c70 con 0x7f796c107620 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.686+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f796c107aa0 con 0x7f7954039260 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.911+0000 7f795b7fe640 1 -- 172.21.15.103:0/3849996605 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f796c107aa0 con 0x7f7954039260 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7954039260 msgr2=0x7f795403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7954039260 0x7f795403b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79600098c0 tx=0x7f7960006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 msgr2=0x7f796c19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c19d8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f795c030560 tx=0x7f795c032670 comp rx=0 tx=0).stop 2024-09-19T07:29:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 shutdown_connections 2024-09-19T07:29:21.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7954039260 0x7f795403b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:21.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 --2- 172.21.15.103:0/3849996605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f796c107620 0x7f796c19d8c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:21.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 >> 172.21.15.103:0/3849996605 conn(0x7f796c102db0 msgr2=0x7f796c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:21.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 shutdown_connections 2024-09-19T07:29:21.293 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:19.913+0000 7f7970ff3640 1 -- 172.21.15.103:0/3849996605 wait complete. 2024-09-19T07:29:21.293 INFO:teuthology.orchestra.run.smithi103.stdout:Fetching dashboard port number... 2024-09-19T07:29:21.407 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:21 smithi103 ceph-mon[31201]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:22.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:22 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2895968385' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T07:29:22.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T07:29:22.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.695+0000 7fb653c27640 1 Processor -- start 2024-09-19T07:29:22.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.695+0000 7fb653c27640 1 -- start start 2024-09-19T07:29:22.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.695+0000 7fb653c27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:22.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.695+0000 7fb653c27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c07bdf0 con 0x7fb64c1080c0 2024-09-19T07:29:22.477 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.696+0000 7fb6519c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:22.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.696+0000 7fb6519c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c10a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40644/0 (socket says 172.21.15.103:40644) 2024-09-19T07:29:22.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.696+0000 7fb6519c3640 1 -- 172.21.15.103:0/2837287859 learned_addr learned my addr 172.21.15.103:0/2837287859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:22.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.696+0000 7fb6519c3640 1 -- 172.21.15.103:0/2837287859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c10a9f0 con 0x7fb64c1080c0 2024-09-19T07:29:22.478 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.696+0000 7fb6519c3640 1 --2- 172.21.15.103:0/2837287859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c10a4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009e80 tx=0x7fb63c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7f5ae9fc576d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb6509c1640 1 -- 172.21.15.103:0/2837287859 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb63c036070 con 0x7fb64c1080c0 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb6509c1640 1 -- 172.21.15.103:0/2837287859 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c0318e0 con 0x7fb64c1080c0 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb6509c1640 1 -- 172.21.15.103:0/2837287859 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb63c007a00 con 0x7fb64c1080c0 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb653c27640 1 -- 172.21.15.103:0/2837287859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 msgr2=0x7fb64c10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb653c27640 1 --2- 172.21.15.103:0/2837287859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c10a4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009e80 tx=0x7fb63c02a7a0 comp rx=0 tx=0).stop 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb653c27640 1 -- 172.21.15.103:0/2837287859 shutdown_connections 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb653c27640 1 --2- 172.21.15.103:0/2837287859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c10a4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb653c27640 1 -- 172.21.15.103:0/2837287859 >> 172.21.15.103:0/2837287859 conn(0x7fb64c101ca0 msgr2=0x7fb64c1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:22.499 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb653c27640 1 -- 172.21.15.103:0/2837287859 shutdown_connections 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.697+0000 7fb653c27640 1 -- 172.21.15.103:0/2837287859 wait complete. 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb653c27640 1 Processor -- start 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb653c27640 1 -- start start 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb653c27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c107940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb653c27640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c10ba10 con 0x7fb64c1080c0 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb6519c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c107940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb6519c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c107940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40648/0 (socket says 172.21.15.103:40648) 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb6519c3640 1 -- 172.21.15.103:0/2895968385 learned_addr learned my addr 172.21.15.103:0/2895968385 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:22.500 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.698+0000 7fb6519c3640 1 -- 172.21.15.103:0/2895968385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c107e80 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.699+0000 7fb6519c3640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c107940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb63c031cf0 tx=0x7fb63c031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.699+0000 7fb64affd640 1 -- 172.21.15.103:0/2895968385 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb63c036050 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.699+0000 7fb64affd640 1 -- 172.21.15.103:0/2895968385 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb63c042070 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.699+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c106070 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.699+0000 7fb64affd640 1 -- 172.21.15.103:0/2895968385 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb63c03d9e0 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.699+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c1065b0 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.700+0000 7fb64affd640 1 -- 172.21.15.103:0/2895968385 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb63c03bd60 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.700+0000 7fb64affd640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb634039200 0x7fb63403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.700+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c108520 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.700+0000 7fb64affd640 1 -- 172.21.15.103:0/2895968385 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb63c06d880 con 0x7fb64c1080c0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.700+0000 7fb6511c2640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb634039200 0x7fb63403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:22.501 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.700+0000 7fb6511c2640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb634039200 0x7fb63403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6400098c0 tx=0x7fb640006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.705+0000 7fb64affd640 1 -- 172.21.15.103:0/2895968385 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb63c042220 con 0x7fb64c1080c0 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.871+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb64c106830 con 0x7fb64c1080c0 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.871+0000 7fb64affd640 1 -- 172.21.15.103:0/2895968385 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb63c036c30 con 0x7fb64c1080c0 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb634039200 msgr2=0x7fb63403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb634039200 0x7fb63403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6400098c0 tx=0x7fb640006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 msgr2=0x7fb64c107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c107940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb63c031cf0 tx=0x7fb63c031d20 comp rx=0 tx=0).stop 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 shutdown_connections 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb634039200 0x7fb63403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 --2- 172.21.15.103:0/2895968385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb64c1080c0 0x7fb64c107940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 >> 172.21.15.103:0/2895968385 conn(0x7fb64c101ca0 msgr2=0x7fb64c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 shutdown_connections 2024-09-19T07:29:22.502 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:21.873+0000 7fb653c27640 1 -- 172.21.15.103:0/2895968385 wait complete. 2024-09-19T07:29:22.503 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T07:29:22.503 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph Dashboard is now available at: 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout: URL: https://smithi103.front.sepia.ceph.com:8443/ 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout: User: admin 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout: Password: emsvdtscu9 2024-09-19T07:29:22.504 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:22.505 INFO:teuthology.orchestra.run.smithi103.stdout:Saving cluster configuration to /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config directory 2024-09-19T07:29:23.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:23 smithi103 ceph-mon[31201]: mgrmap e13: x(active, since 6s) 2024-09-19T07:29:23.820 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcabc84a640 1 Processor -- start 2024-09-19T07:29:23.820 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcabc84a640 1 -- start start 2024-09-19T07:29:23.820 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcabc84a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab8106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:23.820 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcabc84a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8106ef0 con 0x7fcab8106520 2024-09-19T07:29:23.820 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcab659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab8106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcab659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab8106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40662/0 (socket says 172.21.15.103:40662) 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcab659c640 1 -- 172.21.15.103:0/436703336 learned_addr learned my addr 172.21.15.103:0/436703336 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.902+0000 7fcab659c640 1 -- 172.21.15.103:0/436703336 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab8107720 con 0x7fcab8106520 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.903+0000 7fcab659c640 1 --2- 172.21.15.103:0/436703336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab8106920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009820 tx=0x7fcaa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0a260180637a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.903+0000 7fcab5d9b640 1 -- 172.21.15.103:0/436703336 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa8037070 con 0x7fcab8106520 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.903+0000 7fcab5d9b640 1 -- 172.21.15.103:0/436703336 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa8032920 con 0x7fcab8106520 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.903+0000 7fcab5d9b640 1 -- 172.21.15.103:0/436703336 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa8030810 con 0x7fcab8106520 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.904+0000 7fcabc84a640 1 -- 172.21.15.103:0/436703336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 msgr2=0x7fcab8106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.904+0000 7fcabc84a640 1 --2- 172.21.15.103:0/436703336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab8106920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009820 tx=0x7fcaa802a270 comp rx=0 tx=0).stop 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.904+0000 7fcabc84a640 1 -- 172.21.15.103:0/436703336 shutdown_connections 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.904+0000 7fcabc84a640 1 --2- 172.21.15.103:0/436703336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab8106920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.904+0000 7fcabc84a640 1 -- 172.21.15.103:0/436703336 >> 172.21.15.103:0/436703336 conn(0x7fcab8101cb0 msgr2=0x7fcab81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:23.821 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.904+0000 7fcabc84a640 1 -- 172.21.15.103:0/436703336 shutdown_connections 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.904+0000 7fcabc84a640 1 -- 172.21.15.103:0/436703336 wait complete. 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcabc84a640 1 Processor -- start 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcabc84a640 1 -- start start 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcabc84a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab807ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcabc84a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcab810aad0 con 0x7fcab8106520 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcab659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab807ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcab659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab807ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40674/0 (socket says 172.21.15.103:40674) 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcab659c640 1 -- 172.21.15.103:0/2461704606 learned_addr learned my addr 172.21.15.103:0/2461704606 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcab659c640 1 -- 172.21.15.103:0/2461704606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab819e3a0 con 0x7fcab8106520 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcab659c640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab807ba60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8030560 tx=0x7fcaa8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.905+0000 7fcaaf7fe640 1 -- 172.21.15.103:0/2461704606 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa8037040 con 0x7fcab8106520 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.906+0000 7fcaaf7fe640 1 -- 172.21.15.103:0/2461704606 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa8040070 con 0x7fcab8106520 2024-09-19T07:29:23.822 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.906+0000 7fcaaf7fe640 1 -- 172.21.15.103:0/2461704606 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcaa803a6d0 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.906+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab819e570 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.906+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab819e990 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.906+0000 7fcaaf7fe640 1 -- 172.21.15.103:0/2461704606 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fcaa803a870 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.906+0000 7fcaaf7fe640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcaa4039300 0x7fcaa403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.906+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab81069a0 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.907+0000 7fcaaf7fe640 1 -- 172.21.15.103:0/2461704606 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcaa806cab0 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.907+0000 7fcaaffff640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcaa4039300 0x7fcaa403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.907+0000 7fcaaffff640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcaa4039300 0x7fcaa403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00098c0 tx=0x7fcaa0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:22.911+0000 7fcaaf7fe640 1 -- 172.21.15.103:0/2461704606 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcaa80313a0 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.163+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fcab819ebc0 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.165+0000 7fcaaf7fe640 1 -- 172.21.15.103:0/2461704606 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fcaa804a200 con 0x7fcab8106520 2024-09-19T07:29:23.823 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcaa4039300 msgr2=0x7fcaa403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcaa4039300 0x7fcaa403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00098c0 tx=0x7fcaa0006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 msgr2=0x7fcab807ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab807ba60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8030560 tx=0x7fcaa8032670 comp rx=0 tx=0).stop 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 shutdown_connections 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcaa4039300 0x7fcaa403b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 --2- 172.21.15.103:0/2461704606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcab8106520 0x7fcab807ba60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 >> 172.21.15.103:0/2461704606 conn(0x7fcab8101cb0 msgr2=0x7fcab81028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 shutdown_connections 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr 2024-09-19T07:29:23.167+0000 7fcabc84a640 1 -- 172.21.15.103:0/2461704606 wait complete. 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.824 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: ceph telemetry on 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout:For more information see: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:23.825 INFO:teuthology.orchestra.run.smithi103.stdout:Bootstrap complete. 2024-09-19T07:29:23.857 INFO:tasks.cephadm:Fetching config... 2024-09-19T07:29:23.857 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:29:23.857 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T07:29:23.874 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T07:29:23.874 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:29:23.874 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T07:29:23.930 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T07:29:23.930 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:29:23.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T07:29:23.995 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T07:29:23.995 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:29:23.995 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T07:29:24.051 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T07:29:24.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1V0vwYiHnpiNYKb+0SUA1sMizGNTUMVU6itJ4+WEcEXh0YJx7yLcJAXV7B6ii3VkPtmfC/RqeJJovENmcucCYqrTOIyJ5aco7gzEvpQ910AvQRsAji68pYzwpCRQNVCcVV30oW5MuTc37D4Dr+pqbE45mRQbTiV44bJrGqIBvw5F7wytWmrDdw9XdRRNMMHQmRHuk3ngsCo3nVePeRUq0wib/mkHCwnnvdBBT/n1sgpqX7C2B+r3j9ZiW7dz9nwFPc8BCVuYEc+FuMBYojASZtAayCW7yJ+9Pa2Wn2AN8kktEsq7FqXYlbMaXwIU9YmpMHrmLb0zywxX6TKJoW4wURDmsAo8fHcaKUDDKczDZ6TFdByGEKwSpDhdYHVYU702GA5pn8P2CHeXwkM+lrTB//czPh1uoyTp+1b4s3i1qVtf4OeJl5vCh6SuA4ppKH/qTyblugAWQpwX/JkHxQqq+a7czkxqTIhofMfWRACaC+p7W48kS2gMYitRR6XnDWr8= ceph-9fd714d2-7658-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T07:29:24.127 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1V0vwYiHnpiNYKb+0SUA1sMizGNTUMVU6itJ4+WEcEXh0YJx7yLcJAXV7B6ii3VkPtmfC/RqeJJovENmcucCYqrTOIyJ5aco7gzEvpQ910AvQRsAji68pYzwpCRQNVCcVV30oW5MuTc37D4Dr+pqbE45mRQbTiV44bJrGqIBvw5F7wytWmrDdw9XdRRNMMHQmRHuk3ngsCo3nVePeRUq0wib/mkHCwnnvdBBT/n1sgpqX7C2B+r3j9ZiW7dz9nwFPc8BCVuYEc+FuMBYojASZtAayCW7yJ+9Pa2Wn2AN8kktEsq7FqXYlbMaXwIU9YmpMHrmLb0zywxX6TKJoW4wURDmsAo8fHcaKUDDKczDZ6TFdByGEKwSpDhdYHVYU702GA5pn8P2CHeXwkM+lrTB//czPh1uoyTp+1b4s3i1qVtf4OeJl5vCh6SuA4ppKH/qTyblugAWQpwX/JkHxQqq+a7czkxqTIhofMfWRACaC+p7W48kS2gMYitRR6XnDWr8= ceph-9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:24.146 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1V0vwYiHnpiNYKb+0SUA1sMizGNTUMVU6itJ4+WEcEXh0YJx7yLcJAXV7B6ii3VkPtmfC/RqeJJovENmcucCYqrTOIyJ5aco7gzEvpQ910AvQRsAji68pYzwpCRQNVCcVV30oW5MuTc37D4Dr+pqbE45mRQbTiV44bJrGqIBvw5F7wytWmrDdw9XdRRNMMHQmRHuk3ngsCo3nVePeRUq0wib/mkHCwnnvdBBT/n1sgpqX7C2B+r3j9ZiW7dz9nwFPc8BCVuYEc+FuMBYojASZtAayCW7yJ+9Pa2Wn2AN8kktEsq7FqXYlbMaXwIU9YmpMHrmLb0zywxX6TKJoW4wURDmsAo8fHcaKUDDKczDZ6TFdByGEKwSpDhdYHVYU702GA5pn8P2CHeXwkM+lrTB//czPh1uoyTp+1b4s3i1qVtf4OeJl5vCh6SuA4ppKH/qTyblugAWQpwX/JkHxQqq+a7czkxqTIhofMfWRACaC+p7W48kS2gMYitRR6XnDWr8= ceph-9fd714d2-7658-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T07:29:24.189 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1V0vwYiHnpiNYKb+0SUA1sMizGNTUMVU6itJ4+WEcEXh0YJx7yLcJAXV7B6ii3VkPtmfC/RqeJJovENmcucCYqrTOIyJ5aco7gzEvpQ910AvQRsAji68pYzwpCRQNVCcVV30oW5MuTc37D4Dr+pqbE45mRQbTiV44bJrGqIBvw5F7wytWmrDdw9XdRRNMMHQmRHuk3ngsCo3nVePeRUq0wib/mkHCwnnvdBBT/n1sgpqX7C2B+r3j9ZiW7dz9nwFPc8BCVuYEc+FuMBYojASZtAayCW7yJ+9Pa2Wn2AN8kktEsq7FqXYlbMaXwIU9YmpMHrmLb0zywxX6TKJoW4wURDmsAo8fHcaKUDDKczDZ6TFdByGEKwSpDhdYHVYU702GA5pn8P2CHeXwkM+lrTB//czPh1uoyTp+1b4s3i1qVtf4OeJl5vCh6SuA4ppKH/qTyblugAWQpwX/JkHxQqq+a7czkxqTIhofMfWRACaC+p7W48kS2gMYitRR6XnDWr8= ceph-9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:24.201 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1V0vwYiHnpiNYKb+0SUA1sMizGNTUMVU6itJ4+WEcEXh0YJx7yLcJAXV7B6ii3VkPtmfC/RqeJJovENmcucCYqrTOIyJ5aco7gzEvpQ910AvQRsAji68pYzwpCRQNVCcVV30oW5MuTc37D4Dr+pqbE45mRQbTiV44bJrGqIBvw5F7wytWmrDdw9XdRRNMMHQmRHuk3ngsCo3nVePeRUq0wib/mkHCwnnvdBBT/n1sgpqX7C2B+r3j9ZiW7dz9nwFPc8BCVuYEc+FuMBYojASZtAayCW7yJ+9Pa2Wn2AN8kktEsq7FqXYlbMaXwIU9YmpMHrmLb0zywxX6TKJoW4wURDmsAo8fHcaKUDDKczDZ6TFdByGEKwSpDhdYHVYU702GA5pn8P2CHeXwkM+lrTB//czPh1uoyTp+1b4s3i1qVtf4OeJl5vCh6SuA4ppKH/qTyblugAWQpwX/JkHxQqq+a7czkxqTIhofMfWRACaC+p7W48kS2gMYitRR6XnDWr8= ceph-9fd714d2-7658-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T07:29:24.244 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1V0vwYiHnpiNYKb+0SUA1sMizGNTUMVU6itJ4+WEcEXh0YJx7yLcJAXV7B6ii3VkPtmfC/RqeJJovENmcucCYqrTOIyJ5aco7gzEvpQ910AvQRsAji68pYzwpCRQNVCcVV30oW5MuTc37D4Dr+pqbE45mRQbTiV44bJrGqIBvw5F7wytWmrDdw9XdRRNMMHQmRHuk3ngsCo3nVePeRUq0wib/mkHCwnnvdBBT/n1sgpqX7C2B+r3j9ZiW7dz9nwFPc8BCVuYEc+FuMBYojASZtAayCW7yJ+9Pa2Wn2AN8kktEsq7FqXYlbMaXwIU9YmpMHrmLb0zywxX6TKJoW4wURDmsAo8fHcaKUDDKczDZ6TFdByGEKwSpDhdYHVYU702GA5pn8P2CHeXwkM+lrTB//czPh1uoyTp+1b4s3i1qVtf4OeJl5vCh6SuA4ppKH/qTyblugAWQpwX/JkHxQqq+a7czkxqTIhofMfWRACaC+p7W48kS2gMYitRR6XnDWr8= ceph-9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:24.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T07:29:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:24 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2461704606' entity='client.admin' 2024-09-19T07:29:24.505 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.003+0000 7fec5db50640 1 -- 172.21.15.103:0/646363786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 msgr2=0x7fec58103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.003+0000 7fec5db50640 1 --2- 172.21.15.103:0/646363786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58103000 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec48009c20 tx=0x7fec4802aa90 comp rx=0 tx=0).stop 2024-09-19T07:29:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 -- 172.21.15.103:0/646363786 shutdown_connections 2024-09-19T07:29:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 --2- 172.21.15.103:0/646363786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58103000 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 -- 172.21.15.103:0/646363786 >> 172.21.15.103:0/646363786 conn(0x7fec580fb430 msgr2=0x7fec580fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 -- 172.21.15.103:0/646363786 shutdown_connections 2024-09-19T07:29:25.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 -- 172.21.15.103:0/646363786 wait complete. 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 Processor -- start 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 -- start start 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.004+0000 7fec5db50640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec58107730 con 0x7fec58102c00 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40684/0 (socket says 172.21.15.103:40684) 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec577fe640 1 -- 172.21.15.103:0/4241889256 learned_addr learned my addr 172.21.15.103:0/4241889256 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec577fe640 1 -- 172.21.15.103:0/4241889256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581972e0 con 0x7fec58102c00 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec577fe640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec48005cc0 tx=0x7fec4802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec54ff9640 1 -- 172.21.15.103:0/4241889256 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec48047070 con 0x7fec58102c00 2024-09-19T07:29:25.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec58197570 con 0x7fec58102c00 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec54ff9640 1 -- 172.21.15.103:0/4241889256 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec48003e50 con 0x7fec58102c00 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec54ff9640 1 -- 172.21.15.103:0/4241889256 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec48033050 con 0x7fec58102c00 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.005+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5819a220 con 0x7fec58102c00 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.006+0000 7fec54ff9640 1 -- 172.21.15.103:0/4241889256 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fec4804a070 con 0x7fec58102c00 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.006+0000 7fec54ff9640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec400392a0 0x7fec4003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.006+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58107eb0 con 0x7fec58102c00 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.006+0000 7fec54ff9640 1 -- 172.21.15.103:0/4241889256 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec4803e030 con 0x7fec58102c00 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.006+0000 7fec56ffd640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec400392a0 0x7fec4003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.007+0000 7fec56ffd640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec400392a0 0x7fec4003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0098c0 tx=0x7fec4c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:25.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.011+0000 7fec54ff9640 1 -- 172.21.15.103:0/4241889256 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec48051020 con 0x7fec58102c00 2024-09-19T07:29:25.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.181+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fec5802d8d0 con 0x7fec58102c00 2024-09-19T07:29:25.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.184+0000 7fec54ff9640 1 -- 172.21.15.103:0/4241889256 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fec480373b0 con 0x7fec58102c00 2024-09-19T07:29:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec400392a0 msgr2=0x7fec4003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec400392a0 0x7fec4003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0098c0 tx=0x7fec4c006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 msgr2=0x7fec58196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec48005cc0 tx=0x7fec4802fc10 comp rx=0 tx=0).stop 2024-09-19T07:29:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 shutdown_connections 2024-09-19T07:29:25.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec400392a0 0x7fec4003b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 --2- 172.21.15.103:0/4241889256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec58102c00 0x7fec58196da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 >> 172.21.15.103:0/4241889256 conn(0x7fec580fb430 msgr2=0x7fec580fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 shutdown_connections 2024-09-19T07:29:25.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:25.186+0000 7fec5db50640 1 -- 172.21.15.103:0/4241889256 wait complete. 2024-09-19T07:29:25.795 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T07:29:25.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T07:29:26.437 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:26 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4241889256' entity='client.admin' 2024-09-19T07:29:26.794 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.281+0000 7f9016abf640 1 -- 172.21.15.103:0/1800250508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 msgr2=0x7f9010101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.281+0000 7f9016abf640 1 --2- 172.21.15.103:0/1800250508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010101eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9004009880 tx=0x7f900402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.281+0000 7f9016abf640 1 -- 172.21.15.103:0/1800250508 shutdown_connections 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.281+0000 7f9016abf640 1 --2- 172.21.15.103:0/1800250508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010101eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.281+0000 7f9016abf640 1 -- 172.21.15.103:0/1800250508 >> 172.21.15.103:0/1800250508 conn(0x7f90100fb430 msgr2=0x7f90100fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.281+0000 7f9016abf640 1 -- 172.21.15.103:0/1800250508 shutdown_connections 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.281+0000 7f9016abf640 1 -- 172.21.15.103:0/1800250508 wait complete. 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f9016abf640 1 Processor -- start 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f9016abf640 1 -- start start 2024-09-19T07:29:27.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f9016abf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f9016abf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9010106640 con 0x7f9010101ab0 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f901485b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f901485b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010100160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40706/0 (socket says 172.21.15.103:40706) 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f901485b640 1 -- 172.21.15.103:0/3902952112 learned_addr learned my addr 172.21.15.103:0/3902952112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f901485b640 1 -- 172.21.15.103:0/3902952112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90101978f0 con 0x7f9010101ab0 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f901485b640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010100160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9004004f60 tx=0x7f90040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f900dffb640 1 -- 172.21.15.103:0/3902952112 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f900403f070 con 0x7f9010101ab0 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f900dffb640 1 -- 172.21.15.103:0/3902952112 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9004004d40 con 0x7f9010101ab0 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.282+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901019bf00 con 0x7f9010101ab0 2024-09-19T07:29:27.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.283+0000 7f900dffb640 1 -- 172.21.15.103:0/3902952112 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9004035030 con 0x7f9010101ab0 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.283+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f901019c2a0 con 0x7f9010101ab0 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.283+0000 7f900dffb640 1 -- 172.21.15.103:0/3902952112 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f9004042070 con 0x7f9010101ab0 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.284+0000 7f900dffb640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f90000392a0 0x7f900003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.284+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9010106dc0 con 0x7f9010101ab0 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.284+0000 7f900dffb640 1 -- 172.21.15.103:0/3902952112 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f900406d160 con 0x7f9010101ab0 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.284+0000 7f900ffff640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f90000392a0 0x7f900003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.284+0000 7f900ffff640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f90000392a0 0x7f900003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0098c0 tx=0x7f8ffc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:27.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.289+0000 7f900dffb640 1 -- 172.21.15.103:0/3902952112 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f900404ce10 con 0x7f9010101ab0 2024-09-19T07:29:27.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.463+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f901002da40 con 0x7f90000392a0 2024-09-19T07:29:27.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:27.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:27.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:27.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:27.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:27.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:27.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.466+0000 7f900dffb640 1 -- 172.21.15.103:0/3902952112 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f901002da40 con 0x7f90000392a0 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f90000392a0 msgr2=0x7f900003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f90000392a0 0x7f900003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0098c0 tx=0x7f8ffc006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 msgr2=0x7f9010100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010100160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9004004f60 tx=0x7f90040048f0 comp rx=0 tx=0).stop 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 shutdown_connections 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f90000392a0 0x7f900003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 --2- 172.21.15.103:0/3902952112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9010101ab0 0x7f9010100160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 >> 172.21.15.103:0/3902952112 conn(0x7f90100fb430 msgr2=0x7f90100fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 shutdown_connections 2024-09-19T07:29:27.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:27.469+0000 7f9016abf640 1 -- 172.21.15.103:0/3902952112 wait complete. 2024-09-19T07:29:28.183 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-09-19T07:29:28.184 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:29:28.184 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-09-19T07:29:28.203 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:29:28.203 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:28.261 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-09-19T07:29:28.261 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch host add smithi134 2024-09-19T07:29:28.717 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 -- 172.21.15.103:0/1606372609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 msgr2=0x7fb6c4100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 --2- 172.21.15.103:0/1606372609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4100d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009880 tx=0x7fb6b402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 -- 172.21.15.103:0/1606372609 shutdown_connections 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 --2- 172.21.15.103:0/1606372609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4100d70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 -- 172.21.15.103:0/1606372609 >> 172.21.15.103:0/1606372609 conn(0x7fb6c4076250 msgr2=0x7fb6c4076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 -- 172.21.15.103:0/1606372609 shutdown_connections 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 -- 172.21.15.103:0/1606372609 wait complete. 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.141+0000 7fb6c9845640 1 Processor -- start 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c9845640 1 -- start start 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c9845640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:29.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c9845640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c410ab90 con 0x7fb6c4100990 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39628/0 (socket says 172.21.15.103:39628) 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c2ffd640 1 -- 172.21.15.103:0/2074151269 learned_addr learned my addr 172.21.15.103:0/2074151269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c2ffd640 1 -- 172.21.15.103:0/2074151269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c41985a0 con 0x7fb6c4100990 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.142+0000 7fb6c2ffd640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4198060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009230 tx=0x7fb6b4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.143+0000 7fb6c8843640 1 -- 172.21.15.103:0/2074151269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b403f070 con 0x7fb6c4100990 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.143+0000 7fb6c8843640 1 -- 172.21.15.103:0/2074151269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b403bc80 con 0x7fb6c4100990 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.143+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c4198890 con 0x7fb6c4100990 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.143+0000 7fb6c8843640 1 -- 172.21.15.103:0/2074151269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4035030 con 0x7fb6c4100990 2024-09-19T07:29:29.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.143+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c4198d50 con 0x7fb6c4100990 2024-09-19T07:29:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.143+0000 7fb6c8843640 1 -- 172.21.15.103:0/2074151269 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fb6b4042070 con 0x7fb6c4100990 2024-09-19T07:29:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.144+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4101c40 con 0x7fb6c4100990 2024-09-19T07:29:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.144+0000 7fb6c8843640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb6b0039300 0x7fb6b003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:29.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.144+0000 7fb6c8843640 1 -- 172.21.15.103:0/2074151269 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb6b406cf70 con 0x7fb6c4100990 2024-09-19T07:29:29.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.144+0000 7fb6c27fc640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb6b0039300 0x7fb6b003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:29.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.144+0000 7fb6c27fc640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb6b0039300 0x7fb6b003b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0098c0 tx=0x7fb6ac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:29.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.148+0000 7fb6c8843640 1 -- 172.21.15.103:0/2074151269 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb6b4031dd0 con 0x7fb6c4100990 2024-09-19T07:29:29.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:29.318+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) -- 0x7fb6c4100df0 con 0x7fb6b0039300 2024-09-19T07:29:30.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:30 smithi103 ceph-mon[31201]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:30.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:30 smithi103 ceph-mon[31201]: Deploying cephadm binary to smithi134 2024-09-19T07:29:32.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.427+0000 7fb6c8843640 1 -- 172.21.15.103:0/2074151269 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb6c4100df0 con 0x7fb6b0039300 2024-09-19T07:29:32.427 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-09-19T07:29:32.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb6b0039300 msgr2=0x7fb6b003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:32.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb6b0039300 0x7fb6b003b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0098c0 tx=0x7fb6ac006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:32.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 msgr2=0x7fb6c4198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:32.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4198060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009230 tx=0x7fb6b4003d40 comp rx=0 tx=0).stop 2024-09-19T07:29:32.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 shutdown_connections 2024-09-19T07:29:32.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb6b0039300 0x7fb6b003b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:32.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 --2- 172.21.15.103:0/2074151269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6c4100990 0x7fb6c4198060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:32.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.429+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 >> 172.21.15.103:0/2074151269 conn(0x7fb6c4076250 msgr2=0x7fb6c41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:32.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.430+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 shutdown_connections 2024-09-19T07:29:32.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:32.430+0000 7fb6c9845640 1 -- 172.21.15.103:0/2074151269 wait complete. 2024-09-19T07:29:33.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T07:29:33.264 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:33.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:33.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:33 smithi103 ceph-mon[31201]: Added host smithi134 2024-09-19T07:29:33.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:33.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:33.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:33.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.746+0000 7fe8df5a4640 1 -- 172.21.15.103:0/3623588455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 msgr2=0x7fe8d8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:33.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.746+0000 7fe8df5a4640 1 --2- 172.21.15.103:0/3623588455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009880 tx=0x7fe8d402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:29:33.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.746+0000 7fe8df5a4640 1 -- 172.21.15.103:0/3623588455 shutdown_connections 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.746+0000 7fe8df5a4640 1 --2- 172.21.15.103:0/3623588455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.746+0000 7fe8df5a4640 1 -- 172.21.15.103:0/3623588455 >> 172.21.15.103:0/3623588455 conn(0x7fe8d80fd2b0 msgr2=0x7fe8d80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.747+0000 7fe8df5a4640 1 -- 172.21.15.103:0/3623588455 shutdown_connections 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.747+0000 7fe8df5a4640 1 -- 172.21.15.103:0/3623588455 wait complete. 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.747+0000 7fe8df5a4640 1 Processor -- start 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.747+0000 7fe8df5a4640 1 -- start start 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.747+0000 7fe8df5a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.747+0000 7fe8df5a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d810bc70 con 0x7fe8d8107660 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8dd340640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8dd340640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39656/0 (socket says 172.21.15.103:39656) 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8dd340640 1 -- 172.21.15.103:0/1521272755 learned_addr learned my addr 172.21.15.103:0/1521272755 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:33.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8dd340640 1 -- 172.21.15.103:0/1521272755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d8197650 con 0x7fe8d8107660 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8dd340640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8197110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009230 tx=0x7fe8d4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8ce7fc640 1 -- 172.21.15.103:0/1521272755 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d403f070 con 0x7fe8d8107660 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8ce7fc640 1 -- 172.21.15.103:0/1521272755 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d403bc80 con 0x7fe8d8107660 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d8197940 con 0x7fe8d8107660 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8ce7fc640 1 -- 172.21.15.103:0/1521272755 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d4035030 con 0x7fe8d8107660 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.748+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d819b9e0 con 0x7fe8d8107660 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.749+0000 7fe8ce7fc640 1 -- 172.21.15.103:0/1521272755 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe8d4042070 con 0x7fe8d8107660 2024-09-19T07:29:33.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.749+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d8102d10 con 0x7fe8d8107660 2024-09-19T07:29:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.750+0000 7fe8ce7fc640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe8c40392a0 0x7fe8c403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.750+0000 7fe8ce7fc640 1 -- 172.21.15.103:0/1521272755 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe8d406d070 con 0x7fe8d8107660 2024-09-19T07:29:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.750+0000 7fe8dcb3f640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe8c40392a0 0x7fe8c403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:33.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.751+0000 7fe8dcb3f640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe8c40392a0 0x7fe8c403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098c0 tx=0x7fe8c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:33.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.755+0000 7fe8ce7fc640 1 -- 172.21.15.103:0/1521272755 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe8d4031e20 con 0x7fe8d8107660 2024-09-19T07:29:33.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.928+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8d8198470 con 0x7fe8c40392a0 2024-09-19T07:29:33.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.929+0000 7fe8ce7fc640 1 -- 172.21.15.103:0/1521272755 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fe8d8198470 con 0x7fe8c40392a0 2024-09-19T07:29:33.929 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:33.929 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-09-19T07:29:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe8c40392a0 msgr2=0x7fe8c403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe8c40392a0 0x7fe8c403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098c0 tx=0x7fe8c8006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 msgr2=0x7fe8d8197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:33.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8197110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009230 tx=0x7fe8d4003d40 comp rx=0 tx=0).stop 2024-09-19T07:29:33.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 shutdown_connections 2024-09-19T07:29:33.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe8c40392a0 0x7fe8c403b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:33.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 --2- 172.21.15.103:0/1521272755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8d8107660 0x7fe8d8197110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:33.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 >> 172.21.15.103:0/1521272755 conn(0x7fe8d80fd2b0 msgr2=0x7fe8d80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:33.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 shutdown_connections 2024-09-19T07:29:33.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:33.931+0000 7fe8df5a4640 1 -- 172.21.15.103:0/1521272755 wait complete. 2024-09-19T07:29:34.531 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-09-19T07:29:34.531 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:29:34.531 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-09-19T07:29:34.551 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:29:34.551 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:34.609 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-09-19T07:29:34.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch host add smithi174 2024-09-19T07:29:34.850 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:35.074 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:35.074 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:34 smithi103 ceph-mon[31201]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 -- 172.21.15.103:0/2045516904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 msgr2=0x7f3554107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 --2- 172.21.15.103:0/2045516904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3544009880 tx=0x7f354402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 -- 172.21.15.103:0/2045516904 shutdown_connections 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 --2- 172.21.15.103:0/2045516904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 -- 172.21.15.103:0/2045516904 >> 172.21.15.103:0/2045516904 conn(0x7f35540fd2b0 msgr2=0x7f35540ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 -- 172.21.15.103:0/2045516904 shutdown_connections 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 -- 172.21.15.103:0/2045516904 wait complete. 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 Processor -- start 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.280+0000 7f355a8b6640 1 -- start start 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.281+0000 7f355a8b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:35.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.281+0000 7f355a8b6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f355410bc70 con 0x7f3554107660 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.281+0000 7f3553fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.281+0000 7f3553fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39668/0 (socket says 172.21.15.103:39668) 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.281+0000 7f3553fff640 1 -- 172.21.15.103:0/2127401050 learned_addr learned my addr 172.21.15.103:0/2127401050 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.281+0000 7f3553fff640 1 -- 172.21.15.103:0/2127401050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3554197650 con 0x7f3554107660 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.281+0000 7f3553fff640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3544009230 tx=0x7f3544003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.282+0000 7f35517fa640 1 -- 172.21.15.103:0/2127401050 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f354403f070 con 0x7f3554107660 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.282+0000 7f35517fa640 1 -- 172.21.15.103:0/2127401050 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f354403bc80 con 0x7f3554107660 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.282+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3554197940 con 0x7f3554107660 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.282+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355419b9e0 con 0x7f3554107660 2024-09-19T07:29:35.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.282+0000 7f35517fa640 1 -- 172.21.15.103:0/2127401050 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3544035030 con 0x7f3554107660 2024-09-19T07:29:35.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.282+0000 7f35517fa640 1 -- 172.21.15.103:0/2127401050 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f3544042070 con 0x7f3554107660 2024-09-19T07:29:35.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.283+0000 7f35517fa640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3540039300 0x7f354003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:35.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.283+0000 7f35517fa640 1 -- 172.21.15.103:0/2127401050 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f354406cf70 con 0x7f3554107660 2024-09-19T07:29:35.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.283+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3554102d10 con 0x7f3554107660 2024-09-19T07:29:35.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.283+0000 7f35537fe640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3540039300 0x7f354003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:35.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.283+0000 7f35537fe640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3540039300 0x7f354003b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f353c0098c0 tx=0x7f353c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:35.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.288+0000 7f35517fa640 1 -- 172.21.15.103:0/2127401050 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35440314c0 con 0x7f3554107660 2024-09-19T07:29:35.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:35.456+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) -- 0x7f3554107b20 con 0x7f3540039300 2024-09-19T07:29:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:37.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:37.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:37.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:38 smithi103 ceph-mon[31201]: Deploying cephadm binary to smithi174 2024-09-19T07:29:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:38 smithi103 ceph-mon[31201]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T07:29:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:38 smithi103 ceph-mon[31201]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:29:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:38 smithi103 ceph-mon[31201]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:38 smithi103 ceph-mon[31201]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:29:38.639 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-09-19T07:29:38.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.639+0000 7f35517fa640 1 -- 172.21.15.103:0/2127401050 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3554107b20 con 0x7f3540039300 2024-09-19T07:29:38.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.641+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3540039300 msgr2=0x7f354003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:38.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.641+0000 7f355a8b6640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3540039300 0x7f354003b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f353c0098c0 tx=0x7f353c006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:38.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 msgr2=0x7f3554197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:38.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3544009230 tx=0x7f3544003d40 comp rx=0 tx=0).stop 2024-09-19T07:29:38.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 shutdown_connections 2024-09-19T07:29:38.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3540039300 0x7f354003b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:38.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 --2- 172.21.15.103:0/2127401050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3554107660 0x7f3554197110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:38.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 >> 172.21.15.103:0/2127401050 conn(0x7f35540fd2b0 msgr2=0x7f35540feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:38.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 shutdown_connections 2024-09-19T07:29:38.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:38.642+0000 7f355a8b6640 1 -- 172.21.15.103:0/2127401050 wait complete. 2024-09-19T07:29:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:39 smithi103 ceph-mon[31201]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:29:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:39.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T07:29:39.520 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.978+0000 7f6307bc9640 1 -- 172.21.15.103:0/3758389415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 msgr2=0x7f6300107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.978+0000 7f6307bc9640 1 --2- 172.21.15.103:0/3758389415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300107a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009c20 tx=0x7f62f002aa90 comp rx=0 tx=0).stop 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 -- 172.21.15.103:0/3758389415 shutdown_connections 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 --2- 172.21.15.103:0/3758389415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300107a70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 -- 172.21.15.103:0/3758389415 >> 172.21.15.103:0/3758389415 conn(0x7f6300076250 msgr2=0x7f6300076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 -- 172.21.15.103:0/3758389415 shutdown_connections 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 -- 172.21.15.103:0/3758389415 wait complete. 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 Processor -- start 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 -- start start 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:39.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6307bc9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f630010bce0 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.979+0000 7f6305965640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f6305965640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38066/0 (socket says 172.21.15.103:38066) 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f6305965640 1 -- 172.21.15.103:0/1167842214 learned_addr learned my addr 172.21.15.103:0/1167842214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f6305965640 1 -- 172.21.15.103:0/1167842214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6300197650 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f6305965640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300197110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005e30 tx=0x7f62f002fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f62feffd640 1 -- 172.21.15.103:0/1167842214 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f0047070 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f62feffd640 1 -- 172.21.15.103:0/1167842214 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f0003df0 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63001978e0 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f62feffd640 1 -- 172.21.15.103:0/1167842214 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f0033050 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.980+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630019b9e0 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.981+0000 7f62feffd640 1 -- 172.21.15.103:0/1167842214 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f62f004a070 con 0x7f6300107690 2024-09-19T07:29:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.981+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6300102d30 con 0x7f6300107690 2024-09-19T07:29:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.981+0000 7f62feffd640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62e80392a0 0x7f62e803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.981+0000 7f62feffd640 1 -- 172.21.15.103:0/1167842214 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f62f003e030 con 0x7f6300107690 2024-09-19T07:29:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.981+0000 7f6305164640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62e80392a0 0x7f62e803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.982+0000 7f6305164640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62e80392a0 0x7f62e803b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f62f40098c0 tx=0x7f62f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:39.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:39.986+0000 7f62feffd640 1 -- 172.21.15.103:0/1167842214 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62f004a360 con 0x7f6300107690 2024-09-19T07:29:40.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.169+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6300198470 con 0x7f62e80392a0 2024-09-19T07:29:40.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.170+0000 7f62feffd640 1 -- 172.21.15.103:0/1167842214 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f6300198470 con 0x7f62e80392a0 2024-09-19T07:29:40.170 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:29:40.170 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-09-19T07:29:40.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62e80392a0 msgr2=0x7f62e803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62e80392a0 0x7f62e803b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f62f40098c0 tx=0x7f62f4006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 msgr2=0x7f6300197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300197110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62f0005e30 tx=0x7f62f002fc10 comp rx=0 tx=0).stop 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 shutdown_connections 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62e80392a0 0x7f62e803b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 --2- 172.21.15.103:0/1167842214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300107690 0x7f6300197110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 >> 172.21.15.103:0/1167842214 conn(0x7f6300076250 msgr2=0x7f63000fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 shutdown_connections 2024-09-19T07:29:40.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:40.173+0000 7f6307bc9640 1 -- 172.21.15.103:0/1167842214 wait complete. 2024-09-19T07:29:40.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:40 smithi103 ceph-mon[31201]: Added host smithi174 2024-09-19T07:29:40.454 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:40.454 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:40.795 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T07:29:40.795 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T07:29:41.050 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:29:41.465 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:41 smithi103 ceph-mon[31201]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:41.465 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:41 smithi103 ceph-mon[31201]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:29:41.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 -- 172.21.15.103:0/3941072172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 msgr2=0x7f4338102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:41.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 --2- 172.21.15.103:0/3941072172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338102bc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f432c009880 tx=0x7f432c02a670 comp rx=0 tx=0).stop 2024-09-19T07:29:41.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 -- 172.21.15.103:0/3941072172 shutdown_connections 2024-09-19T07:29:41.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 --2- 172.21.15.103:0/3941072172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338102bc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 -- 172.21.15.103:0/3941072172 >> 172.21.15.103:0/3941072172 conn(0x7f43380fc1a0 msgr2=0x7f43380fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 -- 172.21.15.103:0/3941072172 shutdown_connections 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 -- 172.21.15.103:0/3941072172 wait complete. 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.490+0000 7f43407a0640 1 Processor -- start 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f43407a0640 1 -- start start 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f43407a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f43407a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f433810bce0 con 0x7f43381027e0 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f433e53c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f433e53c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38082/0 (socket says 172.21.15.103:38082) 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f433e53c640 1 -- 172.21.15.103:0/109786756 learned_addr learned my addr 172.21.15.103:0/109786756 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f433e53c640 1 -- 172.21.15.103:0/109786756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43381976c0 con 0x7f43381027e0 2024-09-19T07:29:41.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.491+0000 7f433e53c640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338197180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f432c009230 tx=0x7f432c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.492+0000 7f432b7fe640 1 -- 172.21.15.103:0/109786756 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f432c03f070 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.492+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4338197950 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.492+0000 7f432b7fe640 1 -- 172.21.15.103:0/109786756 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432c03bc80 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.492+0000 7f432b7fe640 1 -- 172.21.15.103:0/109786756 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f432c035030 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.492+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4338197db0 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.492+0000 7f432b7fe640 1 -- 172.21.15.103:0/109786756 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f432c042070 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.493+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433806a760 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.493+0000 7f432b7fe640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f43240392a0 0x7f432403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.493+0000 7f432b7fe640 1 -- 172.21.15.103:0/109786756 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f432c06cff0 con 0x7f43381027e0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.493+0000 7f433dd3b640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f43240392a0 0x7f432403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.493+0000 7f433dd3b640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f43240392a0 0x7f432403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43340098c0 tx=0x7f4334006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.497+0000 7f432b7fe640 1 -- 172.21.15.103:0/109786756 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f432c031e20 con 0x7f43381027e0 2024-09-19T07:29:41.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:41.666+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4338198540 con 0x7f43381027e0 2024-09-19T07:29:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.210+0000 7f432b7fe640 1 -- 172.21.15.103:0/109786756 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f432c031570 con 0x7f43381027e0 2024-09-19T07:29:42.211 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f43240392a0 msgr2=0x7f432403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f43240392a0 0x7f432403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f43340098c0 tx=0x7f4334006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 msgr2=0x7f4338197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338197180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f432c009230 tx=0x7f432c003d40 comp rx=0 tx=0).stop 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 shutdown_connections 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f43240392a0 0x7f432403b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 --2- 172.21.15.103:0/109786756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43381027e0 0x7f4338197180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 >> 172.21.15.103:0/109786756 conn(0x7f43380fc1a0 msgr2=0x7f4338103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 shutdown_connections 2024-09-19T07:29:42.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:29:42.212+0000 7f43407a0640 1 -- 172.21.15.103:0/109786756 wait complete. 2024-09-19T07:29:42.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:42 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/109786756' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T07:29:42.841 INFO:tasks.cephadm:Adding mon.a on smithi103 2024-09-19T07:29:42.841 INFO:tasks.cephadm:Adding mon.b on smithi134 2024-09-19T07:29:42.841 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-09-19T07:29:42.841 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c' 2024-09-19T07:29:43.082 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T07:29:43.142 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T07:29:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/109786756' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:43.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:43.677 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 -- 172.21.15.174:0/2888780106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 msgr2=0x7ff378101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 --2- 172.21.15.174:0/2888780106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378101a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff374009880 tx=0x7ff37402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 -- 172.21.15.174:0/2888780106 shutdown_connections 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 --2- 172.21.15.174:0/2888780106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378101a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 -- 172.21.15.174:0/2888780106 >> 172.21.15.174:0/2888780106 conn(0x7ff3780fcdf0 msgr2=0x7ff3780ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 -- 172.21.15.174:0/2888780106 shutdown_connections 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 -- 172.21.15.174:0/2888780106 wait complete. 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 Processor -- start 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 -- start start 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.644+0000 7ff37ea7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff378107750 con 0x7ff378101640 2024-09-19T07:29:43.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff37c817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff37c817640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.174:54594/0 (socket says 172.21.15.174:54594) 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff37c817640 1 -- 172.21.15.174:0/1406702907 learned_addr learned my addr 172.21.15.174:0/1406702907 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff37c817640 1 -- 172.21.15.174:0/1406702907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff378197390 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff37c817640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378196e50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff374009230 tx=0x7ff3740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff36dffb640 1 -- 172.21.15.174:0/1406702907 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff37403f070 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff378197620 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff36dffb640 1 -- 172.21.15.174:0/1406702907 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff374003ad0 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.645+0000 7ff36dffb640 1 -- 172.21.15.174:0/1406702907 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff374035030 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.646+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37819a310 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.646+0000 7ff36dffb640 1 -- 172.21.15.174:0/1406702907 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff374042070 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.647+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff378107ed0 con 0x7ff378101640 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.647+0000 7ff36dffb640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff360039300 0x7ff36003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:43.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.647+0000 7ff36dffb640 1 -- 172.21.15.174:0/1406702907 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff37406e0c0 con 0x7ff378101640 2024-09-19T07:29:43.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.647+0000 7ff36ffff640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff360039300 0x7ff36003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:43.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.647+0000 7ff36ffff640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff360039300 0x7ff36003b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff3680098c0 tx=0x7ff368006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:43.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.651+0000 7ff36dffb640 1 -- 172.21.15.174:0/1406702907 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff374039350 con 0x7ff378101640 2024-09-19T07:29:43.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.844+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}) -- 0x7ff37802d8f0 con 0x7ff360039300 2024-09-19T07:29:43.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.849+0000 7ff36dffb640 1 -- 172.21.15.174:0/1406702907 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff37802d8f0 con 0x7ff360039300 2024-09-19T07:29:43.849 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff360039300 msgr2=0x7ff36003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff360039300 0x7ff36003b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff3680098c0 tx=0x7ff368006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 msgr2=0x7ff378196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378196e50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff374009230 tx=0x7ff3740048f0 comp rx=0 tx=0).stop 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 shutdown_connections 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff360039300 0x7ff36003b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 --2- 172.21.15.174:0/1406702907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff378101640 0x7ff378196e50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 >> 172.21.15.174:0/1406702907 conn(0x7ff3780fcdf0 msgr2=0x7ff3780fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:43.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.851+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 shutdown_connections 2024-09-19T07:29:43.853 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:43.852+0000 7ff37ea7b640 1 -- 172.21.15.174:0/1406702907 wait complete. 2024-09-19T07:29:44.448 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T07:29:44.452 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T07:29:44.454 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:29:44.455 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:29:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:45 smithi103 ceph-mon[31201]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:45 smithi103 ceph-mon[31201]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:29:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:45 smithi103 ceph-mon[31201]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:45 smithi103 ceph-mon[31201]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c;count:3 2024-09-19T07:29:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:45 smithi103 ceph-mon[31201]: Deploying daemon mon.c on smithi174 2024-09-19T07:29:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:45 smithi103 ceph-mon[31201]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:45.915 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:29:46.986 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.929+0000 7ff496f6b640 1 -- 172.21.15.174:0/3373264597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 msgr2=0x7ff490106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.929+0000 7ff496f6b640 1 --2- 172.21.15.174:0/3373264597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff490106930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff484009880 tx=0x7ff48402a5d0 comp rx=0 tx=0).stop 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.929+0000 7ff496f6b640 1 -- 172.21.15.174:0/3373264597 shutdown_connections 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.929+0000 7ff496f6b640 1 --2- 172.21.15.174:0/3373264597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff490106930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.929+0000 7ff496f6b640 1 -- 172.21.15.174:0/3373264597 >> 172.21.15.174:0/3373264597 conn(0x7ff4900fb920 msgr2=0x7ff4900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.929+0000 7ff496f6b640 1 -- 172.21.15.174:0/3373264597 shutdown_connections 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.929+0000 7ff496f6b640 1 -- 172.21.15.174:0/3373264597 wait complete. 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.930+0000 7ff496f6b640 1 Processor -- start 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.930+0000 7ff496f6b640 1 -- start start 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.930+0000 7ff496f6b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff4900fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:46.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.930+0000 7ff496f6b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff49010aaf0 con 0x7ff490106550 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.930+0000 7ff494d07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff4900fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.930+0000 7ff494d07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff4900fbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.174:54608/0 (socket says 172.21.15.174:54608) 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.930+0000 7ff494d07640 1 -- 172.21.15.174:0/3003187055 learned_addr learned my addr 172.21.15.174:0/3003187055 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.931+0000 7ff494d07640 1 -- 172.21.15.174:0/3003187055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900fdcc0 con 0x7ff490106550 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.931+0000 7ff494d07640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff4900fbdd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff484009230 tx=0x7ff484003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.931+0000 7ff48dffb640 1 -- 172.21.15.174:0/3003187055 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff48403f070 con 0x7ff490106550 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.931+0000 7ff48dffb640 1 -- 172.21.15.174:0/3003187055 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff48403bc80 con 0x7ff490106550 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.931+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4900fc370 con 0x7ff490106550 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.931+0000 7ff48dffb640 1 -- 172.21.15.174:0/3003187055 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff484035030 con 0x7ff490106550 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.931+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4900fc830 con 0x7ff490106550 2024-09-19T07:29:46.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.932+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff490101d40 con 0x7ff490106550 2024-09-19T07:29:46.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.932+0000 7ff48dffb640 1 -- 172.21.15.174:0/3003187055 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff484042070 con 0x7ff490106550 2024-09-19T07:29:46.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.933+0000 7ff48dffb640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4800392a0 0x7ff48003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:46.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.933+0000 7ff48dffb640 1 -- 172.21.15.174:0/3003187055 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff48406d090 con 0x7ff490106550 2024-09-19T07:29:46.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.933+0000 7ff48ffff640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4800392a0 0x7ff48003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:46.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.933+0000 7ff48ffff640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4800392a0 0x7ff48003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0098c0 tx=0x7ff47c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:46.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:46.938+0000 7ff48dffb640 1 -- 172.21.15.174:0/3003187055 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff484031e20 con 0x7ff490106550 2024-09-19T07:29:47.192 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.191+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff4900fd300 con 0x7ff490106550 2024-09-19T07:29:47.193 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.192+0000 7ff48dffb640 1 -- 172.21.15.174:0/3003187055 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7ff4840372d0 con 0x7ff490106550 2024-09-19T07:29:47.193 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:29:47.193 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","modified":"2024-09-19T07:28:26.970477Z","created":"2024-09-19T07:28:26.970477Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T07:29:47.193 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-09-19T07:29:47.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4800392a0 msgr2=0x7ff48003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4800392a0 0x7ff48003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0098c0 tx=0x7ff47c006db0 comp rx=0 tx=0).stop 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 msgr2=0x7ff4900fbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff4900fbdd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff484009230 tx=0x7ff484003d40 comp rx=0 tx=0).stop 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 shutdown_connections 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4800392a0 0x7ff48003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 --2- 172.21.15.174:0/3003187055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff490106550 0x7ff4900fbdd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 >> 172.21.15.174:0/3003187055 conn(0x7ff4900fb920 msgr2=0x7ff4900ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 shutdown_connections 2024-09-19T07:29:47.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:47.194+0000 7ff496f6b640 1 -- 172.21.15.174:0/3003187055 wait complete. 2024-09-19T07:29:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:47 smithi103 ceph-mon[31201]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:47 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/3003187055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:47.565 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:47 smithi174 systemd[1]: Starting Ceph mon.c for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:29:48.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 podman[35351]: 2024-09-19 07:29:48.066315392 +0000 UTC m=+0.102690414 container create aa0db486266247b03bbf226190a50d7a6569ddafbf3ca8c3947e307bb76f4b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:29:48.076 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 podman[35351]: 2024-09-19 07:29:47.979281771 +0000 UTC m=+0.015656746 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:29:48.327 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 podman[35351]: 2024-09-19 07:29:48.302054631 +0000 UTC m=+0.338429607 container init aa0db486266247b03bbf226190a50d7a6569ddafbf3ca8c3947e307bb76f4b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-19T07:29:48.327 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 podman[35351]: 2024-09-19 07:29:48.305148891 +0000 UTC m=+0.341523872 container start aa0db486266247b03bbf226190a50d7a6569ddafbf3ca8c3947e307bb76f4b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T07:29:48.327 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T07:29:48.327 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T07:29:48.327 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: pidfile_write: ignore empty --pid-file 2024-09-19T07:29:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: load: jerasure load: lrc 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: RocksDB version: 7.9.2 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Git sha 0 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: DB SUMMARY 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: DB Session ID: 1IFRVZBWDWBDBIOQGCO7 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: CURRENT file: CURRENT 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T07:29:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.error_if_exists: 0 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.create_if_missing: 0 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.paranoid_checks: 1 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.env: 0x55716c0f5940 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.info_log: 0x55716e7276e0 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.statistics: (nil) 2024-09-19T07:29:48.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.use_fsync: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_log_file_size: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.allow_fallocate: 1 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.use_direct_reads: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T07:29:48.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.db_log_dir: 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.wal_dir: 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.write_buffer_manager: 0x55716e736dc0 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T07:29:48.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.unordered_write: 0 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T07:29:48.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.row_cache: None 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.wal_filter: None 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.two_write_queues: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.wal_compression: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.atomic_flush: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.log_readahead_size: 0 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T07:29:48.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_background_jobs: 2 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_background_compactions: -1 2024-09-19T07:29:48.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_subcompactions: 1 2024-09-19T07:29:48.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_open_files: -1 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T07:29:48.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_background_flushes: -1 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Compression algorithms supported: 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kZSTD supported: 0 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kXpressCompression supported: 0 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kBZip2Compression supported: 0 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kLZ4Compression supported: 1 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kZlibCompression supported: 1 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: kSnappyCompression supported: 1 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-19T07:29:48.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.merge_operator: 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_filter: None 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55716e727320) 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T07:29:48.676 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_type: 0 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_index_type: 0 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_shortening: 1 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: checksum: 4 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: no_block_cache: 0 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache: 0x55716e71b1f0 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_options: 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: capacity : 536870912 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_shard_bits : 4 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: strict_capacity_limit : 0 2024-09-19T07:29:48.677 INFO:journalctl@ceph.mon.c.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_compressed: (nil) 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: persistent_cache: (nil) 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size: 4096 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size_deviation: 10 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_restart_interval: 16 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_block_restart_interval: 1 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: metadata_block_size: 4096 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: partition_filters: 0 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: use_delta_encoding: 1 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: filter_policy: bloomfilter 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: whole_key_filtering: 1 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: verify_compression: 0 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-09-19T07:29:48.678 INFO:journalctl@ceph.mon.c.smithi174.stdout: format_version: 5 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_index_compression: 1 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_align: 0 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout: max_auto_readahead_size: 262144 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout: prepopulate_block_cache: 0 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout: initial_auto_readahead_size: 8192 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression: NoCompression 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T07:29:48.679 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.num_levels: 7 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:29:48.680 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T07:29:48.681 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T07:29:48.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T07:29:48.683 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.inplace_update_support: 0 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.bloom_locality: 0 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.max_successive_merges: 0 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T07:29:48.684 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.ttl: 2592000 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.enable_blob_files: false 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.min_blob_size: 0 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T07:29:48.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c9ba7a41-c811-4db6-9db2-2ba04d03f9d4 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730988332741, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730988333183, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726730988, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c9ba7a41-c811-4db6-9db2-2ba04d03f9d4", "db_session_id": "1IFRVZBWDWBDBIOQGCO7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730988333286, "job": 1, "event": "recovery_finished"} 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T07:29:48.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55716e748e00 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: DB pointer 0x55716e844000 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** DB Stats ** 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:29:48.687 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache BinnedLRUCache@0x55716e71b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-19T07:29:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: starting mon.c rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(???) e0 preinit fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).mds e1 new map 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout: e1 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout: btime 2024-09-19T07:28:30:723424+0000 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:29:48.690 INFO:journalctl@ceph.mon.c.smithi174.stdout: legacy client fscid: -1 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout: No filesystems configured 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:29:48.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/109786756' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/109786756' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T07:29:48.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c;count:3 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: Deploying daemon mon.c on smithi174 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:48.693 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/3003187055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:48.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 ceph-mon[35370]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T07:29:48.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 bash[35351]: aa0db486266247b03bbf226190a50d7a6569ddafbf3ca8c3947e307bb76f4b69 2024-09-19T07:29:48.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:48 smithi174 systemd[1]: Started Ceph mon.c for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:29:48.878 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:29:48.879 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:29:49.129 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:29:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:29:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: mon.a calling monitor election 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: mon.c calling monitor election 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: monmap epoch 2 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: last_changed 2024-09-19T07:29:48.357809+0000 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: min_mon_release 19 (squid) 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: election_strategy: 1 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: fsmap 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: mgrmap e13: x(active, since 37s) 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: overall HEALTH_OK 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:53.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:29:53.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:53.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:29:53.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: mon.a calling monitor election 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: mon.c calling monitor election 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: monmap epoch 2 2024-09-19T07:29:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: last_changed 2024-09-19T07:29:48.357809+0000 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: min_mon_release 19 (squid) 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: election_strategy: 1 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: fsmap 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: mgrmap e13: x(active, since 37s) 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: overall HEALTH_OK 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:53.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:53.918 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:29:53.918 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:53.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.938+0000 7fabe246c640 1 -- 172.21.15.174:0/33943056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabc4005740 msgr2=0x7fabc4007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:53.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.938+0000 7fabe246c640 1 --2- 172.21.15.174:0/33943056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabc4005740 0x7fabc4007bd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabdc1016a0 tx=0x7fabcc02c490 comp rx=0 tx=0).stop 2024-09-19T07:29:53.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.939+0000 7fabe246c640 1 -- 172.21.15.174:0/33943056 shutdown_connections 2024-09-19T07:29:53.939 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.939+0000 7fabe246c640 1 --2- 172.21.15.174:0/33943056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabc4005740 0x7fabc4007bd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:53.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.939+0000 7fabe246c640 1 -- 172.21.15.174:0/33943056 >> 172.21.15.174:0/33943056 conn(0x7fabdc0fc220 msgr2=0x7fabdc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:53.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.939+0000 7fabe246c640 1 -- 172.21.15.174:0/33943056 shutdown_connections 2024-09-19T07:29:53.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.939+0000 7fabe246c640 1 -- 172.21.15.174:0/33943056 wait complete. 2024-09-19T07:29:53.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.939+0000 7fabe246c640 1 Processor -- start 2024-09-19T07:29:53.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabe246c640 1 -- start start 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabe246c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 0x7fabdc195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabe246c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabdc195560 0x7fabdc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabe246c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc1075a0 con 0x7fabdc195560 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabe246c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc107420 con 0x7fabc4005740 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabdbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 0x7fabdc195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabdbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 0x7fabdc195020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43516/0 (socket says 172.21.15.174:43516) 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabdbfff640 1 -- 172.21.15.174:0/3684448938 learned_addr learned my addr 172.21.15.174:0/3684448938 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:29:53.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.940+0000 7fabdb7fe640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabdc195560 0x7fabdc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdbfff640 1 -- 172.21.15.174:0/3684448938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 msgr2=0x7fabdc195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdbfff640 1 -- 172.21.15.174:0/3684448938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 msgr2=0x7fabdc195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdbfff640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 0x7fabdc195020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdbfff640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 0x7fabdc195020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdb7fe640 1 -- 172.21.15.174:0/3684448938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 msgr2=0x7fabdc195020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdb7fe640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 0x7fabdc195020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdb7fe640 1 -- 172.21.15.174:0/3684448938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc195b80 con 0x7fabdc195560 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabdb7fe640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabdc195560 0x7fabdc1998f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabd000b3d0 tx=0x7fabd000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabd97fa640 1 -- 172.21.15.174:0/3684448938 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fabd0011630 con 0x7fabdc195560 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabd97fa640 1 -- 172.21.15.174:0/3684448938 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd0011cb0 con 0x7fabdc195560 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc199e30 con 0x7fabdc195560 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabd97fa640 1 -- 172.21.15.174:0/3684448938 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fabd0010df0 con 0x7fabdc195560 2024-09-19T07:29:53.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.941+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc19a3f0 con 0x7fabdc195560 2024-09-19T07:29:53.944 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.942+0000 7fabd97fa640 1 -- 172.21.15.174:0/3684448938 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fabd0018020 con 0x7fabdc195560 2024-09-19T07:29:53.944 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.943+0000 7fabd97fa640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fabc8039040 0x7fabc803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:53.944 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.943+0000 7fabd97fa640 1 -- 172.21.15.174:0/3684448938 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fabd004bf40 con 0x7fabdc195560 2024-09-19T07:29:53.944 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.943+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc19a700 con 0x7fabdc195560 2024-09-19T07:29:53.945 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.943+0000 7fabdbfff640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fabc8039040 0x7fabc803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:53.945 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.943+0000 7fabdbfff640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fabc8039040 0x7fabc803b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fabcc012c10 tx=0x7fabcc0323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:53.948 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:53.948+0000 7fabd97fa640 1 -- 172.21.15.174:0/3684448938 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fabd001f020 con 0x7fabdc195560 2024-09-19T07:29:54.181 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.181+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fabdc18ed40 con 0x7fabdc195560 2024-09-19T07:29:54.182 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.182+0000 7fabd97fa640 1 -- 172.21.15.174:0/3684448938 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fabd0018310 con 0x7fabdc195560 2024-09-19T07:29:54.182 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:29:54.183 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","modified":"2024-09-19T07:29:48.357809Z","created":"2024-09-19T07:28:26.970477Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T07:29:54.183 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-19T07:29:54.184 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fabc8039040 msgr2=0x7fabc803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:54.184 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fabc8039040 0x7fabc803b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fabcc012c10 tx=0x7fabcc0323a0 comp rx=0 tx=0).stop 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabdc195560 msgr2=0x7fabdc1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabdc195560 0x7fabdc1998f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabd000b3d0 tx=0x7fabd000b890 comp rx=0 tx=0).stop 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 shutdown_connections 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fabc8039040 0x7fabc803b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabdc195560 0x7fabdc1998f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 --2- 172.21.15.174:0/3684448938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc4005740 0x7fabdc195020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 >> 172.21.15.174:0/3684448938 conn(0x7fabdc0fc220 msgr2=0x7fabdc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 shutdown_connections 2024-09-19T07:29:54.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:54.184+0000 7fabe246c640 1 -- 172.21.15.174:0/3684448938 wait complete. 2024-09-19T07:29:54.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:54 smithi174 ceph-mon[35370]: Deploying daemon mon.b on smithi134 2024-09-19T07:29:54.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:54 smithi174 ceph-mon[35370]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:54.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:54 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/3684448938' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:54.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:54 smithi103 ceph-mon[31201]: Deploying daemon mon.b on smithi134 2024-09-19T07:29:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:54 smithi103 ceph-mon[31201]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:54 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/3684448938' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:55.827 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:29:55.828 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:29:56.078 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 -- 172.21.15.174:0/199935083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c107660 msgr2=0x7fe55c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 --2- 172.21.15.174:0/199935083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c107660 0x7fe55c107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009880 tx=0x7fe54c02a5d0 comp rx=0 tx=0).stop 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 -- 172.21.15.174:0/199935083 shutdown_connections 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 --2- 172.21.15.174:0/199935083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c107660 0x7fe55c107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 -- 172.21.15.174:0/199935083 >> 172.21.15.174:0/199935083 conn(0x7fe55c0fd320 msgr2=0x7fe55c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 -- 172.21.15.174:0/199935083 shutdown_connections 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 -- 172.21.15.174:0/199935083 wait complete. 2024-09-19T07:29:56.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.523+0000 7fe561d3c640 1 Processor -- start 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe561d3c640 1 -- start start 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe561d3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 0x7fe55c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe561d3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c197900 0x7fe55c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe561d3c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c10bd20 con 0x7fe55c197900 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe561d3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c10bba0 con 0x7fe55c107660 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 0x7fe55c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 0x7fe55c1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43532/0 (socket says 172.21.15.174:43532) 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55b7fe640 1 -- 172.21.15.174:0/2648665638 learned_addr learned my addr 172.21.15.174:0/2648665638 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55affd640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c197900 0x7fe55c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55b7fe640 1 -- 172.21.15.174:0/2648665638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 msgr2=0x7fe55c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55b7fe640 1 -- 172.21.15.174:0/2648665638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 msgr2=0x7fe55c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:29:56.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55b7fe640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 0x7fe55c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:29:56.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55b7fe640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 0x7fe55c1973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:29:56.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55affd640 1 -- 172.21.15.174:0/2648665638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 msgr2=0x7fe55c1973c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:29:56.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55affd640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 0x7fe55c1973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:56.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.524+0000 7fe55affd640 1 -- 172.21.15.174:0/2648665638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c197f20 con 0x7fe55c197900 2024-09-19T07:29:56.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.525+0000 7fe55affd640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c197900 0x7fe55c19bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe55000b3d0 tx=0x7fe55000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:56.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.525+0000 7fe558ff9640 1 -- 172.21.15.174:0/2648665638 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe550011630 con 0x7fe55c197900 2024-09-19T07:29:56.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.525+0000 7fe558ff9640 1 -- 172.21.15.174:0/2648665638 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe550011cb0 con 0x7fe55c197900 2024-09-19T07:29:56.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.525+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c19c1d0 con 0x7fe55c197900 2024-09-19T07:29:56.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.525+0000 7fe558ff9640 1 -- 172.21.15.174:0/2648665638 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe550010df0 con 0x7fe55c197900 2024-09-19T07:29:56.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.525+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c19c710 con 0x7fe55c197900 2024-09-19T07:29:56.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.526+0000 7fe558ff9640 1 -- 172.21.15.174:0/2648665638 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fe550018020 con 0x7fe55c197900 2024-09-19T07:29:56.529 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.526+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c102da0 con 0x7fe55c197900 2024-09-19T07:29:56.529 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.527+0000 7fe558ff9640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe544039090 0x7fe54403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:29:56.529 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.527+0000 7fe558ff9640 1 -- 172.21.15.174:0/2648665638 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe55004be80 con 0x7fe55c197900 2024-09-19T07:29:56.529 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.527+0000 7fe55b7fe640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe544039090 0x7fe54403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:29:56.529 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.527+0000 7fe55b7fe640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe544039090 0x7fe54403b550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0096c0 tx=0x7fe54c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:29:56.532 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.532+0000 7fe558ff9640 1 -- 172.21.15.174:0/2648665638 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe55001f020 con 0x7fe55c197900 2024-09-19T07:29:56.763 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.762+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe55c19ca20 con 0x7fe55c197900 2024-09-19T07:29:56.763 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.763+0000 7fe558ff9640 1 -- 172.21.15.174:0/2648665638 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fe550017500 con 0x7fe55c197900 2024-09-19T07:29:56.763 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:29:56.764 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","modified":"2024-09-19T07:29:48.357809Z","created":"2024-09-19T07:28:26.970477Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T07:29:56.764 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe544039090 msgr2=0x7fe54403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe544039090 0x7fe54403b550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0096c0 tx=0x7fe54c031400 comp rx=0 tx=0).stop 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c197900 msgr2=0x7fe55c19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c197900 0x7fe55c19bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe55000b3d0 tx=0x7fe55000b890 comp rx=0 tx=0).stop 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 shutdown_connections 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe544039090 0x7fe54403b550 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe55c197900 0x7fe55c19bc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 --2- 172.21.15.174:0/2648665638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55c107660 0x7fe55c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 >> 172.21.15.174:0/2648665638 conn(0x7fe55c0fd320 msgr2=0x7fe55c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 shutdown_connections 2024-09-19T07:29:56.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:29:56.765+0000 7fe561d3c640 1 -- 172.21.15.174:0/2648665638 wait complete. 2024-09-19T07:29:57.237 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 podman[35393]: 2024-09-19 07:29:57.215244914 +0000 UTC m=+0.346606410 container start 8cb15dd05f85aa199f948b1537c2e8a130ac6180e569f7bb6173112da351e85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-b, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:29:57.238 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T07:29:57.238 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T07:29:57.238 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pidfile_write: ignore empty --pid-file 2024-09-19T07:29:57.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:57 smithi174 ceph-mon[35370]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:29:57 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/2648665638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:57.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:57 smithi103 ceph-mon[31201]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:29:57 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/2648665638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:57.545 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: load: jerasure load: lrc 2024-09-19T07:29:57.545 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: RocksDB version: 7.9.2 2024-09-19T07:29:57.545 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Git sha 0 2024-09-19T07:29:57.545 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: DB SUMMARY 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: DB Session ID: RCK0YMJE071OR33IKBD1 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: CURRENT file: CURRENT 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.error_if_exists: 0 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.create_if_missing: 0 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.paranoid_checks: 1 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T07:29:57.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.env: 0x558bfefdf940 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.info_log: 0x558c0064b6e0 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.statistics: (nil) 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.use_fsync: 0 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_log_file_size: 0 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T07:29:57.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.allow_fallocate: 1 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.use_direct_reads: 0 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.db_log_dir: 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.wal_dir: 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T07:29:57.548 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.write_buffer_manager: 0x558c0065adc0 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.unordered_write: 0 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T07:29:57.549 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.row_cache: None 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.wal_filter: None 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.two_write_queues: 0 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.wal_compression: 0 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.atomic_flush: 0 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T07:29:57.550 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.log_readahead_size: 0 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_background_jobs: 2 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_background_compactions: -1 2024-09-19T07:29:57.551 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_subcompactions: 1 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_open_files: -1 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T07:29:57.553 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_background_flushes: -1 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Compression algorithms supported: 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kZSTD supported: 0 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kXpressCompression supported: 0 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kBZip2Compression supported: 0 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kLZ4Compression supported: 1 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kZlibCompression supported: 1 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: kSnappyCompression supported: 1 2024-09-19T07:29:57.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.merge_operator: 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_filter: None 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558c0064b320) 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks: 1 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T07:29:57.555 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_top_level_index_and_filter: 1 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_type: 0 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_index_type: 0 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_shortening: 1 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: checksum: 4 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: no_block_cache: 0 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache: 0x558c0063f1f0 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_name: BinnedLRUCache 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_options: 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: capacity : 536870912 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_shard_bits : 4 2024-09-19T07:29:57.556 INFO:journalctl@ceph.mon.b.smithi134.stdout: strict_capacity_limit : 0 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: high_pri_pool_ratio: 0.000 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_compressed: (nil) 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: persistent_cache: (nil) 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size: 4096 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size_deviation: 10 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_restart_interval: 16 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_block_restart_interval: 1 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: metadata_block_size: 4096 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: partition_filters: 0 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: use_delta_encoding: 1 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: filter_policy: bloomfilter 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: whole_key_filtering: 1 2024-09-19T07:29:57.557 INFO:journalctl@ceph.mon.b.smithi134.stdout: verify_compression: 0 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: read_amp_bytes_per_bit: 0 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: format_version: 5 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_index_compression: 1 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_align: 0 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: max_auto_readahead_size: 262144 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: prepopulate_block_cache: 0 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: initial_auto_readahead_size: 8192 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression: NoCompression 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T07:29:57.558 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.num_levels: 7 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:29:57.559 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T07:29:57.560 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T07:29:57.561 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T07:29:57.562 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.inplace_update_support: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.bloom_locality: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.max_successive_merges: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.ttl: 2592000 2024-09-19T07:29:57.563 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.enable_blob_files: false 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.min_blob_size: 0 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-19T07:29:57.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b5fbb0f0-9995-4da9-a55b-72a06de2e26e 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730997242670, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730997243086, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726730997, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b5fbb0f0-9995-4da9-a55b-72a06de2e26e", "db_session_id": "RCK0YMJE071OR33IKBD1", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726730997243179, "job": 1, "event": "recovery_finished"} 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558c0066ce00 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: DB pointer 0x558c00768000 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** DB Stats ** 2024-09-19T07:29:57.565 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:57.566 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T07:29:57.567 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache BinnedLRUCache@0x558c0063f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: starting mon.b rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(???) e0 preinit fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).mds e1 new map 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-19T07:29:57.568 INFO:journalctl@ceph.mon.b.smithi134.stdout: e1 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout: btime 2024-09-19T07:28:30:723424+0000 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout: legacy client fscid: -1 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout: No filesystems configured 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T07:29:57.569 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/109786756' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/109786756' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:57.570 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c;count:3 2024-09-19T07:29:57.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: Deploying daemon mon.c on smithi174 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/3003187055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.a calling monitor election 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.c calling monitor election 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:57.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: monmap epoch 2 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: last_changed 2024-09-19T07:29:48.357809+0000 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: min_mon_release 19 (squid) 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: election_strategy: 1 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: fsmap 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mgrmap e13: x(active, since 37s) 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: overall HEALTH_OK 2024-09-19T07:29:57.573 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: Deploying daemon mon.b on smithi134 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/3684448938' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/2648665638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 ceph-mon[35414]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 bash[35393]: 8cb15dd05f85aa199f948b1537c2e8a130ac6180e569f7bb6173112da351e85e 2024-09-19T07:29:57.574 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:29:57 smithi134 systemd[1]: Started Ceph mon.b for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:29:58.225 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T07:29:58.226 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T07:29:58.490 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:30:02.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.521+0000 7f141f256640 1 -- 172.21.15.174:0/4206910433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408003530 msgr2=0x7f14080059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.521+0000 7f141f256640 1 --2- 172.21.15.174:0/4206910433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408003530 0x7f14080059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f14181016c0 tx=0x7f140c02c3f0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.522+0000 7f141f256640 1 -- 172.21.15.174:0/4206910433 shutdown_connections 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.522+0000 7f141f256640 1 --2- 172.21.15.174:0/4206910433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408003530 0x7f14080059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.522+0000 7f141f256640 1 -- 172.21.15.174:0/4206910433 >> 172.21.15.174:0/4206910433 conn(0x7f14180fc280 msgr2=0x7f14180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.522+0000 7f141f256640 1 -- 172.21.15.174:0/4206910433 shutdown_connections 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.522+0000 7f141f256640 1 -- 172.21.15.174:0/4206910433 wait complete. 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.522+0000 7f141f256640 1 Processor -- start 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.522+0000 7f141f256640 1 -- start start 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141f256640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 0x7f1418199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141f256640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 0x7f141819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141f256640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141819e390 0x7f141819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141f256640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1418108680 con 0x7f141819e390 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141f256640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1418108500 con 0x7f1408003530 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141f256640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1418108800 con 0x7f1418199ac0 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f1417fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 0x7f141819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f1417fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 0x7f141819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:52314/0 (socket says 172.21.15.174:52314) 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f1417fff640 1 -- 172.21.15.174:0/2421865850 learned_addr learned my addr 172.21.15.174:0/2421865850 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141cff2640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 0x7f1418199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f1417fff640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 msgr2=0x7f141819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f1417fff640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 msgr2=0x7f141819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f1417fff640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 0x7f141819de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141d7f3640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141819e390 0x7f141819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f1417fff640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 0x7f141819de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141cff2640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 msgr2=0x7f1418199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141cff2640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 msgr2=0x7f1418199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141cff2640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 0x7f1418199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141cff2640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 0x7f1418199580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141d7f3640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 msgr2=0x7f141819de50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141d7f3640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 0x7f141819de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141d7f3640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 msgr2=0x7f1418199580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141d7f3640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 0x7f1418199580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141d7f3640 1 -- 172.21.15.174:0/2421865850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14181a6100 con 0x7f141819e390 2024-09-19T07:30:02.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.523+0000 7f141d7f3640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141819e390 0x7f141819e840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f141000bd50 tx=0x7f141000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:02.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.524+0000 7f1415ffb640 1 -- 172.21.15.174:0/2421865850 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410019070 con 0x7f141819e390 2024-09-19T07:30:02.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.524+0000 7f1415ffb640 1 -- 172.21.15.174:0/2421865850 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1410014920 con 0x7f141819e390 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.524+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14181a6330 con 0x7f141819e390 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.524+0000 7f1415ffb640 1 -- 172.21.15.174:0/2421865850 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410012990 con 0x7f141819e390 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.524+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14181a67f0 con 0x7f141819e390 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.525+0000 7f1415ffb640 1 -- 172.21.15.174:0/2421865850 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f141001c070 con 0x7f141819e390 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.525+0000 7f1415ffb640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f13fc039730 0x7f13fc03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.525+0000 7f1415ffb640 1 -- 172.21.15.174:0/2421865850 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f141004c080 con 0x7f141819e390 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.525+0000 7f141cff2640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f13fc039730 0x7f13fc03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.525+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1418109770 con 0x7f141819e390 2024-09-19T07:30:02.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.526+0000 7f141cff2640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f13fc039730 0x7f13fc03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:02.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.530+0000 7f1415ffb640 1 -- 172.21.15.174:0/2421865850 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f141000e0a0 con 0x7f141819e390 2024-09-19T07:30:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:30:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: mon.c calling monitor election 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: mon.a calling monitor election 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:30:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: monmap epoch 3 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: last_changed 2024-09-19T07:29:57.267497+0000 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: min_mon_release 19 (squid) 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: election_strategy: 1 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: fsmap 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: mgrmap e13: x(active, since 46s) 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: overall HEALTH_OK 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:02 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:30:02.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: mon.c calling monitor election 2024-09-19T07:30:02.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: mon.a calling monitor election 2024-09-19T07:30:02.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: monmap epoch 3 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:02.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: last_changed 2024-09-19T07:29:57.267497+0000 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: min_mon_release 19 (squid) 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: election_strategy: 1 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: fsmap 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: mgrmap e13: x(active, since 46s) 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: overall HEALTH_OK 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:02 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:02.872 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.872+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f141802d7c0 con 0x7f141819e390 2024-09-19T07:30:02.873 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.873+0000 7f1415ffb640 1 -- 172.21.15.174:0/2421865850 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f14100139a0 con 0x7f141819e390 2024-09-19T07:30:02.873 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:30:02.873 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","modified":"2024-09-19T07:29:57.267497Z","created":"2024-09-19T07:28:26.970477Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T07:30:02.873 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-09-19T07:30:02.875 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f13fc039730 msgr2=0x7f13fc03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.875 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f13fc039730 0x7f13fc03bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c005dc0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.875 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141819e390 msgr2=0x7f141819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141819e390 0x7f141819e840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f141000bd50 tx=0x7f141000be40 comp rx=0 tx=0).stop 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 shutdown_connections 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f13fc039730 0x7f13fc03bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141819e390 0x7f141819e840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418199ac0 0x7f141819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 --2- 172.21.15.174:0/2421865850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1408003530 0x7f1418199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 >> 172.21.15.174:0/2421865850 conn(0x7f14180fc280 msgr2=0x7f141806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 shutdown_connections 2024-09-19T07:30:02.876 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:02.875+0000 7f141f256640 1 -- 172.21.15.174:0/2421865850 wait complete. 2024-09-19T07:30:03.379 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:03.379 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:03 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/2421865850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:30:03.379 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:03.623 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T07:30:03.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T07:30:03.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:03.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:03 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/2421865850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:30:03.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.358+0000 7f108c399640 1 -- 172.21.15.103:0/1339095944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 msgr2=0x7f1084107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.358+0000 7f108c399640 1 --2- 172.21.15.103:0/1339095944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084107a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1080009880 tx=0x7f108002a5d0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.358+0000 7f108c399640 1 -- 172.21.15.103:0/1339095944 shutdown_connections 2024-09-19T07:30:04.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.358+0000 7f108c399640 1 --2- 172.21.15.103:0/1339095944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084107a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.358+0000 7f108c399640 1 -- 172.21.15.103:0/1339095944 >> 172.21.15.103:0/1339095944 conn(0x7f10840fd2b0 msgr2=0x7f10840ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.358+0000 7f108c399640 1 -- 172.21.15.103:0/1339095944 shutdown_connections 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.358+0000 7f108c399640 1 -- 172.21.15.103:0/1339095944 wait complete. 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 Processor -- start 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 -- start start 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084197880 0x7f108419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f108419c150 0x7f108419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f108410bc20 con 0x7f1084107660 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f108410baa0 con 0x7f1084197880 2024-09-19T07:30:04.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108c399640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f108410bda0 con 0x7f108419c150 2024-09-19T07:30:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108a135640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108a135640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36126/0 (socket says 172.21.15.103:36126) 2024-09-19T07:30:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108a135640 1 -- 172.21.15.103:0/2397491864 learned_addr learned my addr 172.21.15.103:0/2397491864 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:30:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f108a936640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f108419c150 0x7f108419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.359+0000 7f1089934640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084197880 0x7f108419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108a135640 1 -- 172.21.15.103:0/2397491864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f108419c150 msgr2=0x7f108419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:04.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108a135640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f108419c150 0x7f108419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108a135640 1 -- 172.21.15.103:0/2397491864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084197880 msgr2=0x7f108419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108a135640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084197880 0x7f108419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108a135640 1 -- 172.21.15.103:0/2397491864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10841a3ec0 con 0x7f1084107660 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108a135640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084197340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1080031550 tx=0x7f10800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f107b7fe640 1 -- 172.21.15.103:0/2397491864 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108003f030 con 0x7f1084107660 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f107b7fe640 1 -- 172.21.15.103:0/2397491864 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1080003ce0 con 0x7f1084107660 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10841a4090 con 0x7f1084107660 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f107b7fe640 1 -- 172.21.15.103:0/2397491864 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080035070 con 0x7f1084107660 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.360+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10841a4620 con 0x7f1084107660 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.361+0000 7f107b7fe640 1 -- 172.21.15.103:0/2397491864 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f1080042030 con 0x7f1084107660 2024-09-19T07:30:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.361+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1084102d10 con 0x7f1084107660 2024-09-19T07:30:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.361+0000 7f107b7fe640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f1070039730 0x7f107003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.361+0000 7f107b7fe640 1 -- 172.21.15.103:0/2397491864 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f108006cca0 con 0x7f1084107660 2024-09-19T07:30:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.361+0000 7f1089934640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f1070039730 0x7f107003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.362+0000 7f1089934640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f1070039730 0x7f107003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1074009a30 tx=0x7f1074009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.366+0000 7f107b7fe640 1 -- 172.21.15.103:0/2397491864 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f108003bce0 con 0x7f1084107660 2024-09-19T07:30:04.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.565+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f10841a4910 con 0x7f1084107660 2024-09-19T07:30:04.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.565+0000 7f107b7fe640 1 -- 172.21.15.103:0/2397491864 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f1080044410 con 0x7f1084107660 2024-09-19T07:30:04.567 INFO:teuthology.orchestra.run.smithi103.stdout:# minimal ceph.conf for 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:04.567 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-09-19T07:30:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: mon_host = [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f1070039730 msgr2=0x7f107003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f1070039730 0x7f107003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1074009a30 tx=0x7f1074009190 comp rx=0 tx=0).stop 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 msgr2=0x7f1084197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084197340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1080031550 tx=0x7f10800042f0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 shutdown_connections 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f1070039730 0x7f107003bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f108419c150 0x7f108419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1084197880 0x7f108419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 --2- 172.21.15.103:0/2397491864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1084107660 0x7f1084197340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 >> 172.21.15.103:0/2397491864 conn(0x7f10840fd2b0 msgr2=0x7f10840fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 shutdown_connections 2024-09-19T07:30:04.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:04.568+0000 7f108c399640 1 -- 172.21.15.103:0/2397491864 wait complete. 2024-09-19T07:30:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:30:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: mon.c calling monitor election 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: mon.a calling monitor election 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: monmap epoch 3 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: last_changed 2024-09-19T07:29:57.267497+0000 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: min_mon_release 19 (squid) 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: election_strategy: 1 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:30:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: fsmap 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: mgrmap e13: x(active, since 46s) 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: overall HEALTH_OK 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/2421865850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T07:30:04.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:05.169 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T07:30:05.169 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:30:05.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T07:30:05.237 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:30:05.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:30:05.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: mon.b calling monitor election 2024-09-19T07:30:05.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: mon.b calling monitor election 2024-09-19T07:30:05.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: mon.a calling monitor election 2024-09-19T07:30:05.299 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: mon.c calling monitor election 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: monmap epoch 3 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: last_changed 2024-09-19T07:29:57.267497+0000 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: min_mon_release 19 (squid) 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: election_strategy: 1 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: fsmap 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:30:05.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: mgrmap e13: x(active, since 48s) 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: overall HEALTH_OK 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2397491864' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.301 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:05.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:05.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:05.303 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:30:05.303 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: mon.b calling monitor election 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: mon.b calling monitor election 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: mon.a calling monitor election 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: mon.c calling monitor election 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: monmap epoch 3 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: last_changed 2024-09-19T07:29:57.267497+0000 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: min_mon_release 19 (squid) 2024-09-19T07:30:05.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: election_strategy: 1 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: fsmap 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: mgrmap e13: x(active, since 48s) 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: overall HEALTH_OK 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2397491864' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:05.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:05.332 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:30:05.332 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:30:05.398 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:30:05.398 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T07:30:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: mon.b calling monitor election 2024-09-19T07:30:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: mon.b calling monitor election 2024-09-19T07:30:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: mon.a calling monitor election 2024-09-19T07:30:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: mon.c calling monitor election 2024-09-19T07:30:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T07:30:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: monmap epoch 3 2024-09-19T07:30:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: last_changed 2024-09-19T07:29:57.267497+0000 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: created 2024-09-19T07:28:26.970477+0000 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: min_mon_release 19 (squid) 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: election_strategy: 1 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: 2: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: fsmap 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: mgrmap e13: x(active, since 48s) 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: overall HEALTH_OK 2024-09-19T07:30:05.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2397491864' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:05.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:05.427 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:30:05.428 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:30:05.495 INFO:tasks.cephadm:Adding mgr.x on smithi103 2024-09-19T07:30:05.495 INFO:tasks.cephadm:Adding mgr.y on smithi134 2024-09-19T07:30:05.495 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-09-19T07:30:05.495 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi103=x;smithi134=y;smithi174=z' 2024-09-19T07:30:05.773 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:30:06.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:06 smithi174 ceph-mon[35370]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:06 smithi174 ceph-mon[35370]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:06 smithi174 ceph-mon[35370]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:06 smithi174 ceph-mon[35370]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T07:30:06.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:06 smithi174 ceph-mon[35370]: Reconfiguring daemon mon.a on smithi103 2024-09-19T07:30:06.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:06.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.199+0000 7f61cd028640 1 -- 172.21.15.174:0/2004393563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c81005f0 msgr2=0x7f61c81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:06.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.199+0000 7f61cd028640 1 --2- 172.21.15.174:0/2004393563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c81005f0 0x7f61c81009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009880 tx=0x7f61b802a5d0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.199+0000 7f61cd028640 1 -- 172.21.15.174:0/2004393563 shutdown_connections 2024-09-19T07:30:06.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.199+0000 7f61cd028640 1 --2- 172.21.15.174:0/2004393563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c81005f0 0x7f61c81009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.199+0000 7f61cd028640 1 -- 172.21.15.174:0/2004393563 >> 172.21.15.174:0/2004393563 conn(0x7f61c80fbda0 msgr2=0x7f61c80fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:06.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.199+0000 7f61cd028640 1 -- 172.21.15.174:0/2004393563 shutdown_connections 2024-09-19T07:30:06.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.199+0000 7f61cd028640 1 -- 172.21.15.174:0/2004393563 wait complete. 2024-09-19T07:30:06.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 Processor -- start 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 -- start start 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c81005f0 0x7f61c818d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c818bf40 0x7f61c818c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c818c900 0x7f61c81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61c81066e0 con 0x7f61c818bf40 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61c8106560 con 0x7f61c818c900 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61cd028640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61c8106860 con 0x7f61c81005f0 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c81005f0 0x7f61c818d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c81005f0 0x7f61c818d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:52324/0 (socket says 172.21.15.174:52324) 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61c6d9d640 1 -- 172.21.15.174:0/854249940 learned_addr learned my addr 172.21.15.174:0/854249940 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.200+0000 7f61c659c640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c818bf40 0x7f61c818c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:06.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c759e640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c818c900 0x7f61c81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c6d9d640 1 -- 172.21.15.174:0/854249940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c818c900 msgr2=0x7f61c81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c6d9d640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c818c900 0x7f61c81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c6d9d640 1 -- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c818bf40 msgr2=0x7f61c818c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c6d9d640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c818bf40 0x7f61c818c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c6d9d640 1 -- 172.21.15.174:0/854249940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c81a15f0 con 0x7f61c81005f0 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c659c640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c818bf40 0x7f61c818c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61c6d9d640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c81005f0 0x7f61c818d8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61b8031610 tx=0x7f61b80043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61bffff640 1 -- 172.21.15.174:0/854249940 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b803f030 con 0x7f61c81005f0 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61bffff640 1 -- 172.21.15.174:0/854249940 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b8003ce0 con 0x7f61c81005f0 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61bffff640 1 -- 172.21.15.174:0/854249940 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8035070 con 0x7f61c81005f0 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c81a1880 con 0x7f61c81005f0 2024-09-19T07:30:06.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.201+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c81a1de0 con 0x7f61c81005f0 2024-09-19T07:30:06.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.202+0000 7f61bffff640 1 -- 172.21.15.174:0/854249940 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f61b8042030 con 0x7f61c81005f0 2024-09-19T07:30:06.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.202+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c81a2040 con 0x7f61c81005f0 2024-09-19T07:30:06.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.202+0000 7f61bffff640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f61b4039730 0x7f61b403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:06.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.202+0000 7f61bffff640 1 -- 172.21.15.174:0/854249940 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f61b806cf60 con 0x7f61c81005f0 2024-09-19T07:30:06.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.202+0000 7f61c659c640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f61b4039730 0x7f61b403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:06.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.203+0000 7f61c659c640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f61b4039730 0x7f61b403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61c818d670 tx=0x7f61b0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:06.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.207+0000 7f61bffff640 1 -- 172.21.15.174:0/854249940 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f61b8036260 con 0x7f61c81005f0 2024-09-19T07:30:06.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:06 smithi103 ceph-mon[31201]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:06 smithi103 ceph-mon[31201]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:06 smithi103 ceph-mon[31201]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:06 smithi103 ceph-mon[31201]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T07:30:06.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:06 smithi103 ceph-mon[31201]: Reconfiguring daemon mon.a on smithi103 2024-09-19T07:30:06.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:06 smithi134 ceph-mon[35414]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:06 smithi134 ceph-mon[35414]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:06 smithi134 ceph-mon[35414]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:30:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:06 smithi134 ceph-mon[35414]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T07:30:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:06 smithi134 ceph-mon[35414]: Reconfiguring daemon mon.a on smithi103 2024-09-19T07:30:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:30:06.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.385+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}) -- 0x7f61c806ab40 con 0x7f61b4039730 2024-09-19T07:30:06.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.389+0000 7f61bffff640 1 -- 172.21.15.174:0/854249940 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f61c806ab40 con 0x7f61b4039730 2024-09-19T07:30:06.389 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-09-19T07:30:06.391 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f61b4039730 msgr2=0x7f61b403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f61b4039730 0x7f61b403bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61c818d670 tx=0x7f61b0009190 comp rx=0 tx=0).stop 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c81005f0 msgr2=0x7f61c818d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c81005f0 0x7f61c818d8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61b8031610 tx=0x7f61b80043c0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 shutdown_connections 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f61b4039730 0x7f61b403bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c818c900 0x7f61c81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61c818bf40 0x7f61c818c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 --2- 172.21.15.174:0/854249940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c81005f0 0x7f61c818d8f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 >> 172.21.15.174:0/854249940 conn(0x7f61c80fbda0 msgr2=0x7f61c806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 shutdown_connections 2024-09-19T07:30:06.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:30:06.391+0000 7f61cd028640 1 -- 172.21.15.174:0/854249940 wait complete. 2024-09-19T07:30:06.948 DEBUG:teuthology.orchestra.run.smithi134:mgr.y> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T07:30:06.951 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T07:30:06.953 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T07:30:06.954 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:30:06.954 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:30:06.971 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:30:06.971 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 07:29:27.081302209 +0000 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 07:24:42.269625910 +0000 2024-09-19T07:30:07.028 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 07:24:42.269625910 +0000 2024-09-19T07:30:07.029 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 07:24:42.269625910 +0000 2024-09-19T07:30:07.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:30:07.094 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T07:30:07.094 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T07:30:07.095 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000325113 s, 1.6 MB/s 2024-09-19T07:30:07.096 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:30:07.153 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-19T07:30:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:07.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:07 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.209 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 07:29:27.107301679 +0000 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 07:24:42.598619392 +0000 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 07:24:42.598619392 +0000 2024-09-19T07:30:07.210 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 07:24:42.598619392 +0000 2024-09-19T07:30:07.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:30:07.274 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T07:30:07.274 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T07:30:07.274 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000192056 s, 2.7 MB/s 2024-09-19T07:30:07.275 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:30:07.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:07.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:07.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:07.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:07.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:07 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.332 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-19T07:30:07.388 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:30:07.388 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:07.388 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T07:30:07.388 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:07.389 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:07.389 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 07:29:27.133301149 +0000 2024-09-19T07:30:07.389 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 07:24:42.925612912 +0000 2024-09-19T07:30:07.389 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 07:24:42.925612912 +0000 2024-09-19T07:30:07.389 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 07:24:42.925612912 +0000 2024-09-19T07:30:07.389 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:30:07.450 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:07.450 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.450 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.450 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.450 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:07.450 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:07.450 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:07.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:07 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:07.452 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T07:30:07.453 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T07:30:07.453 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000168689 s, 3.0 MB/s 2024-09-19T07:30:07.454 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:30:07.510 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-19T07:30:07.566 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:30:07.566 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T07:30:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-19 07:29:27.158300640 +0000 2024-09-19T07:30:07.566 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-19 07:24:43.275605977 +0000 2024-09-19T07:30:07.567 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-19 07:24:43.275605977 +0000 2024-09-19T07:30:07.567 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-19 07:24:43.275605977 +0000 2024-09-19T07:30:07.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:30:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-19T07:30:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-19T07:30:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000195159 s, 2.6 MB/s 2024-09-19T07:30:07.631 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:30:07.688 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-19T07:30:07.688 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:30:07.704 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:30:07.704 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 07:29:56.701200448 +0000 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 07:24:58.218083757 +0000 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 07:24:58.218083757 +0000 2024-09-19T07:30:07.760 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 07:24:58.218083757 +0000 2024-09-19T07:30:07.760 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:30:07.824 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T07:30:07.824 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T07:30:07.824 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000287525 s, 1.8 MB/s 2024-09-19T07:30:07.825 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:30:07.881 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-19T07:30:07.937 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:30:07.938 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:07.938 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-19T07:30:07.938 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:07.938 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:07.938 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 07:29:56.701200448 +0000 2024-09-19T07:30:07.938 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 07:24:58.546075891 +0000 2024-09-19T07:30:07.938 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 07:24:58.546075891 +0000 2024-09-19T07:30:07.939 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 07:24:58.546075891 +0000 2024-09-19T07:30:07.939 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:30:08.001 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T07:30:08.001 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T07:30:08.001 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000165462 s, 3.1 MB/s 2024-09-19T07:30:08.002 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:30:08.059 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-19T07:30:08.115 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 07:29:56.702200425 +0000 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 07:24:58.882067834 +0000 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 07:24:58.882067834 +0000 2024-09-19T07:30:08.116 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 07:24:58.882067834 +0000 2024-09-19T07:30:08.117 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:30:08.180 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T07:30:08.180 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T07:30:08.180 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000175328 s, 2.9 MB/s 2024-09-19T07:30:08.181 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:30:08.237 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-19T07:30:08.293 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:30:08.293 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:08.294 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T07:30:08.294 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:08.294 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:08.294 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-19 07:29:56.702200425 +0000 2024-09-19T07:30:08.294 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-19 07:24:59.212059921 +0000 2024-09-19T07:30:08.294 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-19 07:24:59.212059921 +0000 2024-09-19T07:30:08.294 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-19 07:24:59.212059921 +0000 2024-09-19T07:30:08.294 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:30:08.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: Reconfiguring mon.b (monmap changed)... 2024-09-19T07:30:08.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: Reconfiguring daemon mon.b on smithi134 2024-09-19T07:30:08.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:08.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: Saving service mgr spec with placement smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T07:30:08.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:08.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:08.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:08.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:08.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:08 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:08.352 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-19T07:30:08.353 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-19T07:30:08.353 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.0001873 s, 2.7 MB/s 2024-09-19T07:30:08.354 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:30:08.365 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: Reconfiguring mon.b (monmap changed)... 2024-09-19T07:30:08.365 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: Reconfiguring daemon mon.b on smithi134 2024-09-19T07:30:08.365 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:08.366 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: Saving service mgr spec with placement smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T07:30:08.366 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:08.366 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:08.366 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:08.366 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:08.366 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:08 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:08.410 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-19T07:30:08.410 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T07:30:08.427 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T07:30:08.427 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: Reconfiguring mon.b (monmap changed)... 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: Reconfiguring daemon mon.b on smithi134 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi134=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: Saving service mgr spec with placement smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T07:30:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T07:30:08.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:08 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 07:29:47.349465773 +0000 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 07:25:00.676459870 +0000 2024-09-19T07:30:08.484 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 07:25:00.676459870 +0000 2024-09-19T07:30:08.485 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 07:25:00.676459870 +0000 2024-09-19T07:30:08.485 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T07:30:08.548 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T07:30:08.549 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T07:30:08.549 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000342376 s, 1.5 MB/s 2024-09-19T07:30:08.550 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T07:30:08.606 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-09-19T07:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T07:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-19T07:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 07:29:47.349465773 +0000 2024-09-19T07:30:08.663 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 07:25:01.002455415 +0000 2024-09-19T07:30:08.664 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 07:25:01.002455415 +0000 2024-09-19T07:30:08.664 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 07:25:01.002455415 +0000 2024-09-19T07:30:08.664 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T07:30:08.727 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T07:30:08.727 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T07:30:08.727 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.00021211 s, 2.4 MB/s 2024-09-19T07:30:08.728 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T07:30:08.785 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-09-19T07:30:08.841 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 07:29:47.349465773 +0000 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 07:25:01.336450849 +0000 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 07:25:01.336450849 +0000 2024-09-19T07:30:08.842 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 07:25:01.336450849 +0000 2024-09-19T07:30:08.842 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T07:30:08.907 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T07:30:08.907 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T07:30:08.907 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000199638 s, 2.6 MB/s 2024-09-19T07:30:08.908 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T07:30:08.969 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-19 07:29:47.350465758 +0000 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-19 07:25:01.668446311 +0000 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-19 07:25:01.668446311 +0000 2024-09-19T07:30:09.039 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-19 07:25:01.668446311 +0000 2024-09-19T07:30:09.039 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T07:30:09.104 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-19T07:30:09.104 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-19T07:30:09.104 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000173988 s, 2.9 MB/s 2024-09-19T07:30:09.106 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T07:30:09.163 INFO:tasks.cephadm:Deploying osd.0 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-19T07:30:09.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T07:30:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: Reconfiguring mon.c (monmap changed)... 2024-09-19T07:30:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: Reconfiguring daemon mon.c on smithi174 2024-09-19T07:30:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:09.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: Reconfiguring mon.c (monmap changed)... 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: Reconfiguring daemon mon.c on smithi174 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:09.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:09.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:09.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:30:09.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:09.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:09.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: Reconfiguring mon.c (monmap changed)... 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: Reconfiguring daemon mon.c on smithi174 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:30:09.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:09.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:09.417 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:10.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:10 smithi134 ceph-mon[35414]: Deploying daemon mgr.z on smithi174 2024-09-19T07:30:10.341 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:10 smithi174 ceph-mon[35370]: Deploying daemon mgr.z on smithi174 2024-09-19T07:30:10.449 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:10 smithi103 ceph-mon[31201]: Deploying daemon mgr.z on smithi174 2024-09-19T07:30:10.915 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:10 smithi174 systemd[1]: Starting Ceph mgr.z for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:30:11.279 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-mon[35370]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:11.279 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 podman[37163]: 2024-09-19 07:30:11.00236734 +0000 UTC m=+0.100273331 container create 15ac4ad71a815f7d3ed43074b60cb42a6e952b2c1f88ab016fea79bc25630158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:30:11.279 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 podman[37163]: 2024-09-19 07:30:10.918195299 +0000 UTC m=+0.016101287 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:11.280 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 podman[37163]: 2024-09-19 07:30:11.205006668 +0000 UTC m=+0.302912656 container init 15ac4ad71a815f7d3ed43074b60cb42a6e952b2c1f88ab016fea79bc25630158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:30:11.280 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 podman[37163]: 2024-09-19 07:30:11.208523507 +0000 UTC m=+0.306429503 container start 15ac4ad71a815f7d3ed43074b60cb42a6e952b2c1f88ab016fea79bc25630158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:30:11.280 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-mgr[37182]: -- 172.21.15.174:0/1022213517 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5704230e0 con 0x55c570401000 2024-09-19T07:30:11.280 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 bash[37163]: 15ac4ad71a815f7d3ed43074b60cb42a6e952b2c1f88ab016fea79bc25630158 2024-09-19T07:30:11.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:11 smithi134 ceph-mon[35414]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:11 smithi103 ceph-mon[31201]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:11.538 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 systemd[1]: Started Ceph mgr.z for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:30:11.538 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:11.329+0000 7f7d55ccc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:30:11.538 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:11.388+0000 7f7d55ccc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:30:11.538 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:11.538+0000 7f7d55ccc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:30:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:30:11.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-19T07:30:11.889 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:11.603+0000 7f7d55ccc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:30:11.889 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:11.750+0000 7f7d55ccc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:30:11.889 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:11.888+0000 7f7d55ccc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:30:12.068 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:30:12.452 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: Deploying daemon mgr.y on smithi134 2024-09-19T07:30:12.453 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:12 smithi103 ceph-mon[31201]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:12.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 -- 172.21.15.103:0/124890289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c1037b0 msgr2=0x7f504c10a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:12.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 --2- 172.21.15.103:0/124890289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c1037b0 0x7f504c10a040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5044009ca0 tx=0x7f504402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:30:12.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 -- 172.21.15.103:0/124890289 shutdown_connections 2024-09-19T07:30:12.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 --2- 172.21.15.103:0/124890289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c1037b0 0x7f504c10a040 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:12.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 --2- 172.21.15.103:0/124890289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f504c102df0 0x7f504c103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 --2- 172.21.15.103:0/124890289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f504c101bf0 0x7f504c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 -- 172.21.15.103:0/124890289 >> 172.21.15.103:0/124890289 conn(0x7f504c0fd3c0 msgr2=0x7f504c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 -- 172.21.15.103:0/124890289 shutdown_connections 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 -- 172.21.15.103:0/124890289 wait complete. 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 Processor -- start 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.536+0000 7f5053671640 1 -- start start 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f504c101bf0 0x7f504c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c102df0 0x7f504c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f504c1037b0 0x7f504c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10d850 con 0x7f504c102df0 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10d6d0 con 0x7f504c1037b0 2024-09-19T07:30:12.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10d9d0 con 0x7f504c101bf0 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c102df0 0x7f504c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c102df0 0x7f504c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49462/0 (socket says 172.21.15.103:49462) 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 -- 172.21.15.103:0/3076600101 learned_addr learned my addr 172.21.15.103:0/3076600101 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f505140d640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f504c101bf0 0x7f504c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5051c0e640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f504c1037b0 0x7f504c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 -- 172.21.15.103:0/3076600101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f504c101bf0 msgr2=0x7f504c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f504c101bf0 0x7f504c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 -- 172.21.15.103:0/3076600101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f504c1037b0 msgr2=0x7f504c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f504c1037b0 0x7f504c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 -- 172.21.15.103:0/3076600101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c19fa70 con 0x7f504c102df0 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5050c0c640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c102df0 0x7f504c198380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f503c00d950 tx=0x7f503c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:12.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c017030 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503c004020 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c19fd60 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0047a0 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.537+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c1a0240 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.538+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f503c009a30 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.539+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c108fa0 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.539+0000 7f50427fc640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5038039500 0x7f503803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.539+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f503c04bfe0 con 0x7f504c102df0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.539+0000 7f505140d640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5038039500 0x7f503803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.539+0000 7f505140d640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5038039500 0x7f503803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f504c102c50 tx=0x7f504800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:12.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.544+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f503c009d20 con 0x7f504c102df0 2024-09-19T07:30:12.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:12.204+0000 7f7d55ccc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:30:12.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:12.263+0000 7f7d55ccc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:30:12.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:12.448+0000 7f7d55ccc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: Deploying daemon mgr.y on smithi134 2024-09-19T07:30:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:12 smithi174 ceph-mon[35370]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:12.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:12.718+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f504c062630 con 0x7f5038039500 2024-09-19T07:30:12.719 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.719 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T07:30:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: Deploying daemon mgr.y on smithi134 2024-09-19T07:30:12.720 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:12 smithi134 ceph-mon[35414]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:13.329 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 systemd[1]: Starting Ceph mgr.y for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:30:13.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:13 smithi103 ceph-mon[31201]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:13.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:30:13.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:30:13.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:13 smithi174 ceph-mon[35370]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:30:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:30:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:13.666 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:13 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:13.325+0000 7f7d55ccc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:30:13.666 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:13 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:13.394+0000 7f7d55ccc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:30:13.674 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:13 smithi134 ceph-mon[35414]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:13.675 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:30:13.675 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:30:13.675 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:13.675 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 podman[36767]: 2024-09-19 07:30:13.290390701 +0000 UTC m=+0.015883445 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:30:13.675 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 podman[36767]: 2024-09-19 07:30:13.40603976 +0000 UTC m=+0.131532494 container create e8078c7d2cfdd0b5c3cf62c8b4484691f5518594283b06e104d9974617d6469a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T07:30:13.675 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 podman[36767]: 2024-09-19 07:30:13.675094224 +0000 UTC m=+0.400586966 container init e8078c7d2cfdd0b5c3cf62c8b4484691f5518594283b06e104d9974617d6469a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:30:14.012 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 podman[36767]: 2024-09-19 07:30:13.677947961 +0000 UTC m=+0.403440695 container start e8078c7d2cfdd0b5c3cf62c8b4484691f5518594283b06e104d9974617d6469a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T07:30:14.012 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 ceph-mgr[36786]: -- 172.21.15.134:0/64957471 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579a094b0e0 con 0x5579a0929000 2024-09-19T07:30:14.013 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 bash[36767]: e8078c7d2cfdd0b5c3cf62c8b4484691f5518594283b06e104d9974617d6469a 2024-09-19T07:30:14.013 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 systemd[1]: Started Ceph mgr.y for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:30:14.013 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:13.795+0000 7f3ef8636040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:30:14.013 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:13 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:13.856+0000 7f3ef8636040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:30:14.013 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:14.011+0000 7f3ef8636040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:30:14.329 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:14.081+0000 7f3ef8636040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:30:14.329 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:14.238+0000 7f3ef8636040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:30:14.404 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.120+0000 7f7d55ccc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:30:14.404 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.245+0000 7f7d55ccc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:30:14.404 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.403+0000 7f7d55ccc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:30:14.656 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:14.396+0000 7f3ef8636040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:30:14.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.465+0000 7f7d55ccc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:30:14.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.619+0000 7f7d55ccc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:30:14.976 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:14.722+0000 7f3ef8636040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:30:14.977 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:14.784+0000 7f3ef8636040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:30:14.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:14.977 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-mon[35414]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:14.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.992 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:14.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:14.993 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-mon[35370]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:14.993 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.692+0000 7f7d55ccc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:30:14.993 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.752+0000 7f7d55ccc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:30:14.993 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.924+0000 7f7d55ccc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:30:14.993 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:14.992+0000 7f7d55ccc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:30:15.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:15.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:15.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:15.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:15.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:15.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:14 smithi103 ceph-mon[31201]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:15.262 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:15.059+0000 7f7d55ccc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:30:15.263 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:15.145+0000 7f7d55ccc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:30:15.263 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:15.204+0000 7f7d55ccc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:30:15.263 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:15.262+0000 7f7d55ccc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:30:15.329 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:14.976+0000 7f3ef8636040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:30:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1276017365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25f6411-3656-482d-bd89-a94dd43726b9"}]: dispatch 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1276017365' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d25f6411-3656-482d-bd89-a94dd43726b9"}]': finished 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:16.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:16.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: Reconfiguring daemon mgr.x on smithi103 2024-09-19T07:30:16.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3585954792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:30:16.329 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:15.865+0000 7f3ef8636040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:30:16.329 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:15.930+0000 7f3ef8636040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:30:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1276017365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25f6411-3656-482d-bd89-a94dd43726b9"}]: dispatch 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1276017365' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d25f6411-3656-482d-bd89-a94dd43726b9"}]': finished 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:16.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:16.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: Reconfiguring daemon mgr.x on smithi103 2024-09-19T07:30:16.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3585954792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:30:16.415 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:30:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:30:16.041+0000 7f7d55ccc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:30:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1276017365' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25f6411-3656-482d-bd89-a94dd43726b9"}]: dispatch 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1276017365' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d25f6411-3656-482d-bd89-a94dd43726b9"}]': finished 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T07:30:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T07:30:16.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T07:30:16.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:16.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: Reconfiguring daemon mgr.x on smithi103 2024-09-19T07:30:16.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3585954792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:30:16.900 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:16.644+0000 7f3ef8636040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:30:16.900 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:16.758+0000 7f3ef8636040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:30:16.900 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:16.899+0000 7f3ef8636040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:30:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:17.046+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7f503c01ac60 con 0x7f504c102df0 2024-09-19T07:30:17.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-mon[35414]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:17.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:17.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-mon[35414]: Standby manager daemon z started 2024-09-19T07:30:17.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:30:17.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:30:17.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:30:17.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:30:17.177 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:16.959+0000 7f3ef8636040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:30:17.178 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.109+0000 7f3ef8636040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:30:17.178 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.176+0000 7f3ef8636040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:30:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:17 smithi174 ceph-mon[35370]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:17.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:17.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:17 smithi174 ceph-mon[35370]: Standby manager daemon z started 2024-09-19T07:30:17.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:17 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:30:17.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:17 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:30:17.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:17 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:30:17.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:17 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:30:17.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:17 smithi103 ceph-mon[31201]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:17.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:17.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:17 smithi103 ceph-mon[31201]: Standby manager daemon z started 2024-09-19T07:30:17.613 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:17 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:30:17.614 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:17 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:30:17.614 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:17 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:30:17.614 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:17 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/138416445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:30:17.614 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.235+0000 7f3ef8636040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:30:17.614 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.391+0000 7f3ef8636040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:30:17.614 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.455+0000 7f3ef8636040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:30:17.724 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.522+0000 7f3ef8636040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:30:17.725 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.603+0000 7f3ef8636040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:30:17.725 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.663+0000 7f3ef8636040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:30:17.725 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:17.722+0000 7f3ef8636040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:30:18.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:18 smithi134 ceph-mon[35414]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T07:30:18.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:18 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:30:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:18 smithi174 ceph-mon[35370]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T07:30:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:18 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:30:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:18 smithi103 ceph-mon[31201]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T07:30:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:18 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:30:18.829 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:30:18 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:30:18.507+0000 7f3ef8636040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:30:19.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:19.066+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f503c017c10 con 0x7f504c102df0 2024-09-19T07:30:19.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:19 smithi134 ceph-mon[35414]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:19.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:19 smithi134 ceph-mon[35414]: Standby manager daemon y started 2024-09-19T07:30:19.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:19 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:30:19.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:19 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:30:19.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:19 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:30:19.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:19 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:30:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:19 smithi174 ceph-mon[35370]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:19 smithi174 ceph-mon[35370]: Standby manager daemon y started 2024-09-19T07:30:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:19 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:30:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:19 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:30:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:19 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:30:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:19 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:30:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:19 smithi103 ceph-mon[31201]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:19 smithi103 ceph-mon[31201]: Standby manager daemon y started 2024-09-19T07:30:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:19 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:30:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:19 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:30:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:19 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:30:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:19 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/484160975' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:30:20.178 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:20 smithi103 ceph-mon[31201]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T07:30:20.178 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:30:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:20 smithi134 ceph-mon[35414]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T07:30:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:30:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:20 smithi174 ceph-mon[35370]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-19T07:30:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:30:21.204 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:21 smithi103 ceph-mon[31201]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:21 smithi134 ceph-mon[35414]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:21.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:21 smithi174 ceph-mon[35370]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:23.133 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:23 smithi103 ceph-mon[31201]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:23.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:23.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:23.134 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:23.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:23 smithi134 ceph-mon[35414]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:23.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:23.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:23.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:23.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:23 smithi174 ceph-mon[35370]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:23.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:23.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:23.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:24.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:24.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:24.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:24.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:24.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:24.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:25.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:25 smithi103 ceph-mon[31201]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:25.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T07:30:25.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:25.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:25 smithi103 ceph-mon[31201]: Deploying daemon osd.0 on smithi103 2024-09-19T07:30:25.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:25 smithi134 ceph-mon[35414]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:25.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T07:30:25.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:25.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:25 smithi134 ceph-mon[35414]: Deploying daemon osd.0 on smithi103 2024-09-19T07:30:25.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:25 smithi174 ceph-mon[35370]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:25.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T07:30:25.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:25.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:25 smithi174 ceph-mon[35370]: Deploying daemon osd.0 on smithi103 2024-09-19T07:30:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:27 smithi103 ceph-mon[31201]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:27 smithi134 ceph-mon[35414]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:27 smithi174 ceph-mon[35370]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:29.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:29 smithi134 ceph-mon[35414]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:29.384 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:29 smithi103 ceph-mon[31201]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:29.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:29 smithi174 ceph-mon[35370]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:31.320 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:31 smithi103 ceph-mon[31201]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:31.320 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:31.320 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:31.320 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:31 smithi174 ceph-mon[35370]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:31 smithi134 ceph-mon[35414]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:33.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:33 smithi103 ceph-mon[31201]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:33 smithi174 ceph-mon[35370]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:33 smithi134 ceph-mon[35414]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:34.391 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:34.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:34.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:34.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:34.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:34.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.680+0000 7f50427fc640 1 -- 172.21.15.103:0/3076600101 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f504c062630 con 0x7f5038039500 2024-09-19T07:30:34.681 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 0 on host 'smithi103' 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5038039500 msgr2=0x7f503803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5038039500 0x7f503803b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f504c102c50 tx=0x7f504800b390 comp rx=0 tx=0).stop 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c102df0 msgr2=0x7f504c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c102df0 0x7f504c198380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f503c00d950 tx=0x7f503c00de10 comp rx=0 tx=0).stop 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 shutdown_connections 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5038039500 0x7f503803b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f504c1037b0 0x7f504c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f504c102df0 0x7f504c198380 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 --2- 172.21.15.103:0/3076600101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f504c101bf0 0x7f504c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 >> 172.21.15.103:0/3076600101 conn(0x7f504c0fd3c0 msgr2=0x7f504c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:34.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 shutdown_connections 2024-09-19T07:30:34.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:30:34.682+0000 7f5053671640 1 -- 172.21.15.103:0/3076600101 wait complete. 2024-09-19T07:30:35.327 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:35 smithi103 ceph-mon[31201]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:35.327 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:35.327 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:35.327 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:35.329 DEBUG:teuthology.orchestra.run.smithi103:osd.0> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T07:30:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:35 smithi134 ceph-mon[35414]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:35.331 INFO:tasks.cephadm:Deploying osd.1 on smithi134 with /dev/vg_nvme/lv_4... 2024-09-19T07:30:35.331 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T07:30:35.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:35 smithi174 ceph-mon[35370]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:35.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:35.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:35.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:35.572 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:30:36.598 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:30:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0[44902]: 2024-09-19T07:30:36.338+0000 7f0d4ed7b640 -1 osd.0 0 log_to_monitors true 2024-09-19T07:30:37.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:37 smithi103 ceph-mon[31201]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:37.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:37 smithi103 ceph-mon[31201]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:30:37.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:37 smithi134 ceph-mon[35414]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:37.385 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:37 smithi134 ceph-mon[35414]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:30:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:37 smithi174 ceph-mon[35370]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:37 smithi174 ceph-mon[35370]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T07:30:37.944 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:30:37.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-09-19T07:30:38.223 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:30:38.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:38 smithi134 ceph-mon[35414]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T07:30:38.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:38 smithi134 ceph-mon[35414]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T07:30:38.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:38 smithi134 ceph-mon[35414]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:30:38.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:38.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:38 smithi134 ceph-mon[35414]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:30:38.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:38 smithi134 ceph-mon[35414]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T07:30:38.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:38 smithi174 ceph-mon[35370]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T07:30:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:38 smithi174 ceph-mon[35370]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T07:30:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:38 smithi174 ceph-mon[35370]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:30:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:38.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:38 smithi174 ceph-mon[35370]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:30:38.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:38 smithi174 ceph-mon[35370]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T07:30:38.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:38.470 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0[44902]: 2024-09-19T07:30:38.115+0000 7f0d4acfe640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T07:30:38.470 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0[44902]: 2024-09-19T07:30:38.118+0000 7f0d46b26640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:30:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-mon[31201]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T07:30:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-mon[31201]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T07:30:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-mon[31201]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:30:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:38.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-mon[31201]: from='osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:30:38.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-mon[31201]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T07:30:38.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.753+0000 7f628df0a640 1 -- 172.21.15.134:0/1395380316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 msgr2=0x7f6288100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.753+0000 7f628df0a640 1 --2- 172.21.15.134:0/1395380316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288100f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6278009900 tx=0x7f627802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.753+0000 7f628df0a640 1 -- 172.21.15.134:0/1395380316 shutdown_connections 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.753+0000 7f628df0a640 1 --2- 172.21.15.134:0/1395380316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62881026d0 0x7f6288108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 --2- 172.21.15.134:0/1395380316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d10 0x7f6288102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 --2- 172.21.15.134:0/1395380316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288100f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 -- 172.21.15.134:0/1395380316 >> 172.21.15.134:0/1395380316 conn(0x7f62880fc2a0 msgr2=0x7f62880fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:38.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 -- 172.21.15.134:0/1395380316 shutdown_connections 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 -- 172.21.15.134:0/1395380316 wait complete. 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 Processor -- start 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 -- start start 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d10 0x7f6288198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.754+0000 7f628df0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62881026d0 0x7f628819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f628df0a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f628810d880 con 0x7f62881026d0 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f628df0a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f628810d700 con 0x7f6288100b10 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f628df0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f628810da00 con 0x7f6288101d10 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:45410/0 (socket says 172.21.15.134:45410) 2024-09-19T07:30:38.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 -- 172.21.15.134:0/4223134019 learned_addr learned my addr 172.21.15.134:0/4223134019 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f6287fff640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62881026d0 0x7f628819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f627effd640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d10 0x7f6288198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 -- 172.21.15.134:0/4223134019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d10 msgr2=0x7f6288198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d10 0x7f6288198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 -- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62881026d0 msgr2=0x7f628819f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62881026d0 0x7f628819f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 -- 172.21.15.134:0/4223134019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628819fa70 con 0x7f6288100b10 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62877fe640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288197e40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6278031610 tx=0x7f62780043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62857fa640 1 -- 172.21.15.134:0/4223134019 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627803f030 con 0x7f6288100b10 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62857fa640 1 -- 172.21.15.134:0/4223134019 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6278003ce0 con 0x7f6288100b10 2024-09-19T07:30:38.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628819fd00 con 0x7f6288100b10 2024-09-19T07:30:38.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f62857fa640 1 -- 172.21.15.134:0/4223134019 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278035070 con 0x7f6288100b10 2024-09-19T07:30:38.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.755+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62881a0180 con 0x7f6288100b10 2024-09-19T07:30:38.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.756+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6288107f10 con 0x7f6288100b10 2024-09-19T07:30:38.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.758+0000 7f62857fa640 1 -- 172.21.15.134:0/4223134019 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f6278042030 con 0x7f6288100b10 2024-09-19T07:30:38.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.758+0000 7f62857fa640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62740a3260 0x7f62740a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:30:38.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.758+0000 7f62857fa640 1 -- 172.21.15.134:0/4223134019 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f62780ee4f0 con 0x7f6288100b10 2024-09-19T07:30:38.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.759+0000 7f627effd640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62740a3260 0x7f62740a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:30:38.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.759+0000 7f627effd640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62740a3260 0x7f62740a5720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6288199360 tx=0x7f627000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:30:38.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.762+0000 7f62857fa640 1 -- 172.21.15.134:0/4223134019 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f62780bcda0 con 0x7f6288100b10 2024-09-19T07:30:38.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:38.936+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6288062630 con 0x7f62740a3260 2024-09-19T07:30:39.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi103 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:30:39.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:30:39.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:39.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi103 2024-09-19T07:30:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi103 2024-09-19T07:30:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:30:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:39.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:40.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:40 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:30:40.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:40 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:30:40.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:40 smithi134 ceph-mon[35414]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:40.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:40 smithi134 ceph-mon[35414]: osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] boot 2024-09-19T07:30:40.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:40 smithi134 ceph-mon[35414]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T07:30:40.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:40 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:30:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:40 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:30:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:40 smithi174 ceph-mon[35370]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:40 smithi174 ceph-mon[35370]: osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] boot 2024-09-19T07:30:40.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:40 smithi174 ceph-mon[35370]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T07:30:40.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:40 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:30:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:40 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:30:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:40 smithi103 ceph-mon[31201]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:30:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:40 smithi103 ceph-mon[31201]: osd.0 [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] boot 2024-09-19T07:30:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:40 smithi103 ceph-mon[31201]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T07:30:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:30:41.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:41 smithi134 ceph-mon[35414]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:41.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:41 smithi134 ceph-mon[35414]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T07:30:41.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:41 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/267693750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]: dispatch 2024-09-19T07:30:41.400 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:41 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]: dispatch 2024-09-19T07:30:41.401 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:41 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]': finished 2024-09-19T07:30:41.401 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:41 smithi134 ceph-mon[35414]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T07:30:41.401 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:41 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:30:41.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:41 smithi174 ceph-mon[35370]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:41.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:41 smithi174 ceph-mon[35370]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T07:30:41.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:41 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/267693750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]: dispatch 2024-09-19T07:30:41.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:41 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]: dispatch 2024-09-19T07:30:41.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:41 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]': finished 2024-09-19T07:30:41.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:41 smithi174 ceph-mon[35370]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T07:30:41.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:41 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:30:41.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:41 smithi103 ceph-mon[31201]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T07:30:41.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:41 smithi103 ceph-mon[31201]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T07:30:41.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:41 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/267693750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]: dispatch 2024-09-19T07:30:41.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:41 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]: dispatch 2024-09-19T07:30:41.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:41 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f51a85b8-e79c-496f-a662-59b58fdb0f58"}]': finished 2024-09-19T07:30:41.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:41 smithi103 ceph-mon[31201]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T07:30:41.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:41 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:30:42.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:42 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/2246698212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:30:42.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:42 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/2246698212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:30:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:42 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/2246698212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:30:43.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:43 smithi174 ceph-mon[35370]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:43 smithi103 ceph-mon[31201]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:43.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:43 smithi134 ceph-mon[35414]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:45.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:45 smithi174 ceph-mon[35370]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:45 smithi103 ceph-mon[31201]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:45 smithi134 ceph-mon[35414]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:47.322 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:47 smithi134 ceph-mon[35414]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:47.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:47 smithi174 ceph-mon[35370]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:47 smithi103 ceph-mon[31201]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:49.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:49 smithi134 ceph-mon[35414]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:49 smithi174 ceph-mon[35370]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:49.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:49 smithi103 ceph-mon[31201]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:50 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T07:30:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:50 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:50.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:50 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T07:30:50.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:50 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:50.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:50 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T07:30:50.487 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:50 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:51.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:51 smithi134 ceph-mon[35414]: Deploying daemon osd.1 on smithi134 2024-09-19T07:30:51.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:51 smithi134 ceph-mon[35414]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:51 smithi174 ceph-mon[35370]: Deploying daemon osd.1 on smithi134 2024-09-19T07:30:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:51 smithi174 ceph-mon[35370]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:51 smithi103 ceph-mon[31201]: Deploying daemon osd.1 on smithi134 2024-09-19T07:30:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:51 smithi103 ceph-mon[31201]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:53.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:53 smithi134 ceph-mon[35414]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:53.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:53 smithi174 ceph-mon[35370]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:53 smithi103 ceph-mon[31201]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:55.405 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:55 smithi134 ceph-mon[35414]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:55.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:55 smithi174 ceph-mon[35370]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:55.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:55 smithi103 ceph-mon[31201]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:56.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:56.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:56.443 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:30:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:57.392 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:57 smithi134 ceph-mon[35414]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:57.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:57 smithi174 ceph-mon[35370]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:57 smithi103 ceph-mon[31201]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:59.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:59 smithi134 ceph-mon[35414]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:59.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:59.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:59.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:30:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.319 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 1 on host 'smithi134' 2024-09-19T07:30:59.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.318+0000 7f62857fa640 1 -- 172.21.15.134:0/4223134019 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6288062630 con 0x7f62740a3260 2024-09-19T07:30:59.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62740a3260 msgr2=0x7f62740a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62740a3260 0x7f62740a5720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6288199360 tx=0x7f627000a2d0 comp rx=0 tx=0).stop 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 msgr2=0x7f6288197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288197e40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6278031610 tx=0x7f62780043b0 comp rx=0 tx=0).stop 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 shutdown_connections 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62740a3260 0x7f62740a5720 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62881026d0 0x7f628819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d10 0x7f6288198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 --2- 172.21.15.134:0/4223134019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6288100b10 0x7f6288197e40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 >> 172.21.15.134:0/4223134019 conn(0x7f62880fc2a0 msgr2=0x7f62881050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 shutdown_connections 2024-09-19T07:30:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:30:59.321+0000 7f628df0a640 1 -- 172.21.15.134:0/4223134019 wait complete. 2024-09-19T07:30:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:59 smithi103 ceph-mon[31201]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:30:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:59 smithi174 ceph-mon[35370]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:30:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:30:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:30:59.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:30:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:30:59.911 DEBUG:teuthology.orchestra.run.smithi134:osd.1> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T07:30:59.913 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-09-19T07:30:59.914 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T07:31:00.165 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:31:00.286 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:00.286 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:00.286 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:00.495 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:00.496 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:00.496 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:01.293 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:01 smithi134 ceph-mon[35414]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:01.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:01 smithi103 ceph-mon[31201]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:01.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:01 smithi174 ceph-mon[35370]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:02.079 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1[39868]: 2024-09-19T07:31:01.816+0000 7eff2652b640 -1 osd.1 0 log_to_monitors true 2024-09-19T07:31:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:02 smithi103 ceph-mon[31201]: from='osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:31:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:02 smithi103 ceph-mon[31201]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:31:02.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:02 smithi174 ceph-mon[35370]: from='osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:31:02.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:02 smithi174 ceph-mon[35370]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:31:02.546 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:31:02.571 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-09-19T07:31:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:02 smithi134 ceph-mon[35414]: from='osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:31:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:02 smithi134 ceph-mon[35414]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T07:31:02.812 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:31:03.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:03.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T07:31:03.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:31:03.221 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:03.222 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 -- 172.21.15.174:0/2928422258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 msgr2=0x7f8578105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 --2- 172.21.15.174:0/2928422258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f8578105c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8570009ca0 tx=0x7f857002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 -- 172.21.15.174:0/2928422258 shutdown_connections 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 --2- 172.21.15.174:0/2928422258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f8578105c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 --2- 172.21.15.174:0/2928422258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578102de0 0x7f8578105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 --2- 172.21.15.174:0/2928422258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8578100490 0x7f85781028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 -- 172.21.15.174:0/2928422258 >> 172.21.15.174:0/2928422258 conn(0x7f85780fc2a0 msgr2=0x7f85780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 -- 172.21.15.174:0/2928422258 shutdown_connections 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.232+0000 7f857dafe640 1 -- 172.21.15.174:0/2928422258 wait complete. 2024-09-19T07:31:03.233 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 Processor -- start 2024-09-19T07:31:03.235 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 -- start start 2024-09-19T07:31:03.235 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8578100490 0x7f857819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578102de0 0x7f857819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f85781a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8578113ea0 con 0x7f8578100490 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8578113d20 con 0x7f8578102de0 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f857dafe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8578114020 con 0x7f8578105790 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f8577fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f85781a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f8577fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f85781a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:57820/0 (socket says 172.21.15.174:57820) 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f85777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8578100490 0x7f857819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f8577fff640 1 -- 172.21.15.174:0/2578531319 learned_addr learned my addr 172.21.15.174:0/2578531319 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.233+0000 7f8576ffd640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578102de0 0x7f857819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8577fff640 1 -- 172.21.15.174:0/2578531319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578102de0 msgr2=0x7f857819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8577fff640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578102de0 0x7f857819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8577fff640 1 -- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8578100490 msgr2=0x7f857819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8577fff640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8578100490 0x7f857819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8577fff640 1 -- 172.21.15.174:0/2578531319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85781a6000 con 0x7f8578105790 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8577fff640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f85781a5990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8570005e30 tx=0x7f8570031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8574ff9640 1 -- 172.21.15.174:0/2578531319 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570003b70 con 0x7f8578105790 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85781a6290 con 0x7f8578105790 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8574ff9640 1 -- 172.21.15.174:0/2578531319 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8570003d10 con 0x7f8578105790 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f8574ff9640 1 -- 172.21.15.174:0/2578531319 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857002fd30 con 0x7f8578105790 2024-09-19T07:31:03.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.234+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85781a67d0 con 0x7f8578105790 2024-09-19T07:31:03.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.235+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85780fdd90 con 0x7f8578105790 2024-09-19T07:31:03.238 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.236+0000 7f8574ff9640 1 -- 172.21.15.174:0/2578531319 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f8570004050 con 0x7f8578105790 2024-09-19T07:31:03.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.237+0000 7f8574ff9640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85600a31e0 0x7f85600a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:03.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.237+0000 7f8574ff9640 1 -- 172.21.15.174:0/2578531319 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f85700ef110 con 0x7f8578105790 2024-09-19T07:31:03.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.237+0000 7f85777fe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85600a31e0 0x7f85600a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:03.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.237+0000 7f85777fe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85600a31e0 0x7f85600a56a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8568005d00 tx=0x7f856800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:03.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.241+0000 7f8574ff9640 1 -- 172.21.15.174:0/2578531319 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f85700bd8c0 con 0x7f8578105790 2024-09-19T07:31:03.414 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:03.414+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8578062630 con 0x7f85600a31e0 2024-09-19T07:31:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T07:31:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:31:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T07:31:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:03.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.579 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1[39868]: 2024-09-19T07:31:03.188+0000 7eff224ae640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T07:31:03.580 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1[39868]: 2024-09-19T07:31:03.191+0000 7eff1e2d6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:03.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:03.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:03.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:03.586 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi134 2024-09-19T07:31:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-19T07:31:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:31:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T07:31:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi134 2024-09-19T07:31:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-19T07:31:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:31:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T07:31:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:04.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:04.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:04.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi134 2024-09-19T07:31:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi134 to 17088M 2024-09-19T07:31:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:31:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T07:31:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:05.369 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:05 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:31:05.369 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:05 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:31:05.369 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:05 smithi174 ceph-mon[35370]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:05.370 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:05 smithi174 ceph-mon[35370]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:05.370 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:05 smithi174 ceph-mon[35370]: osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] boot 2024-09-19T07:31:05.370 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:05 smithi174 ceph-mon[35370]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T07:31:05.370 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:05 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:31:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:05 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:31:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:05 smithi103 ceph-mon[31201]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:05 smithi103 ceph-mon[31201]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:05 smithi103 ceph-mon[31201]: osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] boot 2024-09-19T07:31:05.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:05 smithi103 ceph-mon[31201]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T07:31:05.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:05 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:31:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:05 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:31:05.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:05 smithi134 ceph-mon[35414]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:05.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:05 smithi134 ceph-mon[35414]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T07:31:05.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:05 smithi134 ceph-mon[35414]: osd.1 [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] boot 2024-09-19T07:31:05.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:05 smithi134 ceph-mon[35414]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T07:31:05.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:31:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:06 smithi103 ceph-mon[31201]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T07:31:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:06 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/118685268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]: dispatch 2024-09-19T07:31:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:06 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]: dispatch 2024-09-19T07:31:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:06 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]': finished 2024-09-19T07:31:06.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:06 smithi103 ceph-mon[31201]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T07:31:06.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:06 smithi134 ceph-mon[35414]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T07:31:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:06 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/118685268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]: dispatch 2024-09-19T07:31:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:06 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]: dispatch 2024-09-19T07:31:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:06 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]': finished 2024-09-19T07:31:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:06 smithi134 ceph-mon[35414]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T07:31:06.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:06 smithi174 ceph-mon[35370]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T07:31:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:06 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/118685268' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]: dispatch 2024-09-19T07:31:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:06 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]: dispatch 2024-09-19T07:31:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:06 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f82ba336-4618-4e9c-825e-d51abe0ab8c5"}]': finished 2024-09-19T07:31:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:06 smithi174 ceph-mon[35370]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T07:31:06.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:07.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:07 smithi103 ceph-mon[31201]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:07.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:07 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/1686390139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:07 smithi134 ceph-mon[35414]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:07 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/1686390139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:07 smithi174 ceph-mon[35370]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:07 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/1686390139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:09 smithi103 ceph-mon[31201]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:09 smithi134 ceph-mon[35414]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:09 smithi174 ceph-mon[35370]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:11 smithi103 ceph-mon[31201]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:11.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:11 smithi174 ceph-mon[35370]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:11 smithi134 ceph-mon[35414]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:13.336 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:13 smithi174 ceph-mon[35370]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:13 smithi103 ceph-mon[31201]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:13 smithi134 ceph-mon[35414]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:14.356 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T07:31:14.356 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T07:31:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T07:31:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:15 smithi103 ceph-mon[31201]: Deploying daemon osd.2 on smithi174 2024-09-19T07:31:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:15 smithi103 ceph-mon[31201]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:15 smithi134 ceph-mon[35414]: Deploying daemon osd.2 on smithi174 2024-09-19T07:31:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:15 smithi134 ceph-mon[35414]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:15 smithi174 ceph-mon[35370]: Deploying daemon osd.2 on smithi174 2024-09-19T07:31:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:15 smithi174 ceph-mon[35370]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:17 smithi174 ceph-mon[35370]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:17 smithi103 ceph-mon[31201]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:17.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:17 smithi134 ceph-mon[35414]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:19.502 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:19 smithi174 ceph-mon[35370]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:19 smithi134 ceph-mon[35414]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:19 smithi103 ceph-mon[31201]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:20.545 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:20.545 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:20.545 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:21.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:21 smithi174 ceph-mon[35370]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:21 smithi134 ceph-mon[35414]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:21 smithi103 ceph-mon[31201]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:23.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:23 smithi174 ceph-mon[35370]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:23.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:23.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:23.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:23 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:23 smithi134 ceph-mon[35414]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:23.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:23.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:23 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.584 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-09-19T07:31:23.584 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.583+0000 7f8574ff9640 1 -- 172.21.15.174:0/2578531319 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8578062630 con 0x7f85600a31e0 2024-09-19T07:31:23.586 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85600a31e0 msgr2=0x7f85600a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:23.586 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85600a31e0 0x7f85600a56a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8568005d00 tx=0x7f856800b410 comp rx=0 tx=0).stop 2024-09-19T07:31:23.586 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 msgr2=0x7f85781a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:23.586 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f85781a5990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8570005e30 tx=0x7f8570031ee0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.586 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 shutdown_connections 2024-09-19T07:31:23.586 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85600a31e0 0x7f85600a56a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.587 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8578105790 0x7f85781a5990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.587 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578102de0 0x7f857819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.587 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 --2- 172.21.15.174:0/2578531319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8578100490 0x7f857819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:23.587 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 >> 172.21.15.174:0/2578531319 conn(0x7f85780fc2a0 msgr2=0x7f8578100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:23.587 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 shutdown_connections 2024-09-19T07:31:23.587 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:31:23.586+0000 7f857dafe640 1 -- 172.21.15.174:0/2578531319 wait complete. 2024-09-19T07:31:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:23 smithi103 ceph-mon[31201]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:23 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:24.171 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T07:31:24.173 INFO:tasks.cephadm:Deploying osd.3 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-19T07:31:24.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T07:31:24.418 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:24.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:24.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:24.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:24.496 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:24.496 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:24.496 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:25.484 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:25 smithi174 ceph-mon[35370]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:25.509 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:25 smithi134 ceph-mon[35414]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:25.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:25 smithi103 ceph-mon[31201]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:26 smithi174 ceph-mon[35370]: from='osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:31:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:26 smithi174 ceph-mon[35370]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:31:26.416 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:26 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2[40354]: 2024-09-19T07:31:26.136+0000 7fcf5583c640 -1 osd.2 0 log_to_monitors true 2024-09-19T07:31:26.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:26 smithi103 ceph-mon[31201]: from='osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:31:26.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:26 smithi103 ceph-mon[31201]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:31:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:26 smithi134 ceph-mon[35414]: from='osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:31:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:26 smithi134 ceph-mon[35414]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T07:31:27.020 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:31:27.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-19T07:31:27.298 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:31:27.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:27.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T07:31:27.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.561 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2[40354]: 2024-09-19T07:31:27.244+0000 7fcf517bf640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T07:31:27.561 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2[40354]: 2024-09-19T07:31:27.248+0000 7fcf4d5e7640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:27.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:27.563 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T07:31:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T07:31:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:27.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:27.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 -- 172.21.15.103:0/3564735494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 msgr2=0x7f9ad8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/3564735494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad8108f60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009ca0 tx=0x7f9ad002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 -- 172.21.15.103:0/3564735494 shutdown_connections 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/3564735494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad8108f60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/3564735494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad8101d10 0x7f9ad8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/3564735494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ad8100b10 0x7f9ad8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 -- 172.21.15.103:0/3564735494 >> 172.21.15.103:0/3564735494 conn(0x7f9ad80fc2a0 msgr2=0x7f9ad80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 -- 172.21.15.103:0/3564735494 shutdown_connections 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 -- 172.21.15.103:0/3564735494 wait complete. 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 Processor -- start 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.761+0000 7f9adfb6f640 1 -- start start 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9adfb6f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad8100b10 0x7f9ad810b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9adfb6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ad8101d10 0x7f9ad8109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9adfb6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad810a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9adfb6f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d940 con 0x7f9ad81026d0 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9adfb6f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d7c0 con 0x7f9ad8100b10 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9adfb6f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810dac0 con 0x7f9ad8101d10 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad810a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad810a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53520/0 (socket says 172.21.15.103:53520) 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 -- 172.21.15.103:0/1825956992 learned_addr learned my addr 172.21.15.103:0/1825956992 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9add10a640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ad8101d10 0x7f9ad8109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9add90b640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad8100b10 0x7f9ad810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 -- 172.21.15.103:0/1825956992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ad8101d10 msgr2=0x7f9ad8109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ad8101d10 0x7f9ad8109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 -- 172.21.15.103:0/1825956992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad8100b10 msgr2=0x7f9ad810b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad8100b10 0x7f9ad810b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 -- 172.21.15.103:0/1825956992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad810a8e0 con 0x7f9ad81026d0 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.762+0000 7f9ade10c640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad810a020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00319f0 tx=0x7f9ad002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.763+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0003c70 con 0x7f9ad81026d0 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.763+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad81a2030 con 0x7f9ad81026d0 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.763+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ad0003e10 con 0x7f9ad81026d0 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.763+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad002fcc0 con 0x7f9ad81026d0 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.763+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81a2610 con 0x7f9ad81026d0 2024-09-19T07:31:27.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.764+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad8107ec0 con 0x7f9ad81026d0 2024-09-19T07:31:27.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.765+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f9ad0004040 con 0x7f9ad81026d0 2024-09-19T07:31:27.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.765+0000 7f9aceffd640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9ac40a3230 0x7f9ac40a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:27.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.765+0000 7f9add90b640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9ac40a3230 0x7f9ac40a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:27.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.765+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f9ad00ef7d0 con 0x7f9ad81026d0 2024-09-19T07:31:27.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.766+0000 7f9add90b640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9ac40a3230 0x7f9ac40a56f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8101b70 tx=0x7f9ad400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:27.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.769+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9ad00bdc40 con 0x7f9ad81026d0 2024-09-19T07:31:27.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:27.943+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9ad8062630 con 0x7f9ac40a3230 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi174 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:28.513 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi174 2024-09-19T07:31:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T07:31:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:31:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T07:31:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi174 2024-09-19T07:31:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-19T07:31:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:31:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T07:31:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:28.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:28.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:28.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:28.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:29.428 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:31:29.428 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:31:29.429 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:29.429 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:29.429 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:29.429 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] boot 2024-09-19T07:31:29.429 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T07:31:29.429 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:29 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:31:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:31:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:29.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:29.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] boot 2024-09-19T07:31:29.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T07:31:29.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:29 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:31:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:31:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T07:31:29.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:29.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: osd.2 [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] boot 2024-09-19T07:31:29.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T07:31:29.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:29 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:31:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:30 smithi103 ceph-mon[31201]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T07:31:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:30 smithi134 ceph-mon[35414]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T07:31:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:30 smithi174 ceph-mon[35370]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T07:31:31.064 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:30 smithi103 sudo[49614]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:31:31.064 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:30 smithi103 sudo[49614]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:31.064 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:30 smithi103 sudo[49614]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:31.064 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:30 smithi103 sudo[49614]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:31.470 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:31 smithi103 sudo[49625]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:31:31.470 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:31 smithi103 sudo[49625]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:31.470 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:31 smithi103 sudo[49625]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:31.470 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:31:31 smithi103 sudo[49625]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:31:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T07:31:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2203241304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b1b00f7-68c9-4888-81f7-2f2544a97c0d"}]: dispatch 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2203241304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b1b00f7-68c9-4888-81f7-2f2544a97c0d"}]': finished 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3097397044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:31.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:31.579 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:31:31.579 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:31.579 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44003]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:31.579 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44003]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2203241304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b1b00f7-68c9-4888-81f7-2f2544a97c0d"}]: dispatch 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2203241304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b1b00f7-68c9-4888-81f7-2f2544a97c0d"}]': finished 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3097397044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:31.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:31:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: osdmap e20: 3 total, 3 up, 3 in 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2203241304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b1b00f7-68c9-4888-81f7-2f2544a97c0d"}]: dispatch 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2203241304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b1b00f7-68c9-4888-81f7-2f2544a97c0d"}]': finished 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3097397044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:31.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:32.079 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:31:32.079 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:32.079 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44006]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:32.079 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:31:31 smithi134 sudo[44006]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:32.186 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:31 smithi174 sudo[44144]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:31:32.186 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:31 smithi174 sudo[44144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:32.186 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:31 smithi174 sudo[44144]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:32.186 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:31 smithi174 sudo[44144]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:32.665 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44147]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:31:32.665 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44147]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:32.665 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44147]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:32.665 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44147]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:32.711 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:31:32.711 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:32.711 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50000]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:32.711 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50000]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:32.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:32.728+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9ad00b9bb0 con 0x7f9ad81026d0 2024-09-19T07:31:32.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50038]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:31:32.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50038]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:32.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50038]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:32.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 sudo[50038]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:32.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 ceph-mon[31201]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T07:31:32.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:32.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:32.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:32.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:32 smithi103 ceph-mon[31201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 ceph-mon[35370]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 ceph-mon[35370]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44150]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44150]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44150]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:32.976 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:32 smithi174 sudo[44150]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:33.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:32 smithi134 ceph-mon[35414]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T07:31:33.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:32 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:33.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:32 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:33.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:32 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:33.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:32 smithi134 ceph-mon[35414]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 sudo[44153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:31:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 sudo[44153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 sudo[44153]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:33.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 sudo[44153]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:33.716 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44009]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T07:31:33.716 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:33.716 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44009]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:33.716 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44009]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44012]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44012]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44012]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 sudo[44012]: pam_unix(sudo:session): session closed for user root 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:33 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:34.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:34.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:33 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:34.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T07:31:34.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:34.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:31:34.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:31:34.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:33 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:31:35.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:34 smithi134 ceph-mon[35414]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:35.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:34 smithi134 ceph-mon[35414]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:35.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:34 smithi134 ceph-mon[35414]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:35.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:34 smithi174 ceph-mon[35370]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:35.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:34 smithi174 ceph-mon[35370]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:35.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:34 smithi174 ceph-mon[35370]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:35.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:34 smithi103 ceph-mon[31201]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T07:31:35.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:34 smithi103 ceph-mon[31201]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T07:31:35.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:34 smithi103 ceph-mon[31201]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:37.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:37 smithi103 ceph-mon[31201]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:37 smithi174 ceph-mon[35370]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:37 smithi134 ceph-mon[35414]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:39 smithi174 ceph-mon[35370]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T07:31:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:39.435 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:39 smithi103 ceph-mon[31201]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:39.435 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T07:31:39.436 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:39 smithi134 ceph-mon[35414]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T07:31:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:40 smithi174 ceph-mon[35370]: Deploying daemon osd.3 on smithi103 2024-09-19T07:31:40.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:40 smithi103 ceph-mon[31201]: Deploying daemon osd.3 on smithi103 2024-09-19T07:31:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:40 smithi134 ceph-mon[35414]: Deploying daemon osd.3 on smithi103 2024-09-19T07:31:41.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:41 smithi174 ceph-mon[35370]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:41.458 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:41 smithi103 ceph-mon[31201]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:41.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:41 smithi134 ceph-mon[35414]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:43.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:43 smithi174 ceph-mon[35370]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:43 smithi103 ceph-mon[31201]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:43.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:43 smithi134 ceph-mon[35414]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:45.193 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:45 smithi103 ceph-mon[31201]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:45.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:45 smithi174 ceph-mon[35370]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:45 smithi134 ceph-mon[35414]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:46.160 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:46 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:46.161 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:46 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:46.161 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:46 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:46 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:46 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:46 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:46 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:46 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:46 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:47.176 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:47 smithi103 ceph-mon[31201]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:47.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:47 smithi174 ceph-mon[35370]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:47.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:47 smithi134 ceph-mon[35414]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:49.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:49 smithi134 ceph-mon[35414]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:49.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:49 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:49 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:49 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:49.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:49 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:49.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:49 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.629 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:49 smithi103 ceph-mon[31201]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:49.629 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:49 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.629 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:49 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.629 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:49 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:49.629 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:49 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:49.629 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:49 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:49 smithi174 ceph-mon[35370]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:49.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:49 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:49 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:49 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:49.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:49 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:49.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:49 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:49.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.668+0000 7f9aceffd640 1 -- 172.21.15.103:0/1825956992 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ad8062630 con 0x7f9ac40a3230 2024-09-19T07:31:49.669 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 3 on host 'smithi103' 2024-09-19T07:31:49.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9ac40a3230 msgr2=0x7f9ac40a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:49.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9ac40a3230 0x7f9ac40a56f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8101b70 tx=0x7f9ad400b390 comp rx=0 tx=0).stop 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 msgr2=0x7f9ad810a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad810a020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00319f0 tx=0x7f9ad002f450 comp rx=0 tx=0).stop 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 shutdown_connections 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9ac40a3230 0x7f9ac40a56f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ad81026d0 0x7f9ad810a020 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ad8101d10 0x7f9ad8109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 --2- 172.21.15.103:0/1825956992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad8100b10 0x7f9ad810b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:49.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 >> 172.21.15.103:0/1825956992 conn(0x7f9ad80fc2a0 msgr2=0x7f9ad8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:49.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 shutdown_connections 2024-09-19T07:31:49.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:31:49.670+0000 7f9adfb6f640 1 -- 172.21.15.103:0/1825956992 wait complete. 2024-09-19T07:31:50.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:50 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:50.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:50 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:50.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:50 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:50.416 DEBUG:teuthology.orchestra.run.smithi103:osd.3> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T07:31:50.419 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_3... 2024-09-19T07:31:50.419 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T07:31:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:50 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:50 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:50 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:50.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:50 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:31:50.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:50 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:50.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:50 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:50.673 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:31:51.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:51 smithi134 ceph-mon[35414]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:51.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:51 smithi174 ceph-mon[35370]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:52.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:51 smithi103 ceph-mon[31201]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:52.290 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:31:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3[51654]: 2024-09-19T07:31:51.260+0000 7f98dcbdd640 -1 osd.3 0 log_to_monitors true 2024-09-19T07:31:52.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:52 smithi103 ceph-mon[31201]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T07:31:52.525 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:52 smithi134 ceph-mon[35414]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T07:31:52.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:52 smithi174 ceph-mon[35370]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T07:31:53.262 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:31:53.292 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-09-19T07:31:53.470 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:31:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3[51654]: 2024-09-19T07:31:53.263+0000 7f98d8b60640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T07:31:53.470 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:31:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3[51654]: 2024-09-19T07:31:53.267+0000 7f98d4988640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:31:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:53 smithi103 ceph-mon[31201]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:53 smithi103 ceph-mon[31201]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T07:31:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:53 smithi103 ceph-mon[31201]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T07:31:53.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:53 smithi103 ceph-mon[31201]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:31:53.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:53.539 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:31:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:53 smithi134 ceph-mon[35414]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:53 smithi134 ceph-mon[35414]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T07:31:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:53 smithi134 ceph-mon[35414]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T07:31:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:53 smithi134 ceph-mon[35414]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:31:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:53.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:53 smithi174 ceph-mon[35370]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:53.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:53 smithi174 ceph-mon[35370]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T07:31:53.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:53 smithi174 ceph-mon[35370]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T07:31:53.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:53 smithi174 ceph-mon[35370]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:31:53.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:54.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 -- 172.21.15.134:0/3287519195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 msgr2=0x7fe3bc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:54.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 --2- 172.21.15.134:0/3287519195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc102190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009900 tx=0x7fe3ac02a410 comp rx=0 tx=0).stop 2024-09-19T07:31:54.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 -- 172.21.15.134:0/3287519195 shutdown_connections 2024-09-19T07:31:54.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 --2- 172.21.15.134:0/3287519195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3bc1026d0 0x7fe3bc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 --2- 172.21.15.134:0/3287519195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc102190 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 --2- 172.21.15.134:0/3287519195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3bc100b10 0x7fe3bc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 -- 172.21.15.134:0/3287519195 >> 172.21.15.134:0/3287519195 conn(0x7fe3bc0fc2a0 msgr2=0x7fe3bc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 -- 172.21.15.134:0/3287519195 shutdown_connections 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.001+0000 7fe3c458b640 1 -- 172.21.15.134:0/3287519195 wait complete. 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 Processor -- start 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 -- start start 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3bc100b10 0x7fe3bc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3bc1026d0 0x7fe3bc19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc10d8c0 con 0x7fe3bc100b10 2024-09-19T07:31:54.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc10d740 con 0x7fe3bc101d10 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c458b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc10da40 con 0x7fe3bc1026d0 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c1b26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.002+0000 7fe3c1b26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37768/0 (socket says 172.21.15.134:37768) 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c1b26640 1 -- 172.21.15.134:0/9187035 learned_addr learned my addr 172.21.15.134:0/9187035 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c2b28640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3bc1026d0 0x7fe3bc19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c2327640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3bc100b10 0x7fe3bc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c1b26640 1 -- 172.21.15.134:0/9187035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3bc1026d0 msgr2=0x7fe3bc19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c1b26640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3bc1026d0 0x7fe3bc19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c1b26640 1 -- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3bc100b10 msgr2=0x7fe3bc197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c1b26640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3bc100b10 0x7fe3bc197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c1b26640 1 -- 172.21.15.134:0/9187035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc19fb50 con 0x7fe3bc101d10 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c2327640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3bc100b10 0x7fe3bc197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c1b26640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc198460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009230 tx=0x7fe3ac004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3b37fe640 1 -- 172.21.15.134:0/9187035 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac03f030 con 0x7fe3bc101d10 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3b37fe640 1 -- 172.21.15.134:0/9187035 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3ac003ce0 con 0x7fe3bc101d10 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc19fde0 con 0x7fe3bc101d10 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3b37fe640 1 -- 172.21.15.134:0/9187035 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac035070 con 0x7fe3bc101d10 2024-09-19T07:31:54.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.003+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc1a0260 con 0x7fe3bc101d10 2024-09-19T07:31:54.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.004+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc107f60 con 0x7fe3bc101d10 2024-09-19T07:31:54.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.005+0000 7fe3b37fe640 1 -- 172.21.15.134:0/9187035 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe3ac042030 con 0x7fe3bc101d10 2024-09-19T07:31:54.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.006+0000 7fe3b37fe640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe3a80a3270 0x7fe3a80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:31:54.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.006+0000 7fe3b37fe640 1 -- 172.21.15.134:0/9187035 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7fe3ac0eebe0 con 0x7fe3bc101d10 2024-09-19T07:31:54.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.006+0000 7fe3c2327640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe3a80a3270 0x7fe3a80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:31:54.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.006+0000 7fe3c2327640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe3a80a3270 0x7fe3a80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc101b70 tx=0x7fe3b800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:31:54.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.010+0000 7fe3b37fe640 1 -- 172.21.15.134:0/9187035 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3ac0bcc40 con 0x7fe3bc101d10 2024-09-19T07:31:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:31:54.183+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe3bc062630 con 0x7fe3a80a3270 2024-09-19T07:31:54.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:31:54.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T07:31:54.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:54.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:54.468 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi103 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:54.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:54.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:54.470 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:31:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T07:31:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi103 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:54.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:54.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:54.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi103 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:31:54.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-19T07:31:54.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:54.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:31:54.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:31:54.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-19T07:31:54.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:31:54.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:31:54.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:31:54.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:31:55.543 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:55 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:31:55.543 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:55 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:31:55.543 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:55 smithi134 ceph-mon[35414]: osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] boot 2024-09-19T07:31:55.543 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:55 smithi134 ceph-mon[35414]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T07:31:55.543 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:55 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:55.544 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:55 smithi134 ceph-mon[35414]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T07:31:55.566 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:55 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:31:55.566 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:55 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:31:55.566 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:55 smithi174 ceph-mon[35370]: osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] boot 2024-09-19T07:31:55.566 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:55 smithi174 ceph-mon[35370]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T07:31:55.567 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:55 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:55.567 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:55 smithi174 ceph-mon[35370]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T07:31:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:55 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:31:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:55 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:31:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:55 smithi103 ceph-mon[31201]: osd.3 [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] boot 2024-09-19T07:31:55.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:55 smithi103 ceph-mon[31201]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T07:31:55.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:55 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:31:55.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:55 smithi103 ceph-mon[31201]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T07:31:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:56 smithi134 ceph-mon[35414]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:31:56.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:56 smithi134 ceph-mon[35414]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T07:31:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:56 smithi174 ceph-mon[35370]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:31:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:56 smithi174 ceph-mon[35370]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T07:31:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:56 smithi103 ceph-mon[31201]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:31:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:56 smithi103 ceph-mon[31201]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T07:31:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/3849291527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]: dispatch 2024-09-19T07:31:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:57 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]: dispatch 2024-09-19T07:31:57.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:57 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]': finished 2024-09-19T07:31:57.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:57 smithi134 ceph-mon[35414]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T07:31:57.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:57.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/679191804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:57.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:57 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/3849291527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]: dispatch 2024-09-19T07:31:57.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:57 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]: dispatch 2024-09-19T07:31:57.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:57 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]': finished 2024-09-19T07:31:57.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:57 smithi174 ceph-mon[35370]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T07:31:57.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:57 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:57.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:57 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/679191804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:57.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:57 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/3849291527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]: dispatch 2024-09-19T07:31:57.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:57 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]: dispatch 2024-09-19T07:31:57.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:57 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee"}]': finished 2024-09-19T07:31:57.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:57 smithi103 ceph-mon[31201]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T07:31:57.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:57 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:57.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:57 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/679191804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:31:58.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:58 smithi134 ceph-mon[35414]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T07:31:58.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:58 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:58.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:58 smithi134 ceph-mon[35414]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:31:58.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:58 smithi174 ceph-mon[35370]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T07:31:58.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:58 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:58.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:58 smithi174 ceph-mon[35370]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:31:58.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:58 smithi103 ceph-mon[31201]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T07:31:58.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:58 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:58.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:58 smithi103 ceph-mon[31201]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:31:59.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:59 smithi134 ceph-mon[35414]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T07:31:59.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:31:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:59.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:59 smithi174 ceph-mon[35370]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T07:31:59.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:31:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:31:59.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:59 smithi103 ceph-mon[31201]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T07:31:59.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:31:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:00.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:00 smithi134 ceph-mon[35414]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:00.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:00 smithi174 ceph-mon[35370]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:00.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:00 smithi103 ceph-mon[31201]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:03.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:03 smithi134 ceph-mon[35414]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-19T07:32:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:03 smithi174 ceph-mon[35370]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-19T07:32:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:03 smithi103 ceph-mon[31201]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-19T07:32:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:05 smithi134 ceph-mon[35414]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T07:32:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T07:32:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:05 smithi174 ceph-mon[35370]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T07:32:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T07:32:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:05 smithi103 ceph-mon[31201]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-19T07:32:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T07:32:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:06.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:06 smithi134 ceph-mon[35414]: Deploying daemon osd.4 on smithi134 2024-09-19T07:32:06.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:06 smithi174 ceph-mon[35370]: Deploying daemon osd.4 on smithi134 2024-09-19T07:32:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:06 smithi103 ceph-mon[31201]: Deploying daemon osd.4 on smithi134 2024-09-19T07:32:07.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:07 smithi174 ceph-mon[35370]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T07:32:07.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:07 smithi134 ceph-mon[35414]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T07:32:07.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:07 smithi103 ceph-mon[31201]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-19T07:32:09.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:09 smithi174 ceph-mon[35370]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:32:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:09 smithi103 ceph-mon[31201]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:32:09.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:09 smithi134 ceph-mon[35414]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:32:11.281 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:11 smithi134 ceph-mon[35414]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T07:32:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:11 smithi174 ceph-mon[35370]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T07:32:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:11 smithi103 ceph-mon[31201]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T07:32:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:13.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:13 smithi174 ceph-mon[35370]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:32:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:13 smithi103 ceph-mon[31201]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:32:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:13 smithi134 ceph-mon[35414]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:32:15.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:15 smithi134 ceph-mon[35414]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:15.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:15.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:15.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:15 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:15 smithi174 ceph-mon[35370]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:15 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.462+0000 7fe3b37fe640 1 -- 172.21.15.134:0/9187035 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe3bc062630 con 0x7fe3a80a3270 2024-09-19T07:32:15.464 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2024-09-19T07:32:15.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe3a80a3270 msgr2=0x7fe3a80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:15.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe3a80a3270 0x7fe3a80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc101b70 tx=0x7fe3b800a560 comp rx=0 tx=0).stop 2024-09-19T07:32:15.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 msgr2=0x7fe3bc198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:15.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc198460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009230 tx=0x7fe3ac004360 comp rx=0 tx=0).stop 2024-09-19T07:32:15.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 shutdown_connections 2024-09-19T07:32:15.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe3a80a3270 0x7fe3a80a5730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:15.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3bc1026d0 0x7fe3bc19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:15.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3bc101d10 0x7fe3bc198460 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:15.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 --2- 172.21.15.134:0/9187035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3bc100b10 0x7fe3bc197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:15.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 >> 172.21.15.134:0/9187035 conn(0x7fe3bc0fc2a0 msgr2=0x7fe3bc105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:15.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 shutdown_connections 2024-09-19T07:32:15.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:32:15.465+0000 7fe3c458b640 1 -- 172.21.15.134:0/9187035 wait complete. 2024-09-19T07:32:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:15 smithi103 ceph-mon[31201]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:15 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:16.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:16.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:16.169 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:16.201 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T07:32:16.204 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-09-19T07:32:16.204 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T07:32:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:16.456 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:32:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:17.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:17 smithi174 ceph-mon[35370]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:17.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:17 smithi174 ceph-mon[35370]: from='osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:32:17.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:17 smithi174 ceph-mon[35370]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:32:17.209 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:17 smithi134 ceph-mon[35414]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:17.209 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:17 smithi134 ceph-mon[35414]: from='osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:32:17.209 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:17 smithi134 ceph-mon[35414]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:32:17.209 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:32:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4[46757]: 2024-09-19T07:32:16.967+0000 7f88c7dde640 -1 osd.4 0 log_to_monitors true 2024-09-19T07:32:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:17 smithi103 ceph-mon[31201]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:17 smithi103 ceph-mon[31201]: from='osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:32:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:17 smithi103 ceph-mon[31201]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T07:32:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:18 smithi174 ceph-mon[35370]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T07:32:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:18 smithi174 ceph-mon[35370]: from='osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:32:18.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:18 smithi174 ceph-mon[35370]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T07:32:18.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:18 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:18.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:18 smithi174 ceph-mon[35370]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:32:18.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:18 smithi174 ceph-mon[35370]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:18.775 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-mon[35414]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T07:32:18.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-mon[35414]: from='osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:32:18.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-mon[35414]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T07:32:18.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:18.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-mon[35414]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:32:18.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-mon[35414]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:18.829 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:32:18.856 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-09-19T07:32:18.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:18 smithi103 ceph-mon[31201]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T07:32:18.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:18 smithi103 ceph-mon[31201]: from='osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:32:18.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:18 smithi103 ceph-mon[31201]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T07:32:18.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:18 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:18.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:18 smithi103 ceph-mon[31201]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:32:18.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:18 smithi103 ceph-mon[31201]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:19.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4[46757]: 2024-09-19T07:32:18.869+0000 7f88c3d61640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T07:32:19.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:32:18 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4[46757]: 2024-09-19T07:32:18.875+0000 7f88bbb81640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:32:19.106 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:32:19.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:32:19.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T07:32:19.528 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi134 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.529 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:19 smithi174 ceph-mon[35370]: from='osd.4 ' entity='osd.4' 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 -- 172.21.15.174:0/2771297689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494105030 msgr2=0x7f0494107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 --2- 172.21.15.174:0/2771297689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494105030 0x7f0494107420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0484009900 tx=0x7f048402a410 comp rx=0 tx=0).stop 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 -- 172.21.15.174:0/2771297689 shutdown_connections 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 --2- 172.21.15.174:0/2771297689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0494107960 0x7f0494109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 --2- 172.21.15.174:0/2771297689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494105030 0x7f0494107420 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 --2- 172.21.15.174:0/2771297689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0494102700 0x7f0494104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 -- 172.21.15.174:0/2771297689 >> 172.21.15.174:0/2771297689 conn(0x7f04940fc2a0 msgr2=0x7f04940fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 -- 172.21.15.174:0/2771297689 shutdown_connections 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 -- 172.21.15.174:0/2771297689 wait complete. 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 Processor -- start 2024-09-19T07:32:19.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.613+0000 7f049ba11640 1 -- start start 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f049ba11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494102700 0x7f0494102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f049ba11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0494105030 0x7f04941006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f049ba11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0494107960 0x7f0494100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f049ba11640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f049410d890 con 0x7f0494107960 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f049ba11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f049410d710 con 0x7f0494105030 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f049ba11640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f049410da10 con 0x7f0494102700 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494102700 0x7f0494102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494102700 0x7f0494102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:50488/0 (socket says 172.21.15.174:50488) 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 -- 172.21.15.174:0/3568181647 learned_addr learned my addr 172.21.15.174:0/3568181647 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f0498fac640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0494105030 0x7f04941006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:19.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f0499fae640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0494107960 0x7f0494100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 -- 172.21.15.174:0/3568181647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0494105030 msgr2=0x7f04941006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0494105030 0x7f04941006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 -- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0494107960 msgr2=0x7f0494100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0494107960 0x7f0494100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 -- 172.21.15.174:0/3568181647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04941014b0 con 0x7f0494102700 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f04997ad640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494102700 0x7f0494102060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f049000d950 tx=0x7f049000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f048a7fc640 1 -- 172.21.15.174:0/3568181647 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490017070 con 0x7f0494102700 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.614+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04941a20a0 con 0x7f0494102700 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.615+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04941a2600 con 0x7f0494102700 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.615+0000 7f048a7fc640 1 -- 172.21.15.174:0/3568181647 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0490009c30 con 0x7f0494102700 2024-09-19T07:32:19.617 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.615+0000 7f048a7fc640 1 -- 172.21.15.174:0/3568181647 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049000f6b0 con 0x7f0494102700 2024-09-19T07:32:19.618 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.616+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04940fdd90 con 0x7f0494102700 2024-09-19T07:32:19.618 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.617+0000 7f048a7fc640 1 -- 172.21.15.174:0/3568181647 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f04900109d0 con 0x7f0494102700 2024-09-19T07:32:19.618 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.617+0000 7f048a7fc640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04800a3240 0x7f04800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:19.618 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.617+0000 7f048a7fc640 1 -- 172.21.15.174:0/3568181647 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 3821+0+0 (secure 0 0 0) 0x7f04900ce3a0 con 0x7f0494102700 2024-09-19T07:32:19.618 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.618+0000 7f0498fac640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04800a3240 0x7f04800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:19.619 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.619+0000 7f0498fac640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04800a3240 0x7f04800a5700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0484005ab0 tx=0x7f0484000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:19.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.622+0000 7f048a7fc640 1 -- 172.21.15.174:0/3568181647 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0490009de0 con 0x7f0494102700 2024-09-19T07:32:19.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:19.796+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0494062630 con 0x7f04800a3240 2024-09-19T07:32:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:32:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T07:32:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi134 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:19 smithi134 ceph-mon[35414]: from='osd.4 ' entity='osd.4' 2024-09-19T07:32:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:32:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: osdmap e32: 5 total, 4 up, 5 in 2024-09-19T07:32:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi134 2024-09-19T07:32:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi134 to 8544M 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:19.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:19 smithi103 ceph-mon[31201]: from='osd.4 ' entity='osd.4' 2024-09-19T07:32:20.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:32:20.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] boot 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:20.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:20 smithi134 ceph-mon[35414]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:20.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] boot 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:20.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:20 smithi174 ceph-mon[35370]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:32:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:32:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:32:20.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:32:20.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:20.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: osd.4 [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] boot 2024-09-19T07:32:20.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T07:32:20.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:32:20.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:20 smithi103 ceph-mon[31201]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-19T07:32:22.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:21 smithi174 ceph-mon[35370]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T07:32:22.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:21 smithi103 ceph-mon[31201]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T07:32:22.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:21 smithi134 ceph-mon[35414]: osdmap e34: 5 total, 5 up, 5 in 2024-09-19T07:32:23.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:22 smithi174 ceph-mon[35370]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:23.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:22 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/436591294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]: dispatch 2024-09-19T07:32:23.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:22 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]: dispatch 2024-09-19T07:32:23.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:22 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]': finished 2024-09-19T07:32:23.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:22 smithi174 ceph-mon[35370]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T07:32:23.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:22 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:23.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:22 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/34979967' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:32:23.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:22 smithi103 ceph-mon[31201]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:23.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:22 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/436591294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]: dispatch 2024-09-19T07:32:23.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:22 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]: dispatch 2024-09-19T07:32:23.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:22 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]': finished 2024-09-19T07:32:23.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:22 smithi103 ceph-mon[31201]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T07:32:23.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:22 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:23.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:22 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/34979967' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:32:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:22 smithi134 ceph-mon[35414]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:22 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/436591294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]: dispatch 2024-09-19T07:32:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:22 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]: dispatch 2024-09-19T07:32:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:22 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14cf312f-9b50-4214-8f5d-2b4d68af438f"}]': finished 2024-09-19T07:32:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:22 smithi134 ceph-mon[35414]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T07:32:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:22 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:23.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:22 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/34979967' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:32:25.408 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:25 smithi134 ceph-mon[35414]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:25 smithi174 ceph-mon[35370]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:25 smithi103 ceph-mon[31201]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:27.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:27 smithi174 ceph-mon[35370]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:27 smithi103 ceph-mon[31201]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:27 smithi134 ceph-mon[35414]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:29.363 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:29 smithi174 ceph-mon[35370]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:29 smithi103 ceph-mon[31201]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:29 smithi134 ceph-mon[35414]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:31.300 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:31 smithi174 ceph-mon[35370]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:31.300 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T07:32:31.300 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:31 smithi103 ceph-mon[31201]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T07:32:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:31 smithi134 ceph-mon[35414]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T07:32:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:32.248 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:32 smithi174 ceph-mon[35370]: Deploying daemon osd.5 on smithi174 2024-09-19T07:32:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:32 smithi103 ceph-mon[31201]: Deploying daemon osd.5 on smithi174 2024-09-19T07:32:32.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:32 smithi134 ceph-mon[35414]: Deploying daemon osd.5 on smithi174 2024-09-19T07:32:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:33 smithi174 ceph-mon[35370]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:33.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:33 smithi103 ceph-mon[31201]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:33 smithi134 ceph-mon[35414]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:35.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:35 smithi174 ceph-mon[35370]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:35 smithi103 ceph-mon[31201]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:35.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:35 smithi134 ceph-mon[35414]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:37 smithi174 ceph-mon[35370]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:37 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:37 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:37 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:37 smithi103 ceph-mon[31201]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:37 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:37 smithi134 ceph-mon[35414]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:37 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:37 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:37 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:39 smithi174 ceph-mon[35370]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:39 smithi103 ceph-mon[31201]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:39 smithi134 ceph-mon[35414]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:40.988 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:40.988 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:40.988 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:40.988 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:40.988 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:40.988 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:40 smithi174 ceph-mon[35370]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:41.029 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.029+0000 7f048a7fc640 1 -- 172.21.15.174:0/3568181647 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0494062630 con 0x7f04800a3240 2024-09-19T07:32:41.030 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-09-19T07:32:41.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04800a3240 msgr2=0x7f04800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:41.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04800a3240 0x7f04800a5700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0484005ab0 tx=0x7f0484000bc0 comp rx=0 tx=0).stop 2024-09-19T07:32:41.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494102700 msgr2=0x7f0494102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:41.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494102700 0x7f0494102060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f049000d950 tx=0x7f049000de10 comp rx=0 tx=0).stop 2024-09-19T07:32:41.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 shutdown_connections 2024-09-19T07:32:41.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04800a3240 0x7f04800a5700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:41.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0494107960 0x7f0494100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:41.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0494105030 0x7f04941006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:41.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 --2- 172.21.15.174:0/3568181647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0494102700 0x7f0494102060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:41.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 >> 172.21.15.174:0/3568181647 conn(0x7f04940fc2a0 msgr2=0x7f0494103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:41.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.031+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 shutdown_connections 2024-09-19T07:32:41.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:32:41.032+0000 7f049ba11640 1 -- 172.21.15.174:0/3568181647 wait complete. 2024-09-19T07:32:41.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:41.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:41.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:40 smithi134 ceph-mon[35414]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:41.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:41.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:40 smithi103 ceph-mon[31201]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:41.669 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T07:32:41.672 INFO:tasks.cephadm:Deploying osd.6 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-19T07:32:41.672 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T07:32:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:41 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:41 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:41 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:41.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:41 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:41.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:41 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:41.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:41 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:42.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:41 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:32:42.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:41 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:42.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:41 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:42.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:32:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:32:42.126+0000 7fbf40230640 -1 osd.5 0 log_to_monitors true 2024-09-19T07:32:43.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:42 smithi134 ceph-mon[35414]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:43.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:42 smithi134 ceph-mon[35414]: from='osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:32:43.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:42 smithi134 ceph-mon[35414]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:32:43.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:42 smithi103 ceph-mon[31201]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:43.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:42 smithi103 ceph-mon[31201]: from='osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:32:43.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:42 smithi103 ceph-mon[31201]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:32:43.138 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:42 smithi174 ceph-mon[35370]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:43.138 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:42 smithi174 ceph-mon[35370]: from='osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:32:43.138 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:42 smithi174 ceph-mon[35370]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T07:32:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:43 smithi134 ceph-mon[35414]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T07:32:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:43 smithi134 ceph-mon[35414]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T07:32:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:43 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:43 smithi134 ceph-mon[35414]: from='osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:32:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:43 smithi134 ceph-mon[35414]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:32:44.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:43 smithi174 ceph-mon[35370]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T07:32:44.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:43 smithi174 ceph-mon[35370]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T07:32:44.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:43 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:44.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:43 smithi174 ceph-mon[35370]: from='osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:32:44.088 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:43 smithi174 ceph-mon[35370]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:32:44.088 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:32:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:32:43.809+0000 7fbf3c1b3640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T07:32:44.088 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:32:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:32:43.818+0000 7fbf37fdb640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:32:44.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:43 smithi103 ceph-mon[31201]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T07:32:44.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:43 smithi103 ceph-mon[31201]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T07:32:44.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:44.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:43 smithi103 ceph-mon[31201]: from='osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:32:44.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:43 smithi103 ceph-mon[31201]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:32:44.542 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:32:44.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-19T07:32:44.812 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:44.934 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] boot 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T07:32:44.935 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:45.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:32:45.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] boot 2024-09-19T07:32:45.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T07:32:45.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:45.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:32:45.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: osdmap e37: 6 total, 5 up, 6 in 2024-09-19T07:32:45.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:45.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: osd.5 [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] boot 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T07:32:45.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 -- 172.21.15.103:0/595314884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c1026d0 msgr2=0x7fb07c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 --2- 172.21.15.103:0/595314884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c1026d0 0x7fb07c108f60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb074009ca0 tx=0x7fb07402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 -- 172.21.15.103:0/595314884 shutdown_connections 2024-09-19T07:32:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 --2- 172.21.15.103:0/595314884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c1026d0 0x7fb07c108f60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 --2- 172.21.15.103:0/595314884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb07c101d10 0x7fb07c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 --2- 172.21.15.103:0/595314884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c100b10 0x7fb07c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 -- 172.21.15.103:0/595314884 >> 172.21.15.103:0/595314884 conn(0x7fb07c0fc2a0 msgr2=0x7fb07c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 -- 172.21.15.103:0/595314884 shutdown_connections 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.292+0000 7fb082dcf640 1 -- 172.21.15.103:0/595314884 wait complete. 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 Processor -- start 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 -- start start 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c100b10 0x7fb07c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101d10 0x7fb07c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb07c1026d0 0x7fb07c19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c10c770 con 0x7fb07c101d10 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c10c5f0 con 0x7fb07c100b10 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb082dcf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c10c8f0 con 0x7fb07c1026d0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101d10 0x7fb07c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101d10 0x7fb07c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55274/0 (socket says 172.21.15.103:55274) 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 -- 172.21.15.103:0/4196810587 learned_addr learned my addr 172.21.15.103:0/4196810587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb08136c640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb07c1026d0 0x7fb07c19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb080b6b640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c100b10 0x7fb07c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 -- 172.21.15.103:0/4196810587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb07c1026d0 msgr2=0x7fb07c19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb07c1026d0 0x7fb07c19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 -- 172.21.15.103:0/4196810587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c100b10 msgr2=0x7fb07c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:32:45.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c100b10 0x7fb07c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb07bfff640 1 -- 172.21.15.103:0/4196810587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c19fa30 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.293+0000 7fb080b6b640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c100b10 0x7fb07c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.294+0000 7fb07bfff640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101d10 0x7fb07c198340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb06800c920 tx=0x7fb06800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.294+0000 7fb079ffb640 1 -- 172.21.15.103:0/4196810587 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068007c70 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.294+0000 7fb079ffb640 1 -- 172.21.15.103:0/4196810587 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb068007e10 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.294+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c19fd20 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.294+0000 7fb079ffb640 1 -- 172.21.15.103:0/4196810587 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068018890 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.294+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c1a0250 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.295+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07c1a04e0 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.296+0000 7fb079ffb640 1 -- 172.21.15.103:0/4196810587 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb068018a30 con 0x7fb07c101d10 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.297+0000 7fb079ffb640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb06c0a32f0 0x7fb06c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.297+0000 7fb080b6b640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb06c0a32f0 0x7fb06c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:32:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.297+0000 7fb079ffb640 1 -- 172.21.15.103:0/4196810587 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4566+0+0 (secure 0 0 0) 0x7fb0680cedd0 con 0x7fb07c101d10 2024-09-19T07:32:45.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.297+0000 7fb080b6b640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb06c0a32f0 0x7fb06c0a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb0700058c0 tx=0x7fb070005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:32:45.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.301+0000 7fb079ffb640 1 -- 172.21.15.103:0/4196810587 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb068014030 con 0x7fb07c101d10 2024-09-19T07:32:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:32:45.478+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb07c1a06b0 con 0x7fb06c0a32f0 2024-09-19T07:32:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:32:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:32:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi174 2024-09-19T07:32:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T07:32:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:32:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:32:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:45 smithi134 ceph-mon[35414]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T07:32:46.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:32:46.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:32:46.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi174 2024-09-19T07:32:46.139 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T07:32:46.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:32:46.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:32:46.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:46.140 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:45 smithi103 ceph-mon[31201]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T07:32:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:32:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:32:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi174 2024-09-19T07:32:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-19T07:32:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:32:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:32:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:46.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:45 smithi174 ceph-mon[35370]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T07:32:47.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:46 smithi134 ceph-mon[35414]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:47.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:46 smithi134 ceph-mon[35414]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:47.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:46 smithi134 ceph-mon[35414]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T07:32:47.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:46 smithi103 ceph-mon[31201]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:47.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:46 smithi103 ceph-mon[31201]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:47.083 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:46 smithi103 ceph-mon[31201]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T07:32:47.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:46 smithi174 ceph-mon[35370]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:32:47.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:46 smithi174 ceph-mon[35370]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:47.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:46 smithi174 ceph-mon[35370]: osdmap e40: 6 total, 6 up, 6 in 2024-09-19T07:32:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:48 smithi134 ceph-mon[35414]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T07:32:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/564874532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaf8bd47-9f95-4e26-b8e0-387cb89c1de3"}]: dispatch 2024-09-19T07:32:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/564874532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaf8bd47-9f95-4e26-b8e0-387cb89c1de3"}]': finished 2024-09-19T07:32:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:48 smithi134 ceph-mon[35414]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T07:32:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:32:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:48 smithi134 ceph-mon[35414]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2450844294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:32:49.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:48 smithi174 ceph-mon[35370]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T07:32:49.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/564874532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaf8bd47-9f95-4e26-b8e0-387cb89c1de3"}]: dispatch 2024-09-19T07:32:49.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/564874532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaf8bd47-9f95-4e26-b8e0-387cb89c1de3"}]': finished 2024-09-19T07:32:49.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:48 smithi174 ceph-mon[35370]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T07:32:49.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:32:49.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:48 smithi174 ceph-mon[35370]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:49.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2450844294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:32:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:48 smithi103 ceph-mon[31201]: osdmap e41: 6 total, 6 up, 6 in 2024-09-19T07:32:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/564874532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaf8bd47-9f95-4e26-b8e0-387cb89c1de3"}]: dispatch 2024-09-19T07:32:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/564874532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaf8bd47-9f95-4e26-b8e0-387cb89c1de3"}]': finished 2024-09-19T07:32:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:48 smithi103 ceph-mon[31201]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T07:32:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:32:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:48 smithi103 ceph-mon[31201]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2450844294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:32:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:51 smithi174 ceph-mon[35370]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:51 smithi103 ceph-mon[31201]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:51.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:51 smithi134 ceph-mon[35414]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:32:53.303 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:53 smithi103 ceph-mon[31201]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:32:53.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:53 smithi174 ceph-mon[35370]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:32:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:53 smithi134 ceph-mon[35414]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:32:55.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:55 smithi134 ceph-mon[35414]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T07:32:55.412 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:55 smithi103 ceph-mon[31201]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T07:32:55.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:55 smithi174 ceph-mon[35370]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T07:32:57.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:57 smithi174 ceph-mon[35370]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T07:32:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:57 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T07:32:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:57 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:57 smithi174 ceph-mon[35370]: Deploying daemon osd.6 on smithi103 2024-09-19T07:32:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:57 smithi103 ceph-mon[31201]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T07:32:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:57 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T07:32:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:57 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:57 smithi103 ceph-mon[31201]: Deploying daemon osd.6 on smithi103 2024-09-19T07:32:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:57 smithi134 ceph-mon[35414]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-19T07:32:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T07:32:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:32:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:57 smithi134 ceph-mon[35414]: Deploying daemon osd.6 on smithi103 2024-09-19T07:32:59.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:32:59 smithi103 ceph-mon[31201]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T07:32:59.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:32:59 smithi174 ceph-mon[35370]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T07:32:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:32:59 smithi134 ceph-mon[35414]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T07:33:01.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:01 smithi174 ceph-mon[35370]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:33:01.459 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:01 smithi103 ceph-mon[31201]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:33:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:01 smithi134 ceph-mon[35414]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:33:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:03 smithi174 ceph-mon[35370]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:33:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:03 smithi103 ceph-mon[31201]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:33:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:03 smithi134 ceph-mon[35414]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:33:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:05.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:05 smithi103 ceph-mon[31201]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:05 smithi174 ceph-mon[35370]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:05 smithi134 ceph-mon[35414]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:06.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.230+0000 7fb079ffb640 1 -- 172.21.15.103:0/4196810587 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb07c1a06b0 con 0x7fb06c0a32f0 2024-09-19T07:33:06.231 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 6 on host 'smithi103' 2024-09-19T07:33:06.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb06c0a32f0 msgr2=0x7fb06c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:06.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb06c0a32f0 0x7fb06c0a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb0700058c0 tx=0x7fb070005850 comp rx=0 tx=0).stop 2024-09-19T07:33:06.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101d10 msgr2=0x7fb07c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:06.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101d10 0x7fb07c198340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb06800c920 tx=0x7fb06800cde0 comp rx=0 tx=0).stop 2024-09-19T07:33:06.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 shutdown_connections 2024-09-19T07:33:06.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb06c0a32f0 0x7fb06c0a57b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:06.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb07c1026d0 0x7fb07c19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:06.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101d10 0x7fb07c198340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:06.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 --2- 172.21.15.103:0/4196810587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb07c100b10 0x7fb07c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:06.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 >> 172.21.15.103:0/4196810587 conn(0x7fb07c0fc2a0 msgr2=0x7fb07c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:06.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 shutdown_connections 2024-09-19T07:33:06.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:33:06.233+0000 7fb082dcf640 1 -- 172.21.15.103:0/4196810587 wait complete. 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:06.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:06.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:06.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:06.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:06.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:06.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:06.919 DEBUG:teuthology.orchestra.run.smithi103:osd.6> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T07:33:06.922 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_2... 2024-09-19T07:33:06.922 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T07:33:07.176 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:33:08.408 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:33:08 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:33:08.215+0000 7f4874d23640 -1 osd.6 0 log_to_monitors true 2024-09-19T07:33:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:09 smithi134 ceph-mon[35414]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:09 smithi134 ceph-mon[35414]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T07:33:09.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:09 smithi174 ceph-mon[35370]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:09.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:09 smithi174 ceph-mon[35370]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T07:33:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:09 smithi103 ceph-mon[31201]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:09 smithi103 ceph-mon[31201]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T07:33:09.808 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:33:09.834 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-09-19T07:33:10.087 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:33:10.220 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:33:10.108+0000 7f4870ca6640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T07:33:10.220 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:33:10.116+0000 7f486cace640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:33:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T07:33:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T07:33:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:33:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:10.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:10 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:10.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:10 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T07:33:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T07:33:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:33:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:10 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 -- 172.21.15.134:0/2775086994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550100b10 msgr2=0x7f6550100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 --2- 172.21.15.134:0/2775086994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550100b10 0x7f6550100f10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6540009900 tx=0x7f654002a410 comp rx=0 tx=0).stop 2024-09-19T07:33:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 -- 172.21.15.134:0/2775086994 shutdown_connections 2024-09-19T07:33:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 --2- 172.21.15.134:0/2775086994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65501026d0 0x7f6550108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 --2- 172.21.15.134:0/2775086994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550101d10 0x7f6550102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 --2- 172.21.15.134:0/2775086994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550100b10 0x7f6550100f10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 -- 172.21.15.134:0/2775086994 >> 172.21.15.134:0/2775086994 conn(0x7f65500fc2a0 msgr2=0x7f65500fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 -- 172.21.15.134:0/2775086994 shutdown_connections 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.590+0000 7f6554b4d640 1 -- 172.21.15.134:0/2775086994 wait complete. 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 Processor -- start 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 -- start start 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550100b10 0x7f6550197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101d10 0x7f6550198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65501026d0 0x7f655019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f655010c770 con 0x7f6550100b10 2024-09-19T07:33:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f655010c5f0 con 0x7f6550101d10 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f6554b4d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f655010c8f0 con 0x7f65501026d0 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f654dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101d10 0x7f6550198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f654dd9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101d10 0x7f6550198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39190/0 (socket says 172.21.15.134:39190) 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.591+0000 7f654dd9b640 1 -- 172.21.15.134:0/4195754149 learned_addr learned my addr 172.21.15.134:0/4195754149 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654e59c640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550100b10 0x7f6550197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654ed9d640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65501026d0 0x7f655019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654dd9b640 1 -- 172.21.15.134:0/4195754149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65501026d0 msgr2=0x7f655019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654dd9b640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65501026d0 0x7f655019f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654dd9b640 1 -- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550100b10 msgr2=0x7f6550197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654dd9b640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550100b10 0x7f6550197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654dd9b640 1 -- 172.21.15.134:0/4195754149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655019fb70 con 0x7f6550101d10 2024-09-19T07:33:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f654dd9b640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101d10 0x7f6550198480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f654400d950 tx=0x7f654400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f653f7fe640 1 -- 172.21.15.134:0/4195754149 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544017030 con 0x7f6550101d10 2024-09-19T07:33:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f653f7fe640 1 -- 172.21.15.134:0/4195754149 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6544004020 con 0x7f6550101d10 2024-09-19T07:33:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655019fe60 con 0x7f6550101d10 2024-09-19T07:33:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f653f7fe640 1 -- 172.21.15.134:0/4195754149 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440047a0 con 0x7f6550101d10 2024-09-19T07:33:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.592+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a0310 con 0x7f6550101d10 2024-09-19T07:33:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.593+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65501a05a0 con 0x7f6550101d10 2024-09-19T07:33:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.594+0000 7f653f7fe640 1 -- 172.21.15.134:0/4195754149 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f65440041d0 con 0x7f6550101d10 2024-09-19T07:33:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.595+0000 7f653f7fe640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65380a3270 0x7f65380a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.595+0000 7f653f7fe640 1 -- 172.21.15.134:0/4195754149 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4685+0+0 (secure 0 0 0) 0x7f65440ce480 con 0x7f6550101d10 2024-09-19T07:33:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.595+0000 7f654e59c640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65380a3270 0x7f65380a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.596+0000 7f654e59c640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65380a3270 0x7f65380a5730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6540005ab0 tx=0x7f6540000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.599+0000 7f653f7fe640 1 -- 172.21.15.134:0/4195754149 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f654409bfd0 con 0x7f6550101d10 2024-09-19T07:33:10.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:10.773+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f65501a0770 con 0x7f65380a3270 2024-09-19T07:33:11.284 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi103 2024-09-19T07:33:11.284 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T07:33:11.284 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:11.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:33:11.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T07:33:11.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:11.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:11.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:33:11.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:33:11.285 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:11 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi103 2024-09-19T07:33:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T07:33:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:33:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T07:33:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:11.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:11.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:33:11.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:33:11.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:11 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:11.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi103 2024-09-19T07:33:11.436 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: from='osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: osdmap e44: 7 total, 6 up, 7 in 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:33:11.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:11 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:12.410 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:12 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:33:12.411 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:12 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:33:12.411 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:12 smithi134 ceph-mon[35414]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:12.411 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:12 smithi134 ceph-mon[35414]: osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] boot 2024-09-19T07:33:12.411 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:12 smithi134 ceph-mon[35414]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T07:33:12.411 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:12 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:33:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:12 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:33:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:12 smithi174 ceph-mon[35370]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:12 smithi174 ceph-mon[35370]: osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] boot 2024-09-19T07:33:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:12 smithi174 ceph-mon[35370]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T07:33:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:12 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:33:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:12 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:33:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:12 smithi103 ceph-mon[31201]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:12 smithi103 ceph-mon[31201]: osd.6 [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] boot 2024-09-19T07:33:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:12 smithi103 ceph-mon[31201]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T07:33:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:33:13.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:13 smithi134 ceph-mon[35414]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:13.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:13 smithi134 ceph-mon[35414]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T07:33:13.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:13 smithi174 ceph-mon[35370]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:13.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:13 smithi174 ceph-mon[35370]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T07:33:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:13 smithi103 ceph-mon[31201]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-19T07:33:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:13 smithi103 ceph-mon[31201]: osdmap e46: 7 total, 7 up, 7 in 2024-09-19T07:33:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:14 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/4197066547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]: dispatch 2024-09-19T07:33:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:14 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]: dispatch 2024-09-19T07:33:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:14 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]': finished 2024-09-19T07:33:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:14 smithi174 ceph-mon[35370]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T07:33:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:14 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:14.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:14 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/3439063391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:33:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:14 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/4197066547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]: dispatch 2024-09-19T07:33:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:14 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]: dispatch 2024-09-19T07:33:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:14 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]': finished 2024-09-19T07:33:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:14 smithi103 ceph-mon[31201]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T07:33:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:14 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:14 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/3439063391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:33:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:14 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/4197066547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]: dispatch 2024-09-19T07:33:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:14 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]: dispatch 2024-09-19T07:33:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:14 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5ac551b-6bfa-45bb-a12c-7684b5d30079"}]': finished 2024-09-19T07:33:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:14 smithi134 ceph-mon[35414]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T07:33:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:14 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:14 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/3439063391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:33:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:15 smithi174 ceph-mon[35370]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:15 smithi103 ceph-mon[31201]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:15 smithi134 ceph-mon[35414]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:17 smithi174 ceph-mon[35370]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:17 smithi103 ceph-mon[31201]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:17.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:17 smithi134 ceph-mon[35414]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:19 smithi174 ceph-mon[35370]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:19.453 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:19 smithi134 ceph-mon[35414]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:19 smithi103 ceph-mon[31201]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:21.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:21 smithi174 ceph-mon[35370]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:21.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:21 smithi134 ceph-mon[35414]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:21 smithi103 ceph-mon[31201]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:22.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:22 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T07:33:22.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:22 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:22.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:22 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T07:33:22.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:22 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:22 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T07:33:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:22 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:23.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:23 smithi174 ceph-mon[35370]: Deploying daemon osd.7 on smithi134 2024-09-19T07:33:23.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:23 smithi174 ceph-mon[35370]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:23.433 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:23 smithi134 ceph-mon[35414]: Deploying daemon osd.7 on smithi134 2024-09-19T07:33:23.433 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:23 smithi134 ceph-mon[35414]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:23 smithi103 ceph-mon[31201]: Deploying daemon osd.7 on smithi134 2024-09-19T07:33:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:23 smithi103 ceph-mon[31201]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:25.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:25 smithi134 ceph-mon[35414]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:25 smithi174 ceph-mon[35370]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:25 smithi103 ceph-mon[31201]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:27 smithi134 ceph-mon[35414]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:27 smithi174 ceph-mon[35370]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:27 smithi103 ceph-mon[31201]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:28.453 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:28.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:28.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:29.393 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:29 smithi134 ceph-mon[35414]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:29.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:29 smithi174 ceph-mon[35370]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:29 smithi103 ceph-mon[31201]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:31 smithi174 ceph-mon[35370]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:31 smithi103 ceph-mon[31201]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:31 smithi134 ceph-mon[35414]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:31.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:31.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:31.488 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:31.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.532+0000 7f653f7fe640 1 -- 172.21.15.134:0/4195754149 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f65501a0770 con 0x7f65380a3270 2024-09-19T07:33:31.537 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2024-09-19T07:33:31.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65380a3270 msgr2=0x7f65380a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:31.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65380a3270 0x7f65380a5730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6540005ab0 tx=0x7f6540000bc0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101d10 msgr2=0x7f6550198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101d10 0x7f6550198480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f654400d950 tx=0x7f654400de10 comp rx=0 tx=0).stop 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 shutdown_connections 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65380a3270 0x7f65380a5730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65501026d0 0x7f655019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101d10 0x7f6550198480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 --2- 172.21.15.134:0/4195754149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550100b10 0x7f6550197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 >> 172.21.15.134:0/4195754149 conn(0x7f65500fc2a0 msgr2=0x7f655006a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 shutdown_connections 2024-09-19T07:33:31.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:33:31.535+0000 7f6554b4d640 1 -- 172.21.15.134:0/4195754149 wait complete. 2024-09-19T07:33:32.193 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T07:33:32.196 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-09-19T07:33:32.196 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T07:33:32.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:32 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:32.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:32 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:32.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:32 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:32.447 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:33:32.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:32 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:32.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:32 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:32.469 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:32 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:32.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:32 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:32 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:32 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:33.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:33 smithi134 ceph-mon[35414]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:33 smithi174 ceph-mon[35370]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:33.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:33 smithi103 ceph-mon[31201]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:33.653 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:33:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:33:33.535+0000 7f8c30bf8640 -1 osd.7 0 log_to_monitors true 2024-09-19T07:33:34.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:34 smithi174 ceph-mon[35370]: from='osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:33:34.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:34 smithi174 ceph-mon[35370]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:33:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:34 smithi103 ceph-mon[31201]: from='osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:33:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:34 smithi103 ceph-mon[31201]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:33:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:34 smithi134 ceph-mon[35414]: from='osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:33:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:34 smithi134 ceph-mon[35414]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T07:33:35.178 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:33:35.204 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-09-19T07:33:35.329 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:33:35.173+0000 7f8c2cb7b640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T07:33:35.329 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:33:35.183+0000 7f8c289a3640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:33:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:35.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:35.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:35 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.448 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:33:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T07:33:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:33:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T07:33:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:33:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:35.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:35 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.568 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:35.568 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:35.569 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:35 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:35.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 -- 172.21.15.174:0/3915544178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 msgr2=0x7f8c18102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 --2- 172.21.15.174:0/3915544178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18102000 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009900 tx=0x7f8c1402a410 comp rx=0 tx=0).stop 2024-09-19T07:33:35.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 -- 172.21.15.174:0/3915544178 shutdown_connections 2024-09-19T07:33:35.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 --2- 172.21.15.174:0/3915544178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181037c0 0x7f8c1810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 --2- 172.21.15.174:0/3915544178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c18102e00 0x7f8c18103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 --2- 172.21.15.174:0/3915544178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18102000 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.907 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 -- 172.21.15.174:0/3915544178 >> 172.21.15.174:0/3915544178 conn(0x7f8c180fd390 msgr2=0x7f8c180ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 -- 172.21.15.174:0/3915544178 shutdown_connections 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.904+0000 7f8c202e0640 1 -- 172.21.15.174:0/3915544178 wait complete. 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 Processor -- start 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 -- start start 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c18102e00 0x7f8c18198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181037c0 0x7f8c1819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1810d860 con 0x7f8c18102e00 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1810d6e0 con 0x7f8c181037c0 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c202e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1810d9e0 con 0x7f8c18101c00 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c1e07c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c1e07c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48486/0 (socket says 172.21.15.174:48486) 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.905+0000 7f8c1e07c640 1 -- 172.21.15.174:0/2355556312 learned_addr learned my addr 172.21.15.174:0/2355556312 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1e87d640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181037c0 0x7f8c1819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1d87b640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c18102e00 0x7f8c18198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1e07c640 1 -- 172.21.15.174:0/2355556312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181037c0 msgr2=0x7f8c1819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1e07c640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181037c0 0x7f8c1819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1e07c640 1 -- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c18102e00 msgr2=0x7f8c18198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1e07c640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c18102e00 0x7f8c18198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1e07c640 1 -- 172.21.15.174:0/2355556312 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1819fb40 con 0x7f8c18101c00 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1d87b640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c18102e00 0x7f8c18198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c1e07c640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18197f10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c14031610 tx=0x7f8c140043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:35.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c0f7fe640 1 -- 172.21.15.174:0/2355556312 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1403f030 con 0x7f8c18101c00 2024-09-19T07:33:35.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c0f7fe640 1 -- 172.21.15.174:0/2355556312 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c14003ce0 con 0x7f8c18101c00 2024-09-19T07:33:35.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c0f7fe640 1 -- 172.21.15.174:0/2355556312 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14035070 con 0x7f8c18101c00 2024-09-19T07:33:35.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1819fdd0 con 0x7f8c18101c00 2024-09-19T07:33:35.910 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.906+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c181a01a0 con 0x7f8c18101c00 2024-09-19T07:33:35.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.907+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c18109050 con 0x7f8c18101c00 2024-09-19T07:33:35.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.908+0000 7f8c0f7fe640 1 -- 172.21.15.174:0/2355556312 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8c14042030 con 0x7f8c18101c00 2024-09-19T07:33:35.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.909+0000 7f8c0f7fe640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8c040a32f0 0x7f8c040a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:33:35.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.909+0000 7f8c0f7fe640 1 -- 172.21.15.174:0/2355556312 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5430+0+0 (secure 0 0 0) 0x7f8c140ef510 con 0x7f8c18101c00 2024-09-19T07:33:35.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.909+0000 7f8c1d87b640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8c040a32f0 0x7f8c040a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:33:35.911 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.910+0000 7f8c1d87b640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8c040a32f0 0x7f8c040a57b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c18199430 tx=0x7f8c08005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:33:35.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:35.912+0000 7f8c0f7fe640 1 -- 172.21.15.174:0/2355556312 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c140bcd70 con 0x7f8c18101c00 2024-09-19T07:33:36.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:36.087+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8c18062630 con 0x7f8c040a32f0 2024-09-19T07:33:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi134 2024-09-19T07:33:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-19T07:33:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:33:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T07:33:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] boot 2024-09-19T07:33:36.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T07:33:36.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:33:36.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:33:36.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:36 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi134 2024-09-19T07:33:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-19T07:33:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:33:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T07:33:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] boot 2024-09-19T07:33:36.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T07:33:36.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:33:36.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:33:36.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:36 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi134 2024-09-19T07:33:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi134 to 5696M 2024-09-19T07:33:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:33:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: osdmap e49: 8 total, 7 up, 8 in 2024-09-19T07:33:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: osd.7 [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] boot 2024-09-19T07:33:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T07:33:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:33:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:33:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:33:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:36 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:37 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:33:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:37 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:33:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:37 smithi174 ceph-mon[35370]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:37 smithi174 ceph-mon[35370]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:37 smithi174 ceph-mon[35370]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T07:33:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:37 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:33:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:37 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:33:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:37 smithi103 ceph-mon[31201]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:37 smithi103 ceph-mon[31201]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:37 smithi103 ceph-mon[31201]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T07:33:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:37 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:33:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:37 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:33:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:37 smithi134 ceph-mon[35414]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-09-19T07:33:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:37 smithi134 ceph-mon[35414]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:33:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:37 smithi134 ceph-mon[35414]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T07:33:38.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:38 smithi174 ceph-mon[35370]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T07:33:38.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:38 smithi174 ceph-mon[35370]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:38.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:38 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/2442450897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]: dispatch 2024-09-19T07:33:38.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:38 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]: dispatch 2024-09-19T07:33:38.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:38 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]': finished 2024-09-19T07:33:38.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:38 smithi174 ceph-mon[35370]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T07:33:38.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:33:39.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:38 smithi134 ceph-mon[35414]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T07:33:39.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:38 smithi134 ceph-mon[35414]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:39.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:38 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/2442450897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]: dispatch 2024-09-19T07:33:39.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:38 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]: dispatch 2024-09-19T07:33:39.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:38 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]': finished 2024-09-19T07:33:39.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:38 smithi134 ceph-mon[35414]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T07:33:39.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:33:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:38 smithi103 ceph-mon[31201]: osdmap e52: 8 total, 8 up, 8 in 2024-09-19T07:33:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:38 smithi103 ceph-mon[31201]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:38 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/2442450897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]: dispatch 2024-09-19T07:33:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:38 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]: dispatch 2024-09-19T07:33:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:38 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3aa078e4-5626-4b10-8727-a060b52ffb58"}]': finished 2024-09-19T07:33:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:38 smithi103 ceph-mon[31201]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T07:33:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:33:39.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:39 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/3541281215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:33:39.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:39 smithi103 ceph-mon[31201]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T07:33:39.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:39 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:33:40.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:39 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/3541281215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:33:40.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:39 smithi134 ceph-mon[35414]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T07:33:40.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:39 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:33:40.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:39 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/3541281215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:33:40.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:39 smithi174 ceph-mon[35370]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T07:33:40.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:39 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:33:41.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:40 smithi134 ceph-mon[35414]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:41.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:40 smithi174 ceph-mon[35370]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:40 smithi103 ceph-mon[31201]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:43.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:43 smithi174 ceph-mon[35370]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:43 smithi103 ceph-mon[31201]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:43.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:43 smithi134 ceph-mon[35414]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:45.367 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:45 smithi174 ceph-mon[35370]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:45 smithi103 ceph-mon[31201]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:45 smithi134 ceph-mon[35414]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:47.219 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:47 smithi174 ceph-mon[35370]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:47.220 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T07:33:47.220 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:47 smithi103 ceph-mon[31201]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T07:33:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:47.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:47 smithi134 ceph-mon[35414]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:47.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T07:33:47.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:48.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:48 smithi174 ceph-mon[35370]: Deploying daemon osd.8 on smithi174 2024-09-19T07:33:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:48 smithi103 ceph-mon[31201]: Deploying daemon osd.8 on smithi174 2024-09-19T07:33:48.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:48 smithi134 ceph-mon[35414]: Deploying daemon osd.8 on smithi174 2024-09-19T07:33:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:49 smithi174 ceph-mon[35370]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:49.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:49 smithi103 ceph-mon[31201]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:49.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:49 smithi134 ceph-mon[35414]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:51.218 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:51 smithi174 ceph-mon[35370]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:51 smithi103 ceph-mon[31201]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:51.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:51 smithi134 ceph-mon[35414]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:53.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:53 smithi174 ceph-mon[35370]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:53.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:53.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:53.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:53 smithi103 ceph-mon[31201]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:53.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:53 smithi134 ceph-mon[35414]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:55.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:55 smithi134 ceph-mon[35414]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:55.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:55 smithi174 ceph-mon[35370]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:55.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:55 smithi103 ceph-mon[31201]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:56.249 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-09-19T07:33:56.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.248+0000 7f8c0f7fe640 1 -- 172.21.15.174:0/2355556312 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c18062630 con 0x7f8c040a32f0 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8c040a32f0 msgr2=0x7f8c040a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8c040a32f0 0x7f8c040a57b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c18199430 tx=0x7f8c08005ec0 comp rx=0 tx=0).stop 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 msgr2=0x7f8c18197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18197f10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c14031610 tx=0x7f8c140043b0 comp rx=0 tx=0).stop 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 shutdown_connections 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8c040a32f0 0x7f8c040a57b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c181037c0 0x7f8c1819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:56.252 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8c18102e00 0x7f8c18198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:56.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 --2- 172.21.15.174:0/2355556312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c18101c00 0x7f8c18197f10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:33:56.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.251+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 >> 172.21.15.174:0/2355556312 conn(0x7f8c180fd390 msgr2=0x7f8c18106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:33:56.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.252+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 shutdown_connections 2024-09-19T07:33:56.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:33:56.252+0000 7f8c202e0640 1 -- 172.21.15.174:0/2355556312 wait complete. 2024-09-19T07:33:56.486 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.486 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.487 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:56.487 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:56.487 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:56.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:56.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:33:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:33:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:56.952 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T07:33:56.955 INFO:tasks.cephadm:Deploying osd.9 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-19T07:33:56.955 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T07:33:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:33:57.232 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:57 smithi103 ceph-mon[31201]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:57.233 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:57 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:57.233 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:57 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:57.233 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:57 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:57 smithi134 ceph-mon[35414]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:57.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:57.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:57.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:57 smithi174 ceph-mon[35370]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:57.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:57 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:33:57.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:57 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:57.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:57 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:33:58.916 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:33:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:33:58.529+0000 7f5b5012f640 -1 osd.8 0 log_to_monitors true 2024-09-19T07:33:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:59 smithi134 ceph-mon[35414]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:59 smithi134 ceph-mon[35414]: from='osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:33:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:33:59 smithi134 ceph-mon[35414]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:33:59.585 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:59 smithi103 ceph-mon[31201]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:59.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:59 smithi103 ceph-mon[31201]: from='osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:33:59.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:33:59 smithi103 ceph-mon[31201]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:33:59.663 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:59 smithi174 ceph-mon[35370]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:33:59.663 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:59 smithi174 ceph-mon[35370]: from='osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:33:59.663 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:33:59 smithi174 ceph-mon[35370]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T07:33:59.950 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:33:59.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-19T07:34:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:34:00.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T07:34:00.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T07:34:00.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:00.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:34:00.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:34:00.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:00.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:00.519 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T07:34:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T07:34:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:34:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.626 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:00.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:00.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:00.627 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 -- 172.21.15.103:0/586718878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 msgr2=0x7fac58108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 --2- 172.21.15.103:0/586718878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac58108f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac50009ca0 tx=0x7fac5002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 -- 172.21.15.103:0/586718878 shutdown_connections 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 --2- 172.21.15.103:0/586718878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac58108f20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 --2- 172.21.15.103:0/586718878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac58101cd0 0x7fac58102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 --2- 172.21.15.103:0/586718878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac58100ad0 0x7fac58100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 -- 172.21.15.103:0/586718878 >> 172.21.15.103:0/586718878 conn(0x7fac580fc280 msgr2=0x7fac580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:34:00.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 -- 172.21.15.103:0/586718878 shutdown_connections 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.770+0000 7fac5f526640 1 -- 172.21.15.103:0/586718878 wait complete. 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 Processor -- start 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 -- start start 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac58100ad0 0x7fac5806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac58101cd0 0x7fac5819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac581a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac5810c730 con 0x7fac58102690 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac5810c5b0 con 0x7fac58100ad0 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5f526640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac5810c8b0 con 0x7fac58101cd0 2024-09-19T07:34:00.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5dac3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac581a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5dac3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac581a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39192/0 (socket says 172.21.15.103:39192) 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5dac3640 1 -- 172.21.15.103:0/2300109137 learned_addr learned my addr 172.21.15.103:0/2300109137 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5d2c2640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac58100ad0 0x7fac5806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.771+0000 7fac5cac1640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac58101cd0 0x7fac5819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5dac3640 1 -- 172.21.15.103:0/2300109137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac58101cd0 msgr2=0x7fac5819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5dac3640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac58101cd0 0x7fac5819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5dac3640 1 -- 172.21.15.103:0/2300109137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac58100ad0 msgr2=0x7fac5806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5dac3640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac58100ad0 0x7fac5806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5dac3640 1 -- 172.21.15.103:0/2300109137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac581a1d20 con 0x7fac58102690 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5dac3640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac581a1530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac50031ae0 tx=0x7fac50031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac4e7fc640 1 -- 172.21.15.103:0/2300109137 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50003c30 con 0x7fac58102690 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac4e7fc640 1 -- 172.21.15.103:0/2300109137 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac50003dd0 con 0x7fac58102690 2024-09-19T07:34:00.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac581a1f50 con 0x7fac58102690 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac4e7fc640 1 -- 172.21.15.103:0/2300109137 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5002fcb0 con 0x7fac58102690 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.772+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac581a24b0 con 0x7fac58102690 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.773+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac58107ef0 con 0x7fac58102690 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.774+0000 7fac4e7fc640 1 -- 172.21.15.103:0/2300109137 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fac50004050 con 0x7fac58102690 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.775+0000 7fac4e7fc640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fac440a32f0 0x7fac440a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.775+0000 7fac5d2c2640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fac440a32f0 0x7fac440a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.775+0000 7fac4e7fc640 1 -- 172.21.15.103:0/2300109137 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7fac500f00f0 con 0x7fac58102690 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.775+0000 7fac5d2c2640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fac440a32f0 0x7fac440a57b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac58101b30 tx=0x7fac54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:34:00.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.779+0000 7fac4e7fc640 1 -- 172.21.15.103:0/2300109137 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fac500bd8c0 con 0x7fac58102690 2024-09-19T07:34:00.915 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:34:00.624+0000 7f5b4c0b2640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T07:34:00.916 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:34:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:34:00.634+0000 7f5b43ed2640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:34:00.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:00.955+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fac58062630 con 0x7fac440a32f0 2024-09-19T07:34:01.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi174 2024-09-19T07:34:01.504 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T07:34:01.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:34:01.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:34:01.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T07:34:01.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='osd.8 ' entity='osd.8' 2024-09-19T07:34:01.505 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] boot 2024-09-19T07:34:01.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T07:34:01.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:01.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:01.506 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi174 2024-09-19T07:34:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T07:34:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='osd.8 ' entity='osd.8' 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] boot 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:01.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi174 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: osdmap e56: 9 total, 8 up, 9 in 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='osd.8 ' entity='osd.8' 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: osd.8 [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] boot 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:34:01.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:01.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:01.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:02 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:34:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:02 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:34:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:02 smithi103 ceph-mon[31201]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:02 smithi103 ceph-mon[31201]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T07:34:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:02 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:34:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:02 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:34:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:02 smithi134 ceph-mon[35414]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:02 smithi134 ceph-mon[35414]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T07:34:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:02 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:34:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:02 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:34:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:02 smithi174 ceph-mon[35370]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:02 smithi174 ceph-mon[35370]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T07:34:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:03 smithi103 ceph-mon[31201]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:03 smithi103 ceph-mon[31201]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T07:34:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:03 smithi134 ceph-mon[35414]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:03 smithi134 ceph-mon[35414]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T07:34:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:03 smithi174 ceph-mon[35370]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:03 smithi174 ceph-mon[35370]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T07:34:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:04 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/987802413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f63e40-aa84-48be-b99b-2f16545d058f"}]: dispatch 2024-09-19T07:34:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:04 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/987802413' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f63e40-aa84-48be-b99b-2f16545d058f"}]': finished 2024-09-19T07:34:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:04 smithi103 ceph-mon[31201]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T07:34:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:04 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2598224052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:04 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/987802413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f63e40-aa84-48be-b99b-2f16545d058f"}]: dispatch 2024-09-19T07:34:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:04 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/987802413' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f63e40-aa84-48be-b99b-2f16545d058f"}]': finished 2024-09-19T07:34:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:04 smithi134 ceph-mon[35414]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T07:34:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:04 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2598224052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:04 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/987802413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9f63e40-aa84-48be-b99b-2f16545d058f"}]: dispatch 2024-09-19T07:34:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:04 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/987802413' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9f63e40-aa84-48be-b99b-2f16545d058f"}]': finished 2024-09-19T07:34:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:04 smithi174 ceph-mon[35370]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T07:34:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:04 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2598224052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:05 smithi134 ceph-mon[35414]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:05 smithi134 ceph-mon[35414]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T07:34:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:05 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:05 smithi174 ceph-mon[35370]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:05 smithi174 ceph-mon[35370]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T07:34:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:05 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:05.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:05 smithi103 ceph-mon[31201]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:05.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:05 smithi103 ceph-mon[31201]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T07:34:05.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:05 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:07 smithi134 ceph-mon[35414]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:07 smithi174 ceph-mon[35370]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:07.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:07 smithi103 ceph-mon[31201]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:09 smithi103 ceph-mon[31201]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:34:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:09 smithi134 ceph-mon[35414]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:34:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:09 smithi174 ceph-mon[35370]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T07:34:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:11 smithi103 ceph-mon[31201]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T07:34:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:11 smithi134 ceph-mon[35414]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T07:34:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:11 smithi174 ceph-mon[35370]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-19T07:34:13.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:13 smithi103 ceph-mon[31201]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T07:34:13.464 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T07:34:13.465 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:13 smithi134 ceph-mon[35414]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T07:34:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T07:34:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:13 smithi174 ceph-mon[35370]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-19T07:34:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T07:34:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:14.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:14 smithi103 ceph-mon[31201]: Deploying daemon osd.9 on smithi103 2024-09-19T07:34:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:14 smithi134 ceph-mon[35414]: Deploying daemon osd.9 on smithi103 2024-09-19T07:34:14.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:14 smithi174 ceph-mon[35370]: Deploying daemon osd.9 on smithi103 2024-09-19T07:34:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:15 smithi103 ceph-mon[31201]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:34:15.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:15 smithi134 ceph-mon[35414]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:34:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:15 smithi174 ceph-mon[35370]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-19T07:34:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:17 smithi103 ceph-mon[31201]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T07:34:17.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:17 smithi134 ceph-mon[35414]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T07:34:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:17 smithi174 ceph-mon[35370]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-19T07:34:19.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:19 smithi103 ceph-mon[31201]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:34:19.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:19.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:19.293 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:19 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:19 smithi134 ceph-mon[35414]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:34:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:19 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:19 smithi174 ceph-mon[35370]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:34:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:19 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:21 smithi134 ceph-mon[35414]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:21.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:21 smithi174 ceph-mon[35370]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:21 smithi103 ceph-mon[31201]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:22.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.291+0000 7fac4e7fc640 1 -- 172.21.15.103:0/2300109137 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fac58062630 con 0x7fac440a32f0 2024-09-19T07:34:22.293 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 9 on host 'smithi103' 2024-09-19T07:34:22.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fac440a32f0 msgr2=0x7fac440a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fac440a32f0 0x7fac440a57b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac58101b30 tx=0x7fac54005f90 comp rx=0 tx=0).stop 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 msgr2=0x7fac581a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac581a1530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac50031ae0 tx=0x7fac50031ee0 comp rx=0 tx=0).stop 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 shutdown_connections 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fac440a32f0 0x7fac440a57b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac58102690 0x7fac581a1530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac58101cd0 0x7fac5819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 --2- 172.21.15.103:0/2300109137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac58100ad0 0x7fac5806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 >> 172.21.15.103:0/2300109137 conn(0x7fac580fc280 msgr2=0x7fac581050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 shutdown_connections 2024-09-19T07:34:22.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:34:22.294+0000 7fac5f526640 1 -- 172.21.15.103:0/2300109137 wait complete. 2024-09-19T07:34:22.624 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:22 smithi103 ceph-mon[31201]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:22.625 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:22 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:22.625 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:22 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:22 smithi174 ceph-mon[35370]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:22 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:22 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:22 smithi134 ceph-mon[35414]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:22 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:22 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:22.943 DEBUG:teuthology.orchestra.run.smithi103:osd.9> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T07:34:22.946 INFO:tasks.cephadm:Deploying osd.10 on smithi134 with /dev/vg_nvme/lv_1... 2024-09-19T07:34:22.946 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T07:34:23.200 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:34:24.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:24.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:24 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:24.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:24 smithi103 ceph-mon[31201]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T07:34:24.221 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:34:24 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:34:24.086+0000 7f50b6177640 -1 osd.9 0 log_to_monitors true 2024-09-19T07:34:24.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:24.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:24 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:24.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:24 smithi134 ceph-mon[35414]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T07:34:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:24 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:24 smithi174 ceph-mon[35370]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T07:34:25.345 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:25 smithi134 ceph-mon[35414]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:25.345 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:25 smithi134 ceph-mon[35414]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T07:34:25.345 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:25 smithi134 ceph-mon[35414]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T07:34:25.345 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:25 smithi134 ceph-mon[35414]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:34:25.345 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:25 smithi174 ceph-mon[35370]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:25 smithi174 ceph-mon[35370]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T07:34:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:25 smithi174 ceph-mon[35370]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T07:34:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:25 smithi174 ceph-mon[35370]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:34:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:25.470 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:34:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:34:25.157+0000 7f50b20fa640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T07:34:25.470 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:34:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:34:25.167+0000 7f50adf22640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:34:25.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:25 smithi103 ceph-mon[31201]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:25.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:25 smithi103 ceph-mon[31201]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T07:34:25.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:25 smithi103 ceph-mon[31201]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T07:34:25.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:25 smithi103 ceph-mon[31201]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-19T07:34:25.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:25.665 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-19T07:34:25.691 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-09-19T07:34:25.942 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:34:26.316 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:26 smithi134 ceph-mon[35414]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:34:26.316 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:26 smithi134 ceph-mon[35414]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T07:34:26.316 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:26 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:26.316 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:26 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:26 smithi174 ceph-mon[35370]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:34:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:26 smithi174 ceph-mon[35370]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T07:34:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:26 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:26 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 -- 172.21.15.134:0/587400622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 msgr2=0x7ff56c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 --2- 172.21.15.134:0/587400622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c108f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff564009ca0 tx=0x7ff56402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 -- 172.21.15.134:0/587400622 shutdown_connections 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 --2- 172.21.15.134:0/587400622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c108f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 --2- 172.21.15.134:0/587400622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c101cd0 0x7ff56c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 --2- 172.21.15.134:0/587400622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56c100ad0 0x7ff56c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 -- 172.21.15.134:0/587400622 >> 172.21.15.134:0/587400622 conn(0x7ff56c0fc280 msgr2=0x7ff56c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 -- 172.21.15.134:0/587400622 shutdown_connections 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.400+0000 7ff5736bf640 1 -- 172.21.15.134:0/587400622 wait complete. 2024-09-19T07:34:26.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 Processor -- start 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 -- start start 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c100ad0 0x7ff56c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56c101cd0 0x7ff56c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c10d880 con 0x7ff56c101cd0 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c10d700 con 0x7ff56c102690 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff5736bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c10da00 con 0x7ff56c100ad0 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff571c5c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c100ad0 0x7ff56c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c100ad0 0x7ff56c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.134:51998/0 (socket says 172.21.15.134:51998) 2024-09-19T07:34:26.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 -- 172.21.15.134:0/2090132280 learned_addr learned my addr 172.21.15.134:0/2090132280 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff571c5c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c19f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48618/0 (socket says 172.21.15.134:48618) 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff570c5a640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56c101cd0 0x7ff56c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 -- 172.21.15.134:0/2090132280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 msgr2=0x7ff56c19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 -- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56c101cd0 msgr2=0x7ff56c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56c101cd0 0x7ff56c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 -- 172.21.15.134:0/2090132280 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c19fae0 con 0x7ff56c100ad0 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff571c5c640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff570c5a640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56c101cd0 0x7ff56c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.401+0000 7ff57145b640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c100ad0 0x7ff56c197eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d950 tx=0x7ff56800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.402+0000 7ff5627fc640 1 -- 172.21.15.134:0/2090132280 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568017020 con 0x7ff56c100ad0 2024-09-19T07:34:26.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.402+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56c19fdd0 con 0x7ff56c100ad0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.402+0000 7ff5627fc640 1 -- 172.21.15.134:0/2090132280 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff568009c30 con 0x7ff56c100ad0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.402+0000 7ff5627fc640 1 -- 172.21.15.134:0/2090132280 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56800f6b0 con 0x7ff56c100ad0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.402+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56c1a0220 con 0x7ff56c100ad0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.403+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56c107ea0 con 0x7ff56c100ad0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.404+0000 7ff5627fc640 1 -- 172.21.15.134:0/2090132280 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff56801a020 con 0x7ff56c100ad0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.405+0000 7ff5627fc640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5580a3270 0x7ff5580a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.405+0000 7ff5627fc640 1 -- 172.21.15.134:0/2090132280 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6261+0+0 (secure 0 0 0) 0x7ff5680cf540 con 0x7ff56c100ad0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.405+0000 7ff570c5a640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5580a3270 0x7ff5580a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.405+0000 7ff570c5a640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5580a3270 0x7ff5580a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff56c1993d0 tx=0x7ff55c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:34:26.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.408+0000 7ff5627fc640 1 -- 172.21.15.134:0/2090132280 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff568021020 con 0x7ff56c100ad0 2024-09-19T07:34:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:26 smithi103 ceph-mon[31201]: from='osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-19T07:34:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:26 smithi103 ceph-mon[31201]: osdmap e63: 10 total, 9 up, 10 in 2024-09-19T07:34:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:26 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:26 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:26.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:26.589+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff56c062630 con 0x7ff5580a3270 2024-09-19T07:34:27.327 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] boot 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.328 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:27 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:34:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:34:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] boot 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:27.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:27 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:34:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:34:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-09-19T07:34:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: osd.9 [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] boot 2024-09-19T07:34:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T07:34:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:27.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:27.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:27.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:27.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:27 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:28.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:28 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi103 2024-09-19T07:34:28.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:28 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T07:34:28.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:28 smithi134 ceph-mon[35414]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:28.280 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:28 smithi134 ceph-mon[35414]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T07:34:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:28 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi103 2024-09-19T07:34:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:28 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T07:34:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:28 smithi174 ceph-mon[35370]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:28 smithi174 ceph-mon[35370]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T07:34:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:28 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi103 2024-09-19T07:34:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:28 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-19T07:34:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:28 smithi103 ceph-mon[31201]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:28 smithi103 ceph-mon[31201]: osdmap e65: 10 total, 10 up, 10 in 2024-09-19T07:34:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:29 smithi103 ceph-mon[31201]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:29 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/918595163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]: dispatch 2024-09-19T07:34:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:29 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]: dispatch 2024-09-19T07:34:29.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:29 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]': finished 2024-09-19T07:34:29.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:29 smithi103 ceph-mon[31201]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T07:34:29.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:29 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:29.567 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:29 smithi134 ceph-mon[35414]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:29.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:29 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/918595163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]: dispatch 2024-09-19T07:34:29.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:29 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]: dispatch 2024-09-19T07:34:29.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:29 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]': finished 2024-09-19T07:34:29.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:29 smithi134 ceph-mon[35414]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T07:34:29.568 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:29 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:29 smithi174 ceph-mon[35370]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:29 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/918595163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]: dispatch 2024-09-19T07:34:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:29 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]: dispatch 2024-09-19T07:34:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:29 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60776c91-f34f-499f-825d-43691743cd2d"}]': finished 2024-09-19T07:34:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:29 smithi174 ceph-mon[35370]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T07:34:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:29 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:30 smithi103 ceph-mon[31201]: from='client.? 172.21.15.134:0/2848532257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:30 smithi134 ceph-mon[35414]: from='client.? 172.21.15.134:0/2848532257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:30 smithi174 ceph-mon[35370]: from='client.? 172.21.15.134:0/2848532257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:31.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:31 smithi103 ceph-mon[31201]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:31 smithi134 ceph-mon[35414]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:31 smithi174 ceph-mon[35370]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:33.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:33 smithi103 ceph-mon[31201]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:33.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:33 smithi134 ceph-mon[35414]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:33 smithi174 ceph-mon[35370]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:35.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:35 smithi134 ceph-mon[35414]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:35 smithi103 ceph-mon[31201]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:35.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:35 smithi174 ceph-mon[35370]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:37 smithi103 ceph-mon[31201]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:37.575 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:37 smithi134 ceph-mon[35414]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:37.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:37 smithi174 ceph-mon[35370]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:38.303 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T07:34:38.304 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T07:34:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T07:34:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:39.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:39 smithi134 ceph-mon[35414]: Deploying daemon osd.10 on smithi134 2024-09-19T07:34:39.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:39 smithi134 ceph-mon[35414]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:39 smithi103 ceph-mon[31201]: Deploying daemon osd.10 on smithi134 2024-09-19T07:34:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:39 smithi103 ceph-mon[31201]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:39 smithi174 ceph-mon[35370]: Deploying daemon osd.10 on smithi134 2024-09-19T07:34:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:39 smithi174 ceph-mon[35370]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:41.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:41 smithi134 ceph-mon[35414]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:41.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:41 smithi103 ceph-mon[31201]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:41.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:41 smithi174 ceph-mon[35370]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:43.371 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:43 smithi134 ceph-mon[35414]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:43 smithi103 ceph-mon[31201]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:43.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:43 smithi174 ceph-mon[35370]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:44 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:44 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:44.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:44 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:45 smithi103 ceph-mon[31201]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:45 smithi134 ceph-mon[35414]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:45.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:45 smithi174 ceph-mon[35370]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:46.885 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 10 on host 'smithi134' 2024-09-19T07:34:46.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.884+0000 7ff5627fc640 1 -- 172.21.15.134:0/2090132280 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff56c062630 con 0x7ff5580a3270 2024-09-19T07:34:46.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.886+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5580a3270 msgr2=0x7ff5580a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:46.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5580a3270 0x7ff5580a5730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff56c1993d0 tx=0x7ff55c005f90 comp rx=0 tx=0).stop 2024-09-19T07:34:46.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c100ad0 msgr2=0x7ff56c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:46.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c100ad0 0x7ff56c197eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff56800d950 tx=0x7ff56800de10 comp rx=0 tx=0).stop 2024-09-19T07:34:46.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 shutdown_connections 2024-09-19T07:34:46.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5580a3270 0x7ff5580a5730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:46.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff56c102690 0x7ff56c19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:46.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff56c101cd0 0x7ff56c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:46.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 --2- 172.21.15.134:0/2090132280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff56c100ad0 0x7ff56c197eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:46.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 >> 172.21.15.134:0/2090132280 conn(0x7ff56c0fc280 msgr2=0x7ff56c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:34:46.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 shutdown_connections 2024-09-19T07:34:46.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-19T07:34:46.887+0000 7ff5736bf640 1 -- 172.21.15.134:0/2090132280 wait complete. 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:47.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:47 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:47.571 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.572 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:47 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.603 DEBUG:teuthology.orchestra.run.smithi134:osd.10> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T07:34:47.605 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-09-19T07:34:47.606 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T07:34:47.634 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.635 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:47 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:47.863 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:34:49.247 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:49 smithi174 ceph-mon[35370]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:49.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:49 smithi134 ceph-mon[35414]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:49.330 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:34:49 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:34:49.220+0000 7fe476573640 -1 osd.10 0 log_to_monitors true 2024-09-19T07:34:49.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:49 smithi103 ceph-mon[31201]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:50.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:50 smithi134 ceph-mon[35414]: from='osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:34:50.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:50 smithi134 ceph-mon[35414]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:34:50.460 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:50 smithi174 ceph-mon[35370]: from='osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:34:50.460 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:50 smithi174 ceph-mon[35370]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:34:50.463 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-19T07:34:50.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:50 smithi103 ceph-mon[31201]: from='osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:34:50.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:50 smithi103 ceph-mon[31201]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T07:34:50.492 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-09-19T07:34:50.789 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:34:51.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 -- 172.21.15.174:0/63351321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 msgr2=0x7fde44109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:51.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 --2- 172.21.15.174:0/63351321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde44109d50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009ca0 tx=0x7fde3c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:34:51.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 -- 172.21.15.174:0/63351321 shutdown_connections 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 --2- 172.21.15.174:0/63351321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde44109d50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 --2- 172.21.15.174:0/63351321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde44105030 0x7fde44107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 --2- 172.21.15.174:0/63351321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde44102700 0x7fde44104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 -- 172.21.15.174:0/63351321 >> 172.21.15.174:0/63351321 conn(0x7fde440fc2a0 msgr2=0x7fde440fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 -- 172.21.15.174:0/63351321 shutdown_connections 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.283+0000 7fde4b158640 1 -- 172.21.15.174:0/63351321 wait complete. 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 Processor -- start 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 -- start start 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde44102700 0x7fde44197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:51.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde44105030 0x7fde44198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde4419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410d8a0 con 0x7fde44102700 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410d720 con 0x7fde44105030 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde4b158640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde4410da20 con 0x7fde44107960 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde496f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde4419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde496f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde4419f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:57556/0 (socket says 172.21.15.174:57556) 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde496f5640 1 -- 172.21.15.174:0/1058315990 learned_addr learned my addr 172.21.15.174:0/1058315990 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde48ef4640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde44102700 0x7fde44197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.284+0000 7fde43fff640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde44105030 0x7fde44198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde496f5640 1 -- 172.21.15.174:0/1058315990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde44105030 msgr2=0x7fde44198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde496f5640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde44105030 0x7fde44198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:51.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde496f5640 1 -- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde44102700 msgr2=0x7fde44197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde496f5640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde44102700 0x7fde44197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde496f5640 1 -- 172.21.15.174:0/1058315990 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4419fb00 con 0x7fde44107960 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde43fff640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde44105030 0x7fde44198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde496f5640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde4419f490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005e30 tx=0x7fde3c031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde41ffb640 1 -- 172.21.15.174:0/1058315990 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c003c30 con 0x7fde44107960 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde41ffb640 1 -- 172.21.15.174:0/1058315990 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde3c003dd0 con 0x7fde44107960 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde41ffb640 1 -- 172.21.15.174:0/1058315990 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c02fde0 con 0x7fde44107960 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4419fd90 con 0x7fde44107960 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.285+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441a0270 con 0x7fde44107960 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.286+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde440fdd90 con 0x7fde44107960 2024-09-19T07:34:51.300 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.287+0000 7fde41ffb640 1 -- 172.21.15.174:0/1058315990 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fde3c004050 con 0x7fde44107960 2024-09-19T07:34:51.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.288+0000 7fde41ffb640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde340a3270 0x7fde340a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:34:51.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.289+0000 7fde41ffb640 1 -- 172.21.15.174:0/1058315990 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6414+0+0 (secure 0 0 0) 0x7fde3c0f0470 con 0x7fde44107960 2024-09-19T07:34:51.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.289+0000 7fde48ef4640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde340a3270 0x7fde340a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:34:51.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.289+0000 7fde48ef4640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde340a3270 0x7fde340a5730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fde3800bbe0 tx=0x7fde3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:34:51.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.293+0000 7fde41ffb640 1 -- 172.21.15.174:0/1058315990 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fde3c0bd860 con 0x7fde44107960 2024-09-19T07:34:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T07:34:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T07:34:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:34:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.473 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:34:51.473+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fde44062630 con 0x7fde340a3270 2024-09-19T07:34:51.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:34:51.225+0000 7fe4724f6640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T07:34:51.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:34:51.238+0000 7fe46e31e640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:34:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T07:34:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:51.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:51.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:34:51.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi134 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:52 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:52.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi134 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:52 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi134 2024-09-19T07:34:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi134 to 4272M 2024-09-19T07:34:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-09-19T07:34:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: osdmap e68: 11 total, 10 up, 11 in 2024-09-19T07:34:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:52.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:52.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T07:34:52.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T07:34:52.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:52 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:34:53.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:34:53.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:34:53.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:53.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:53.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:53.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] boot 2024-09-19T07:34:53.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T07:34:53.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:34:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:34:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:53.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:53.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] boot 2024-09-19T07:34:53.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T07:34:53.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:34:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:34:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:34:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-09-19T07:34:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: osd.10 [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] boot 2024-09-19T07:34:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T07:34:53.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:34:54.574 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:54 smithi174 ceph-mon[35370]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T07:34:54.574 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:54 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/3293675176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]: dispatch 2024-09-19T07:34:54.575 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:54 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]: dispatch 2024-09-19T07:34:54.575 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:54 smithi174 ceph-mon[35370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]': finished 2024-09-19T07:34:54.575 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:54 smithi174 ceph-mon[35370]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T07:34:54.575 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:54 smithi134 ceph-mon[35414]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T07:34:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:54 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/3293675176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]: dispatch 2024-09-19T07:34:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:54 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]: dispatch 2024-09-19T07:34:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:54 smithi134 ceph-mon[35414]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]': finished 2024-09-19T07:34:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:54 smithi134 ceph-mon[35414]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T07:34:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:54 smithi103 ceph-mon[31201]: osdmap e70: 11 total, 11 up, 11 in 2024-09-19T07:34:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:54 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/3293675176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]: dispatch 2024-09-19T07:34:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:54 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]: dispatch 2024-09-19T07:34:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:54 smithi103 ceph-mon[31201]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d731972a-f6ca-4e9d-b1a7-9b1b82f0e381"}]': finished 2024-09-19T07:34:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:54 smithi103 ceph-mon[31201]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T07:34:54.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:55.493 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:55 smithi134 ceph-mon[35414]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:34:55.493 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:55 smithi134 ceph-mon[35414]: from='client.? 172.21.15.174:0/3287390969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:55.493 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:55 smithi134 ceph-mon[35414]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T07:34:55.493 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:55 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:55.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:55 smithi174 ceph-mon[35370]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:34:55.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:55 smithi174 ceph-mon[35370]: from='client.? 172.21.15.174:0/3287390969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:55.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:55 smithi174 ceph-mon[35370]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T07:34:55.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:55 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:55 smithi103 ceph-mon[31201]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:34:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:55 smithi103 ceph-mon[31201]: from='client.? 172.21.15.174:0/3287390969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T07:34:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:55 smithi103 ceph-mon[31201]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T07:34:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:55 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:57 smithi134 ceph-mon[35414]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T07:34:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:57 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:57 smithi134 ceph-mon[35414]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:34:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:57 smithi174 ceph-mon[35370]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T07:34:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:57 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:57 smithi174 ceph-mon[35370]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:34:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:57 smithi103 ceph-mon[31201]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T07:34:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:57 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:34:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:57 smithi103 ceph-mon[31201]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:34:59.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:34:59 smithi174 ceph-mon[35370]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-19T07:34:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:34:59 smithi103 ceph-mon[31201]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-19T07:34:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:34:59 smithi134 ceph-mon[35414]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-19T07:35:01.271 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:01 smithi174 ceph-mon[35370]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-19T07:35:01.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:01 smithi103 ceph-mon[31201]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-19T07:35:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:01 smithi134 ceph-mon[35414]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-19T07:35:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:03 smithi174 ceph-mon[35370]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:35:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T07:35:03.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:03 smithi103 ceph-mon[31201]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:35:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T07:35:03.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:03 smithi134 ceph-mon[35414]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T07:35:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T07:35:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:04 smithi174 ceph-mon[35370]: Deploying daemon osd.11 on smithi174 2024-09-19T07:35:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:04 smithi103 ceph-mon[31201]: Deploying daemon osd.11 on smithi174 2024-09-19T07:35:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:04 smithi134 ceph-mon[35414]: Deploying daemon osd.11 on smithi174 2024-09-19T07:35:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:05 smithi174 ceph-mon[35370]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T07:35:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:05 smithi103 ceph-mon[31201]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T07:35:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:05 smithi134 ceph-mon[35414]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-19T07:35:07.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:07 smithi174 ceph-mon[35370]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T07:35:07.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:07 smithi103 ceph-mon[31201]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T07:35:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:07 smithi134 ceph-mon[35414]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-19T07:35:09.310 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:09 smithi174 ceph-mon[35370]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:35:09.310 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:09.311 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:09.311 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:09 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:09 smithi103 ceph-mon[31201]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:35:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:09.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:09 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:09 smithi134 ceph-mon[35414]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T07:35:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:09 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:11.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:11 smithi174 ceph-mon[35370]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:11 smithi103 ceph-mon[31201]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:11 smithi134 ceph-mon[35414]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:12.444 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-09-19T07:35:12.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.442+0000 7fde41ffb640 1 -- 172.21.15.174:0/1058315990 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fde44062630 con 0x7fde340a3270 2024-09-19T07:35:12.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde340a3270 msgr2=0x7fde340a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:12.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde340a3270 0x7fde340a5730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fde3800bbe0 tx=0x7fde3800b390 comp rx=0 tx=0).stop 2024-09-19T07:35:12.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 msgr2=0x7fde4419f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:12.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde4419f490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005e30 tx=0x7fde3c031ee0 comp rx=0 tx=0).stop 2024-09-19T07:35:12.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 shutdown_connections 2024-09-19T07:35:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde340a3270 0x7fde340a5730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde44107960 0x7fde4419f490 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde44105030 0x7fde44198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 --2- 172.21.15.174:0/1058315990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde44102700 0x7fde44197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 >> 172.21.15.174:0/1058315990 conn(0x7fde440fc2a0 msgr2=0x7fde44102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 shutdown_connections 2024-09-19T07:35:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:12.445+0000 7fde4b158640 1 -- 172.21.15.174:0/1058315990 wait complete. 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:12 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:13.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:13.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:12 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.099 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:12 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:13.125 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T07:35:13.128 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T07:35:13.128 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T07:35:13.379 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:13.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 -- 172.21.15.103:0/3799289330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1554100350 msgr2=0x7f155410a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:13.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 --2- 172.21.15.103:0/3799289330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1554100350 0x7f155410a1e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f154c009ca0 tx=0x7f154c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:35:13.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 -- 172.21.15.103:0/3799289330 shutdown_connections 2024-09-19T07:35:13.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 --2- 172.21.15.103:0/3799289330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1554100350 0x7f155410a1e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:13.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 --2- 172.21.15.103:0/3799289330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1554103bc0 0x7f15540ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:13.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 --2- 172.21.15.103:0/3799289330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15541031f0 0x7f15541035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 -- 172.21.15.103:0/3799289330 >> 172.21.15.103:0/3799289330 conn(0x7f15540fba40 msgr2=0x7f15540fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 -- 172.21.15.103:0/3799289330 shutdown_connections 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.882+0000 7f155c485640 1 -- 172.21.15.103:0/3799289330 wait complete. 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 Processor -- start 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 -- start start 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1554100350 0x7f1554197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15541031f0 0x7f1554198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1554103bc0 0x7f155419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f155410d960 con 0x7f15541031f0 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f155410d7e0 con 0x7f1554100350 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155c485640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f155410dae0 con 0x7f1554103bc0 2024-09-19T07:35:13.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15541031f0 0x7f1554198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15541031f0 0x7f1554198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46816/0 (socket says 172.21.15.103:46816) 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 -- 172.21.15.103:0/2601662339 learned_addr learned my addr 172.21.15.103:0/2601662339 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155a221640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1554100350 0x7f1554197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155aa22640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1554103bc0 0x7f155419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 -- 172.21.15.103:0/2601662339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1554103bc0 msgr2=0x7f155419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1554103bc0 0x7f155419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 -- 172.21.15.103:0/2601662339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1554100350 msgr2=0x7f1554197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1554100350 0x7f1554197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f1559a20640 1 -- 172.21.15.103:0/2601662339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155419fc20 con 0x7f15541031f0 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155aa22640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1554103bc0 0x7f155419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:35:13.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.883+0000 7f155a221640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1554100350 0x7f1554197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.884+0000 7f1559a20640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15541031f0 0x7f1554198530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f154400d740 tx=0x7f154400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.884+0000 7f154b7fe640 1 -- 172.21.15.103:0/2601662339 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544017070 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.884+0000 7f154b7fe640 1 -- 172.21.15.103:0/2601662339 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154400de50 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.884+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155419ff10 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.884+0000 7f154b7fe640 1 -- 172.21.15.103:0/2601662339 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544004e00 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.884+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15541a03f0 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.885+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15541091b0 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.886+0000 7f154b7fe640 1 -- 172.21.15.103:0/2601662339 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f1544004fa0 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.887+0000 7f154b7fe640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f15400a3270 0x7f15400a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.887+0000 7f155a221640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f15400a3270 0x7f15400a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.887+0000 7f154b7fe640 1 -- 172.21.15.103:0/2601662339 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7f15440cf0b0 con 0x7f15541031f0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.887+0000 7f155a221640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f15400a3270 0x7f15400a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f155406a4b0 tx=0x7f1550008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:13.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:13.890+0000 7f154b7fe640 1 -- 172.21.15.103:0/2601662339 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f154409c3b0 con 0x7f15541031f0 2024-09-19T07:35:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.056+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1554062e20 con 0x7f15541031f0 2024-09-19T07:35:14.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.056+0000 7f154b7fe640 1 -- 172.21.15.103:0/2601662339 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f154409bcb0 con 0x7f15541031f0 2024-09-19T07:35:14.057 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f15400a3270 msgr2=0x7f15400a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f15400a3270 0x7f15400a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f155406a4b0 tx=0x7f1550008040 comp rx=0 tx=0).stop 2024-09-19T07:35:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15541031f0 msgr2=0x7f1554198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15541031f0 0x7f1554198530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f154400d740 tx=0x7f154400dc00 comp rx=0 tx=0).stop 2024-09-19T07:35:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 shutdown_connections 2024-09-19T07:35:14.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f15400a3270 0x7f15400a5730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1554103bc0 0x7f155419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15541031f0 0x7f1554198530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 --2- 172.21.15.103:0/2601662339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1554100350 0x7f1554197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 >> 172.21.15.103:0/2601662339 conn(0x7f15540fba40 msgr2=0x7f1554106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 shutdown_connections 2024-09-19T07:35:14.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:14.058+0000 7f155c485640 1 -- 172.21.15.103:0/2601662339 wait complete. 2024-09-19T07:35:14.389 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:14 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2601662339' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:14.431 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:14 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2601662339' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:14 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2601662339' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:14.886 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726731292,"num_in_osds":12,"osd_in_since":1726731294,"num_remapped_pgs":0} 2024-09-19T07:35:15.123 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:35:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:35:14.883+0000 7f03d72dd640 -1 osd.11 0 log_to_monitors true 2024-09-19T07:35:15.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:15 smithi174 ceph-mon[35370]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:15.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:15 smithi174 ceph-mon[35370]: from='osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:35:15.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:15 smithi174 ceph-mon[35370]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:35:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:15 smithi103 ceph-mon[31201]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:15 smithi103 ceph-mon[31201]: from='osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:35:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:15 smithi103 ceph-mon[31201]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:35:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:15 smithi134 ceph-mon[35414]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:15 smithi134 ceph-mon[35414]: from='osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:35:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:15 smithi134 ceph-mon[35414]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T07:35:15.887 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T07:35:16.139 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:16 smithi103 ceph-mon[31201]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T07:35:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:16 smithi103 ceph-mon[31201]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T07:35:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:16 smithi103 ceph-mon[31201]: from='osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:35:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:16 smithi103 ceph-mon[31201]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:35:16.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:35:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:35:16.133+0000 7f03d3260640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T07:35:16.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:35:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:35:16.148+0000 7f03cf088640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T07:35:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:16 smithi174 ceph-mon[35370]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T07:35:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:16 smithi174 ceph-mon[35370]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T07:35:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:16 smithi174 ceph-mon[35370]: from='osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:35:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:16 smithi174 ceph-mon[35370]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:35:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:16 smithi134 ceph-mon[35414]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T07:35:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:16 smithi134 ceph-mon[35414]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T07:35:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:16 smithi134 ceph-mon[35414]: from='osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:35:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:16 smithi134 ceph-mon[35414]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-19T07:35:16.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 -- 172.21.15.103:0/968629729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf8102e00 msgr2=0x7f0bf8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:16.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 --2- 172.21.15.103:0/968629729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf8102e00 0x7f0bf8103280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009900 tx=0x7f0be402a410 comp rx=0 tx=0).stop 2024-09-19T07:35:16.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 -- 172.21.15.103:0/968629729 shutdown_connections 2024-09-19T07:35:16.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 --2- 172.21.15.103:0/968629729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf81037c0 0x7f0bf810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 --2- 172.21.15.103:0/968629729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf8102e00 0x7f0bf8103280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 --2- 172.21.15.103:0/968629729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8101c00 0x7f0bf8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 -- 172.21.15.103:0/968629729 >> 172.21.15.103:0/968629729 conn(0x7f0bf80fd390 msgr2=0x7f0bf80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 -- 172.21.15.103:0/968629729 shutdown_connections 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.646+0000 7f0bff08a640 1 -- 172.21.15.103:0/968629729 wait complete. 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 Processor -- start 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 -- start start 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8101c00 0x7f0bf8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf8102e00 0x7f0bf81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81037c0 0x7f0bf819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:16.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf810d860 con 0x7f0bf81037c0 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf810d6e0 con 0x7f0bf8102e00 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bff08a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf810d9e0 con 0x7f0bf8101c00 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81037c0 0x7f0bf819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfce26640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8101c00 0x7f0bf8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81037c0 0x7f0bf819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46830/0 (socket says 172.21.15.103:46830) 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfce26640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8101c00 0x7f0bf8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37314/0 (socket says 172.21.15.103:37314) 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 -- 172.21.15.103:0/1002779859 learned_addr learned my addr 172.21.15.103:0/1002779859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bf7fff640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf8102e00 0x7f0bf81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 -- 172.21.15.103:0/1002779859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8101c00 msgr2=0x7f0bf8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8101c00 0x7f0bf8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 -- 172.21.15.103:0/1002779859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf8102e00 msgr2=0x7f0bf81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:16.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf8102e00 0x7f0bf81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.647+0000 7f0bfd627640 1 -- 172.21.15.103:0/1002779859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf819fad0 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.648+0000 7f0bfd627640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81037c0 0x7f0bf819f460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000eaa0 tx=0x7f0bf000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.648+0000 7f0bf5ffb640 1 -- 172.21.15.103:0/1002779859 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000cdb0 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.648+0000 7f0bf5ffb640 1 -- 172.21.15.103:0/1002779859 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf0005020 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.648+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf819fdc0 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.648+0000 7f0bf5ffb640 1 -- 172.21.15.103:0/1002779859 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf00116b0 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.648+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf81a0220 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.649+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf8108fa0 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.650+0000 7f0bf5ffb640 1 -- 172.21.15.103:0/1002779859 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0bf0005560 con 0x7f0bf81037c0 2024-09-19T07:35:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.651+0000 7f0bf5ffb640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f0be80a3270 0x7f0be80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:16.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.651+0000 7f0bfce26640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f0be80a3270 0x7f0be80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:16.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.651+0000 7f0bf5ffb640 1 -- 172.21.15.103:0/1002779859 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7f0bf00cf580 con 0x7f0bf81037c0 2024-09-19T07:35:16.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.651+0000 7f0bfce26640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f0be80a3270 0x7f0be80a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8102c60 tx=0x7f0bec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:16.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.655+0000 7f0bf5ffb640 1 -- 172.21.15.103:0/1002779859 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0bf0016030 con 0x7f0bf81037c0 2024-09-19T07:35:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.822+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0bf8062e90 con 0x7f0bf81037c0 2024-09-19T07:35:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.823+0000 7f0bf5ffb640 1 -- 172.21.15.103:0/1002779859 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f0bf009c860 con 0x7f0bf81037c0 2024-09-19T07:35:16.823 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:16.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f0be80a3270 msgr2=0x7f0be80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:16.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f0be80a3270 0x7f0be80a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8102c60 tx=0x7f0bec00b390 comp rx=0 tx=0).stop 2024-09-19T07:35:16.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81037c0 msgr2=0x7f0bf819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:16.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81037c0 0x7f0bf819f460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000eaa0 tx=0x7f0bf000ef60 comp rx=0 tx=0).stop 2024-09-19T07:35:16.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 shutdown_connections 2024-09-19T07:35:16.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f0be80a3270 0x7f0be80a5730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf81037c0 0x7f0bf819f460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf8102e00 0x7f0bf81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 --2- 172.21.15.103:0/1002779859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf8101c00 0x7f0bf8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:16.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 >> 172.21.15.103:0/1002779859 conn(0x7f0bf80fd390 msgr2=0x7f0bf8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:16.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 shutdown_connections 2024-09-19T07:35:16.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:16.825+0000 7f0bff08a640 1 -- 172.21.15.103:0/1002779859 wait complete. 2024-09-19T07:35:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: Detected new or changed devices on smithi174 2024-09-19T07:35:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T07:35:17.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:17.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:17.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:17 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1002779859' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: Detected new or changed devices on smithi174 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.461 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.462 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.462 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T07:35:17.462 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.462 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:17.462 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:17.462 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.462 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:17 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1002779859' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:17.487 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726731292,"num_in_osds":12,"osd_in_since":1726731294,"num_remapped_pgs":0} 2024-09-19T07:35:17.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:17.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: Detected new or changed devices on smithi174 2024-09-19T07:35:17.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-19T07:35:17.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: osdmap e75: 12 total, 11 up, 12 in 2024-09-19T07:35:17.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:17.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:17.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:35:17.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-19T07:35:17.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:17.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:17.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:17.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:17 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1002779859' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:18 smithi174 ceph-mon[35370]: purged_snaps scrub starts 2024-09-19T07:35:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:18 smithi174 ceph-mon[35370]: purged_snaps scrub ok 2024-09-19T07:35:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:18 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:18 smithi174 ceph-mon[35370]: osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] boot 2024-09-19T07:35:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:18 smithi174 ceph-mon[35370]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T07:35:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:18 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:18 smithi103 ceph-mon[31201]: purged_snaps scrub starts 2024-09-19T07:35:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:18 smithi103 ceph-mon[31201]: purged_snaps scrub ok 2024-09-19T07:35:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:18 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:18 smithi103 ceph-mon[31201]: osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] boot 2024-09-19T07:35:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:18 smithi103 ceph-mon[31201]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T07:35:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:18 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:18.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T07:35:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:18 smithi134 ceph-mon[35414]: purged_snaps scrub starts 2024-09-19T07:35:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:18 smithi134 ceph-mon[35414]: purged_snaps scrub ok 2024-09-19T07:35:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:18 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:18 smithi134 ceph-mon[35414]: osd.11 [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] boot 2024-09-19T07:35:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:18 smithi134 ceph-mon[35414]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T07:35:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:18 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:35:18.741 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:19.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 -- 172.21.15.103:0/3438122845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c1078c0 msgr2=0x7ff96c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:19.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 --2- 172.21.15.103:0/3438122845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c1078c0 0x7ff96c109d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff964009ca0 tx=0x7ff96402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 -- 172.21.15.103:0/3438122845 shutdown_connections 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 --2- 172.21.15.103:0/3438122845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c1078c0 0x7ff96c109d10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 --2- 172.21.15.103:0/3438122845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff96c104f90 0x7ff96c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 --2- 172.21.15.103:0/3438122845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff96c102660 0x7ff96c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 -- 172.21.15.103:0/3438122845 >> 172.21.15.103:0/3438122845 conn(0x7ff96c0fc280 msgr2=0x7ff96c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 -- 172.21.15.103:0/3438122845 shutdown_connections 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 -- 172.21.15.103:0/3438122845 wait complete. 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 Processor -- start 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.221+0000 7ff972768640 1 -- start start 2024-09-19T07:35:19.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff972768640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c102660 0x7ff96c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff972768640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff96c104f90 0x7ff96c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff972768640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff96c1078c0 0x7ff96c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff972768640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c10d8f0 con 0x7ff96c102660 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff972768640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c10d770 con 0x7ff96c1078c0 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff972768640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c10da70 con 0x7ff96c104f90 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c102660 0x7ff96c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c102660 0x7ff96c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44194/0 (socket says 172.21.15.103:44194) 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 -- 172.21.15.103:0/3764736479 learned_addr learned my addr 172.21.15.103:0/3764736479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff970d05640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff96c1078c0 0x7ff96c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96b7fe640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff96c104f90 0x7ff96c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:19.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 -- 172.21.15.103:0/3764736479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff96c104f90 msgr2=0x7ff96c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff96c104f90 0x7ff96c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 -- 172.21.15.103:0/3764736479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff96c1078c0 msgr2=0x7ff96c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff96c1078c0 0x7ff96c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 -- 172.21.15.103:0/3764736479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c19fb70 con 0x7ff96c102660 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff970d05640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff96c1078c0 0x7ff96c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.222+0000 7ff96bfff640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c102660 0x7ff96c197f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00d770 tx=0x7ff95c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.223+0000 7ff9697fa640 1 -- 172.21.15.103:0/3764736479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c017070 con 0x7ff96c102660 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.223+0000 7ff9697fa640 1 -- 172.21.15.103:0/3764736479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff95c005500 con 0x7ff96c102660 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.223+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c19fe60 con 0x7ff96c102660 2024-09-19T07:35:19.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.223+0000 7ff9697fa640 1 -- 172.21.15.103:0/3764736479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004e00 con 0x7ff96c102660 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.223+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1a02e0 con 0x7ff96c102660 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.224+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c0fdd70 con 0x7ff96c102660 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.226+0000 7ff9697fa640 1 -- 172.21.15.103:0/3764736479 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff95c004fa0 con 0x7ff96c102660 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.226+0000 7ff9697fa640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9540a3270 0x7ff9540a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.226+0000 7ff96b7fe640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9540a3270 0x7ff9540a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.226+0000 7ff9697fa640 1 -- 172.21.15.103:0/3764736479 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff95c0cf330 con 0x7ff96c102660 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.227+0000 7ff96b7fe640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9540a3270 0x7ff9540a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff96c199460 tx=0x7ff960008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:19.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.229+0000 7ff9697fa640 1 -- 172.21.15.103:0/3764736479 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff95c09c4f0 con 0x7ff96c102660 2024-09-19T07:35:19.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.395+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff96c02da30 con 0x7ff96c102660 2024-09-19T07:35:19.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:19 smithi103 ceph-mon[31201]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:19.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:19 smithi103 ceph-mon[31201]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T07:35:19.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.395+0000 7ff9697fa640 1 -- 172.21.15.103:0/3764736479 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7ff95c09bdf0 con 0x7ff96c102660 2024-09-19T07:35:19.395 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:19.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9540a3270 msgr2=0x7ff9540a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:19.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9540a3270 0x7ff9540a5730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff96c199460 tx=0x7ff960008040 comp rx=0 tx=0).stop 2024-09-19T07:35:19.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c102660 msgr2=0x7ff96c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:19.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c102660 0x7ff96c197f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00d770 tx=0x7ff95c00dc30 comp rx=0 tx=0).stop 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 shutdown_connections 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9540a3270 0x7ff9540a5730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff96c1078c0 0x7ff96c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff96c104f90 0x7ff96c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 --2- 172.21.15.103:0/3764736479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff96c102660 0x7ff96c197f40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.397+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 >> 172.21.15.103:0/3764736479 conn(0x7ff96c0fc280 msgr2=0x7ff96c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.398+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 shutdown_connections 2024-09-19T07:35:19.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:19.398+0000 7ff972768640 1 -- 172.21.15.103:0/3764736479 wait complete. 2024-09-19T07:35:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:19 smithi174 ceph-mon[35370]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:19 smithi174 ceph-mon[35370]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T07:35:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:19 smithi134 ceph-mon[35414]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 983 GiB / 983 GiB avail 2024-09-19T07:35:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:19 smithi134 ceph-mon[35414]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T07:35:20.089 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726731317,"num_in_osds":12,"osd_in_since":1726731294,"num_remapped_pgs":0} 2024-09-19T07:35:20.090 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:35:20.337 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:20 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3764736479' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:20.458 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:20 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3764736479' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:20.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:20 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3764736479' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T07:35:20.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 -- 172.21.15.103:0/3014157188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 msgr2=0x7fb01010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 --2- 172.21.15.103:0/3014157188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01010ea90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb008009ca0 tx=0x7fb00802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 -- 172.21.15.103:0/3014157188 shutdown_connections 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 --2- 172.21.15.103:0/3014157188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01010ea90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 --2- 172.21.15.103:0/3014157188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb010102790 0x7fb0101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 --2- 172.21.15.103:0/3014157188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb010101de0 0x7fb0101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 -- 172.21.15.103:0/3014157188 >> 172.21.15.103:0/3014157188 conn(0x7fb0100fd8c0 msgr2=0x7fb0100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 -- 172.21.15.103:0/3014157188 shutdown_connections 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.833+0000 7fb016be0640 1 -- 172.21.15.103:0/3014157188 wait complete. 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 Processor -- start 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 -- start start 2024-09-19T07:35:20.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb010101de0 0x7fb010198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb010102790 0x7fb0101989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb010111fb0 con 0x7fb010106f10 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb010111e30 con 0x7fb010102790 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.834+0000 7fb016be0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb010112130 con 0x7fb010101de0 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44210/0 (socket says 172.21.15.103:44210) 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 -- 172.21.15.103:0/4248712342 learned_addr learned my addr 172.21.15.103:0/4248712342 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01497c640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb010101de0 0x7fb010198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb00ffff640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb010102790 0x7fb0101989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:20.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 -- 172.21.15.103:0/4248712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb010101de0 msgr2=0x7fb010198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb010101de0 0x7fb010198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 -- 172.21.15.103:0/4248712342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb010102790 msgr2=0x7fb0101989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb010102790 0x7fb0101989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 -- 172.21.15.103:0/4248712342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01019d440 con 0x7fb010106f10 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb00ffff640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb010102790 0x7fb0101989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01497c640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb010101de0 0x7fb010198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb01517d640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01019cd60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb008031a20 tx=0x7fb008031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb00dffb640 1 -- 172.21.15.103:0/4248712342 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008003c30 con 0x7fb010106f10 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb00dffb640 1 -- 172.21.15.103:0/4248712342 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb008003dd0 con 0x7fb010106f10 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb00dffb640 1 -- 172.21.15.103:0/4248712342 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00802fcf0 con 0x7fb010106f10 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01019d6d0 con 0x7fb010106f10 2024-09-19T07:35:20.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.835+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101a5010 con 0x7fb010106f10 2024-09-19T07:35:20.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.836+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0101021c0 con 0x7fb010106f10 2024-09-19T07:35:20.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.837+0000 7fb00dffb640 1 -- 172.21.15.103:0/4248712342 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb008004050 con 0x7fb010106f10 2024-09-19T07:35:20.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.838+0000 7fb00dffb640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb0000a32c0 0x7fb0000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:20.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.838+0000 7fb01497c640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb0000a32c0 0x7fb0000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:20.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.838+0000 7fb00dffb640 1 -- 172.21.15.103:0/4248712342 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fb0080f0860 con 0x7fb010106f10 2024-09-19T07:35:20.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.839+0000 7fb01497c640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb0000a32c0 0x7fb0000a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb004009930 tx=0x7fb004008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:20.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:20.843+0000 7fb00dffb640 1 -- 172.21.15.103:0/4248712342 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb0080bda90 con 0x7fb010106f10 2024-09-19T07:35:21.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.026+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb010109b60 con 0x7fb010106f10 2024-09-19T07:35:21.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.027+0000 7fb00dffb640 1 -- 172.21.15.103:0/4248712342 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18526 (secure 0 0 0) 0x7fb00802fe90 con 0x7fb010106f10 2024-09-19T07:35:21.027 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:21.027 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":77,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:35:18.152557+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:35:21.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb0000a32c0 msgr2=0x7fb0000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:21.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb0000a32c0 0x7fb0000a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb004009930 tx=0x7fb004008040 comp rx=0 tx=0).stop 2024-09-19T07:35:21.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 msgr2=0x7fb01019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:21.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01019cd60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb008031a20 tx=0x7fb008031a50 comp rx=0 tx=0).stop 2024-09-19T07:35:21.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 shutdown_connections 2024-09-19T07:35:21.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb0000a32c0 0x7fb0000a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:21.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb010106f10 0x7fb01019cd60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:21.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb010102790 0x7fb0101989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:21.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 --2- 172.21.15.103:0/4248712342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb010101de0 0x7fb010198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:21.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 >> 172.21.15.103:0/4248712342 conn(0x7fb0100fd8c0 msgr2=0x7fb0100fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:21.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 shutdown_connections 2024-09-19T07:35:21.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:21.029+0000 7fb016be0640 1 -- 172.21.15.103:0/4248712342 wait complete. 2024-09-19T07:35:21.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:21 smithi103 ceph-mon[31201]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:21.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:21 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4248712342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:35:21.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:21 smithi174 ceph-mon[35370]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:21.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:21 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4248712342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:35:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:21 smithi134 ceph-mon[35414]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:21 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4248712342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:35:21.827 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:31:30.307307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:35:21.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T07:35:22.068 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 -- 172.21.15.103:0/2298203908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 msgr2=0x7f85881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 --2- 172.21.15.103:0/2298203908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881029b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f857c009900 tx=0x7f857c02a480 comp rx=0 tx=0).stop 2024-09-19T07:35:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 -- 172.21.15.103:0/2298203908 shutdown_connections 2024-09-19T07:35:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 --2- 172.21.15.103:0/2298203908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8588102ef0 0x7f858810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 --2- 172.21.15.103:0/2298203908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881029b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 --2- 172.21.15.103:0/2298203908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588100e20 0x7f8588101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 -- 172.21.15.103:0/2298203908 >> 172.21.15.103:0/2298203908 conn(0x7f85880fc7d0 msgr2=0x7f85880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 -- 172.21.15.103:0/2298203908 shutdown_connections 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.587+0000 7f858d1c3640 1 -- 172.21.15.103:0/2298203908 wait complete. 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 Processor -- start 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 -- start start 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588100e20 0x7f85881994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8588102ef0 0x7f85881a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8588111020 con 0x7f8588102570 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8588110ea0 con 0x7f8588100e20 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858d1c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85881111a0 con 0x7f8588102ef0 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44224/0 (socket says 172.21.15.103:44224) 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858659c640 1 -- 172.21.15.103:0/3856343748 learned_addr learned my addr 172.21.15.103:0/3856343748 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858759e640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8588102ef0 0x7f85881a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f8586d9d640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588100e20 0x7f85881994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858659c640 1 -- 172.21.15.103:0/3856343748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8588102ef0 msgr2=0x7f85881a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.588+0000 7f858659c640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8588102ef0 0x7f85881a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f858659c640 1 -- 172.21.15.103:0/3856343748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588100e20 msgr2=0x7f85881994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:22.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f858659c640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588100e20 0x7f85881994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f858659c640 1 -- 172.21.15.103:0/3856343748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85881a4a70 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f858659c640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881999f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f857c006010 tx=0x7f857c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f856ffff640 1 -- 172.21.15.103:0/3856343748 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c03f070 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f856ffff640 1 -- 172.21.15.103:0/3856343748 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c031e30 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85881a4c40 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f856ffff640 1 -- 172.21.15.103:0/3856343748 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c035030 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.589+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881a5090 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.590+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8588101280 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.591+0000 7f856ffff640 1 -- 172.21.15.103:0/3856343748 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f857c042070 con 0x7f8588102570 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.592+0000 7f856ffff640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85700a3240 0x7f85700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.592+0000 7f8586d9d640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85700a3240 0x7f85700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.592+0000 7f856ffff640 1 -- 172.21.15.103:0/3856343748 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f857c0f04a0 con 0x7f8588102570 2024-09-19T07:35:22.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.592+0000 7f8586d9d640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85700a3240 0x7f85700a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8578009990 tx=0x7f8578009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:22.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.595+0000 7f856ffff640 1 -- 172.21.15.103:0/3856343748 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f857c0bd660 con 0x7f8588102570 2024-09-19T07:35:22.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.762+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8588109090 con 0x7f8588102570 2024-09-19T07:35:22.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.763+0000 7f856ffff640 1 -- 172.21.15.103:0/3856343748 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f857c0bcf60 con 0x7f8588102570 2024-09-19T07:35:22.763 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-19T07:35:22.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.765+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85700a3240 msgr2=0x7f85700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:22.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.765+0000 7f858d1c3640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85700a3240 0x7f85700a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8578009990 tx=0x7f8578009110 comp rx=0 tx=0).stop 2024-09-19T07:35:22.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.765+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 msgr2=0x7f85881999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:22.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.765+0000 7f858d1c3640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881999f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f857c006010 tx=0x7f857c003c80 comp rx=0 tx=0).stop 2024-09-19T07:35:22.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 shutdown_connections 2024-09-19T07:35:22.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f85700a3240 0x7f85700a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8588102ef0 0x7f85881a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8588102570 0x7f85881999f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 --2- 172.21.15.103:0/3856343748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588100e20 0x7f85881994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:22.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 >> 172.21.15.103:0/3856343748 conn(0x7f85880fc7d0 msgr2=0x7f85880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:22.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 shutdown_connections 2024-09-19T07:35:22.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:22.766+0000 7f858d1c3640 1 -- 172.21.15.103:0/3856343748 wait complete. 2024-09-19T07:35:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:23 smithi103 ceph-mon[31201]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:23 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3856343748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.a on smithi103 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.d on smithi103 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.g on smithi103 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.j on smithi103 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.b on smithi134 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.e on smithi134 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.h on smithi134 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.k on smithi134 2024-09-19T07:35:23.537 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-09-19T07:35:23.538 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-09-19T07:35:23.538 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-09-19T07:35:23.538 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-09-19T07:35:23.538 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-09-19T07:35:23.567 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:23 smithi174 ceph-mon[35370]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:23.567 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:23 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3856343748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:35:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:23 smithi134 ceph-mon[35414]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:23 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3856343748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:35:23.797 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:35:24.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 -- 172.21.15.174:0/306379515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c102690 msgr2=0x7fd39c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:24.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/306379515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c102690 0x7fd39c108f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd394009ca0 tx=0x7fd39402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 -- 172.21.15.174:0/306379515 shutdown_connections 2024-09-19T07:35:24.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/306379515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c102690 0x7fd39c108f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/306379515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd39c101cd0 0x7fd39c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/306379515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c100ad0 0x7fd39c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 -- 172.21.15.174:0/306379515 >> 172.21.15.174:0/306379515 conn(0x7fd39c0fc280 msgr2=0x7fd39c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 -- 172.21.15.174:0/306379515 shutdown_connections 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.293+0000 7fd3a1d10640 1 -- 172.21.15.174:0/306379515 wait complete. 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 Processor -- start 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 -- start start 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c100ad0 0x7fd39c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd39c101cd0 0x7fd39c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c102690 0x7fd39c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10c7d0 con 0x7fd39c101cd0 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10c650 con 0x7fd39c102690 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd3a1d10640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10c950 con 0x7fd39c100ad0 2024-09-19T07:35:24.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd39b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c100ad0 0x7fd39c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd39b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c100ad0 0x7fd39c196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47378/0 (socket says 172.21.15.174:47378) 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.294+0000 7fd39b7fe640 1 -- 172.21.15.174:0/837363355 learned_addr learned my addr 172.21.15.174:0/837363355 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd39bfff640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c102690 0x7fd39c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd39b7fe640 1 -- 172.21.15.174:0/837363355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c102690 msgr2=0x7fd39c19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd39b7fe640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c102690 0x7fd39c19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd39b7fe640 1 -- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd39c101cd0 msgr2=0x7fd39c197330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd39b7fe640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd39c101cd0 0x7fd39c197330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd39b7fe640 1 -- 172.21.15.174:0/837363355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c19ea20 con 0x7fd39c100ad0 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd39b7fe640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c100ad0 0x7fd39c196df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00b450 tx=0x7fd38c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd398ff9640 1 -- 172.21.15.174:0/837363355 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c015020 con 0x7fd39c100ad0 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c19ed10 con 0x7fd39c100ad0 2024-09-19T07:35:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd398ff9640 1 -- 172.21.15.174:0/837363355 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd38c011d70 con 0x7fd39c100ad0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd398ff9640 1 -- 172.21.15.174:0/837363355 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00f340 con 0x7fd39c100ad0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.295+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c19f220 con 0x7fd39c100ad0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.296+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39c107e20 con 0x7fd39c100ad0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.297+0000 7fd398ff9640 1 -- 172.21.15.174:0/837363355 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd38c018020 con 0x7fd39c100ad0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.298+0000 7fd398ff9640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd3840a3270 0x7fd3840a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.299+0000 7fd398ff9640 1 -- 172.21.15.174:0/837363355 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd38c0ceff0 con 0x7fd39c100ad0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.299+0000 7fd39affd640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd3840a3270 0x7fd3840a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.299+0000 7fd39affd640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd3840a3270 0x7fd3840a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd39c198310 tx=0x7fd39000b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:24.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.302+0000 7fd398ff9640 1 -- 172.21.15.174:0/837363355 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd38c01f020 con 0x7fd39c100ad0 2024-09-19T07:35:24.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.483+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) -- 0x7fd39c062830 con 0x7fd3840a3270 2024-09-19T07:35:24.492 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.491+0000 7fd398ff9640 1 -- 172.21.15.174:0/837363355 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd39c062830 con 0x7fd3840a3270 2024-09-19T07:35:24.492 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-09-19T07:35:24.494 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd3840a3270 msgr2=0x7fd3840a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:24.494 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd3840a3270 0x7fd3840a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd39c198310 tx=0x7fd39000b370 comp rx=0 tx=0).stop 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c100ad0 msgr2=0x7fd39c196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c100ad0 0x7fd39c196df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00b450 tx=0x7fd38c00b910 comp rx=0 tx=0).stop 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 shutdown_connections 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd3840a3270 0x7fd3840a5730 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c102690 0x7fd39c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd39c101cd0 0x7fd39c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 --2- 172.21.15.174:0/837363355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd39c100ad0 0x7fd39c196df0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 >> 172.21.15.174:0/837363355 conn(0x7fd39c0fc280 msgr2=0x7fd39c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 shutdown_connections 2024-09-19T07:35:24.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-19T07:35:24.493+0000 7fd3a1d10640 1 -- 172.21.15.174:0/837363355 wait complete. 2024-09-19T07:35:25.211 DEBUG:teuthology.orchestra.run.smithi103:mds.a> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T07:35:25.214 DEBUG:teuthology.orchestra.run.smithi103:mds.d> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T07:35:25.259 DEBUG:teuthology.orchestra.run.smithi103:mds.g> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T07:35:25.301 DEBUG:teuthology.orchestra.run.smithi103:mds.j> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T07:35:25.346 DEBUG:teuthology.orchestra.run.smithi134:mds.b> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T07:35:25.349 DEBUG:teuthology.orchestra.run.smithi134:mds.e> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T07:35:25.377 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:25.378 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:25.378 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:25.378 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:25.378 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:25.378 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:25.378 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:25.378 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:25.379 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:25 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:25.392 DEBUG:teuthology.orchestra.run.smithi134:mds.h> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T07:35:25.436 DEBUG:teuthology.orchestra.run.smithi134:mds.k> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T07:35:25.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:25.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:25.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:25.455 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:25.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:25.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:25.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:25.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:25.456 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:25 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:25.457 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T07:35:25.459 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T07:35:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:35:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:25.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:25.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:25 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:25.503 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T07:35:25.546 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T07:35:25.589 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T07:35:25.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:25.836 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:26.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:26 smithi103 ceph-mon[31201]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:35:26.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:26 smithi103 ceph-mon[31201]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T07:35:26.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:26 smithi103 ceph-mon[31201]: Deploying daemon mds.l on smithi174 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 -- 172.21.15.103:0/3033868732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc102ef0 msgr2=0x7f92fc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 --2- 172.21.15.103:0/3033868732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc102ef0 0x7f92fc10da40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009ca0 tx=0x7f92f402a950 comp rx=0 tx=0).stop 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 -- 172.21.15.103:0/3033868732 shutdown_connections 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 --2- 172.21.15.103:0/3033868732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc102ef0 0x7f92fc10da40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 --2- 172.21.15.103:0/3033868732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc102570 0x7f92fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 --2- 172.21.15.103:0/3033868732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc100e20 0x7f92fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 -- 172.21.15.103:0/3033868732 >> 172.21.15.103:0/3033868732 conn(0x7f92fc0fc7d0 msgr2=0x7f92fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 -- 172.21.15.103:0/3033868732 shutdown_connections 2024-09-19T07:35:26.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 -- 172.21.15.103:0/3033868732 wait complete. 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 Processor -- start 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.335+0000 7f930158f640 1 -- start start 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc100e20 0x7f92fc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc102570 0x7f92fc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc102ef0 0x7f92fc19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc110f80 con 0x7f92fc100e20 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc110e00 con 0x7f92fc102ef0 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc111100 con 0x7f92fc102570 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc100e20 0x7f92fc197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc100e20 0x7f92fc197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44242/0 (socket says 172.21.15.103:44242) 2024-09-19T07:35:26.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 -- 172.21.15.103:0/3067977544 learned_addr learned my addr 172.21.15.103:0/3067977544 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92f3fff640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc102570 0x7f92fc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92fb7fe640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc102ef0 0x7f92fc19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 -- 172.21.15.103:0/3067977544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc102570 msgr2=0x7f92fc1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc102570 0x7f92fc1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 -- 172.21.15.103:0/3067977544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc102ef0 msgr2=0x7f92fc19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc102ef0 0x7f92fc19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 -- 172.21.15.103:0/3067977544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc19c410 con 0x7f92fc100e20 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92faffd640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc100e20 0x7f92fc197460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d770 tx=0x7f92ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92f8ff9640 1 -- 172.21.15.103:0/3067977544 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec017070 con 0x7f92fc100e20 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92f8ff9640 1 -- 172.21.15.103:0/3067977544 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ec005500 con 0x7f92fc100e20 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc19c700 con 0x7f92fc100e20 2024-09-19T07:35:26.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f92f8ff9640 1 -- 172.21.15.103:0/3067977544 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec004e00 con 0x7f92fc100e20 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.336+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a3fe0 con 0x7f92fc100e20 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.337+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc101200 con 0x7f92fc100e20 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.338+0000 7f92f8ff9640 1 -- 172.21.15.103:0/3067977544 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f92ec004fa0 con 0x7f92fc100e20 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.339+0000 7f92f8ff9640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92e80a3240 0x7f92e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.339+0000 7f92f3fff640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92e80a3240 0x7f92e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.339+0000 7f92f8ff9640 1 -- 172.21.15.103:0/3067977544 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f92ec0cedf0 con 0x7f92fc100e20 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.339+0000 7f92f3fff640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92e80a3240 0x7f92e80a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0fece0 tx=0x7f92e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:26.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.343+0000 7f92f8ff9640 1 -- 172.21.15.103:0/3067977544 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92ec09b960 con 0x7f92fc100e20 2024-09-19T07:35:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:26 smithi174 ceph-mon[35370]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:35:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:26 smithi174 ceph-mon[35370]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T07:35:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:26 smithi174 ceph-mon[35370]: Deploying daemon mds.l on smithi174 2024-09-19T07:35:26.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.545+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92fc109090 con 0x7f92fc100e20 2024-09-19T07:35:26.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.546+0000 7f92f8ff9640 1 -- 172.21.15.103:0/3067977544 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f92ec0ce090 con 0x7f92fc100e20 2024-09-19T07:35:26.546 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:26.546 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"btime":"2024-09-19T07:28:30:723424+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T07:35:26.546 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 1 2024-09-19T07:35:26.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92e80a3240 msgr2=0x7f92e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:26.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92e80a3240 0x7f92e80a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0fece0 tx=0x7f92e4005f90 comp rx=0 tx=0).stop 2024-09-19T07:35:26.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc100e20 msgr2=0x7f92fc197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:26.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc100e20 0x7f92fc197460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d770 tx=0x7f92ec00dc30 comp rx=0 tx=0).stop 2024-09-19T07:35:26.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 shutdown_connections 2024-09-19T07:35:26.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92e80a3240 0x7f92e80a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc102ef0 0x7f92fc19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc102570 0x7f92fc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 --2- 172.21.15.103:0/3067977544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92fc100e20 0x7f92fc197460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:26.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 >> 172.21.15.103:0/3067977544 conn(0x7f92fc0fc7d0 msgr2=0x7f92fc1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:26.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 shutdown_connections 2024-09-19T07:35:26.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:26.548+0000 7f930158f640 1 -- 172.21.15.103:0/3067977544 wait complete. 2024-09-19T07:35:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:26 smithi134 ceph-mon[35414]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:35:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:26 smithi134 ceph-mon[35414]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T07:35:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:26 smithi134 ceph-mon[35414]: Deploying daemon mds.l on smithi174 2024-09-19T07:35:27.157 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:26 smithi174 systemd[1]: Starting Ceph mds.l for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:27 smithi103 ceph-mon[31201]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:27.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:27 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3067977544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:27 smithi174 ceph-mon[35370]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:27 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3067977544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:27.415 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 podman[64828]: 2024-09-19 07:35:27.059556256 +0000 UTC m=+0.016048438 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:27.416 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 podman[64828]: 2024-09-19 07:35:27.183286277 +0000 UTC m=+0.139778464 container create fabe4f01144f6c77be7fc447bed5659ea2b06da35103122b47c6ff89b0463ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:35:27.416 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 podman[64828]: 2024-09-19 07:35:27.402615904 +0000 UTC m=+0.359108087 container init fabe4f01144f6c77be7fc447bed5659ea2b06da35103122b47c6ff89b0463ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-19T07:35:27.416 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 podman[64828]: 2024-09-19 07:35:27.405488471 +0000 UTC m=+0.361980667 container start fabe4f01144f6c77be7fc447bed5659ea2b06da35103122b47c6ff89b0463ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T07:35:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:27 smithi134 ceph-mon[35414]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:27 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3067977544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:27.915 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 ceph-mds[64847]: -- 172.21.15.174:0/3780176005 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a95287860 con 0x559a95264800 2024-09-19T07:35:27.915 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l[64843]: starting mds.l at 2024-09-19T07:35:27.916 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 bash[64828]: fabe4f01144f6c77be7fc447bed5659ea2b06da35103122b47c6ff89b0463ad5 2024-09-19T07:35:27.916 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:35:27 smithi174 systemd[1]: Started Ceph mds.l for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6832/1885230172,v1:172.21.15.174:6833/1885230172] up:boot 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: fsmap 1 up:standby 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: Deploying daemon mds.k on smithi134 2024-09-19T07:35:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:28 smithi134 ceph-mon[35414]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6832/1885230172,v1:172.21.15.174:6833/1885230172] up:boot 2024-09-19T07:35:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: fsmap 1 up:standby 2024-09-19T07:35:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:35:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:28.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:28.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:28.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: Deploying daemon mds.k on smithi134 2024-09-19T07:35:28.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:28 smithi103 ceph-mon[31201]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:28.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6832/1885230172,v1:172.21.15.174:6833/1885230172] up:boot 2024-09-19T07:35:28.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: fsmap 1 up:standby 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: Deploying daemon mds.k on smithi134 2024-09-19T07:35:28.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:28 smithi174 ceph-mon[35370]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:29.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:29.797 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:29 smithi134 systemd[1]: Starting Ceph mds.k for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:29.797 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:29 smithi134 podman[64631]: 2024-09-19 07:35:29.699108801 +0000 UTC m=+0.015475721 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:30.066 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:29 smithi134 podman[64631]: 2024-09-19 07:35:29.821904117 +0000 UTC m=+0.138271036 container create 25a6eb472a196ad6087ec8c7e01f7e97cbf9cc1cfb1d3f1b5d6f508dea54d950 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:35:30.066 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:30 smithi134 podman[64631]: 2024-09-19 07:35:30.066568411 +0000 UTC m=+0.382935330 container init 25a6eb472a196ad6087ec8c7e01f7e97cbf9cc1cfb1d3f1b5d6f508dea54d950 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:35:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.085+0000 7ff4609fb640 1 -- 172.21.15.103:0/2118391552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c102d80 msgr2=0x7ff45c103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.085+0000 7ff4609fb640 1 --2- 172.21.15.103:0/2118391552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c102d80 0x7ff45c103160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff454009ca0 tx=0x7ff45402a950 comp rx=0 tx=0).stop 2024-09-19T07:35:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 -- 172.21.15.103:0/2118391552 shutdown_connections 2024-09-19T07:35:30.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 --2- 172.21.15.103:0/2118391552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff45c10c160 0x7ff45c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 --2- 172.21.15.103:0/2118391552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45c1036a0 0x7ff45c10bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 --2- 172.21.15.103:0/2118391552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c102d80 0x7ff45c103160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 -- 172.21.15.103:0/2118391552 >> 172.21.15.103:0/2118391552 conn(0x7ff45c0fc780 msgr2=0x7ff45c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 -- 172.21.15.103:0/2118391552 shutdown_connections 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 -- 172.21.15.103:0/2118391552 wait complete. 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 Processor -- start 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.086+0000 7ff4609fb640 1 -- start start 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4609fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45c102d80 0x7ff45c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4609fb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff45c1036a0 0x7ff45c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4609fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c10c160 0x7ff45c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:30.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4609fb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c111f40 con 0x7ff45c10c160 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4609fb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c111dc0 con 0x7ff45c102d80 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4609fb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1120c0 con 0x7ff45c1036a0 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c10c160 0x7ff45c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c10c160 0x7ff45c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36782/0 (socket says 172.21.15.103:36782) 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 -- 172.21.15.103:0/1348927388 learned_addr learned my addr 172.21.15.103:0/1348927388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45a59c640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45c102d80 0x7ff45c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff459d9b640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff45c1036a0 0x7ff45c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 -- 172.21.15.103:0/1348927388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff45c1036a0 msgr2=0x7ff45c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff45c1036a0 0x7ff45c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 -- 172.21.15.103:0/1348927388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45c102d80 msgr2=0x7ff45c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45c102d80 0x7ff45c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 -- 172.21.15.103:0/1348927388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c19d430 con 0x7ff45c10c160 2024-09-19T07:35:30.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff45ad9d640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c10c160 0x7ff45c19cd50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c970 tx=0x7ff44c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4537fe640 1 -- 172.21.15.103:0/1348927388 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c007bb0 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c19d720 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4537fe640 1 -- 172.21.15.103:0/1348927388 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44c007d50 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.087+0000 7ff4537fe640 1 -- 172.21.15.103:0/1348927388 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00f450 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.088+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a5000 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.089+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c103160 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.089+0000 7ff4537fe640 1 -- 172.21.15.103:0/1348927388 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff44c004f60 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.090+0000 7ff4537fe640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4480a3240 0x7ff4480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.090+0000 7ff45a59c640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4480a3240 0x7ff4480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.090+0000 7ff4537fe640 1 -- 172.21.15.103:0/1348927388 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff44c0cf110 con 0x7ff45c10c160 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.091+0000 7ff45a59c640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4480a3240 0x7ff4480a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff454004ad0 tx=0x7ff454004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.094+0000 7ff4537fe640 1 -- 172.21.15.103:0/1348927388 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff44c014030 con 0x7ff45c10c160 2024-09-19T07:35:30.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.307+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff45c107260 con 0x7ff45c10c160 2024-09-19T07:35:30.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.307+0000 7ff4537fe640 1 -- 172.21.15.103:0/1348927388 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7ff44c09c2d0 con 0x7ff45c10c160 2024-09-19T07:35:30.308 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:30.308 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"btime":"2024-09-19T07:35:30:148096+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T07:35:30.309 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 3 2024-09-19T07:35:30.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.310+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4480a3240 msgr2=0x7ff4480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:30.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.310+0000 7ff4609fb640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4480a3240 0x7ff4480a5700 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff454004ad0 tx=0x7ff454004a60 comp rx=0 tx=0).stop 2024-09-19T07:35:30.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.310+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c10c160 msgr2=0x7ff45c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:30.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.310+0000 7ff4609fb640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c10c160 0x7ff45c19cd50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00c970 tx=0x7ff44c00ce30 comp rx=0 tx=0).stop 2024-09-19T07:35:30.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 shutdown_connections 2024-09-19T07:35:30.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff4480a3240 0x7ff4480a5700 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff45c10c160 0x7ff45c19cd50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff45c1036a0 0x7ff45c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 --2- 172.21.15.103:0/1348927388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45c102d80 0x7ff45c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:30.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 >> 172.21.15.103:0/1348927388 conn(0x7ff45c0fc780 msgr2=0x7ff45c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:30.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 shutdown_connections 2024-09-19T07:35:30.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:30.311+0000 7ff4609fb640 1 -- 172.21.15.103:0/1348927388 wait complete. 2024-09-19T07:35:30.329 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:30 smithi134 podman[64631]: 2024-09-19 07:35:30.069650709 +0000 UTC m=+0.386017628 container start 25a6eb472a196ad6087ec8c7e01f7e97cbf9cc1cfb1d3f1b5d6f508dea54d950 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T07:35:30.329 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:30 smithi134 ceph-mds[64650]: -- 172.21.15.134:0/1243760511 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e045bdb860 con 0x55e045bb9400 2024-09-19T07:35:30.329 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k[64646]: starting mds.k at 2024-09-19T07:35:30.329 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:30 smithi134 bash[64631]: 25a6eb472a196ad6087ec8c7e01f7e97cbf9cc1cfb1d3f1b5d6f508dea54d950 2024-09-19T07:35:30.329 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:35:30 smithi134 systemd[1]: Started Ceph mds.k for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6832/2440407987,v1:172.21.15.134:6833/2440407987] up:boot 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: fsmap 2 up:standby 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:31.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:31.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:31.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: Deploying daemon mds.j on smithi103 2024-09-19T07:35:31.390 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:31 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1348927388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6832/2440407987,v1:172.21.15.134:6833/2440407987] up:boot 2024-09-19T07:35:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: fsmap 2 up:standby 2024-09-19T07:35:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:35:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: Deploying daemon mds.j on smithi103 2024-09-19T07:35:31.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:31 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1348927388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6832/2440407987,v1:172.21.15.134:6833/2440407987] up:boot 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: fsmap 2 up:standby 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: Deploying daemon mds.j on smithi103 2024-09-19T07:35:31.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:31 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1348927388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:32.970 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:32 smithi103 systemd[1]: Starting Ceph mds.j for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:33.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:33.314 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:33 smithi103 ceph-mon[31201]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:33.315 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 podman[70652]: 2024-09-19 07:35:32.946792053 +0000 UTC m=+0.015795024 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:33.315 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 podman[70652]: 2024-09-19 07:35:33.069826584 +0000 UTC m=+0.138829554 container create 3d9a484773c425f4a780e389ac7db8ba58813c9df1cb725b434ef475b44726b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T07:35:33.315 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 podman[70652]: 2024-09-19 07:35:33.314934086 +0000 UTC m=+0.383937065 container init 3d9a484773c425f4a780e389ac7db8ba58813c9df1cb725b434ef475b44726b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T07:35:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:33 smithi174 ceph-mon[35370]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:33.489 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:33 smithi134 ceph-mon[35414]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:33.611 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 podman[70652]: 2024-09-19 07:35:33.318656147 +0000 UTC m=+0.387659118 container start 3d9a484773c425f4a780e389ac7db8ba58813c9df1cb725b434ef475b44726b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T07:35:33.612 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 ceph-mds[70689]: -- 172.21.15.103:0/271253936 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9f0873860 con 0x55b9f0851000 2024-09-19T07:35:33.612 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j[70685]: starting mds.j at 2024-09-19T07:35:33.612 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 bash[70652]: 3d9a484773c425f4a780e389ac7db8ba58813c9df1cb725b434ef475b44726b7 2024-09-19T07:35:33.612 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:35:33 smithi103 systemd[1]: Started Ceph mds.j for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:34.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 -- 172.21.15.103:0/3537774681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 msgr2=0x7f7aa81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:34.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3537774681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa81067e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009900 tx=0x7f7a9802a410 comp rx=0 tx=0).stop 2024-09-19T07:35:34.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 -- 172.21.15.103:0/3537774681 shutdown_connections 2024-09-19T07:35:34.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3537774681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7aa8106f10 0x7f7aa810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3537774681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa81067e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3537774681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa8101de0 0x7f7aa81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 -- 172.21.15.103:0/3537774681 >> 172.21.15.103:0/3537774681 conn(0x7f7aa80fd8c0 msgr2=0x7f7aa80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 -- 172.21.15.103:0/3537774681 shutdown_connections 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.991+0000 7f7aafd75640 1 -- 172.21.15.103:0/3537774681 wait complete. 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 Processor -- start 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 -- start start 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa8101de0 0x7f7aa81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7aa8106f10 0x7f7aa819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8112010 con 0x7f7aa8102790 2024-09-19T07:35:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8111e90 con 0x7f7aa8106f10 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aafd75640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8112190 con 0x7f7aa8101de0 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aad310640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aad310640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa8198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36806/0 (socket says 172.21.15.103:36806) 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.992+0000 7f7aad310640 1 -- 172.21.15.103:0/3437586040 learned_addr learned my addr 172.21.15.103:0/3437586040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aae312640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7aa8106f10 0x7f7aa819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aadb11640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa8101de0 0x7f7aa81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aad310640 1 -- 172.21.15.103:0/3437586040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa8101de0 msgr2=0x7f7aa81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aad310640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa8101de0 0x7f7aa81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aad310640 1 -- 172.21.15.103:0/3437586040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7aa8106f10 msgr2=0x7f7aa819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aad310640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7aa8106f10 0x7f7aa819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aad310640 1 -- 172.21.15.103:0/3437586040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa819d560 con 0x7f7aa8102790 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aadb11640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa8101de0 0x7f7aa81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:35:34.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aad310640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa8198af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005ab0 tx=0x7f7a980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7a9effd640 1 -- 172.21.15.103:0/3437586040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9803f030 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa819d7f0 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7a9effd640 1 -- 172.21.15.103:0/3437586040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a98003ce0 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7a9effd640 1 -- 172.21.15.103:0/3437586040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98035070 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.993+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa81a5130 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.994+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa81032c0 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.995+0000 7f7a9effd640 1 -- 172.21.15.103:0/3437586040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f7a98042030 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.996+0000 7f7a9effd640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7a940a3240 0x7f7a940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.996+0000 7f7aadb11640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7a940a3240 0x7f7a940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.996+0000 7f7a9effd640 1 -- 172.21.15.103:0/3437586040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f7a980f0060 con 0x7f7aa8102790 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.996+0000 7f7aadb11640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7a940a3240 0x7f7a940a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006fd0 tx=0x7f7aa4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:34.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:33.999+0000 7f7a9effd640 1 -- 172.21.15.103:0/3437586040 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7a980bd220 con 0x7f7aa8102790 2024-09-19T07:35:34.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.213+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aa810a0f0 con 0x7f7aa8102790 2024-09-19T07:35:34.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.214+0000 7f7a9effd640 1 -- 172.21.15.103:0/3437586040 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f7a980bcb20 con 0x7f7aa8102790 2024-09-19T07:35:34.215 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:34.215 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":4,"btime":"2024-09-19T07:35:34:155045+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T07:35:34.215 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 4 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7a940a3240 msgr2=0x7f7a940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7a940a3240 0x7f7a940a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006fd0 tx=0x7f7aa4008040 comp rx=0 tx=0).stop 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 msgr2=0x7f7aa8198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa8198af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005ab0 tx=0x7f7a980042f0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 shutdown_connections 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7a940a3240 0x7f7a940a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7aa8106f10 0x7f7aa819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7aa8102790 0x7f7aa8198af0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 --2- 172.21.15.103:0/3437586040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa8101de0 0x7f7aa81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 >> 172.21.15.103:0/3437586040 conn(0x7f7aa80fd8c0 msgr2=0x7f7aa80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 shutdown_connections 2024-09-19T07:35:34.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:34.216+0000 7f7aafd75640 1 -- 172.21.15.103:0/3437586040 wait complete. 2024-09-19T07:35:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: Deploying daemon mds.d on smithi103 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.103:6834/2425532131,v1:172.21.15.103:6835/2425532131] up:boot 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: fsmap 3 up:standby 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:35:34.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:34 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3437586040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:34.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: Deploying daemon mds.d on smithi103 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.103:6834/2425532131,v1:172.21.15.103:6835/2425532131] up:boot 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: fsmap 3 up:standby 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:35:34.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:34 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3437586040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:34.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:34.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: Deploying daemon mds.d on smithi103 2024-09-19T07:35:34.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:34.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.103:6834/2425532131,v1:172.21.15.103:6835/2425532131] up:boot 2024-09-19T07:35:34.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: fsmap 3 up:standby 2024-09-19T07:35:34.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:35:34.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:34 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3437586040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:36.720 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:36 smithi103 systemd[1]: Starting Ceph mds.d for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:37.100 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:36 smithi103 podman[71066]: 2024-09-19 07:35:36.744341775 +0000 UTC m=+0.015633388 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:37.100 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:36 smithi103 podman[71066]: 2024-09-19 07:35:36.864580309 +0000 UTC m=+0.135871914 container create a0d1b727f2f22ff1505cfba34f1df7439c436df1e8a2798d5c40c10a201cd943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T07:35:37.101 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:37 smithi103 podman[71066]: 2024-09-19 07:35:37.10097226 +0000 UTC m=+0.372263873 container init a0d1b727f2f22ff1505cfba34f1df7439c436df1e8a2798d5c40c10a201cd943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:35:37.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:37.401 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:37 smithi174 ceph-mon[35370]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:37 smithi103 ceph-mon[31201]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:37.470 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:37 smithi103 podman[71066]: 2024-09-19 07:35:37.103817678 +0000 UTC m=+0.375109291 container start a0d1b727f2f22ff1505cfba34f1df7439c436df1e8a2798d5c40c10a201cd943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T07:35:37.470 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:37 smithi103 ceph-mds[71085]: -- 172.21.15.103:0/2007283469 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56260a63b860 con 0x56260a618800 2024-09-19T07:35:37.470 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d[71081]: starting mds.d at 2024-09-19T07:35:37.471 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:37 smithi103 bash[71066]: a0d1b727f2f22ff1505cfba34f1df7439c436df1e8a2798d5c40c10a201cd943 2024-09-19T07:35:37.471 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:35:37 smithi103 systemd[1]: Started Ceph mds.d for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:37 smithi134 ceph-mon[35414]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 -- 172.21.15.103:0/2584974637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e41036c0 msgr2=0x7f67e410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2584974637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e41036c0 0x7f67e410bbb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009900 tx=0x7f67d802a480 comp rx=0 tx=0).stop 2024-09-19T07:35:37.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 -- 172.21.15.103:0/2584974637 shutdown_connections 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2584974637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e410c180 0x7f67e410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2584974637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e41036c0 0x7f67e410bbb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2584974637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e4102da0 0x7f67e4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 -- 172.21.15.103:0/2584974637 >> 172.21.15.103:0/2584974637 conn(0x7f67e40fc7a0 msgr2=0x7f67e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 -- 172.21.15.103:0/2584974637 shutdown_connections 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 -- 172.21.15.103:0/2584974637 wait complete. 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 Processor -- start 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.895+0000 7f67e9e77640 1 -- start start 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e4102da0 0x7f67e4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e41036c0 0x7f67e4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e410c180 0x7f67e419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:37.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4111f60 con 0x7f67e410c180 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4111de0 con 0x7f67e41036c0 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41120e0 con 0x7f67e4102da0 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e410c180 0x7f67e419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e410c180 0x7f67e419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36844/0 (socket says 172.21.15.103:36844) 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 -- 172.21.15.103:0/2029722992 learned_addr learned my addr 172.21.15.103:0/2029722992 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e2ffd640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e41036c0 0x7f67e4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 -- 172.21.15.103:0/2029722992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e4102da0 msgr2=0x7f67e4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e37fe640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e4102da0 0x7f67e4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e4102da0 0x7f67e4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 -- 172.21.15.103:0/2029722992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e41036c0 msgr2=0x7f67e4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:37.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e41036c0 0x7f67e4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 -- 172.21.15.103:0/2029722992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e419d3e0 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e3fff640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e410c180 0x7f67e419cd00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eaa0 tx=0x7f67dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e0ff9640 1 -- 172.21.15.103:0/2029722992 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00cdb0 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e0ff9640 1 -- 172.21.15.103:0/2029722992 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc005020 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e419d6d0 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e0ff9640 1 -- 172.21.15.103:0/2029722992 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0116b0 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.896+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41a4fb0 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.898+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e4103180 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.898+0000 7f67e0ff9640 1 -- 172.21.15.103:0/2029722992 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f67dc005560 con 0x7f67e410c180 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.899+0000 7f67e0ff9640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f67cc0a3240 0x7f67cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.899+0000 7f67e37fe640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f67cc0a3240 0x7f67cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:37.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.899+0000 7f67e0ff9640 1 -- 172.21.15.103:0/2029722992 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f67dc0cf680 con 0x7f67e410c180 2024-09-19T07:35:37.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.899+0000 7f67e37fe640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f67cc0a3240 0x7f67cc0a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009aa0 tx=0x7f67d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:37.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:37.904+0000 7f67e0ff9640 1 -- 172.21.15.103:0/2029722992 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f67dc016030 con 0x7f67e410c180 2024-09-19T07:35:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.105+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67e41071f0 con 0x7f67e410c180 2024-09-19T07:35:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.105+0000 7f67e0ff9640 1 -- 172.21.15.103:0/2029722992 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f67dc09c840 con 0x7f67e410c180 2024-09-19T07:35:38.107 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:38.107 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":4,"btime":"2024-09-19T07:35:34:155045+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T07:35:38.107 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 4 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.107+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f67cc0a3240 msgr2=0x7f67cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.107+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f67cc0a3240 0x7f67cc0a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009aa0 tx=0x7f67d4005f90 comp rx=0 tx=0).stop 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e410c180 msgr2=0x7f67e419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e410c180 0x7f67e419cd00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eaa0 tx=0x7f67dc00ef60 comp rx=0 tx=0).stop 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 shutdown_connections 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f67cc0a3240 0x7f67cc0a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e410c180 0x7f67e419cd00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e41036c0 0x7f67e4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 --2- 172.21.15.103:0/2029722992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e4102da0 0x7f67e4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:38.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 >> 172.21.15.103:0/2029722992 conn(0x7f67e40fc7a0 msgr2=0x7f67e40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:38.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 shutdown_connections 2024-09-19T07:35:38.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:38.108+0000 7f67e9e77640 1 -- 172.21.15.103:0/2029722992 wait complete. 2024-09-19T07:35:38.426 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.426 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: Deploying daemon mds.e on smithi134 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2029722992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.103:6836/1711280456,v1:172.21.15.103:6837/1711280456] up:boot 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: fsmap 4 up:standby 2024-09-19T07:35:38.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:38 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:35:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: Deploying daemon mds.e on smithi134 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2029722992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.103:6836/1711280456,v1:172.21.15.103:6837/1711280456] up:boot 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: fsmap 4 up:standby 2024-09-19T07:35:38.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:38 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:35:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:38.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:38.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: Deploying daemon mds.e on smithi134 2024-09-19T07:35:38.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2029722992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:38.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.103:6836/1711280456,v1:172.21.15.103:6837/1711280456] up:boot 2024-09-19T07:35:38.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: fsmap 4 up:standby 2024-09-19T07:35:38.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:38 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:35:39.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:39 smithi134 ceph-mon[35414]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:39 smithi174 ceph-mon[35370]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:39.698 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:35:39 smithi134 podman[64932]: 2024-09-19 07:35:39.696422108 +0000 UTC m=+0.362154480 container init e3df714ada3e79c5736770a118af942721194a708c2419f95d2147aa9542c1e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T07:35:39.698 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:35:39 smithi134 podman[64932]: 2024-09-19 07:35:39.699089629 +0000 UTC m=+0.364822009 container start e3df714ada3e79c5736770a118af942721194a708c2419f95d2147aa9542c1e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T07:35:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:39 smithi103 ceph-mon[31201]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:40.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:35:39 smithi134 ceph-mds[64951]: -- 172.21.15.134:0/2151209226 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56475c41d860 con 0x56475c3fb000 2024-09-19T07:35:40.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:35:39 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e[64947]: starting mds.e at 2024-09-19T07:35:40.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:35:39 smithi134 bash[64932]: e3df714ada3e79c5736770a118af942721194a708c2419f95d2147aa9542c1e2 2024-09-19T07:35:40.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:35:39 smithi134 systemd[1]: Started Ceph mds.e for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:40.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:41.142 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:41.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:41.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:41.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:41.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: Deploying daemon mds.f on smithi174 2024-09-19T07:35:41.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:41.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6834/19326296,v1:172.21.15.134:6835/19326296] up:boot 2024-09-19T07:35:41.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: fsmap 5 up:standby 2024-09-19T07:35:41.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:40 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: Deploying daemon mds.f on smithi174 2024-09-19T07:35:41.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:41.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6834/19326296,v1:172.21.15.134:6835/19326296] up:boot 2024-09-19T07:35:41.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: fsmap 5 up:standby 2024-09-19T07:35:41.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:40 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:35:41.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:41.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:41.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:41.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:41.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: Deploying daemon mds.f on smithi174 2024-09-19T07:35:41.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:41.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6834/19326296,v1:172.21.15.134:6835/19326296] up:boot 2024-09-19T07:35:41.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: fsmap 5 up:standby 2024-09-19T07:35:41.257 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:40 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:35:41.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 -- 172.21.15.103:0/1153561069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 msgr2=0x7f898c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:41.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1153561069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c103110 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f897c009900 tx=0x7f897c02a480 comp rx=0 tx=0).stop 2024-09-19T07:35:41.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 -- 172.21.15.103:0/1153561069 shutdown_connections 2024-09-19T07:35:41.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1153561069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c10c160 0x7f898c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1153561069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c103650 0x7f898c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1153561069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c103110 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 -- 172.21.15.103:0/1153561069 >> 172.21.15.103:0/1153561069 conn(0x7f898c0fc790 msgr2=0x7f898c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 -- 172.21.15.103:0/1153561069 shutdown_connections 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 -- 172.21.15.103:0/1153561069 wait complete. 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.655+0000 7f8993f4e640 1 Processor -- start 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8993f4e640 1 -- start start 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8993f4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8993f4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c103650 0x7f898c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8993f4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c10c160 0x7f898c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8993f4e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f898c111f60 con 0x7f898c102d30 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8993f4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f898c111de0 con 0x7f898c103650 2024-09-19T07:35:41.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8993f4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1120e0 con 0x7f898c10c160 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54832/0 (socket says 172.21.15.103:54832) 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 -- 172.21.15.103:0/1411624189 learned_addr learned my addr 172.21.15.103:0/1411624189 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f89914e9640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c103650 0x7f898c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f89924eb640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c10c160 0x7f898c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 -- 172.21.15.103:0/1411624189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c10c160 msgr2=0x7f898c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c10c160 0x7f898c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 -- 172.21.15.103:0/1411624189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c103650 msgr2=0x7f898c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c103650 0x7f898c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 -- 172.21.15.103:0/1411624189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c19d3f0 con 0x7f898c102d30 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f89924eb640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c10c160 0x7f898c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:35:41.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8991cea640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c198440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f897c006010 tx=0x7f897c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.656+0000 7f8982ffd640 1 -- 172.21.15.103:0/1411624189 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c03f030 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.657+0000 7f8982ffd640 1 -- 172.21.15.103:0/1411624189 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897c003ce0 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.657+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c19d680 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.657+0000 7f8982ffd640 1 -- 172.21.15.103:0/1411624189 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c035070 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.657+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c1a4fc0 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.657+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898c06b070 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.659+0000 7f8982ffd640 1 -- 172.21.15.103:0/1411624189 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f897c042030 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.659+0000 7f8982ffd640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89780a3240 0x7f89780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.660+0000 7f89914e9640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89780a3240 0x7f89780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.660+0000 7f8982ffd640 1 -- 172.21.15.103:0/1411624189 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f897c0efd80 con 0x7f898c102d30 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.660+0000 7f89914e9640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89780a3240 0x7f89780a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f89880058c0 tx=0x7f8988005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:41.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.663+0000 7f8982ffd640 1 -- 172.21.15.103:0/1411624189 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f897c0bd000 con 0x7f898c102d30 2024-09-19T07:35:41.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.862+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f898c103110 con 0x7f898c102d30 2024-09-19T07:35:41.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.862+0000 7f8982ffd640 1 -- 172.21.15.103:0/1411624189 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7f898c103110 con 0x7f898c102d30 2024-09-19T07:35:41.866 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:41.866 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":6,"btime":"2024-09-19T07:35:40:267712+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T07:35:41.866 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 6 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89780a3240 msgr2=0x7f89780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89780a3240 0x7f89780a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f89880058c0 tx=0x7f8988005850 comp rx=0 tx=0).stop 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 msgr2=0x7f898c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c198440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f897c006010 tx=0x7f897c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 shutdown_connections 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89780a3240 0x7f89780a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c10c160 0x7f898c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f898c103650 0x7f898c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 --2- 172.21.15.103:0/1411624189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f898c102d30 0x7f898c198440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 >> 172.21.15.103:0/1411624189 conn(0x7f898c0fc790 msgr2=0x7f898c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:41.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 shutdown_connections 2024-09-19T07:35:41.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:41.865+0000 7f8993f4e640 1 -- 172.21.15.103:0/1411624189 wait complete. 2024-09-19T07:35:41.915 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:41 smithi174 systemd[1]: Starting Ceph mds.f for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:42.212 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:41 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1411624189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:42.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:41 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1411624189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:42.263 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 podman[65129]: 2024-09-19 07:35:41.912436936 +0000 UTC m=+0.015772993 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:42.263 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 podman[65129]: 2024-09-19 07:35:42.027347258 +0000 UTC m=+0.130683302 container create 01da294647bdc3241b6fec2f5acc962c31d1f30a4bf1dcb88b8a98ffabde0a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T07:35:42.263 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 podman[65129]: 2024-09-19 07:35:42.263243808 +0000 UTC m=+0.366579851 container init 01da294647bdc3241b6fec2f5acc962c31d1f30a4bf1dcb88b8a98ffabde0a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T07:35:42.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:41 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1411624189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:42.665 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 podman[65129]: 2024-09-19 07:35:42.26613348 +0000 UTC m=+0.369469533 container start 01da294647bdc3241b6fec2f5acc962c31d1f30a4bf1dcb88b8a98ffabde0a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T07:35:42.665 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mds[65148]: -- 172.21.15.174:0/3750142037 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5b2b91860 con 0x55b5b2b6e800 2024-09-19T07:35:42.665 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f[65144]: starting mds.f at 2024-09-19T07:35:42.665 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 bash[65129]: 01da294647bdc3241b6fec2f5acc962c31d1f30a4bf1dcb88b8a98ffabde0a8c 2024-09-19T07:35:42.665 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:35:42 smithi174 systemd[1]: Started Ceph mds.f for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:43.089 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mon[35370]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:43.089 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.089 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:43.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:43.090 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:42 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:42 smithi103 ceph-mon[31201]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:42 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:42 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:42 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:42 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:42 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:42 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:43.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:42 smithi134 ceph-mon[35414]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:43.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:42 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:42 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:42 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:43.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:42 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:43.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:42 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:43.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:42 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:44.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:43 smithi174 ceph-mon[35370]: Deploying daemon mds.c on smithi174 2024-09-19T07:35:44.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:43 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] up:boot 2024-09-19T07:35:44.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:43 smithi174 ceph-mon[35370]: fsmap 6 up:standby 2024-09-19T07:35:44.135 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:43 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:35:44.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:43 smithi103 ceph-mon[31201]: Deploying daemon mds.c on smithi174 2024-09-19T07:35:44.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:43 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] up:boot 2024-09-19T07:35:44.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:43 smithi103 ceph-mon[31201]: fsmap 6 up:standby 2024-09-19T07:35:44.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:43 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:35:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:43 smithi134 ceph-mon[35414]: Deploying daemon mds.c on smithi174 2024-09-19T07:35:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:43 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] up:boot 2024-09-19T07:35:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:43 smithi134 ceph-mon[35414]: fsmap 6 up:standby 2024-09-19T07:35:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:43 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:35:44.415 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 systemd[1]: Starting Ceph mds.c for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:44.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:44.802 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:44.811 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 podman[65430]: 2024-09-19 07:35:44.436542731 +0000 UTC m=+0.016008038 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:44.811 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 podman[65430]: 2024-09-19 07:35:44.559124686 +0000 UTC m=+0.138589986 container create b916fbfb0045a1e293bdb48a1ffec3589964361a095c6e7025e6fa0da2143a9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True) 2024-09-19T07:35:44.811 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 podman[65430]: 2024-09-19 07:35:44.811533355 +0000 UTC m=+0.390998663 container init b916fbfb0045a1e293bdb48a1ffec3589964361a095c6e7025e6fa0da2143a9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:35:45.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:44 smithi174 ceph-mon[35370]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:45.165 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 podman[65430]: 2024-09-19 07:35:44.814426657 +0000 UTC m=+0.393891961 container start b916fbfb0045a1e293bdb48a1ffec3589964361a095c6e7025e6fa0da2143a9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-19T07:35:45.165 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 ceph-mds[65449]: -- 172.21.15.174:0/3430860098 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608b9e57860 con 0x5608b9e34800 2024-09-19T07:35:45.165 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c[65445]: starting mds.c at 2024-09-19T07:35:45.166 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 bash[65430]: b916fbfb0045a1e293bdb48a1ffec3589964361a095c6e7025e6fa0da2143a9c 2024-09-19T07:35:45.166 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:35:44 smithi174 systemd[1]: Started Ceph mds.c for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:45.219 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:44 smithi103 ceph-mon[31201]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 -- 172.21.15.103:0/2363135507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c101de0 msgr2=0x7f444c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 --2- 172.21.15.103:0/2363135507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c101de0 0x7f444c1021c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f443c009900 tx=0x7f443c02a410 comp rx=0 tx=0).stop 2024-09-19T07:35:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 -- 172.21.15.103:0/2363135507 shutdown_connections 2024-09-19T07:35:45.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 --2- 172.21.15.103:0/2363135507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444c106f10 0x7f444c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 --2- 172.21.15.103:0/2363135507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f444c102790 0x7f444c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 --2- 172.21.15.103:0/2363135507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c101de0 0x7f444c1021c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 -- 172.21.15.103:0/2363135507 >> 172.21.15.103:0/2363135507 conn(0x7f444c0fd8c0 msgr2=0x7f444c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 -- 172.21.15.103:0/2363135507 shutdown_connections 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 -- 172.21.15.103:0/2363135507 wait complete. 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.299+0000 7f44528d5640 1 Processor -- start 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f44528d5640 1 -- start start 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f44528d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f444c101de0 0x7f444c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f44528d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444c102790 0x7f444c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f44528d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c106f10 0x7f444c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f44528d5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f444c111f30 con 0x7f444c106f10 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f44528d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f444c111db0 con 0x7f444c101de0 2024-09-19T07:35:45.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f44528d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f444c1120b0 con 0x7f444c102790 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c106f10 0x7f444c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f444bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f444c101de0 0x7f444c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c106f10 0x7f444c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54850/0 (socket says 172.21.15.103:54850) 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 -- 172.21.15.103:0/3486710865 learned_addr learned my addr 172.21.15.103:0/3486710865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f444b7fe640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444c102790 0x7f444c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 -- 172.21.15.103:0/3486710865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444c102790 msgr2=0x7f444c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444c102790 0x7f444c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 -- 172.21.15.103:0/3486710865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f444c101de0 msgr2=0x7f444c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f444c101de0 0x7f444c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 -- 172.21.15.103:0/3486710865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c19d3f0 con 0x7f444c106f10 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f444b7fe640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444c102790 0x7f444c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.300+0000 7f4450e72640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c106f10 0x7f444c19cd10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f444400cad0 tx=0x7f444400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:45.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.301+0000 7f44497fa640 1 -- 172.21.15.103:0/3486710865 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444400ae20 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.301+0000 7f44497fa640 1 -- 172.21.15.103:0/3486710865 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4444005020 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.301+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c19d6e0 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.301+0000 7f44497fa640 1 -- 172.21.15.103:0/3486710865 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444400f6f0 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.301+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c1a4fc0 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.302+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c1021c0 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.303+0000 7f44497fa640 1 -- 172.21.15.103:0/3486710865 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f44440051d0 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.303+0000 7f44497fa640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44340a32c0 0x7f44340a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.303+0000 7f444bfff640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44340a32c0 0x7f44340a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.304+0000 7f44497fa640 1 -- 172.21.15.103:0/3486710865 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f44440cf3f0 con 0x7f444c106f10 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.304+0000 7f444bfff640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44340a32c0 0x7f44340a5780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f443c009230 tx=0x7f443c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:45.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.307+0000 7f44497fa640 1 -- 172.21.15.103:0/3486710865 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4444014030 con 0x7f444c106f10 2024-09-19T07:35:45.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:44 smithi134 ceph-mon[35414]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:45.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.509+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f444c109b60 con 0x7f444c106f10 2024-09-19T07:35:45.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.509+0000 7f44497fa640 1 -- 172.21.15.103:0/3486710865 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f444409c5b0 con 0x7f444c106f10 2024-09-19T07:35:45.510 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:45.510 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":8,"btime":"2024-09-19T07:35:44:922188+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-19T07:35:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 8 2024-09-19T07:35:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44340a32c0 msgr2=0x7f44340a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44340a32c0 0x7f44340a5780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f443c009230 tx=0x7f443c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c106f10 msgr2=0x7f444c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:45.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c106f10 0x7f444c19cd10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f444400cad0 tx=0x7f444400cf90 comp rx=0 tx=0).stop 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 shutdown_connections 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44340a32c0 0x7f44340a5780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f444c106f10 0x7f444c19cd10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444c102790 0x7f444c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 --2- 172.21.15.103:0/3486710865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f444c101de0 0x7f444c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 >> 172.21.15.103:0/3486710865 conn(0x7f444c0fd8c0 msgr2=0x7f444c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 shutdown_connections 2024-09-19T07:35:45.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:45.511+0000 7f44528d5640 1 -- 172.21.15.103:0/3486710865 wait complete. 2024-09-19T07:35:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] up:boot 2024-09-19T07:35:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: fsmap 7 up:standby 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: Deploying daemon mds.h on smithi134 2024-09-19T07:35:46.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:45 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3486710865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:46.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] up:boot 2024-09-19T07:35:46.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: fsmap 7 up:standby 2024-09-19T07:35:46.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:35:46.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:46.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:46.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:46.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: Deploying daemon mds.h on smithi134 2024-09-19T07:35:46.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:45 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3486710865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] up:boot 2024-09-19T07:35:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: fsmap 7 up:standby 2024-09-19T07:35:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:35:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: Deploying daemon mds.h on smithi134 2024-09-19T07:35:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:45 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3486710865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:47.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:46 smithi134 ceph-mon[35414]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:47.079 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:46 smithi134 systemd[1]: Starting Ceph mds.h for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:47.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:46 smithi103 ceph-mon[31201]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:47.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:46 smithi174 ceph-mon[35370]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:47.434 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 podman[65231]: 2024-09-19 07:35:47.072263639 +0000 UTC m=+0.015754588 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:47.434 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 podman[65231]: 2024-09-19 07:35:47.190448155 +0000 UTC m=+0.133939093 container create 68564d7e5b73dd07f5245b1e600fc10031542f0d36f7d0bffeda45df92063435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T07:35:47.434 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 podman[65231]: 2024-09-19 07:35:47.434696362 +0000 UTC m=+0.378187296 container init 68564d7e5b73dd07f5245b1e600fc10031542f0d36f7d0bffeda45df92063435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:35:47.829 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 podman[65231]: 2024-09-19 07:35:47.437764439 +0000 UTC m=+0.381255377 container start 68564d7e5b73dd07f5245b1e600fc10031542f0d36f7d0bffeda45df92063435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T07:35:47.829 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 ceph-mds[65250]: -- 172.21.15.134:0/4018348692 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c2de21860 con 0x555c2ddff400 2024-09-19T07:35:47.829 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h[65246]: starting mds.h at 2024-09-19T07:35:47.829 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 bash[65231]: 68564d7e5b73dd07f5245b1e600fc10031542f0d36f7d0bffeda45df92063435 2024-09-19T07:35:47.829 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:35:47 smithi134 systemd[1]: Started Ceph mds.h for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:48.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:48.433 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: Deploying daemon mds.i on smithi174 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] up:boot 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: fsmap 8 up:standby 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:35:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:48 smithi174 ceph-mon[35370]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:48.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:48.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:48.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:48.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:48.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:48.822 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:48.822 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: Deploying daemon mds.i on smithi174 2024-09-19T07:35:48.822 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] up:boot 2024-09-19T07:35:48.822 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: fsmap 8 up:standby 2024-09-19T07:35:48.822 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:35:48.822 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:48 smithi103 ceph-mon[31201]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 -- 172.21.15.103:0/3560386346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c106a80 msgr2=0x7f880c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 --2- 172.21.15.103:0/3560386346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c106a80 0x7f880c106e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009930 tx=0x7f87fc02a420 comp rx=0 tx=0).stop 2024-09-19T07:35:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 -- 172.21.15.103:0/3560386346 shutdown_connections 2024-09-19T07:35:48.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 --2- 172.21.15.103:0/3560386346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f880c101420 0x7f880c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 --2- 172.21.15.103:0/3560386346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f880c100a80 0x7f880c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 --2- 172.21.15.103:0/3560386346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c106a80 0x7f880c106e60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 -- 172.21.15.103:0/3560386346 >> 172.21.15.103:0/3560386346 conn(0x7f880c0fc7a0 msgr2=0x7f880c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 -- 172.21.15.103:0/3560386346 shutdown_connections 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.900+0000 7f8813ca8640 1 -- 172.21.15.103:0/3560386346 wait complete. 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 Processor -- start 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 -- start start 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c100a80 0x7f880c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f880c101420 0x7f880c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f880c106a80 0x7f880c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:48.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f880c110e80 con 0x7f880c100a80 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f880c110d00 con 0x7f880c106a80 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8813ca8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f880c111000 con 0x7f880c101420 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8811a44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c100a80 0x7f880c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8811a44640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c100a80 0x7f880c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58578/0 (socket says 172.21.15.103:58578) 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.901+0000 7f8811a44640 1 -- 172.21.15.103:0/1137420183 learned_addr learned my addr 172.21.15.103:0/1137420183 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8811243640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f880c101420 0x7f880c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8812245640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f880c106a80 0x7f880c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8811a44640 1 -- 172.21.15.103:0/1137420183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f880c101420 msgr2=0x7f880c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8811a44640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f880c101420 0x7f880c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8811a44640 1 -- 172.21.15.103:0/1137420183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f880c106a80 msgr2=0x7f880c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:48.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8811a44640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f880c106a80 0x7f880c1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8811a44640 1 -- 172.21.15.103:0/1137420183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c1a4850 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8812245640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f880c106a80 0x7f880c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8811a44640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c100a80 0x7f880c199290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009230 tx=0x7f87fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8802ffd640 1 -- 172.21.15.103:0/1137420183 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc03f030 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8802ffd640 1 -- 172.21.15.103:0/1137420183 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc003ad0 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1a4a20 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8802ffd640 1 -- 172.21.15.103:0/1137420183 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc035070 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.902+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1a4ed0 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.903+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c100ee0 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.904+0000 7f8802ffd640 1 -- 172.21.15.103:0/1137420183 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f87fc042030 con 0x7f880c100a80 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.905+0000 7f8802ffd640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f87f80a32c0 0x7f87f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:48.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.905+0000 7f8802ffd640 1 -- 172.21.15.103:0/1137420183 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f87fc0ef8f0 con 0x7f880c100a80 2024-09-19T07:35:48.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.905+0000 7f8811243640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f87f80a32c0 0x7f87f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:48.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.905+0000 7f8811243640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f87f80a32c0 0x7f87f80a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f880c06b2e0 tx=0x7f880800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:48.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:48.909+0000 7f8802ffd640 1 -- 172.21.15.103:0/1137420183 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f87fc0bcab0 con 0x7f880c100a80 2024-09-19T07:35:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: Deploying daemon mds.i on smithi174 2024-09-19T07:35:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] up:boot 2024-09-19T07:35:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: fsmap 8 up:standby 2024-09-19T07:35:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:35:49.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:48 smithi134 ceph-mon[35414]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:49.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.109+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f880c108ff0 con 0x7f880c100a80 2024-09-19T07:35:49.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.110+0000 7f8802ffd640 1 -- 172.21.15.103:0/1137420183 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f87fc0bc3b0 con 0x7f880c100a80 2024-09-19T07:35:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:49.111 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":9,"btime":"2024-09-19T07:35:47:936164+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T07:35:49.111 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 9 2024-09-19T07:35:49.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f87f80a32c0 msgr2=0x7f87f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:49.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f87f80a32c0 0x7f87f80a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f880c06b2e0 tx=0x7f880800b390 comp rx=0 tx=0).stop 2024-09-19T07:35:49.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c100a80 msgr2=0x7f880c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:49.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c100a80 0x7f880c199290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009230 tx=0x7f87fc004270 comp rx=0 tx=0).stop 2024-09-19T07:35:49.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 shutdown_connections 2024-09-19T07:35:49.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f87f80a32c0 0x7f87f80a5780 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:49.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f880c106a80 0x7f880c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:49.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f880c101420 0x7f880c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:49.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 --2- 172.21.15.103:0/1137420183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f880c100a80 0x7f880c199290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:49.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 >> 172.21.15.103:0/1137420183 conn(0x7f880c0fc7a0 msgr2=0x7f880c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:49.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 shutdown_connections 2024-09-19T07:35:49.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:49.112+0000 7f8813ca8640 1 -- 172.21.15.103:0/1137420183 wait complete. 2024-09-19T07:35:49.665 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:49 smithi174 systemd[1]: Starting Ceph mds.i for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:49.941 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:49 smithi174 podman[65728]: 2024-09-19 07:35:49.679949218 +0000 UTC m=+0.112719464 container create afc78d30af9271a3b7060a228e5a175ede2fccb9d04137cb8fd39547dc65ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T07:35:49.942 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:49 smithi174 podman[65728]: 2024-09-19 07:35:49.582676948 +0000 UTC m=+0.015447192 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:49.942 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:49 smithi174 podman[65728]: 2024-09-19 07:35:49.916270403 +0000 UTC m=+0.349040648 container init afc78d30af9271a3b7060a228e5a175ede2fccb9d04137cb8fd39547dc65ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T07:35:49.942 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:49 smithi174 podman[65728]: 2024-09-19 07:35:49.919067395 +0000 UTC m=+0.351837641 container start afc78d30af9271a3b7060a228e5a175ede2fccb9d04137cb8fd39547dc65ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T07:35:50.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:49 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1137420183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:50.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:49 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1137420183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:49 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1137420183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:50.415 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:49 smithi174 ceph-mds[65747]: -- 172.21.15.174:0/3108049311 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622a7377860 con 0x5622a7355000 2024-09-19T07:35:50.415 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:49 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i[65743]: starting mds.i at 2024-09-19T07:35:50.416 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:50 smithi174 bash[65728]: afc78d30af9271a3b7060a228e5a175ede2fccb9d04137cb8fd39547dc65ca7a 2024-09-19T07:35:50.416 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:35:50 smithi174 systemd[1]: Started Ceph mds.i for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: Deploying daemon mds.g on smithi103 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] up:boot 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: fsmap 9 up:standby 2024-09-19T07:35:51.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:51 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:35:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:51.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: Deploying daemon mds.g on smithi103 2024-09-19T07:35:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] up:boot 2024-09-19T07:35:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: fsmap 9 up:standby 2024-09-19T07:35:51.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:51 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:35:51.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:51.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: Deploying daemon mds.g on smithi103 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] up:boot 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: fsmap 9 up:standby 2024-09-19T07:35:51.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:51 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:35:51.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:52.188 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:51 smithi103 systemd[1]: Starting Ceph mds.g for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:52.470 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 podman[71856]: 2024-09-19 07:35:52.091202212 +0000 UTC m=+0.017295012 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:52.471 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 podman[71856]: 2024-09-19 07:35:52.227647207 +0000 UTC m=+0.153739994 container create 0e48ee93f0588e8bd54ba79ba672440e72d35e70c3c0ffb71d274358d210f202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-19T07:35:52.662 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:52.784 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 podman[71856]: 2024-09-19 07:35:52.493449675 +0000 UTC m=+0.419542474 container init 0e48ee93f0588e8bd54ba79ba672440e72d35e70c3c0ffb71d274358d210f202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:35:52.784 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 podman[71856]: 2024-09-19 07:35:52.497172423 +0000 UTC m=+0.423265214 container start 0e48ee93f0588e8bd54ba79ba672440e72d35e70c3c0ffb71d274358d210f202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:35:52.784 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 ceph-mds[71900]: -- 172.21.15.103:0/1599203835 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6f0cd9860 con 0x55c6f0cb7400 2024-09-19T07:35:52.784 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g[71883]: starting mds.g at 2024-09-19T07:35:52.784 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 bash[71856]: 0e48ee93f0588e8bd54ba79ba672440e72d35e70c3c0ffb71d274358d210f202 2024-09-19T07:35:52.784 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:35:52 smithi103 systemd[1]: Started Ceph mds.g for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:53.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 -- 172.21.15.103:0/1868588273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908103660 msgr2=0x7f590810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:53.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 --2- 172.21.15.103:0/1868588273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908103660 0x7f590810bb50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009900 tx=0x7f58fc02a480 comp rx=0 tx=0).stop 2024-09-19T07:35:53.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 -- 172.21.15.103:0/1868588273 shutdown_connections 2024-09-19T07:35:53.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 --2- 172.21.15.103:0/1868588273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590810c170 0x7f590810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 --2- 172.21.15.103:0/1868588273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908103660 0x7f590810bb50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 --2- 172.21.15.103:0/1868588273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5908102d40 0x7f5908103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 -- 172.21.15.103:0/1868588273 >> 172.21.15.103:0/1868588273 conn(0x7f59080fc7a0 msgr2=0x7f59080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 -- 172.21.15.103:0/1868588273 shutdown_connections 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.122+0000 7f590d784640 1 -- 172.21.15.103:0/1868588273 wait complete. 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.123+0000 7f590d784640 1 Processor -- start 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.123+0000 7f590d784640 1 -- start start 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908102d40 0x7f59081985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5908103660 0x7f5908198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590810c170 0x7f590819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5908111fb0 con 0x7f5908102d40 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5908111e30 con 0x7f590810c170 2024-09-19T07:35:53.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5908112130 con 0x7f5908103660 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908102d40 0x7f59081985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908102d40 0x7f59081985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58594/0 (socket says 172.21.15.103:58594) 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 -- 172.21.15.103:0/1103747516 learned_addr learned my addr 172.21.15.103:0/1103747516 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f59077fe640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590810c170 0x7f590819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 -- 172.21.15.103:0/1103747516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5908103660 msgr2=0x7f5908198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5908103660 0x7f5908198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 -- 172.21.15.103:0/1103747516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590810c170 msgr2=0x7f590819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590810c170 0x7f590819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 -- 172.21.15.103:0/1103747516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590819d560 con 0x7f5908102d40 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f59077fe640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590810c170 0x7f590819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:35:53.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f5906ffd640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908102d40 0x7f59081985b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58f800b4a0 tx=0x7f58f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f58effff640 1 -- 172.21.15.103:0/1103747516 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8014070 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590819d850 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f58effff640 1 -- 172.21.15.103:0/1103747516 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f8010af0 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f58effff640 1 -- 172.21.15.103:0/1103747516 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8017d10 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.124+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081a5130 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.125+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5908103120 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.126+0000 7f58effff640 1 -- 172.21.15.103:0/1103747516 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f58f8017070 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.127+0000 7f58effff640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f58f00a3240 0x7f58f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.127+0000 7f59067fc640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f58f00a3240 0x7f58f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.127+0000 7f58effff640 1 -- 172.21.15.103:0/1103747516 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f58f80ce960 con 0x7f5908102d40 2024-09-19T07:35:53.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.127+0000 7f59067fc640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f58f00a3240 0x7f58f00a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58fc02a9a0 tx=0x7f58fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:53.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.131+0000 7f58effff640 1 -- 172.21.15.103:0/1103747516 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f58f809bb20 con 0x7f5908102d40 2024-09-19T07:35:53.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:53 smithi103 ceph-mon[31201]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:53.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.324 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.325 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:53.325 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:53.325 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:53 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:53.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.346+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59081071e0 con 0x7f5908102d40 2024-09-19T07:35:53.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.346+0000 7f58effff640 1 -- 172.21.15.103:0/1103747516 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7f58f809b420 con 0x7f5908102d40 2024-09-19T07:35:53.347 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:53.347 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":11,"btime":"2024-09-19T07:35:53:083666+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T07:35:53.348 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 11 2024-09-19T07:35:53.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f58f00a3240 msgr2=0x7f58f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:53.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f58f00a3240 0x7f58f00a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58fc02a9a0 tx=0x7f58fc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908102d40 msgr2=0x7f59081985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:53.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908102d40 0x7f59081985b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58f800b4a0 tx=0x7f58f800b960 comp rx=0 tx=0).stop 2024-09-19T07:35:53.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 shutdown_connections 2024-09-19T07:35:53.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f58f00a3240 0x7f58f00a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590810c170 0x7f590819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5908103660 0x7f5908198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 --2- 172.21.15.103:0/1103747516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5908102d40 0x7f59081985b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:53.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 >> 172.21.15.103:0/1103747516 conn(0x7f59080fc7a0 msgr2=0x7f59080fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:53.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 shutdown_connections 2024-09-19T07:35:53.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:53.348+0000 7f590d784640 1 -- 172.21.15.103:0/1103747516 wait complete. 2024-09-19T07:35:53.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:53 smithi174 ceph-mon[35370]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:53.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:53.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:53.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:53 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:53 smithi134 ceph-mon[35414]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:53.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:53.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:53.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:53 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:54 smithi174 ceph-mon[35370]: Deploying daemon mds.a on smithi103 2024-09-19T07:35:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:54 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.103:6838/1660338093,v1:172.21.15.103:6839/1660338093] up:boot 2024-09-19T07:35:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:54 smithi174 ceph-mon[35370]: fsmap 10 up:standby 2024-09-19T07:35:54.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:54 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:35:54.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:54 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1103747516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:54.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:54 smithi103 ceph-mon[31201]: Deploying daemon mds.a on smithi103 2024-09-19T07:35:54.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:54 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.103:6838/1660338093,v1:172.21.15.103:6839/1660338093] up:boot 2024-09-19T07:35:54.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:54 smithi103 ceph-mon[31201]: fsmap 10 up:standby 2024-09-19T07:35:54.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:54 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:35:54.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:54 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1103747516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:54 smithi134 ceph-mon[35414]: Deploying daemon mds.a on smithi103 2024-09-19T07:35:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:54 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.103:6838/1660338093,v1:172.21.15.103:6839/1660338093] up:boot 2024-09-19T07:35:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:54 smithi134 ceph-mon[35414]: fsmap 10 up:standby 2024-09-19T07:35:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:54 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:35:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:54 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1103747516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:55.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:55 smithi103 ceph-mon[31201]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:55.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:55 smithi174 ceph-mon[35370]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:55.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:55 smithi134 ceph-mon[35414]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:55.575 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 systemd[1]: Starting Ceph mds.a for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:55.844 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 podman[72254]: 2024-09-19 07:35:55.537950742 +0000 UTC m=+0.015883962 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:55.844 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 podman[72254]: 2024-09-19 07:35:55.666772425 +0000 UTC m=+0.144705626 container create 0af034840cb86fb5b01c34ebb5437b1affd9887c4b38de4ea12f0f2e048f6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:35:55.844 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 podman[72254]: 2024-09-19 07:35:55.844620258 +0000 UTC m=+0.322553473 container init 0af034840cb86fb5b01c34ebb5437b1affd9887c4b38de4ea12f0f2e048f6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:35:56.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:56.103 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 podman[72254]: 2024-09-19 07:35:55.847693569 +0000 UTC m=+0.325626779 container start 0af034840cb86fb5b01c34ebb5437b1affd9887c4b38de4ea12f0f2e048f6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T07:35:56.103 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 ceph-mds[72274]: -- 172.21.15.103:0/2840901169 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c14ba1c20 con 0x564c14b7f000 2024-09-19T07:35:56.103 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a[72270]: starting mds.a at 2024-09-19T07:35:56.103 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 bash[72254]: 0af034840cb86fb5b01c34ebb5437b1affd9887c4b38de4ea12f0f2e048f6933 2024-09-19T07:35:56.103 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:35:55 smithi103 systemd[1]: Started Ceph mds.a for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:56.331 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.745+0000 7ff2fbe51640 1 -- 172.21.15.103:0/2860637526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4101450 msgr2=0x7ff2f4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.745+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/2860637526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4101450 0x7ff2f4105920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009900 tx=0x7ff2e402a470 comp rx=0 tx=0).stop 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.745+0000 7ff2fbe51640 1 -- 172.21.15.103:0/2860637526 shutdown_connections 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.745+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/2860637526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f4105f40 0x7ff2f41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.745+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/2860637526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4101450 0x7ff2f4105920 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.745+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/2860637526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4100b30 0x7ff2f4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.745+0000 7ff2fbe51640 1 -- 172.21.15.103:0/2860637526 >> 172.21.15.103:0/2860637526 conn(0x7ff2f40fc7a0 msgr2=0x7ff2f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 -- 172.21.15.103:0/2860637526 shutdown_connections 2024-09-19T07:35:56.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 -- 172.21.15.103:0/2860637526 wait complete. 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 Processor -- start 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 -- start start 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4100b30 0x7ff2f41a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4101450 0x7ff2f41a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f4105f40 0x7ff2f419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41185c0 con 0x7ff2f4100b30 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4118440 con 0x7ff2f4105f40 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2fbe51640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4118740 con 0x7ff2f4101450 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.746+0000 7ff2f9bed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4100b30 0x7ff2f41a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4100b30 0x7ff2f41a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58632/0 (socket says 172.21.15.103:58632) 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 -- 172.21.15.103:0/4209882970 learned_addr learned my addr 172.21.15.103:0/4209882970 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:35:56.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2fa3ee640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f4105f40 0x7ff2f419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f93ec640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4101450 0x7ff2f41a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 -- 172.21.15.103:0/4209882970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4101450 msgr2=0x7ff2f41a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4101450 0x7ff2f41a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 -- 172.21.15.103:0/4209882970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f4105f40 msgr2=0x7ff2f419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f4105f40 0x7ff2f419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 -- 172.21.15.103:0/4209882970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f419f0c0 con 0x7ff2f4100b30 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2f9bed640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4100b30 0x7ff2f41a47d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000b4a0 tx=0x7ff2f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2eaffd640 1 -- 172.21.15.103:0/4209882970 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0014070 con 0x7ff2f4100b30 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f419f3b0 con 0x7ff2f4100b30 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2eaffd640 1 -- 172.21.15.103:0/4209882970 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f0010af0 con 0x7ff2f4100b30 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2eaffd640 1 -- 172.21.15.103:0/4209882970 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0017d10 con 0x7ff2f4100b30 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.747+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41ab6a0 con 0x7ff2f4100b30 2024-09-19T07:35:56.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.748+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f4105920 con 0x7ff2f4100b30 2024-09-19T07:35:56.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.749+0000 7ff2eaffd640 1 -- 172.21.15.103:0/4209882970 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff2f0017070 con 0x7ff2f4100b30 2024-09-19T07:35:56.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.750+0000 7ff2eaffd640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff2e00a3240 0x7ff2e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:35:56.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.750+0000 7ff2eaffd640 1 -- 172.21.15.103:0/4209882970 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff2f00ce960 con 0x7ff2f4100b30 2024-09-19T07:35:56.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.750+0000 7ff2f93ec640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff2e00a3240 0x7ff2e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:35:56.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.750+0000 7ff2f93ec640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff2e00a3240 0x7ff2e00a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40096c0 tx=0x7ff2e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:35:56.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.754+0000 7ff2eaffd640 1 -- 172.21.15.103:0/4209882970 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff2f009bb20 con 0x7ff2f4100b30 2024-09-19T07:35:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.953+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2f41ab8d0 con 0x7ff2f4100b30 2024-09-19T07:35:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.954+0000 7ff2eaffd640 1 -- 172.21.15.103:0/4209882970 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7ff2f009b420 con 0x7ff2f4100b30 2024-09-19T07:35:56.955 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:35:56.955 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":12,"btime":"2024-09-19T07:35:56:095334+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-19T07:35:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 12 2024-09-19T07:35:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff2e00a3240 msgr2=0x7ff2e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff2e00a3240 0x7ff2e00a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40096c0 tx=0x7ff2e4000bc0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4100b30 msgr2=0x7ff2f41a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:35:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4100b30 0x7ff2f41a47d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000b4a0 tx=0x7ff2f000b960 comp rx=0 tx=0).stop 2024-09-19T07:35:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 shutdown_connections 2024-09-19T07:35:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff2e00a3240 0x7ff2e00a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f4105f40 0x7ff2f419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4101450 0x7ff2f41a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 --2- 172.21.15.103:0/4209882970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4100b30 0x7ff2f41a47d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:35:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 >> 172.21.15.103:0/4209882970 conn(0x7ff2f40fc7a0 msgr2=0x7ff2f41038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:35:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 shutdown_connections 2024-09-19T07:35:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:35:56.956+0000 7ff2fbe51640 1 -- 172.21.15.103:0/4209882970 wait complete. 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: Deploying daemon mds.b on smithi134 2024-09-19T07:35:57.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:57.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.103:6840/1521905484,v1:172.21.15.103:6841/1521905484] up:boot 2024-09-19T07:35:57.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: fsmap 11 up:standby 2024-09-19T07:35:57.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:35:57.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:56 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4209882970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:57.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: Deploying daemon mds.b on smithi134 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.103:6840/1521905484,v1:172.21.15.103:6841/1521905484] up:boot 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: fsmap 11 up:standby 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:35:57.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:56 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4209882970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:57.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: Deploying daemon mds.b on smithi134 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.103:6840/1521905484,v1:172.21.15.103:6841/1521905484] up:boot 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: fsmap 11 up:standby 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:35:57.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:56 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4209882970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:35:58.234 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 systemd[1]: Starting Ceph mds.b for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:35:58.506 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 podman[65541]: 2024-09-19 07:35:58.136675541 +0000 UTC m=+0.016163001 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:35:58.506 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 podman[65541]: 2024-09-19 07:35:58.235260186 +0000 UTC m=+0.114747637 container create a76ebc26f0589aa8949ff2573ecd9bb3a46bcd9313105d711551f83065a84273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-19T07:35:58.506 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 podman[65541]: 2024-09-19 07:35:58.480083288 +0000 UTC m=+0.359570734 container init a76ebc26f0589aa8949ff2573ecd9bb3a46bcd9313105d711551f83065a84273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T07:35:58.506 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 podman[65541]: 2024-09-19 07:35:58.483006385 +0000 UTC m=+0.362493832 container start a76ebc26f0589aa8949ff2573ecd9bb3a46bcd9313105d711551f83065a84273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T07:35:58.829 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 ceph-mds[65560]: -- 172.21.15.134:0/1853458721 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d47af2b860 con 0x55d47af09400 2024-09-19T07:35:58.829 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: starting mds.b at 2024-09-19T07:35:58.829 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 bash[65541]: a76ebc26f0589aa8949ff2573ecd9bb3a46bcd9313105d711551f83065a84273 2024-09-19T07:35:58.829 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:35:58 smithi134 systemd[1]: Started Ceph mds.b for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:35:59.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:59 smithi174 ceph-mon[35370]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:59.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:35:59 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:59 smithi103 ceph-mon[31201]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:35:59 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:59.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:59 smithi134 ceph-mon[35414]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:35:59.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:35:59.506 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:35:59 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:35:59.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:35:59.889 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:00.329 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:00 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] up:boot 2024-09-19T07:36:00.329 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:00 smithi103 ceph-mon[31201]: fsmap 12 up:standby 2024-09-19T07:36:00.329 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:00 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:36:00.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 -- 172.21.15.103:0/433013828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea501036e0 msgr2=0x7fea50107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:00.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 --2- 172.21.15.103:0/433013828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea501036e0 0x7fea50107b90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fea44009930 tx=0x7fea4402a420 comp rx=0 tx=0).stop 2024-09-19T07:36:00.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 -- 172.21.15.103:0/433013828 shutdown_connections 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 --2- 172.21.15.103:0/433013828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea501082a0 0x7fea5010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 --2- 172.21.15.103:0/433013828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea501036e0 0x7fea50107b90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 --2- 172.21.15.103:0/433013828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea50102d30 0x7fea50103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 -- 172.21.15.103:0/433013828 >> 172.21.15.103:0/433013828 conn(0x7fea500fc7d0 msgr2=0x7fea500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 -- 172.21.15.103:0/433013828 shutdown_connections 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.328+0000 7fea54c86640 1 -- 172.21.15.103:0/433013828 wait complete. 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 Processor -- start 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 -- start start 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea50102d30 0x7fea50198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea501036e0 0x7fea50198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea501082a0 0x7fea5019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea501120a0 con 0x7fea50102d30 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea50111f20 con 0x7fea501036e0 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea54c86640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea50112220 con 0x7fea501082a0 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea50102d30 0x7fea50198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea50102d30 0x7fea50198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49106/0 (socket says 172.21.15.103:49106) 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4e59c640 1 -- 172.21.15.103:0/3105447950 learned_addr learned my addr 172.21.15.103:0/3105447950 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4ed9d640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea501082a0 0x7fea5019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4dd9b640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea501036e0 0x7fea50198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4ed9d640 1 -- 172.21.15.103:0/3105447950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea501036e0 msgr2=0x7fea50198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4ed9d640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea501036e0 0x7fea50198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4ed9d640 1 -- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea50102d30 msgr2=0x7fea50198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4ed9d640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea50102d30 0x7fea50198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4ed9d640 1 -- 172.21.15.103:0/3105447950 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5019d5b0 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.329+0000 7fea4e59c640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea50102d30 0x7fea50198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.330+0000 7fea4ed9d640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea501082a0 0x7fea5019ced0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea4800d680 tx=0x7fea4800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.330+0000 7fea3f7fe640 1 -- 172.21.15.103:0/3105447950 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48017070 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.330+0000 7fea3f7fe640 1 -- 172.21.15.103:0/3105447950 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea48009820 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.330+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5019d8a0 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.330+0000 7fea3f7fe640 1 -- 172.21.15.103:0/3105447950 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48004e00 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.330+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea501a5180 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.331+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea50103110 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.332+0000 7fea3f7fe640 1 -- 172.21.15.103:0/3105447950 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fea480099d0 con 0x7fea501082a0 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.332+0000 7fea3f7fe640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea380a3240 0x7fea380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.333+0000 7fea4e59c640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea380a3240 0x7fea380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.333+0000 7fea3f7fe640 1 -- 172.21.15.103:0/3105447950 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fea480cecb0 con 0x7fea501082a0 2024-09-19T07:36:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.333+0000 7fea4e59c640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea380a3240 0x7fea380a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fea40009fd0 tx=0x7fea40009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.337+0000 7fea3f7fe640 1 -- 172.21.15.103:0/3105447950 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fea4809b780 con 0x7fea501082a0 2024-09-19T07:36:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:00 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] up:boot 2024-09-19T07:36:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:00 smithi174 ceph-mon[35370]: fsmap 12 up:standby 2024-09-19T07:36:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:00 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:36:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:00 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] up:boot 2024-09-19T07:36:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:00 smithi134 ceph-mon[35414]: fsmap 12 up:standby 2024-09-19T07:36:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:00 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:36:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.601+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5010a2b0 con 0x7fea501082a0 2024-09-19T07:36:00.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.602+0000 7fea3f7fe640 1 -- 172.21.15.103:0/3105447950 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7fea4809fa80 con 0x7fea501082a0 2024-09-19T07:36:00.603 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:00.603 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":13,"btime":"2024-09-19T07:35:59:130814+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14580,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T07:36:00.603 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 13 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea380a3240 msgr2=0x7fea380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea380a3240 0x7fea380a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fea40009fd0 tx=0x7fea40009700 comp rx=0 tx=0).stop 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea501082a0 msgr2=0x7fea5019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea501082a0 0x7fea5019ced0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea4800d680 tx=0x7fea4800db40 comp rx=0 tx=0).stop 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 shutdown_connections 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea380a3240 0x7fea380a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea501082a0 0x7fea5019ced0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea501036e0 0x7fea50198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 --2- 172.21.15.103:0/3105447950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea50102d30 0x7fea50198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:00.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 >> 172.21.15.103:0/3105447950 conn(0x7fea500fc7d0 msgr2=0x7fea50105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:00.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 shutdown_connections 2024-09-19T07:36:00.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:00.604+0000 7fea54c86640 1 -- 172.21.15.103:0/3105447950 wait complete. 2024-09-19T07:36:01.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3105447950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:36:01.305 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:01 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.336 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T07:36:01.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:01.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3105447950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:36:01.482 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:01 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.519 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3105447950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:01.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:01.521 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:36:01.521 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:01 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:02.234 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:02.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 -- 172.21.15.103:0/1591231501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304101420 msgr2=0x7f630410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 --2- 172.21.15.103:0/1591231501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304101420 0x7f630410d950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009cd0 tx=0x7f62fc02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 -- 172.21.15.103:0/1591231501 shutdown_connections 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 --2- 172.21.15.103:0/1591231501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304101420 0x7f630410d950 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 --2- 172.21.15.103:0/1591231501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6304100a80 0x7f6304100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 --2- 172.21.15.103:0/1591231501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6304106a80 0x7f6304106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 -- 172.21.15.103:0/1591231501 >> 172.21.15.103:0/1591231501 conn(0x7f63040fc7a0 msgr2=0x7f63040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 -- 172.21.15.103:0/1591231501 shutdown_connections 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.691+0000 7f630c67c640 1 -- 172.21.15.103:0/1591231501 wait complete. 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 Processor -- start 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 -- start start 2024-09-19T07:36:02.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304100a80 0x7f6304102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6304101420 0x7f6304102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6304106a80 0x7f63041030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6304110ef0 con 0x7f6304100a80 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6304110d70 con 0x7f6304101420 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630c67c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6304111070 con 0x7f6304106a80 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630a418640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304100a80 0x7f6304102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630a418640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304100a80 0x7f6304102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49114/0 (socket says 172.21.15.103:49114) 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630a418640 1 -- 172.21.15.103:0/2624149269 learned_addr learned my addr 172.21.15.103:0/2624149269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f6309c17640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6304101420 0x7f6304102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.692+0000 7f630ac19640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6304106a80 0x7f63041030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630a418640 1 -- 172.21.15.103:0/2624149269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6304106a80 msgr2=0x7f63041030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630a418640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6304106a80 0x7f63041030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630a418640 1 -- 172.21.15.103:0/2624149269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6304101420 msgr2=0x7f6304102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630a418640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6304101420 0x7f6304102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630a418640 1 -- 172.21.15.103:0/2624149269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63041a6900 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630a418640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304100a80 0x7f6304102650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f630000b4a0 tx=0x7f630000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f62fb7fe640 1 -- 172.21.15.103:0/2624149269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300014070 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f62fb7fe640 1 -- 172.21.15.103:0/2624149269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6300010af0 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f62fb7fe640 1 -- 172.21.15.103:0/2624149269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300017d10 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63041a6bf0 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.693+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63041a7180 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.694+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6304100ee0 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.695+0000 7f62fb7fe640 1 -- 172.21.15.103:0/2624149269 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6300017070 con 0x7f6304100a80 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.696+0000 7f62fb7fe640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62f00a32c0 0x7f62f00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.696+0000 7f6309c17640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62f00a32c0 0x7f62f00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:02.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.696+0000 7f62fb7fe640 1 -- 172.21.15.103:0/2624149269 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f63000ce6b0 con 0x7f6304100a80 2024-09-19T07:36:02.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.696+0000 7f6309c17640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62f00a32c0 0x7f62f00a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f63041040e0 tx=0x7f62f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:02.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.700+0000 7f62fb7fe640 1 -- 172.21.15.103:0/2624149269 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f630009b220 con 0x7f6304100a80 2024-09-19T07:36:02.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.924+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f63041a73b0 con 0x7f6304100a80 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.924+0000 7f62fb7fe640 1 -- 172.21.15.103:0/2624149269 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f630009f520 con 0x7f6304100a80 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stdout:[ 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stdout: "poolnum": 1, 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stdout: "poolname": ".mgr" 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:36:02.925 INFO:teuthology.orchestra.run.smithi103.stdout:] 2024-09-19T07:36:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62f00a32c0 msgr2=0x7f62f00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62f00a32c0 0x7f62f00a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f63041040e0 tx=0x7f62f400b390 comp rx=0 tx=0).stop 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304100a80 msgr2=0x7f6304102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304100a80 0x7f6304102650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f630000b4a0 tx=0x7f630000b960 comp rx=0 tx=0).stop 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 shutdown_connections 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f62f00a32c0 0x7f62f00a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6304106a80 0x7f63041030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6304101420 0x7f6304102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 --2- 172.21.15.103:0/2624149269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6304100a80 0x7f6304102650 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 >> 172.21.15.103:0/2624149269 conn(0x7f63040fc7a0 msgr2=0x7f63040fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 shutdown_connections 2024-09-19T07:36:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:02.927+0000 7f630c67c640 1 -- 172.21.15.103:0/2624149269 wait complete. 2024-09-19T07:36:03.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:03 smithi103 ceph-mon[31201]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:03.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:03 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:03 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2624149269' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T07:36:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:03 smithi134 ceph-mon[35414]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:03 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:03 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2624149269' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T07:36:03.605 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T07:36:03.606 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T07:36:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:03 smithi174 ceph-mon[35370]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:03 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:03 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2624149269' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T07:36:03.867 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:04.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/1144991678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28100e20 msgr2=0x7f3a28101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/1144991678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28100e20 0x7f3a28101200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009ca0 tx=0x7f3a1802a950 comp rx=0 tx=0).stop 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/1144991678 shutdown_connections 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/1144991678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a28102ef0 0x7f3a2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/1144991678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28102570 0x7f3a281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/1144991678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28100e20 0x7f3a28101200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/1144991678 >> 172.21.15.103:0/1144991678 conn(0x7f3a280fc7d0 msgr2=0x7f3a280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/1144991678 shutdown_connections 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.356+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/1144991678 wait complete. 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 Processor -- start 2024-09-19T07:36:04.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 -- start start 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a28100e20 0x7f3a281983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28102570 0x7f3a28198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28102ef0 0x7f3a2819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28110f80 con 0x7f3a28102570 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28110e00 con 0x7f3a28100e20 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a2d8c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28111100 con 0x7f3a28102ef0 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.357+0000 7f3a267fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28102570 0x7f3a28198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28102570 0x7f3a28198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49136/0 (socket says 172.21.15.103:49136) 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 -- 172.21.15.103:0/862570859 learned_addr learned my addr 172.21.15.103:0/862570859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a277fe640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28102ef0 0x7f3a2819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a26ffd640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a28100e20 0x7f3a281983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 -- 172.21.15.103:0/862570859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28102ef0 msgr2=0x7f3a2819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:04.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28102ef0 0x7f3a2819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 -- 172.21.15.103:0/862570859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a28100e20 msgr2=0x7f3a281983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a28100e20 0x7f3a281983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 -- 172.21.15.103:0/862570859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2819d380 con 0x7f3a28102570 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a26ffd640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a28100e20 0x7f3a281983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a267fc640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28102570 0x7f3a28198910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c920 tx=0x7f3a1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a2c8c1640 1 -- 172.21.15.103:0/862570859 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c007c70 con 0x7f3a28102570 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2819d670 con 0x7f3a28102570 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a2c8c1640 1 -- 172.21.15.103:0/862570859 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c007e10 con 0x7f3a28102570 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a2c8c1640 1 -- 172.21.15.103:0/862570859 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c018890 con 0x7f3a28102570 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.358+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a281a4f50 con 0x7f3a28102570 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.359+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a28101200 con 0x7f3a28102570 2024-09-19T07:36:04.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.360+0000 7f3a2c8c1640 1 -- 172.21.15.103:0/862570859 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3a1c018a50 con 0x7f3a28102570 2024-09-19T07:36:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.361+0000 7f3a2c8c1640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3a100a3240 0x7f3a100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.361+0000 7f3a26ffd640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3a100a3240 0x7f3a100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.361+0000 7f3a2c8c1640 1 -- 172.21.15.103:0/862570859 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f3a1c0cf850 con 0x7f3a28102570 2024-09-19T07:36:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.361+0000 7f3a26ffd640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3a100a3240 0x7f3a100a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006010 tx=0x7f3a18005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:04.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.364+0000 7f3a2c8c1640 1 -- 172.21.15.103:0/862570859 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a1c014030 con 0x7f3a28102570 2024-09-19T07:36:04.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.532+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3a281a5120 con 0x7f3a28102570 2024-09-19T07:36:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.818+0000 7f3a2c8c1640 1 -- 172.21.15.103:0/862570859 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f3a1c09caa0 con 0x7f3a28102570 2024-09-19T07:36:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' created 2024-09-19T07:36:04.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:04.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:04.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:04.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:04.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:04.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:04 smithi103 ceph-mon[31201]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3a100a3240 msgr2=0x7f3a100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3a100a3240 0x7f3a100a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006010 tx=0x7f3a18005f80 comp rx=0 tx=0).stop 2024-09-19T07:36:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28102570 msgr2=0x7f3a28198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28102570 0x7f3a28198910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c920 tx=0x7f3a1c00cde0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 shutdown_connections 2024-09-19T07:36:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3a100a3240 0x7f3a100a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28102ef0 0x7f3a2819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a28102570 0x7f3a28198910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 --2- 172.21.15.103:0/862570859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a28100e20 0x7f3a281983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:04.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 >> 172.21.15.103:0/862570859 conn(0x7f3a280fc7d0 msgr2=0x7f3a2806a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:04.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 shutdown_connections 2024-09-19T07:36:04.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:04.820+0000 7f3a2d8c3640 1 -- 172.21.15.103:0/862570859 wait complete. 2024-09-19T07:36:05.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:05.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:05.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:05.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:05.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:05.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:04 smithi134 ceph-mon[35414]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:05.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:05.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:05.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:36:05.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:36:05.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:36:05.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:04 smithi174 ceph-mon[35370]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:05.476 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T07:36:05.723 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:06.068 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:05 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/862570859' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T07:36:06.068 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:05 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/862570859' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T07:36:06.068 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:05 smithi103 ceph-mon[31201]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T07:36:06.069 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:05 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/862570859' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T07:36:06.069 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:05 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/862570859' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T07:36:06.069 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:05 smithi174 ceph-mon[35370]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T07:36:06.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:05 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/862570859' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T07:36:06.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:05 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/862570859' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T07:36:06.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:05 smithi134 ceph-mon[35414]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.154+0000 7fbd1b289640 1 -- 172.21.15.103:0/2615990257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd14100480 msgr2=0x7fbd14100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.154+0000 7fbd1b289640 1 --2- 172.21.15.103:0/2615990257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd14100480 0x7fbd14100860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009900 tx=0x7fbd1002a410 comp rx=0 tx=0).stop 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.154+0000 7fbd1b289640 1 -- 172.21.15.103:0/2615990257 shutdown_connections 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.154+0000 7fbd1b289640 1 --2- 172.21.15.103:0/2615990257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd141055b0 0x7fbd1410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.154+0000 7fbd1b289640 1 --2- 172.21.15.103:0/2615990257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14100e30 0x7fbd14104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.154+0000 7fbd1b289640 1 --2- 172.21.15.103:0/2615990257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd14100480 0x7fbd14100860 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 -- 172.21.15.103:0/2615990257 >> 172.21.15.103:0/2615990257 conn(0x7fbd14077210 msgr2=0x7fbd140fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 -- 172.21.15.103:0/2615990257 shutdown_connections 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 -- 172.21.15.103:0/2615990257 wait complete. 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 Processor -- start 2024-09-19T07:36:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 -- start start 2024-09-19T07:36:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd14100480 0x7fbd14197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14100e30 0x7fbd14197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd141055b0 0x7fbd14197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14110630 con 0x7fbd141055b0 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141104b0 con 0x7fbd14100e30 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.155+0000 7fbd1b289640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141107b0 con 0x7fbd14100480 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd141055b0 0x7fbd14197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd141055b0 0x7fbd14197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49148/0 (socket says 172.21.15.103:49148) 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 -- 172.21.15.103:0/3196190396 learned_addr learned my addr 172.21.15.103:0/3196190396 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19025640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd14100480 0x7fbd14197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd18824640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14100e30 0x7fbd14197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 -- 172.21.15.103:0/3196190396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd14100480 msgr2=0x7fbd14197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd14100480 0x7fbd14197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 -- 172.21.15.103:0/3196190396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14100e30 msgr2=0x7fbd14197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:06.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14100e30 0x7fbd14197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 -- 172.21.15.103:0/3196190396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd141a6990 con 0x7fbd141055b0 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19025640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd14100480 0x7fbd14197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd18824640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14100e30 0x7fbd14197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd19826640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd141055b0 0x7fbd14197ba0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b970 tx=0x7fbd0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd0a7fc640 1 -- 172.21.15.103:0/3196190396 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c019070 con 0x7fbd141055b0 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd0a7fc640 1 -- 172.21.15.103:0/3196190396 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd0c00cb40 con 0x7fbd141055b0 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd141a6c80 con 0x7fbd141055b0 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd0a7fc640 1 -- 172.21.15.103:0/3196190396 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c007650 con 0x7fbd141055b0 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.156+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd141a71c0 con 0x7fbd141055b0 2024-09-19T07:36:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.158+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd14100860 con 0x7fbd141055b0 2024-09-19T07:36:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.159+0000 7fbd0a7fc640 1 -- 172.21.15.103:0/3196190396 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbd0c01c070 con 0x7fbd141055b0 2024-09-19T07:36:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.160+0000 7fbd0a7fc640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd000a3240 0x7fbd000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.160+0000 7fbd19025640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd000a3240 0x7fbd000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.160+0000 7fbd0a7fc640 1 -- 172.21.15.103:0/3196190396 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7fbd0c0cf0e0 con 0x7fbd141055b0 2024-09-19T07:36:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.160+0000 7fbd19025640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd000a3240 0x7fbd000a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005ab0 tx=0x7fbd10005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.164+0000 7fbd0a7fc640 1 -- 172.21.15.103:0/3196190396 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd0c09c130 con 0x7fbd141055b0 2024-09-19T07:36:06.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.329+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbd141a7520 con 0x7fbd141055b0 2024-09-19T07:36:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.825+0000 7fbd0a7fc640 1 -- 172.21.15.103:0/3196190396 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7fbd0c09ba30 con 0x7fbd141055b0 2024-09-19T07:36:06.825 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' created 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd000a3240 msgr2=0x7fbd000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd000a3240 0x7fbd000a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005ab0 tx=0x7fbd10005a20 comp rx=0 tx=0).stop 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd141055b0 msgr2=0x7fbd14197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd141055b0 0x7fbd14197ba0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b970 tx=0x7fbd0c00be30 comp rx=0 tx=0).stop 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 shutdown_connections 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd000a3240 0x7fbd000a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd141055b0 0x7fbd14197ba0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14100e30 0x7fbd14197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 --2- 172.21.15.103:0/3196190396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd14100480 0x7fbd14197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:06.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 >> 172.21.15.103:0/3196190396 conn(0x7fbd14077210 msgr2=0x7fbd140fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:06.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 shutdown_connections 2024-09-19T07:36:06.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:06.828+0000 7fbd1b289640 1 -- 172.21.15.103:0/3196190396 wait complete. 2024-09-19T07:36:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:06 smithi174 ceph-mon[35370]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T07:36:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:06 smithi174 ceph-mon[35370]: pgmap v274: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:06 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3196190396' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:36:07.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:06 smithi134 ceph-mon[35414]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T07:36:07.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:06 smithi134 ceph-mon[35414]: pgmap v274: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:07.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:06 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3196190396' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:36:07.122 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:06 smithi103 ceph-mon[31201]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T07:36:07.123 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:06 smithi103 ceph-mon[31201]: pgmap v274: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:07.123 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:06 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3196190396' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T07:36:07.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T07:36:07.697 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:08.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:07 smithi134 ceph-mon[35414]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:36:08.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:07 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3196190396' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T07:36:08.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:07 smithi134 ceph-mon[35414]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T07:36:08.104 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:07 smithi103 ceph-mon[31201]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:36:08.104 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:07 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3196190396' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T07:36:08.104 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:07 smithi103 ceph-mon[31201]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T07:36:08.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:07 smithi174 ceph-mon[35370]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T07:36:08.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:07 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3196190396' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T07:36:08.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:07 smithi174 ceph-mon[35370]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T07:36:08.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 -- 172.21.15.103:0/70036726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88100e20 msgr2=0x7fbf88101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:08.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/70036726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88100e20 0x7fbf88101200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009900 tx=0x7fbf8402a480 comp rx=0 tx=0).stop 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 -- 172.21.15.103:0/70036726 shutdown_connections 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/70036726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf88102ef0 0x7fbf8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/70036726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf88102570 0x7fbf881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/70036726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88100e20 0x7fbf88101200 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 -- 172.21.15.103:0/70036726 >> 172.21.15.103:0/70036726 conn(0x7fbf880fc7d0 msgr2=0x7fbf880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.186+0000 7fbf8fd14640 1 -- 172.21.15.103:0/70036726 shutdown_connections 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 -- 172.21.15.103:0/70036726 wait complete. 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 Processor -- start 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 -- start start 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf88100e20 0x7fbf88198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88102570 0x7fbf88198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:08.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf88102ef0 0x7fbf8819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88110fa0 con 0x7fbf88102570 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88110e20 con 0x7fbf88102ef0 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8fd14640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88111120 con 0x7fbf88100e20 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8d2af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88102570 0x7fbf88198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8d2af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88102570 0x7fbf88198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37870/0 (socket says 172.21.15.103:37870) 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.187+0000 7fbf8d2af640 1 -- 172.21.15.103:0/2489169649 learned_addr learned my addr 172.21.15.103:0/2489169649 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8dab0640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf88100e20 0x7fbf88198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8e2b1640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf88102ef0 0x7fbf8819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8d2af640 1 -- 172.21.15.103:0/2489169649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf88100e20 msgr2=0x7fbf88198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8d2af640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf88100e20 0x7fbf88198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8d2af640 1 -- 172.21.15.103:0/2489169649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf88102ef0 msgr2=0x7fbf8819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:08.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8d2af640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf88102ef0 0x7fbf8819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8d2af640 1 -- 172.21.15.103:0/2489169649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8819d540 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8d2af640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88102570 0x7fbf88198ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b4a0 tx=0x7fbf7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf7effd640 1 -- 172.21.15.103:0/2489169649 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78014070 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf7effd640 1 -- 172.21.15.103:0/2489169649 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78010af0 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8819d830 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf7effd640 1 -- 172.21.15.103:0/2489169649 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78017d10 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.188+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881a5110 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.189+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf88101200 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.190+0000 7fbf7effd640 1 -- 172.21.15.103:0/2489169649 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fbf78017070 con 0x7fbf88102570 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.191+0000 7fbf7effd640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbf740a3240 0x7fbf740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.191+0000 7fbf8dab0640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbf740a3240 0x7fbf740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:08.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.191+0000 7fbf7effd640 1 -- 172.21.15.103:0/2489169649 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7fbf780cf090 con 0x7fbf88102570 2024-09-19T07:36:08.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.192+0000 7fbf8dab0640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbf740a3240 0x7fbf740a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005ab0 tx=0x7fbf84005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:08.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.195+0000 7fbf7effd640 1 -- 172.21.15.103:0/2489169649 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbf7809bec0 con 0x7fbf88102570 2024-09-19T07:36:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.394+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbf881998f0 con 0x7fbf88102570 2024-09-19T07:36:08.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.843+0000 7fbf7effd640 1 -- 172.21.15.103:0/2489169649 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fbf881998f0 con 0x7fbf88102570 2024-09-19T07:36:08.844 INFO:teuthology.orchestra.run.smithi103.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T07:36:08.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbf740a3240 msgr2=0x7fbf740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:08.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbf740a3240 0x7fbf740a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005ab0 tx=0x7fbf84005a20 comp rx=0 tx=0).stop 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88102570 msgr2=0x7fbf88198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88102570 0x7fbf88198ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b4a0 tx=0x7fbf7800b960 comp rx=0 tx=0).stop 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 shutdown_connections 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbf740a3240 0x7fbf740a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf88102ef0 0x7fbf8819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf88102570 0x7fbf88198ad0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 --2- 172.21.15.103:0/2489169649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf88100e20 0x7fbf88198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 >> 172.21.15.103:0/2489169649 conn(0x7fbf880fc7d0 msgr2=0x7fbf8806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 shutdown_connections 2024-09-19T07:36:08.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:08.846+0000 7fbf8fd14640 1 -- 172.21.15.103:0/2489169649 wait complete. 2024-09-19T07:36:09.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:08 smithi174 ceph-mon[35370]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T07:36:09.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:08 smithi174 ceph-mon[35370]: pgmap v277: 97 pgs: 16 active+clean, 17 creating+peering, 64 unknown; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:09.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:08 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2489169649' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:36:09.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:08 smithi174 ceph-mon[35370]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T07:36:09.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:08 smithi103 ceph-mon[31201]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T07:36:09.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:08 smithi103 ceph-mon[31201]: pgmap v277: 97 pgs: 16 active+clean, 17 creating+peering, 64 unknown; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:09.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:08 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2489169649' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:36:09.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:08 smithi103 ceph-mon[31201]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T07:36:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:08 smithi134 ceph-mon[35414]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T07:36:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:08 smithi134 ceph-mon[35414]: pgmap v277: 97 pgs: 16 active+clean, 17 creating+peering, 64 unknown; 577 KiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:36:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:08 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2489169649' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T07:36:09.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:08 smithi134 ceph-mon[35414]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T07:36:09.559 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:36:09.805 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:10.102 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:09 smithi103 ceph-mon[31201]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T07:36:10.102 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:09 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2489169649' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T07:36:10.102 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:09 smithi103 ceph-mon[31201]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T07:36:10.102 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:09 smithi103 ceph-mon[31201]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T07:36:10.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:09 smithi174 ceph-mon[35370]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T07:36:10.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:09 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2489169649' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T07:36:10.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:09 smithi174 ceph-mon[35370]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T07:36:10.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:09 smithi174 ceph-mon[35370]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T07:36:10.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:09 smithi134 ceph-mon[35414]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T07:36:10.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:09 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2489169649' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T07:36:10.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:09 smithi134 ceph-mon[35414]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T07:36:10.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:09 smithi134 ceph-mon[35414]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T07:36:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 -- 172.21.15.103:0/2105211379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754102ef0 msgr2=0x7ff75410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 --2- 172.21.15.103:0/2105211379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754102ef0 0x7ff75410da40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009ca0 tx=0x7ff74c02a950 comp rx=0 tx=0).stop 2024-09-19T07:36:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 -- 172.21.15.103:0/2105211379 shutdown_connections 2024-09-19T07:36:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 --2- 172.21.15.103:0/2105211379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754102ef0 0x7ff75410da40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 --2- 172.21.15.103:0/2105211379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff754102570 0x7ff7541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 --2- 172.21.15.103:0/2105211379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754100e20 0x7ff754101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 -- 172.21.15.103:0/2105211379 >> 172.21.15.103:0/2105211379 conn(0x7ff7540fc7d0 msgr2=0x7ff7540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:10.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 -- 172.21.15.103:0/2105211379 shutdown_connections 2024-09-19T07:36:10.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.353+0000 7ff75a820640 1 -- 172.21.15.103:0/2105211379 wait complete. 2024-09-19T07:36:10.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.354+0000 7ff75a820640 1 Processor -- start 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.354+0000 7ff75a820640 1 -- start start 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.354+0000 7ff75a820640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754100e20 0x7ff754199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.354+0000 7ff75a820640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754102570 0x7ff754199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff75a820640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff754102ef0 0x7ff7541a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff75a820640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff754111020 con 0x7ff754100e20 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff75a820640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff754110ea0 con 0x7ff754102ef0 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff75a820640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541111a0 con 0x7ff754102570 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754100e20 0x7ff754199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754100e20 0x7ff754199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37896/0 (socket says 172.21.15.103:37896) 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 -- 172.21.15.103:0/1732449295 learned_addr learned my addr 172.21.15.103:0/1732449295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff7537fe640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754102570 0x7ff754199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:10.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff758dbd640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff754102ef0 0x7ff7541a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 -- 172.21.15.103:0/1732449295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754102570 msgr2=0x7ff754199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754102570 0x7ff754199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 -- 172.21.15.103:0/1732449295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff754102ef0 msgr2=0x7ff7541a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff754102ef0 0x7ff7541a44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 -- 172.21.15.103:0/1732449295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7541a4a10 con 0x7ff754100e20 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff7537fe640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754102570 0x7ff754199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff758dbd640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff754102ef0 0x7ff7541a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff753fff640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754100e20 0x7ff754199450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b4a0 tx=0x7ff74400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff7517fa640 1 -- 172.21.15.103:0/1732449295 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744014070 con 0x7ff754100e20 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7541a4be0 con 0x7ff754100e20 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff7517fa640 1 -- 172.21.15.103:0/1732449295 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff744010af0 con 0x7ff754100e20 2024-09-19T07:36:10.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.355+0000 7ff7517fa640 1 -- 172.21.15.103:0/1732449295 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744017d10 con 0x7ff754100e20 2024-09-19T07:36:10.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.356+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7541a50a0 con 0x7ff754100e20 2024-09-19T07:36:10.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.356+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff754101280 con 0x7ff754100e20 2024-09-19T07:36:10.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.358+0000 7ff7517fa640 1 -- 172.21.15.103:0/1732449295 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff744017070 con 0x7ff754100e20 2024-09-19T07:36:10.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.358+0000 7ff7517fa640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff73c0a3240 0x7ff73c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:10.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.358+0000 7ff7537fe640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff73c0a3240 0x7ff73c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:10.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.358+0000 7ff7517fa640 1 -- 172.21.15.103:0/1732449295 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7ff7440cec40 con 0x7ff754100e20 2024-09-19T07:36:10.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.359+0000 7ff7537fe640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff73c0a3240 0x7ff73c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff748006fd0 tx=0x7ff748008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:10.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.366+0000 7ff7517fa640 1 -- 172.21.15.103:0/1732449295 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff74409bab0 con 0x7ff754100e20 2024-09-19T07:36:10.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.541+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7541090c0 con 0x7ff754100e20 2024-09-19T07:36:10.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.542+0000 7ff7517fa640 1 -- 172.21.15.103:0/1732449295 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22337 (secure 0 0 0) 0x7ff74409b3b0 con 0x7ff754100e20 2024-09-19T07:36:10.542 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:10.543 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":82,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:08.395046+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff73c0a3240 msgr2=0x7ff73c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff73c0a3240 0x7ff73c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff748006fd0 tx=0x7ff748008040 comp rx=0 tx=0).stop 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754100e20 msgr2=0x7ff754199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754100e20 0x7ff754199450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b4a0 tx=0x7ff74400b960 comp rx=0 tx=0).stop 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 shutdown_connections 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff73c0a3240 0x7ff73c0a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff754102ef0 0x7ff7541a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754102570 0x7ff754199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 --2- 172.21.15.103:0/1732449295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff754100e20 0x7ff754199450 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 >> 172.21.15.103:0/1732449295 conn(0x7ff7540fc7d0 msgr2=0x7ff7540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 shutdown_connections 2024-09-19T07:36:10.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:10.545+0000 7ff75a820640 1 -- 172.21.15.103:0/1732449295 wait complete. 2024-09-19T07:36:11.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:10 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] up:active 2024-09-19T07:36:11.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:10 smithi174 ceph-mon[35370]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:11.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:10 smithi174 ceph-mon[35370]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:11.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:10 smithi174 ceph-mon[35370]: pgmap v279: 97 pgs: 57 active+clean, 12 creating+peering, 28 unknown; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-09-19T07:36:11.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:10 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1732449295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:11.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:10 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] up:active 2024-09-19T07:36:11.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:10 smithi103 ceph-mon[31201]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:11.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:10 smithi103 ceph-mon[31201]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:11.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:10 smithi103 ceph-mon[31201]: pgmap v279: 97 pgs: 57 active+clean, 12 creating+peering, 28 unknown; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-09-19T07:36:11.192 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:10 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1732449295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:11.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:36:11.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:10 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] up:active 2024-09-19T07:36:11.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:10 smithi134 ceph-mon[35414]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:11.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:10 smithi134 ceph-mon[35414]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:11.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:10 smithi134 ceph-mon[35414]: pgmap v279: 97 pgs: 57 active+clean, 12 creating+peering, 28 unknown; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 1 op/s 2024-09-19T07:36:11.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:10 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1732449295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:11.475 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:11.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 -- 172.21.15.103:0/912408634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 msgr2=0x7fa46c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:11.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 --2- 172.21.15.103:0/912408634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c1021c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009900 tx=0x7fa45c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:36:11.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 -- 172.21.15.103:0/912408634 shutdown_connections 2024-09-19T07:36:11.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 --2- 172.21.15.103:0/912408634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa46c106f10 0x7fa46c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:11.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 --2- 172.21.15.103:0/912408634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46c102790 0x7fa46c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 --2- 172.21.15.103:0/912408634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c1021c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 -- 172.21.15.103:0/912408634 >> 172.21.15.103:0/912408634 conn(0x7fa46c0fd8c0 msgr2=0x7fa46c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 -- 172.21.15.103:0/912408634 shutdown_connections 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.968+0000 7fa472412640 1 -- 172.21.15.103:0/912408634 wait complete. 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 Processor -- start 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 -- start start 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46c102790 0x7fa46c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa46c106f10 0x7fa46c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c112010 con 0x7fa46c101de0 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c111e90 con 0x7fa46c102790 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa472412640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c112190 con 0x7fa46c106f10 2024-09-19T07:36:11.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa46bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37922/0 (socket says 172.21.15.103:37922) 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.969+0000 7fa46bfff640 1 -- 172.21.15.103:0/1240951123 learned_addr learned my addr 172.21.15.103:0/1240951123 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46b7fe640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46c102790 0x7fa46c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa4709af640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa46c106f10 0x7fa46c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46bfff640 1 -- 172.21.15.103:0/1240951123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa46c106f10 msgr2=0x7fa46c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46bfff640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa46c106f10 0x7fa46c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46bfff640 1 -- 172.21.15.103:0/1240951123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46c102790 msgr2=0x7fa46c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:11.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46bfff640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46c102790 0x7fa46c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46bfff640 1 -- 172.21.15.103:0/1240951123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c1a6a70 con 0x7fa46c101de0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46b7fe640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46c102790 0x7fa46c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa46bfff640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c0744f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa45c005ab0 tx=0x7fa45c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa4697fa640 1 -- 172.21.15.103:0/1240951123 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c03f030 con 0x7fa46c101de0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa4697fa640 1 -- 172.21.15.103:0/1240951123 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa45c003ce0 con 0x7fa46c101de0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46c1a6d00 con 0x7fa46c101de0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa4697fa640 1 -- 172.21.15.103:0/1240951123 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c035070 con 0x7fa46c101de0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.970+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46c1a7190 con 0x7fa46c101de0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.971+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa46c102240 con 0x7fa46c101de0 2024-09-19T07:36:12.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.972+0000 7fa4697fa640 1 -- 172.21.15.103:0/1240951123 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa45c042030 con 0x7fa46c101de0 2024-09-19T07:36:12.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.973+0000 7fa4697fa640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa4540a3240 0x7fa4540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:12.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.973+0000 7fa46b7fe640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa4540a3240 0x7fa4540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:12.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.973+0000 7fa4697fa640 1 -- 172.21.15.103:0/1240951123 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fa45c0f0180 con 0x7fa46c101de0 2024-09-19T07:36:12.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.973+0000 7fa46b7fe640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa4540a3240 0x7fa4540a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa46c075f80 tx=0x7fa460008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:12.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:11.977+0000 7fa4697fa640 1 -- 172.21.15.103:0/1240951123 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa45c0bcff0 con 0x7fa46c101de0 2024-09-19T07:36:12.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.145+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa46c02d7c0 con 0x7fa46c101de0 2024-09-19T07:36:12.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:11 smithi103 ceph-mon[31201]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T07:36:12.145 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:11 smithi103 ceph-mon[31201]: Cluster is now healthy 2024-09-19T07:36:12.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.147+0000 7fa4697fa640 1 -- 172.21.15.103:0/1240951123 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22337 (secure 0 0 0) 0x7fa45c0bc8f0 con 0x7fa46c101de0 2024-09-19T07:36:12.148 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:12.148 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":82,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:08.395046+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa4540a3240 msgr2=0x7fa4540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa4540a3240 0x7fa4540a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa46c075f80 tx=0x7fa460008040 comp rx=0 tx=0).stop 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 msgr2=0x7fa46c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c0744f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa45c005ab0 tx=0x7fa45c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 shutdown_connections 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa4540a3240 0x7fa4540a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa46c106f10 0x7fa46c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa46c102790 0x7fa46c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:12.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 --2- 172.21.15.103:0/1240951123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa46c101de0 0x7fa46c0744f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 >> 172.21.15.103:0/1240951123 conn(0x7fa46c0fd8c0 msgr2=0x7fa46c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 shutdown_connections 2024-09-19T07:36:12.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:12.150+0000 7fa472412640 1 -- 172.21.15.103:0/1240951123 wait complete. 2024-09-19T07:36:12.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:11 smithi174 ceph-mon[35370]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T07:36:12.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:11 smithi174 ceph-mon[35370]: Cluster is now healthy 2024-09-19T07:36:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:11 smithi134 ceph-mon[35414]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T07:36:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:11 smithi134 ceph-mon[35414]: Cluster is now healthy 2024-09-19T07:36:12.789 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T07:36:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:12 smithi103 ceph-mon[31201]: pgmap v280: 97 pgs: 89 active+clean, 8 creating+peering; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T07:36:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:12 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1240951123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:13.036 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:13.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:12 smithi174 ceph-mon[35370]: pgmap v280: 97 pgs: 89 active+clean, 8 creating+peering; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T07:36:13.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:12 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1240951123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:13.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:12 smithi134 ceph-mon[35414]: pgmap v280: 97 pgs: 89 active+clean, 8 creating+peering; 577 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T07:36:13.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:12 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1240951123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 -- 172.21.15.103:0/2859590992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 msgr2=0x7f491c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 --2- 172.21.15.103:0/2859590992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c106e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4914009930 tx=0x7f491402a420 comp rx=0 tx=0).stop 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 -- 172.21.15.103:0/2859590992 shutdown_connections 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 --2- 172.21.15.103:0/2859590992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f491c101420 0x7f491c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 --2- 172.21.15.103:0/2859590992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f491c100a80 0x7f491c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 --2- 172.21.15.103:0/2859590992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c106e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 -- 172.21.15.103:0/2859590992 >> 172.21.15.103:0/2859590992 conn(0x7f491c0fc7a0 msgr2=0x7f491c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 -- 172.21.15.103:0/2859590992 shutdown_connections 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.540+0000 7f4924015640 1 -- 172.21.15.103:0/2859590992 wait complete. 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 Processor -- start 2024-09-19T07:36:13.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 -- start start 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f491c100a80 0x7f491c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f491c101420 0x7f491c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f491c110ea0 con 0x7f491c106a80 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f491c110d20 con 0x7f491c101420 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4924015640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f491c111020 con 0x7f491c100a80 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f49225b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f49225b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37934/0 (socket says 172.21.15.103:37934) 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f49225b2640 1 -- 172.21.15.103:0/2791989592 learned_addr learned my addr 172.21.15.103:0/2791989592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f4921db1640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f491c100a80 0x7f491c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f49215b0640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f491c101420 0x7f491c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f49225b2640 1 -- 172.21.15.103:0/2791989592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f491c100a80 msgr2=0x7f491c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:13.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.541+0000 7f49225b2640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f491c100a80 0x7f491c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f49225b2640 1 -- 172.21.15.103:0/2791989592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f491c101420 msgr2=0x7f491c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f49225b2640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f491c101420 0x7f491c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f49225b2640 1 -- 172.21.15.103:0/2791989592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c19c290 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f49225b2640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c19bbb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f490c009fd0 tx=0x7f490c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f491affd640 1 -- 172.21.15.103:0/2791989592 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c011830 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f491affd640 1 -- 172.21.15.103:0/2791989592 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c016070 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c19c580 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f491affd640 1 -- 172.21.15.103:0/2791989592 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c00f2e0 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.542+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c1a3e60 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.543+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491c100ee0 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.544+0000 7f491affd640 1 -- 172.21.15.103:0/2791989592 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f490c0119d0 con 0x7f491c106a80 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.545+0000 7f491affd640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f49040a3240 0x7f49040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:13.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.545+0000 7f4921db1640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f49040a3240 0x7f49040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:13.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.545+0000 7f491affd640 1 -- 172.21.15.103:0/2791989592 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f490c0cedc0 con 0x7f491c106a80 2024-09-19T07:36:13.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.545+0000 7f4921db1640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f49040a3240 0x7f49040a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f49140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:13.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.550+0000 7f491affd640 1 -- 172.21.15.103:0/2791989592 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f490c09bc30 con 0x7f491c106a80 2024-09-19T07:36:13.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.752+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f491c1a4090 con 0x7f491c106a80 2024-09-19T07:36:13.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.865+0000 7f491affd640 1 -- 172.21.15.103:0/2791989592 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f490c09b530 con 0x7f491c106a80 2024-09-19T07:36:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.867+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f49040a3240 msgr2=0x7f49040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:13.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.867+0000 7f4924015640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f49040a3240 0x7f49040a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f49140058d0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.867+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 msgr2=0x7f491c19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.867+0000 7f4924015640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c19bbb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f490c009fd0 tx=0x7f490c00b8e0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 shutdown_connections 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f49040a3240 0x7f49040a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f491c106a80 0x7f491c19bbb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f491c101420 0x7f491c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 --2- 172.21.15.103:0/2791989592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f491c100a80 0x7f491c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 >> 172.21.15.103:0/2791989592 conn(0x7f491c0fc7a0 msgr2=0x7f491c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 shutdown_connections 2024-09-19T07:36:13.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:13.868+0000 7f4924015640 1 -- 172.21.15.103:0/2791989592 wait complete. 2024-09-19T07:36:14.140 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:13 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2791989592' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T07:36:14.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:13 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2791989592' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T07:36:14.178 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:13 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2791989592' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T07:36:14.498 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-19T07:36:14.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-19T07:36:14.579 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:36:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: 2024-09-19T07:36:14.139+0000 7f4353afb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T07:36:14.745 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:15.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:14 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2791989592' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T07:36:15.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:14 smithi174 ceph-mon[35370]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:15.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:14 smithi174 ceph-mon[35370]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-19T07:36:15.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:14 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2791989592' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T07:36:15.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:14 smithi103 ceph-mon[31201]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:15.182 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:14 smithi103 ceph-mon[31201]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-19T07:36:15.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 -- 172.21.15.103:0/3794371681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 msgr2=0x7f602c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:15.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 --2- 172.21.15.103:0/3794371681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c103120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f601c009900 tx=0x7f601c02a480 comp rx=0 tx=0).stop 2024-09-19T07:36:15.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 -- 172.21.15.103:0/3794371681 shutdown_connections 2024-09-19T07:36:15.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 --2- 172.21.15.103:0/3794371681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c10c170 0x7f602c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 --2- 172.21.15.103:0/3794371681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c103660 0x7f602c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 --2- 172.21.15.103:0/3794371681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c103120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 -- 172.21.15.103:0/3794371681 >> 172.21.15.103:0/3794371681 conn(0x7f602c0fc7a0 msgr2=0x7f602c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 -- 172.21.15.103:0/3794371681 shutdown_connections 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 -- 172.21.15.103:0/3794371681 wait complete. 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 Processor -- start 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.182+0000 7f6032319640 1 -- start start 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f6032319640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f6032319640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c103660 0x7f602c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f6032319640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c10c170 0x7f602c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f6032319640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f602c112030 con 0x7f602c102d40 2024-09-19T07:36:15.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f6032319640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f602c111eb0 con 0x7f602c10c170 2024-09-19T07:36:15.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f6032319640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1121b0 con 0x7f602c103660 2024-09-19T07:36:15.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:15.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37964/0 (socket says 172.21.15.103:37964) 2024-09-19T07:36:15.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 -- 172.21.15.103:0/3231688574 learned_addr learned my addr 172.21.15.103:0/3231688574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:15.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f60308b6640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c10c170 0x7f602c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:15.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f6023fff640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c103660 0x7f602c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 -- 172.21.15.103:0/3231688574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c103660 msgr2=0x7f602c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c103660 0x7f602c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 -- 172.21.15.103:0/3231688574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c10c170 msgr2=0x7f602c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c10c170 0x7f602c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 -- 172.21.15.103:0/3231688574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c19d560 con 0x7f602c102d40 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.183+0000 7f602bfff640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c1985b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f601c005ab0 tx=0x7f601c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.184+0000 7f6029ffb640 1 -- 172.21.15.103:0/3231688574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c03f030 con 0x7f602c102d40 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.184+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602c19d7f0 con 0x7f602c102d40 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.184+0000 7f6029ffb640 1 -- 172.21.15.103:0/3231688574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c003ce0 con 0x7f602c102d40 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.184+0000 7f6029ffb640 1 -- 172.21.15.103:0/3231688574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c035070 con 0x7f602c102d40 2024-09-19T07:36:15.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.184+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602c1a5130 con 0x7f602c102d40 2024-09-19T07:36:15.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.185+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602c1031a0 con 0x7f602c102d40 2024-09-19T07:36:15.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.186+0000 7f6029ffb640 1 -- 172.21.15.103:0/3231688574 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f601c042030 con 0x7f602c102d40 2024-09-19T07:36:15.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.186+0000 7f6029ffb640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f60180a3240 0x7f60180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:15.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.187+0000 7f6023fff640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f60180a3240 0x7f60180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:15.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.187+0000 7f6029ffb640 1 -- 172.21.15.103:0/3231688574 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f601c0f0090 con 0x7f602c102d40 2024-09-19T07:36:15.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.187+0000 7f6023fff640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f60180a3240 0x7f60180a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f602c0feda0 tx=0x7f6014008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:15.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.191+0000 7f6029ffb640 1 -- 172.21.15.103:0/3231688574 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f601c0bcf00 con 0x7f602c102d40 2024-09-19T07:36:15.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:14 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2791989592' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T07:36:15.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:14 smithi134 ceph-mon[35414]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T07:36:15.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:14 smithi134 ceph-mon[35414]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-19T07:36:15.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.397+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f602c107210 con 0x7f602c102d40 2024-09-19T07:36:15.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.742+0000 7f6029ffb640 1 -- 172.21.15.103:0/3231688574 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f601c0bc800 con 0x7f602c102d40 2024-09-19T07:36:15.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f60180a3240 msgr2=0x7f60180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:15.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f60180a3240 0x7f60180a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f602c0feda0 tx=0x7f6014008040 comp rx=0 tx=0).stop 2024-09-19T07:36:15.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 msgr2=0x7f602c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:15.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c1985b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f601c005ab0 tx=0x7f601c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 shutdown_connections 2024-09-19T07:36:15.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f60180a3240 0x7f60180a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c10c170 0x7f602c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c103660 0x7f602c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 --2- 172.21.15.103:0/3231688574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c102d40 0x7f602c1985b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:15.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 >> 172.21.15.103:0/3231688574 conn(0x7f602c0fc7a0 msgr2=0x7f602c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:15.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 shutdown_connections 2024-09-19T07:36:15.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:15.744+0000 7f6032319640 1 -- 172.21.15.103:0/3231688574 wait complete. 2024-09-19T07:36:16.045 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3231688574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T07:36:16.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:15 smithi103 ceph-mon[31201]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T07:36:16.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:15 smithi103 ceph-mon[31201]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T07:36:16.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3231688574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T07:36:16.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:15 smithi103 ceph-mon[31201]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T07:36:16.046 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:15 smithi103 ceph-mon[31201]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T07:36:16.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3231688574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T07:36:16.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:15 smithi174 ceph-mon[35370]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T07:36:16.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:15 smithi174 ceph-mon[35370]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T07:36:16.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3231688574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T07:36:16.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:15 smithi174 ceph-mon[35370]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T07:36:16.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:15 smithi174 ceph-mon[35370]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T07:36:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3231688574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-19T07:36:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:15 smithi134 ceph-mon[35414]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T07:36:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:15 smithi134 ceph-mon[35414]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T07:36:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3231688574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-19T07:36:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:15 smithi134 ceph-mon[35414]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-19T07:36:16.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:15 smithi134 ceph-mon[35414]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-19T07:36:16.382 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T07:36:16.632 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-19T07:36:17.086 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T07:36:17.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] up:active 2024-09-19T07:36:17.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T07:36:17.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T07:36:17.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T07:36:17.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:16 smithi103 ceph-mon[31201]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T07:36:17.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] up:active 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T07:36:17.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T07:36:17.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T07:36:17.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:16 smithi174 ceph-mon[35370]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T07:36:17.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.168+0000 7f6258ed9640 1 -- 172.21.15.103:0/2547457917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6254106b80 msgr2=0x7f6254106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.168+0000 7f6258ed9640 1 --2- 172.21.15.103:0/2547457917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6254106b80 0x7f6254106f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6244009900 tx=0x7f624402a410 comp rx=0 tx=0).stop 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.168+0000 7f6258ed9640 1 -- 172.21.15.103:0/2547457917 shutdown_connections 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.168+0000 7f6258ed9640 1 --2- 172.21.15.103:0/2547457917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62541005a0 0x7f625410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.168+0000 7f6258ed9640 1 --2- 172.21.15.103:0/2547457917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62540ffc00 0x7f6254100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.168+0000 7f6258ed9640 1 --2- 172.21.15.103:0/2547457917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6254106b80 0x7f6254106f60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 -- 172.21.15.103:0/2547457917 >> 172.21.15.103:0/2547457917 conn(0x7f62540fba90 msgr2=0x7f62540fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 -- 172.21.15.103:0/2547457917 shutdown_connections 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 -- 172.21.15.103:0/2547457917 wait complete. 2024-09-19T07:36:17.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 Processor -- start 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 -- start start 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62540ffc00 0x7f6254198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62541005a0 0x7f6254198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6254106b80 0x7f625419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6254111fa0 con 0x7f62541005a0 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.169+0000 7f6258ed9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6254111e20 con 0x7f62540ffc00 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6258ed9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6254112120 con 0x7f6254106b80 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62541005a0 0x7f6254198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62541005a0 0x7f6254198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37986/0 (socket says 172.21.15.103:37986) 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 -- 172.21.15.103:0/3017445924 learned_addr learned my addr 172.21.15.103:0/3017445924 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6252d9d640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6254106b80 0x7f625419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:17.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f625259c640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62540ffc00 0x7f6254198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 -- 172.21.15.103:0/3017445924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6254106b80 msgr2=0x7f625419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6254106b80 0x7f625419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 -- 172.21.15.103:0/3017445924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62540ffc00 msgr2=0x7f6254198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62540ffc00 0x7f6254198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 -- 172.21.15.103:0/3017445924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625419d3f0 con 0x7f62541005a0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6251d9b640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62541005a0 0x7f6254198980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f624800b4a0 tx=0x7f624800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f62437fe640 1 -- 172.21.15.103:0/3017445924 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248014070 con 0x7f62541005a0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625419d6e0 con 0x7f62541005a0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f62437fe640 1 -- 172.21.15.103:0/3017445924 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6248010af0 con 0x7f62541005a0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f62437fe640 1 -- 172.21.15.103:0/3017445924 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248017d10 con 0x7f62541005a0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.170+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541a4fc0 con 0x7f62541005a0 2024-09-19T07:36:17.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.171+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6254100060 con 0x7f62541005a0 2024-09-19T07:36:17.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.172+0000 7f62437fe640 1 -- 172.21.15.103:0/3017445924 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6248017070 con 0x7f62541005a0 2024-09-19T07:36:17.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.173+0000 7f62437fe640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f623c0a3240 0x7f623c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:17.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.173+0000 7f625259c640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f623c0a3240 0x7f623c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:17.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.173+0000 7f62437fe640 1 -- 172.21.15.103:0/3017445924 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8240+0+0 (secure 0 0 0) 0x7f62480cece0 con 0x7f62541005a0 2024-09-19T07:36:17.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.173+0000 7f625259c640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f623c0a3240 0x7f623c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6244005ab0 tx=0x7f6244005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:17.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.177+0000 7f62437fe640 1 -- 172.21.15.103:0/3017445924 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f624809ba00 con 0x7f62541005a0 2024-09-19T07:36:17.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] up:active 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T07:36:17.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-19T07:36:17.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-19T07:36:17.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:16 smithi134 ceph-mon[35414]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-19T07:36:17.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.378+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f62541a5350 con 0x7f62541005a0 2024-09-19T07:36:17.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.751+0000 7f62437fe640 1 -- 172.21.15.103:0/3017445924 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f624809b300 con 0x7f62541005a0 2024-09-19T07:36:17.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f623c0a3240 msgr2=0x7f623c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:17.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f623c0a3240 0x7f623c0a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6244005ab0 tx=0x7f6244005a20 comp rx=0 tx=0).stop 2024-09-19T07:36:17.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62541005a0 msgr2=0x7f6254198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:17.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62541005a0 0x7f6254198980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f624800b4a0 tx=0x7f624800b960 comp rx=0 tx=0).stop 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 shutdown_connections 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f623c0a3240 0x7f623c0a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6254106b80 0x7f625419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62541005a0 0x7f6254198980 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 --2- 172.21.15.103:0/3017445924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62540ffc00 0x7f6254198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.753+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 >> 172.21.15.103:0/3017445924 conn(0x7f62540fba90 msgr2=0x7f62540fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.754+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 shutdown_connections 2024-09-19T07:36:17.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:17.754+0000 7f6258ed9640 1 -- 172.21.15.103:0/3017445924 wait complete. 2024-09-19T07:36:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T07:36:18.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3017445924' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3017445924' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] up:active 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T07:36:18.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T07:36:18.083 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:17 smithi103 ceph-mon[31201]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-19T07:36:18.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3017445924' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3017445924' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T07:36:18.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] up:active 2024-09-19T07:36:18.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T07:36:18.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T07:36:18.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:17 smithi174 ceph-mon[35370]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-19T07:36:18.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3017445924' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3017445924' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T07:36:18.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] up:active 2024-09-19T07:36:18.334 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-19T07:36:18.334 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-19T07:36:18.334 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:17 smithi134 ceph-mon[35414]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-19T07:36:18.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T07:36:18.671 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:19.070 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:18 smithi103 ceph-mon[31201]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T07:36:19.070 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:18 smithi103 ceph-mon[31201]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T07:36:19.070 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:18 smithi103 ceph-mon[31201]: pgmap v285: 97 pgs: 4 peering, 93 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 7 op/s 2024-09-19T07:36:19.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:18 smithi174 ceph-mon[35370]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T07:36:19.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:18 smithi174 ceph-mon[35370]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T07:36:19.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:18 smithi174 ceph-mon[35370]: pgmap v285: 97 pgs: 4 peering, 93 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 7 op/s 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 -- 172.21.15.103:0/3360313984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01082e0 msgr2=0x7f8af010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 --2- 172.21.15.103:0/3360313984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01082e0 0x7f8af010ec50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009930 tx=0x7f8ae802a440 comp rx=0 tx=0).stop 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 -- 172.21.15.103:0/3360313984 shutdown_connections 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 --2- 172.21.15.103:0/3360313984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af01082e0 0x7f8af010ec50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 --2- 172.21.15.103:0/3360313984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8af0103760 0x7f8af0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 --2- 172.21.15.103:0/3360313984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af0102db0 0x7f8af0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 -- 172.21.15.103:0/3360313984 >> 172.21.15.103:0/3360313984 conn(0x7f8af00fc7d0 msgr2=0x7f8af00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 -- 172.21.15.103:0/3360313984 shutdown_connections 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 -- 172.21.15.103:0/3360313984 wait complete. 2024-09-19T07:36:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 Processor -- start 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.153+0000 7f8af56e0640 1 -- start start 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8af56e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102db0 0x7f8af0100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8af56e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af0103760 0x7f8af0100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8af56e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8af01082e0 0x7f8af0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8af56e0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8af01120f0 con 0x7f8af0102db0 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8af56e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0111f70 con 0x7f8af0103760 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8af56e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0112270 con 0x7f8af01082e0 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102db0 0x7f8af0100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102db0 0x7f8af0100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37244/0 (socket says 172.21.15.103:37244) 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 -- 172.21.15.103:0/1003112319 learned_addr learned my addr 172.21.15.103:0/1003112319 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af0103760 0x7f8af0100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aef7fe640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8af01082e0 0x7f8af0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 -- 172.21.15.103:0/1003112319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8af01082e0 msgr2=0x7f8af0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8af01082e0 0x7f8af0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 -- 172.21.15.103:0/1003112319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af0103760 msgr2=0x7f8af0100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af0103760 0x7f8af0100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 -- 172.21.15.103:0/1003112319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af01a6a50 con 0x7f8af0102db0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8aeeffd640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102db0 0x7f8af0100a10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000cce0 tx=0x7f8ae000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.154+0000 7f8ad7fff640 1 -- 172.21.15.103:0/1003112319 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0004b20 con 0x7f8af0102db0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.155+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af01a6d40 con 0x7f8af0102db0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.155+0000 7f8ad7fff640 1 -- 172.21.15.103:0/1003112319 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0004cc0 con 0x7f8af0102db0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.155+0000 7f8ad7fff640 1 -- 172.21.15.103:0/1003112319 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000f6b0 con 0x7f8af0102db0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.155+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af01a7220 con 0x7f8af0102db0 2024-09-19T07:36:19.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.156+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af0103190 con 0x7f8af0102db0 2024-09-19T07:36:19.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.157+0000 7f8ad7fff640 1 -- 172.21.15.103:0/1003112319 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8ae0005000 con 0x7f8af0102db0 2024-09-19T07:36:19.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.157+0000 7f8ad7fff640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8ad80a3240 0x7f8ad80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:19.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.157+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8ad80a3240 0x7f8ad80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:19.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.157+0000 7f8ad7fff640 1 -- 172.21.15.103:0/1003112319 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8ae00cfa60 con 0x7f8af0102db0 2024-09-19T07:36:19.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.158+0000 7f8aee7fc640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8ad80a3240 0x7f8ad80a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8af01024a0 tx=0x7f8ae4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:19.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.161+0000 7f8ad7fff640 1 -- 172.21.15.103:0/1003112319 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8ae0014030 con 0x7f8af0102db0 2024-09-19T07:36:19.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:18 smithi134 ceph-mon[35414]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T07:36:19.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:18 smithi134 ceph-mon[35414]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-19T07:36:19.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:18 smithi134 ceph-mon[35414]: pgmap v285: 97 pgs: 4 peering, 93 active+clean; 579 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s wr, 7 op/s 2024-09-19T07:36:19.330 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: 2024-09-19T07:36:19.139+0000 7f4353afb640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T07:36:19.330 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: 2024-09-19T07:36:19.139+0000 7f4353afb640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T07:36:19.330 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: 2024-09-19T07:36:19.139+0000 7f4353afb640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-19T07:36:19.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.366+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8af002d7c0 con 0x7f8af0102db0 2024-09-19T07:36:19.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.901+0000 7f8ad7fff640 1 -- 172.21.15.103:0/1003112319 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f8ae009c780 con 0x7f8af0102db0 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8ad80a3240 msgr2=0x7f8ad80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8ad80a3240 0x7f8ad80a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8af01024a0 tx=0x7f8ae4009110 comp rx=0 tx=0).stop 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102db0 msgr2=0x7f8af0100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102db0 0x7f8af0100a10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000cce0 tx=0x7f8ae000a620 comp rx=0 tx=0).stop 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 shutdown_connections 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8ad80a3240 0x7f8ad80a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8af01082e0 0x7f8af0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8af0103760 0x7f8af0100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 --2- 172.21.15.103:0/1003112319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8af0102db0 0x7f8af0100a10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:19.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 >> 172.21.15.103:0/1003112319 conn(0x7f8af00fc7d0 msgr2=0x7f8af0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:19.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 shutdown_connections 2024-09-19T07:36:19.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:19.903+0000 7f8af56e0640 1 -- 172.21.15.103:0/1003112319 wait complete. 2024-09-19T07:36:20.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] up:active 2024-09-19T07:36:20.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T07:36:20.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T07:36:20.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: Cluster is now healthy 2024-09-19T07:36:20.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-19T07:36:20.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T07:36:20.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T07:36:20.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:19 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1003112319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:36:20.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] up:active 2024-09-19T07:36:20.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T07:36:20.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T07:36:20.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: Cluster is now healthy 2024-09-19T07:36:20.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-19T07:36:20.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T07:36:20.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T07:36:20.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:19 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1003112319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:36:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] up:active 2024-09-19T07:36:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-19T07:36:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T07:36:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: Cluster is now healthy 2024-09-19T07:36:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-19T07:36:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-19T07:36:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-19T07:36:20.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:19 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1003112319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T07:36:20.564 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-19T07:36:20.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T07:36:20.810 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:21.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:20 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1003112319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T07:36:21.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:20 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] up:active 2024-09-19T07:36:21.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:20 smithi174 ceph-mon[35370]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T07:36:21.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:20 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:36:21.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:20 smithi174 ceph-mon[35370]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 582 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T07:36:21.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:20 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1003112319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T07:36:21.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:20 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] up:active 2024-09-19T07:36:21.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:20 smithi103 ceph-mon[31201]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T07:36:21.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:20 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:36:21.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:20 smithi103 ceph-mon[31201]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 582 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T07:36:21.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 -- 172.21.15.103:0/938057764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 msgr2=0x7fa25810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:21.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 --2- 172.21.15.103:0/938057764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25810eaa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa250009ca0 tx=0x7fa25002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 -- 172.21.15.103:0/938057764 shutdown_connections 2024-09-19T07:36:21.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 --2- 172.21.15.103:0/938057764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25810eaa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 --2- 172.21.15.103:0/938057764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2581027a0 0x7fa2581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 --2- 172.21.15.103:0/938057764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa258101df0 0x7fa2581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 -- 172.21.15.103:0/938057764 >> 172.21.15.103:0/938057764 conn(0x7fa2580fd8b0 msgr2=0x7fa2580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 -- 172.21.15.103:0/938057764 shutdown_connections 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.279+0000 7fa25dee6640 1 -- 172.21.15.103:0/938057764 wait complete. 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 Processor -- start 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 -- start start 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa258101df0 0x7fa2581984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2581027a0 0x7fa258198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa258112020 con 0x7fa258106f20 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa258111ea0 con 0x7fa258101df0 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa25dee6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581121a0 con 0x7fa2581027a0 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37254/0 (socket says 172.21.15.103:37254) 2024-09-19T07:36:21.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 -- 172.21.15.103:0/1145344345 learned_addr learned my addr 172.21.15.103:0/1145344345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa256ffd640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2581027a0 0x7fa258198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa2577fe640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa258101df0 0x7fa2581984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 -- 172.21.15.103:0/1145344345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2581027a0 msgr2=0x7fa258198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2581027a0 0x7fa258198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 -- 172.21.15.103:0/1145344345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa258101df0 msgr2=0x7fa2581984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa258101df0 0x7fa2581984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.280+0000 7fa257fff640 1 -- 172.21.15.103:0/1145344345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25819d4a0 con 0x7fa258106f20 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.281+0000 7fa257fff640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25819cdc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa250031a60 tx=0x7fa250031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.281+0000 7fa254ff9640 1 -- 172.21.15.103:0/1145344345 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250003b70 con 0x7fa258106f20 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.281+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25819d730 con 0x7fa258106f20 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.281+0000 7fa254ff9640 1 -- 172.21.15.103:0/1145344345 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa250003d10 con 0x7fa258106f20 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.281+0000 7fa254ff9640 1 -- 172.21.15.103:0/1145344345 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250004770 con 0x7fa258106f20 2024-09-19T07:36:21.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.281+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a5070 con 0x7fa258106f20 2024-09-19T07:36:21.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.282+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa258102250 con 0x7fa258106f20 2024-09-19T07:36:21.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.283+0000 7fa254ff9640 1 -- 172.21.15.103:0/1145344345 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa25002f790 con 0x7fa258106f20 2024-09-19T07:36:21.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.284+0000 7fa254ff9640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa2400a3240 0x7fa2400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:21.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.284+0000 7fa2577fe640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa2400a3240 0x7fa2400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:21.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.284+0000 7fa254ff9640 1 -- 172.21.15.103:0/1145344345 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa2500f0c60 con 0x7fa258106f20 2024-09-19T07:36:21.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.284+0000 7fa2577fe640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa2400a3240 0x7fa2400a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa248009890 tx=0x7fa248008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:21.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.288+0000 7fa254ff9640 1 -- 172.21.15.103:0/1145344345 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2500bd980 con 0x7fa258106f20 2024-09-19T07:36:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:20 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1003112319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T07:36:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:20 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] up:active 2024-09-19T07:36:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:20 smithi134 ceph-mon[35414]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-19T07:36:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:20 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:36:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:20 smithi134 ceph-mon[35414]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 582 KiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T07:36:21.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.480+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa258062630 con 0x7fa2400a3240 2024-09-19T07:36:21.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.510+0000 7fa254ff9640 1 -- 172.21.15.103:0/1145344345 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa258062630 con 0x7fa2400a3240 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa2400a3240 msgr2=0x7fa2400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa2400a3240 0x7fa2400a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa248009890 tx=0x7fa248008040 comp rx=0 tx=0).stop 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 msgr2=0x7fa25819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25819cdc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa250031a60 tx=0x7fa250031a90 comp rx=0 tx=0).stop 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 shutdown_connections 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa2400a3240 0x7fa2400a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa258106f20 0x7fa25819cdc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2581027a0 0x7fa258198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 --2- 172.21.15.103:0/1145344345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa258101df0 0x7fa2581984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:21.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 >> 172.21.15.103:0/1145344345 conn(0x7fa2580fd8b0 msgr2=0x7fa2580fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:21.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 shutdown_connections 2024-09-19T07:36:21.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:21.513+0000 7fa25dee6640 1 -- 172.21.15.103:0/1145344345 wait complete. 2024-09-19T07:36:21.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:36:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:36:21.485+0000 7f62af5c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:36:21.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:36:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:36:21.485+0000 7f62af5c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:36:21.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:36:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:36:21.485+0000 7f62af5c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:36:21.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:36:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:36:21.485+0000 7f62af5c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:36:21.793 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:36:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:36:21.485+0000 7f62af5c1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:36:22.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T07:36:22.362 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:22.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 -- 172.21.15.103:0/676767242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 msgr2=0x7fe5d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:22.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/676767242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d010da40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009ca0 tx=0x7fe5c802a950 comp rx=0 tx=0).stop 2024-09-19T07:36:22.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 -- 172.21.15.103:0/676767242 shutdown_connections 2024-09-19T07:36:22.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/676767242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d010da40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:22.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/676767242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5d0102570 0x7fe5d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/676767242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5d0100e20 0x7fe5d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 -- 172.21.15.103:0/676767242 >> 172.21.15.103:0/676767242 conn(0x7fe5d00fc7d0 msgr2=0x7fe5d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 -- 172.21.15.103:0/676767242 shutdown_connections 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.841+0000 7fe5d7e88640 1 -- 172.21.15.103:0/676767242 wait complete. 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 Processor -- start 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 -- start start 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5d0100e20 0x7fe5d00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5d0102570 0x7fe5d00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0110fc0 con 0x7fe5d0102ef0 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0110e40 con 0x7fe5d0102570 2024-09-19T07:36:22.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d7e88640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0111140 con 0x7fe5d0100e20 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d00fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37292/0 (socket says 172.21.15.103:37292) 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 -- 172.21.15.103:0/2404290134 learned_addr learned my addr 172.21.15.103:0/2404290134 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d5423640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5d0102570 0x7fe5d00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d5c24640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5d0100e20 0x7fe5d00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 -- 172.21.15.103:0/2404290134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5d0100e20 msgr2=0x7fe5d00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5d0100e20 0x7fe5d00fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 -- 172.21.15.103:0/2404290134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5d0102570 msgr2=0x7fe5d00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5d0102570 0x7fe5d00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 -- 172.21.15.103:0/2404290134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d01a69e0 con 0x7fe5d0102ef0 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d5423640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5d0102570 0x7fe5d00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.842+0000 7fe5d6425640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d00fd700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006010 tx=0x7fe5c802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:22.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.843+0000 7fe5c6ffd640 1 -- 172.21.15.103:0/2404290134 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8034040 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.843+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d01a6c70 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.843+0000 7fe5c6ffd640 1 -- 172.21.15.103:0/2404290134 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c8003e00 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.843+0000 7fe5c6ffd640 1 -- 172.21.15.103:0/2404290134 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8045480 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.843+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d01a71a0 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.844+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d0101200 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.845+0000 7fe5c6ffd640 1 -- 172.21.15.103:0/2404290134 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fe5c8037020 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.846+0000 7fe5c6ffd640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5bc0a3240 0x7fe5bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.846+0000 7fe5d5c24640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5bc0a3240 0x7fe5bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.846+0000 7fe5c6ffd640 1 -- 172.21.15.103:0/2404290134 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe5c80f0b10 con 0x7fe5d0102ef0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.846+0000 7fe5d5c24640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5bc0a3240 0x7fe5bc0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0058c0 tx=0x7fe5cc005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:22.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:22.850+0000 7fe5c6ffd640 1 -- 172.21.15.103:0/2404290134 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5c80bd830 con 0x7fe5d0102ef0 2024-09-19T07:36:23.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.041+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe5d010a110 con 0x7fe5bc0a3240 2024-09-19T07:36:23.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.069+0000 7fe5c6ffd640 1 -- 172.21.15.103:0/2404290134 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5cc00d9c0 con 0x7fe5bc0a3240 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5bc0a3240 msgr2=0x7fe5bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5bc0a3240 0x7fe5bc0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0058c0 tx=0x7fe5cc005c60 comp rx=0 tx=0).stop 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 msgr2=0x7fe5d00fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d00fd700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8006010 tx=0x7fe5c802f6f0 comp rx=0 tx=0).stop 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 shutdown_connections 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5bc0a3240 0x7fe5bc0a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5d0102ef0 0x7fe5d00fd700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5d0102570 0x7fe5d00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:23.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 --2- 172.21.15.103:0/2404290134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5d0100e20 0x7fe5d00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:23.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 >> 172.21.15.103:0/2404290134 conn(0x7fe5d00fc7d0 msgr2=0x7fe5d006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:23.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 shutdown_connections 2024-09-19T07:36:23.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:23.071+0000 7fe5d7e88640 1 -- 172.21.15.103:0/2404290134 wait complete. 2024-09-19T07:36:23.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:23 smithi174 ceph-mon[35370]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:23.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:23 smithi174 ceph-mon[35370]: pgmap v287: 97 pgs: 4 peering, 93 active+clean; 584 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.0 KiB/s wr, 19 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T07:36:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:23 smithi103 ceph-mon[31201]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:23 smithi103 ceph-mon[31201]: pgmap v287: 97 pgs: 4 peering, 93 active+clean; 584 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.0 KiB/s wr, 19 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T07:36:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:23 smithi134 ceph-mon[35414]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:23 smithi134 ceph-mon[35414]: pgmap v287: 97 pgs: 4 peering, 93 active+clean; 584 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.0 KiB/s wr, 19 op/s; 13 B/s, 1 objects/s recovering 2024-09-19T07:36:23.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T07:36:24.000 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:24 smithi174 ceph-mon[35370]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:24.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:24 smithi174 ceph-mon[35370]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T07:36:24.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:24 smithi103 ceph-mon[31201]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:24.427 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:24 smithi103 ceph-mon[31201]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 -- 172.21.15.103:0/2506253915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102570 msgr2=0x7fe4501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 --2- 172.21.15.103:0/2506253915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102570 0x7fe4501029b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe440009900 tx=0x7fe44002a480 comp rx=0 tx=0).stop 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 -- 172.21.15.103:0/2506253915 shutdown_connections 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 --2- 172.21.15.103:0/2506253915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450102ef0 0x7fe45010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 --2- 172.21.15.103:0/2506253915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102570 0x7fe4501029b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 --2- 172.21.15.103:0/2506253915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe450100e20 0x7fe450101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 -- 172.21.15.103:0/2506253915 >> 172.21.15.103:0/2506253915 conn(0x7fe4500fc7d0 msgr2=0x7fe4500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:24.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 -- 172.21.15.103:0/2506253915 shutdown_connections 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.426+0000 7fe4580ce640 1 -- 172.21.15.103:0/2506253915 wait complete. 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 Processor -- start 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 -- start start 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450100e20 0x7fe450199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe450102570 0x7fe4501998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102ef0 0x7fe4501a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe450110f50 con 0x7fe450102ef0 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe450110dd0 con 0x7fe450100e20 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe4580ce640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4501110d0 con 0x7fe450102570 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102ef0 0x7fe4501a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102ef0 0x7fe4501a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37320/0 (socket says 172.21.15.103:37320) 2024-09-19T07:36:24.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 -- 172.21.15.103:0/1314385810 learned_addr learned my addr 172.21.15.103:0/1314385810 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe455669640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe450102570 0x7fe4501998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe455e6a640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450100e20 0x7fe450199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 -- 172.21.15.103:0/1314385810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe450102570 msgr2=0x7fe4501998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe450102570 0x7fe4501998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 -- 172.21.15.103:0/1314385810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450100e20 msgr2=0x7fe450199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450100e20 0x7fe450199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.427+0000 7fe45666b640 1 -- 172.21.15.103:0/1314385810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4501a4950 con 0x7fe450102ef0 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.428+0000 7fe45666b640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102ef0 0x7fe4501a4410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe44800cad0 tx=0x7fe44800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.428+0000 7fe446ffd640 1 -- 172.21.15.103:0/1314385810 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44800ae20 con 0x7fe450102ef0 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.428+0000 7fe446ffd640 1 -- 172.21.15.103:0/1314385810 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448005020 con 0x7fe450102ef0 2024-09-19T07:36:24.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.428+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501a4b20 con 0x7fe450102ef0 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.428+0000 7fe446ffd640 1 -- 172.21.15.103:0/1314385810 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44800f6f0 con 0x7fe450102ef0 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.428+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501a5030 con 0x7fe450102ef0 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.429+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe450101200 con 0x7fe450102ef0 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.430+0000 7fe446ffd640 1 -- 172.21.15.103:0/1314385810 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe4480051d0 con 0x7fe450102ef0 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.430+0000 7fe446ffd640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe43c0a3350 0x7fe43c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.431+0000 7fe455e6a640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe43c0a3350 0x7fe43c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.431+0000 7fe446ffd640 1 -- 172.21.15.103:0/1314385810 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe4480cf990 con 0x7fe450102ef0 2024-09-19T07:36:24.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.431+0000 7fe455e6a640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe43c0a3350 0x7fe43c0a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00bbe0 tx=0x7fe44c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:24.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.435+0000 7fe446ffd640 1 -- 172.21.15.103:0/1314385810 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe448014030 con 0x7fe450102ef0 2024-09-19T07:36:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:24 smithi134 ceph-mon[35414]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:24 smithi134 ceph-mon[35414]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T07:36:24.579 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:36:24 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: 2024-09-19T07:36:24.140+0000 7f4353afb640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-19T07:36:24.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.630+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe450062630 con 0x7fe43c0a3350 2024-09-19T07:36:24.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.635+0000 7fe446ffd640 1 -- 172.21.15.103:0/1314385810 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe450062630 con 0x7fe43c0a3350 2024-09-19T07:36:24.636 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_0/44d50894-29f1-4db1-ad86-79a0f4a403dc 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe43c0a3350 msgr2=0x7fe43c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe43c0a3350 0x7fe43c0a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00bbe0 tx=0x7fe44c00b390 comp rx=0 tx=0).stop 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102ef0 msgr2=0x7fe4501a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102ef0 0x7fe4501a4410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe44800cad0 tx=0x7fe44800cf90 comp rx=0 tx=0).stop 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 shutdown_connections 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe43c0a3350 0x7fe43c0a5810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe450102ef0 0x7fe4501a4410 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe450102570 0x7fe4501998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 --2- 172.21.15.103:0/1314385810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450100e20 0x7fe450199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 >> 172.21.15.103:0/1314385810 conn(0x7fe4500fc7d0 msgr2=0x7fe4500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 shutdown_connections 2024-09-19T07:36:24.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:24.637+0000 7fe4580ce640 1 -- 172.21.15.103:0/1314385810 wait complete. 2024-09-19T07:36:25.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:25 smithi103 ceph-mon[31201]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.0 KiB/s wr, 27 op/s; 86 B/s, 1 objects/s recovering 2024-09-19T07:36:25.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T07:36:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:25 smithi174 ceph-mon[35370]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.0 KiB/s wr, 27 op/s; 86 B/s, 1 objects/s recovering 2024-09-19T07:36:25.481 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:25 smithi134 ceph-mon[35414]: pgmap v288: 97 pgs: 97 active+clean; 605 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.0 KiB/s wr, 27 op/s; 86 B/s, 1 objects/s recovering 2024-09-19T07:36:25.636 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:26.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 -- 172.21.15.103:0/1755739048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 msgr2=0x7ff05c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:26.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 --2- 172.21.15.103:0/1755739048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c103170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff058009930 tx=0x7ff05802a420 comp rx=0 tx=0).stop 2024-09-19T07:36:26.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 -- 172.21.15.103:0/1755739048 shutdown_connections 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 --2- 172.21.15.103:0/1755739048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c108290 0x7ff05c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 --2- 172.21.15.103:0/1755739048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff05c103740 0x7ff05c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 --2- 172.21.15.103:0/1755739048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c103170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 -- 172.21.15.103:0/1755739048 >> 172.21.15.103:0/1755739048 conn(0x7ff05c0fc7d0 msgr2=0x7ff05c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 -- 172.21.15.103:0/1755739048 shutdown_connections 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.168+0000 7ff0645e6640 1 -- 172.21.15.103:0/1755739048 wait complete. 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 Processor -- start 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 -- start start 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c103740 0x7ff05c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff05c108290 0x7ff05c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1120d0 con 0x7ff05c102d90 2024-09-19T07:36:26.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c111f50 con 0x7ff05c103740 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff0645e6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c112250 con 0x7ff05c108290 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37332/0 (socket says 172.21.15.103:37332) 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 -- 172.21.15.103:0/3314061760 learned_addr learned my addr 172.21.15.103:0/3314061760 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff061b81640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c103740 0x7ff05c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062b83640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff05c108290 0x7ff05c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 -- 172.21.15.103:0/3314061760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff05c108290 msgr2=0x7ff05c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff05c108290 0x7ff05c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 -- 172.21.15.103:0/3314061760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c103740 msgr2=0x7ff05c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c103740 0x7ff05c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.169+0000 7ff062382640 1 -- 172.21.15.103:0/3314061760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c1a6a70 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.170+0000 7ff062382640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c1009f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff05802a940 tx=0x7ff058003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.170+0000 7ff0537fe640 1 -- 172.21.15.103:0/3314061760 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05803f070 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.170+0000 7ff0537fe640 1 -- 172.21.15.103:0/3314061760 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff058031c90 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.170+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c1a6d00 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.170+0000 7ff0537fe640 1 -- 172.21.15.103:0/3314061760 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058035030 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.170+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1a71d0 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.171+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05c06a7f0 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.172+0000 7ff0537fe640 1 -- 172.21.15.103:0/3314061760 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff058042070 con 0x7ff05c102d90 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.172+0000 7ff0537fe640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff0480a3350 0x7ff0480a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.172+0000 7ff061b81640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff0480a3350 0x7ff0480a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:26.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.173+0000 7ff0537fe640 1 -- 172.21.15.103:0/3314061760 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7ff0580f0520 con 0x7ff05c102d90 2024-09-19T07:36:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.173+0000 7ff061b81640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff0480a3350 0x7ff0480a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff05c102480 tx=0x7ff04c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.176+0000 7ff0537fe640 1 -- 172.21.15.103:0/3314061760 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff0580bcba0 con 0x7ff05c102d90 2024-09-19T07:36:26.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.365+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff05c10a310 con 0x7ff0480a3350 2024-09-19T07:36:26.367 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:26 smithi103 ceph-mon[31201]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:26.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.385+0000 7ff0537fe640 1 -- 172.21.15.103:0/3314061760 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff04c003890 con 0x7ff0480a3350 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff0480a3350 msgr2=0x7ff0480a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff0480a3350 0x7ff0480a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff05c102480 tx=0x7ff04c00b390 comp rx=0 tx=0).stop 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 msgr2=0x7ff05c1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c1009f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff05802a940 tx=0x7ff058003b90 comp rx=0 tx=0).stop 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 shutdown_connections 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff0480a3350 0x7ff0480a5810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff05c108290 0x7ff05c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff05c103740 0x7ff05c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 --2- 172.21.15.103:0/3314061760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff05c102d90 0x7ff05c1009f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:26.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 >> 172.21.15.103:0/3314061760 conn(0x7ff05c0fc7d0 msgr2=0x7ff05c106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:26.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 shutdown_connections 2024-09-19T07:36:26.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:26.387+0000 7ff0645e6640 1 -- 172.21.15.103:0/3314061760 wait complete. 2024-09-19T07:36:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:26 smithi174 ceph-mon[35370]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:26 smithi134 ceph-mon[35414]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:27.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T07:36:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:27 smithi174 ceph-mon[35370]: pgmap v289: 97 pgs: 97 active+clean; 605 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 KiB/s wr, 25 op/s; 75 B/s, 1 objects/s recovering 2024-09-19T07:36:27.452 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:27 smithi103 ceph-mon[31201]: pgmap v289: 97 pgs: 97 active+clean; 605 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 KiB/s wr, 25 op/s; 75 B/s, 1 objects/s recovering 2024-09-19T07:36:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:27 smithi134 ceph-mon[35414]: pgmap v289: 97 pgs: 97 active+clean; 605 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 KiB/s wr, 25 op/s; 75 B/s, 1 objects/s recovering 2024-09-19T07:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 -- 172.21.15.103:0/2551134312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93781012e0 msgr2=0x7f937810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 --2- 172.21.15.103:0/2551134312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93781012e0 0x7f937810dfa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9368009900 tx=0x7f936802a410 comp rx=0 tx=0).stop 2024-09-19T07:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 -- 172.21.15.103:0/2551134312 shutdown_connections 2024-09-19T07:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 --2- 172.21.15.103:0/2551134312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f937810e4e0 0x7f93781108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 --2- 172.21.15.103:0/2551134312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93781012e0 0x7f937810dfa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 --2- 172.21.15.103:0/2551134312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9378100930 0x7f9378100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 -- 172.21.15.103:0/2551134312 >> 172.21.15.103:0/2551134312 conn(0x7f93780fc7a0 msgr2=0x7f93780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 -- 172.21.15.103:0/2551134312 shutdown_connections 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.003+0000 7f9380587640 1 -- 172.21.15.103:0/2551134312 wait complete. 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 Processor -- start 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 -- start start 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9378100930 0x7f937819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93781012e0 0x7f937819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f937810e4e0 0x7f937819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9378114180 con 0x7f937810e4e0 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9378114000 con 0x7f9378100930 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f9380587640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9378114300 con 0x7f93781012e0 2024-09-19T07:36:28.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f937eb24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f937810e4e0 0x7f937819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f937eb24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f937810e4e0 0x7f937819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37354/0 (socket says 172.21.15.103:37354) 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f937eb24640 1 -- 172.21.15.103:0/2728063978 learned_addr learned my addr 172.21.15.103:0/2728063978 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f937e323640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9378100930 0x7f937819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.004+0000 7f937db22640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93781012e0 0x7f937819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f937eb24640 1 -- 172.21.15.103:0/2728063978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93781012e0 msgr2=0x7f937819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f937eb24640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93781012e0 0x7f937819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f937eb24640 1 -- 172.21.15.103:0/2728063978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9378100930 msgr2=0x7f937819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f937eb24640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9378100930 0x7f937819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f937eb24640 1 -- 172.21.15.103:0/2728063978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937819f670 con 0x7f937810e4e0 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f937eb24640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f937810e4e0 0x7f937819ef90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f937000eb40 tx=0x7f937000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f936f7fe640 1 -- 172.21.15.103:0/2728063978 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370004b20 con 0x7f937810e4e0 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f936f7fe640 1 -- 172.21.15.103:0/2728063978 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9370004cc0 con 0x7f937810e4e0 2024-09-19T07:36:28.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937819f960 con 0x7f937810e4e0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f936f7fe640 1 -- 172.21.15.103:0/2728063978 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370010850 con 0x7f937810e4e0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.005+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93781a7240 con 0x7f937810e4e0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.006+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9378100d10 con 0x7f937810e4e0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.007+0000 7f936f7fe640 1 -- 172.21.15.103:0/2728063978 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9370005020 con 0x7f937810e4e0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.008+0000 7f936f7fe640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f93640a3350 0x7f93640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.008+0000 7f937e323640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f93640a3350 0x7f93640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.008+0000 7f936f7fe640 1 -- 172.21.15.103:0/2728063978 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f93700d0230 con 0x7f937810e4e0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.008+0000 7f937e323640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f93640a3350 0x7f93640a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9374009bd0 tx=0x7f9374005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:28.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.012+0000 7f936f7fe640 1 -- 172.21.15.103:0/2728063978 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f937009cdf0 con 0x7f937810e4e0 2024-09-19T07:36:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.205+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f93781096d0 con 0x7f93640a3350 2024-09-19T07:36:28.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:28 smithi103 ceph-mon[31201]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:28.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.209+0000 7f936f7fe640 1 -- 172.21.15.103:0/2728063978 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f93781096d0 con 0x7f93640a3350 2024-09-19T07:36:28.210 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_1/7cc5c1f1-01fb-4213-a2c6-8ec0edefbf83 2024-09-19T07:36:28.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f93640a3350 msgr2=0x7f93640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:28.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f93640a3350 0x7f93640a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9374009bd0 tx=0x7f9374005f90 comp rx=0 tx=0).stop 2024-09-19T07:36:28.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f937810e4e0 msgr2=0x7f937819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:28.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f937810e4e0 0x7f937819ef90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f937000eb40 tx=0x7f937000c2d0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 shutdown_connections 2024-09-19T07:36:28.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f93640a3350 0x7f93640a5810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f937810e4e0 0x7f937819ef90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93781012e0 0x7f937819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 --2- 172.21.15.103:0/2728063978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9378100930 0x7f937819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:28.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 >> 172.21.15.103:0/2728063978 conn(0x7f93780fc7a0 msgr2=0x7f93780feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:28.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 shutdown_connections 2024-09-19T07:36:28.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:28.211+0000 7f9380587640 1 -- 172.21.15.103:0/2728063978 wait complete. 2024-09-19T07:36:28.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:28 smithi134 ceph-mon[35414]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:28 smithi174 ceph-mon[35370]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:28.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:36:29.217 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:29.219 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:29 smithi103 ceph-mon[31201]: pgmap v290: 97 pgs: 97 active+clean; 606 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.4 KiB/s wr, 23 op/s; 68 B/s, 1 objects/s recovering 2024-09-19T07:36:29.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:29 smithi103 ceph-mon[31201]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:29 smithi134 ceph-mon[35414]: pgmap v290: 97 pgs: 97 active+clean; 606 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.4 KiB/s wr, 23 op/s; 68 B/s, 1 objects/s recovering 2024-09-19T07:36:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:29 smithi134 ceph-mon[35414]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:29 smithi174 ceph-mon[35370]: pgmap v290: 97 pgs: 97 active+clean; 606 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.4 KiB/s wr, 23 op/s; 68 B/s, 1 objects/s recovering 2024-09-19T07:36:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:29 smithi174 ceph-mon[35370]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:36:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3160866754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941082a0 msgr2=0x7fec9410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3160866754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941082a0 0x7fec9410ec40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009930 tx=0x7fec8c02a440 comp rx=0 tx=0).stop 2024-09-19T07:36:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3160866754 shutdown_connections 2024-09-19T07:36:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3160866754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941082a0 0x7fec9410ec40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3160866754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec941036e0 0x7fec94107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3160866754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec94102d30 0x7fec94103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3160866754 >> 172.21.15.103:0/3160866754 conn(0x7fec940fc7d0 msgr2=0x7fec940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:29.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3160866754 shutdown_connections 2024-09-19T07:36:29.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.734+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3160866754 wait complete. 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 Processor -- start 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 -- start start 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec94102d30 0x7fec941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941036e0 0x7fec94198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec941082a0 0x7fec9419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec94112160 con 0x7fec941036e0 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec94111fe0 con 0x7fec941082a0 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec9ba4f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec941122e0 con 0x7fec94102d30 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941036e0 0x7fec94198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941036e0 0x7fec94198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43806/0 (socket says 172.21.15.103:43806) 2024-09-19T07:36:29.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 -- 172.21.15.103:0/3731594295 learned_addr learned my addr 172.21.15.103:0/3731594295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec997eb640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec94102d30 0x7fec941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec99fec640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec941082a0 0x7fec9419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 -- 172.21.15.103:0/3731594295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec94102d30 msgr2=0x7fec941986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec94102d30 0x7fec941986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 -- 172.21.15.103:0/3731594295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec941082a0 msgr2=0x7fec9419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec941082a0 0x7fec9419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec98fea640 1 -- 172.21.15.103:0/3731594295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9419d690 con 0x7fec941036e0 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.735+0000 7fec99fec640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec941082a0 0x7fec9419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.736+0000 7fec98fea640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941036e0 0x7fec94198c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fec88006fd0 tx=0x7fec8800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:29.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.736+0000 7fec927fc640 1 -- 172.21.15.103:0/3731594295 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88011830 con 0x7fec941036e0 2024-09-19T07:36:29.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.736+0000 7fec927fc640 1 -- 172.21.15.103:0/3731594295 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec88016070 con 0x7fec941036e0 2024-09-19T07:36:29.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.736+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9419d980 con 0x7fec941036e0 2024-09-19T07:36:29.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.736+0000 7fec927fc640 1 -- 172.21.15.103:0/3731594295 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8800f2e0 con 0x7fec941036e0 2024-09-19T07:36:29.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.736+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941a5260 con 0x7fec941036e0 2024-09-19T07:36:29.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.737+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec94103190 con 0x7fec941036e0 2024-09-19T07:36:29.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.738+0000 7fec927fc640 1 -- 172.21.15.103:0/3731594295 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fec8801b460 con 0x7fec941036e0 2024-09-19T07:36:29.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.739+0000 7fec927fc640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec7c0a32d0 0x7fec7c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:29.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.739+0000 7fec997eb640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec7c0a32d0 0x7fec7c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:29.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.739+0000 7fec927fc640 1 -- 172.21.15.103:0/3731594295 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fec880cf030 con 0x7fec941036e0 2024-09-19T07:36:29.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.739+0000 7fec997eb640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec7c0a32d0 0x7fec7c0a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fec8400a400 tx=0x7fec8400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:29.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.743+0000 7fec927fc640 1 -- 172.21.15.103:0/3731594295 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fec8809bd00 con 0x7fec941036e0 2024-09-19T07:36:29.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.945+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec9410a2e0 con 0x7fec941036e0 2024-09-19T07:36:29.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.946+0000 7fec927fc640 1 -- 172.21.15.103:0/3731594295 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11991 (secure 0 0 0) 0x7fec8809b600 con 0x7fec941036e0 2024-09-19T07:36:29.947 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:29.947 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T07:36:19:905223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:36:19.905198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:36:29.947 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T07:36:29.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec7c0a32d0 msgr2=0x7fec7c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:29.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec7c0a32d0 0x7fec7c0a5790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fec8400a400 tx=0x7fec8400d040 comp rx=0 tx=0).stop 2024-09-19T07:36:29.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941036e0 msgr2=0x7fec94198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:29.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941036e0 0x7fec94198c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fec88006fd0 tx=0x7fec8800b8e0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 shutdown_connections 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fec7c0a32d0 0x7fec7c0a5790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec941082a0 0x7fec9419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec941036e0 0x7fec94198c20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 --2- 172.21.15.103:0/3731594295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec94102d30 0x7fec941986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.948+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 >> 172.21.15.103:0/3731594295 conn(0x7fec940fc7d0 msgr2=0x7fec94105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.949+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 shutdown_connections 2024-09-19T07:36:29.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:29.949+0000 7fec9ba4f640 1 -- 172.21.15.103:0/3731594295 wait complete. 2024-09-19T07:36:30.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:30 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3731594295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:36:30.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:36:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:30 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3731594295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:36:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:30 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3731594295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:36:30.821 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:31.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:31 smithi103 ceph-mon[31201]: pgmap v291: 97 pgs: 97 active+clean; 624 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 23 op/s; 57 B/s, 1 objects/s recovering 2024-09-19T07:36:31.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 -- 172.21.15.103:0/3319270191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4100a80 msgr2=0x7f8fa4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:31.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/3319270191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4100a80 0x7f8fa4100ee0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009930 tx=0x7f8f8c02a420 comp rx=0 tx=0).stop 2024-09-19T07:36:31.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 -- 172.21.15.103:0/3319270191 shutdown_connections 2024-09-19T07:36:31.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/3319270191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4101420 0x7f8fa410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/3319270191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4100a80 0x7f8fa4100ee0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/3319270191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4106a80 0x7f8fa4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 -- 172.21.15.103:0/3319270191 >> 172.21.15.103:0/3319270191 conn(0x7f8fa40fc7a0 msgr2=0x7f8fa40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 -- 172.21.15.103:0/3319270191 shutdown_connections 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.254+0000 7f8fa8fea640 1 -- 172.21.15.103:0/3319270191 wait complete. 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 Processor -- start 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 -- start start 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4100a80 0x7f8fa41974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101420 0x7f8fa4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4106a80 0x7f8fa419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4110f20 con 0x7f8fa4101420 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4110da0 con 0x7f8fa4100a80 2024-09-19T07:36:31.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa8fea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41110a0 con 0x7f8fa4106a80 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101420 0x7f8fa4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101420 0x7f8fa4197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43816/0 (socket says 172.21.15.103:43816) 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 -- 172.21.15.103:0/1790329682 learned_addr learned my addr 172.21.15.103:0/1790329682 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa259c640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4100a80 0x7f8fa41974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa2d9d640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4106a80 0x7f8fa419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 -- 172.21.15.103:0/1790329682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4106a80 msgr2=0x7f8fa419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4106a80 0x7f8fa419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 -- 172.21.15.103:0/1790329682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4100a80 msgr2=0x7f8fa41974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4100a80 0x7f8fa41974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa1d9b640 1 -- 172.21.15.103:0/1790329682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa419c470 con 0x7f8fa4101420 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa2d9d640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4106a80 0x7f8fa419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.255+0000 7f8fa259c640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4100a80 0x7f8fa41974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:31.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.256+0000 7f8fa1d9b640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101420 0x7f8fa4197a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009a50 tx=0x7f8f8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.256+0000 7f8f9b7fe640 1 -- 172.21.15.103:0/1790329682 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c03f030 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.256+0000 7f8f9b7fe640 1 -- 172.21.15.103:0/1790329682 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f8c003ce0 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.256+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa419c700 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.256+0000 7f8f9b7fe640 1 -- 172.21.15.103:0/1790329682 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c035070 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.256+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa41a4040 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.257+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa4100f60 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.258+0000 7f8f9b7fe640 1 -- 172.21.15.103:0/1790329682 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8f8c042030 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.259+0000 7f8f9b7fe640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8f900a3350 0x7f8f900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.259+0000 7f8fa259c640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8f900a3350 0x7f8f900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.259+0000 7f8f9b7fe640 1 -- 172.21.15.103:0/1790329682 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f8f8c0f00b0 con 0x7f8fa4101420 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.259+0000 7f8fa259c640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8f900a3350 0x7f8f900a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009930 tx=0x7f8f94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:31.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.263+0000 7f8f9b7fe640 1 -- 172.21.15.103:0/1790329682 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8f8c0bcc70 con 0x7f8fa4101420 2024-09-19T07:36:31.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.431+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fa4109040 con 0x7f8fa4101420 2024-09-19T07:36:31.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.432+0000 7f8f9b7fe640 1 -- 172.21.15.103:0/1790329682 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22818 (secure 0 0 0) 0x7f8f8c0dc090 con 0x7f8fa4101420 2024-09-19T07:36:31.433 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:31.433 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:17.882336+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:36:31.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8f900a3350 msgr2=0x7f8f900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:31.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8f900a3350 0x7f8f900a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009930 tx=0x7f8f94008040 comp rx=0 tx=0).stop 2024-09-19T07:36:31.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101420 msgr2=0x7f8fa4197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:31.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101420 0x7f8fa4197a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009a50 tx=0x7f8f8c004270 comp rx=0 tx=0).stop 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 shutdown_connections 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f8f900a3350 0x7f8f900a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fa4106a80 0x7f8fa419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fa4101420 0x7f8fa4197a00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 --2- 172.21.15.103:0/1790329682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fa4100a80 0x7f8fa41974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.434+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 >> 172.21.15.103:0/1790329682 conn(0x7f8fa40fc7a0 msgr2=0x7f8fa40fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.435+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 shutdown_connections 2024-09-19T07:36:31.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:31.435+0000 7f8fa8fea640 1 -- 172.21.15.103:0/1790329682 wait complete. 2024-09-19T07:36:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:31 smithi134 ceph-mon[35414]: pgmap v291: 97 pgs: 97 active+clean; 624 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 23 op/s; 57 B/s, 1 objects/s recovering 2024-09-19T07:36:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:31 smithi174 ceph-mon[35370]: pgmap v291: 97 pgs: 97 active+clean; 624 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 23 op/s; 57 B/s, 1 objects/s recovering 2024-09-19T07:36:32.126 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T07:36:32.126 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:36:32.371 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:32 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1790329682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:32.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:32 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1790329682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:32.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:32 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1790329682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 -- 172.21.15.103:0/1480015102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710106f10 msgr2=0x7f771010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 --2- 172.21.15.103:0/1480015102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710106f10 0x7f771010ea90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7708009ca0 tx=0x7f770802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:36:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 -- 172.21.15.103:0/1480015102 shutdown_connections 2024-09-19T07:36:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 --2- 172.21.15.103:0/1480015102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710106f10 0x7f771010ea90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 --2- 172.21.15.103:0/1480015102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7710102790 0x7f77101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 --2- 172.21.15.103:0/1480015102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7710101de0 0x7f77101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 -- 172.21.15.103:0/1480015102 >> 172.21.15.103:0/1480015102 conn(0x7f77100fd8c0 msgr2=0x7f77100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 -- 172.21.15.103:0/1480015102 shutdown_connections 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.855+0000 7f7716ce6640 1 -- 172.21.15.103:0/1480015102 wait complete. 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 Processor -- start 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 -- start start 2024-09-19T07:36:32.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7710101de0 0x7f77101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710102790 0x7f7710198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7710106f10 0x7f771019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7710111f70 con 0x7f7710102790 2024-09-19T07:36:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7710111df0 con 0x7f7710101de0 2024-09-19T07:36:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7716ce6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77101120f0 con 0x7f7710106f10 2024-09-19T07:36:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7707fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710102790 0x7f7710198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:32.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7707fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710102790 0x7f7710198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43826/0 (socket says 172.21.15.103:43826) 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7707fff640 1 -- 172.21.15.103:0/3626168788 learned_addr learned my addr 172.21.15.103:0/3626168788 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7714a82640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7710101de0 0x7f77101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.856+0000 7f7715283640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7710106f10 0x7f771019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7707fff640 1 -- 172.21.15.103:0/3626168788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7710106f10 msgr2=0x7f771019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7707fff640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7710106f10 0x7f771019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7707fff640 1 -- 172.21.15.103:0/3626168788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7710101de0 msgr2=0x7f77101983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7707fff640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7710101de0 0x7f77101983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:32.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7707fff640 1 -- 172.21.15.103:0/3626168788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771019d3a0 con 0x7f7710102790 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7715283640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7710106f10 0x7f771019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7707fff640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710102790 0x7f7710198930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f770000d770 tx=0x7f770000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7705ffb640 1 -- 172.21.15.103:0/3626168788 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700017070 con 0x7f7710102790 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7705ffb640 1 -- 172.21.15.103:0/3626168788 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7700005500 con 0x7f7710102790 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771019d690 con 0x7f7710102790 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7705ffb640 1 -- 172.21.15.103:0/3626168788 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700004e00 con 0x7f7710102790 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.857+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101a4f70 con 0x7f7710102790 2024-09-19T07:36:32.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.858+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77101021c0 con 0x7f7710102790 2024-09-19T07:36:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.859+0000 7f7705ffb640 1 -- 172.21.15.103:0/3626168788 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7700004fa0 con 0x7f7710102790 2024-09-19T07:36:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.860+0000 7f7705ffb640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f76f80a3350 0x7f76f80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.860+0000 7f7714a82640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f76f80a3350 0x7f76f80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.860+0000 7f7705ffb640 1 -- 172.21.15.103:0/3626168788 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f77000cf440 con 0x7f7710102790 2024-09-19T07:36:32.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.861+0000 7f7714a82640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f76f80a3350 0x7f76f80a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f770c00bbe0 tx=0x7f770c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:32.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:32.864+0000 7f7705ffb640 1 -- 172.21.15.103:0/3626168788 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f770009c000 con 0x7f7710102790 2024-09-19T07:36:33.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.027+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7710062630 con 0x7f76f80a3350 2024-09-19T07:36:33.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.034+0000 7f7705ffb640 1 -- 172.21.15.103:0/3626168788 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258996 (secure 0 0 0) 0x7f7710062630 con 0x7f76f80a3350 2024-09-19T07:36:33.038 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:33.040 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:36:33.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f76f80a3350 msgr2=0x7f76f80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:33.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f76f80a3350 0x7f76f80a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f770c00bbe0 tx=0x7f770c00b390 comp rx=0 tx=0).stop 2024-09-19T07:36:33.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710102790 msgr2=0x7f7710198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:33.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710102790 0x7f7710198930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f770000d770 tx=0x7f770000dc30 comp rx=0 tx=0).stop 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 shutdown_connections 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f76f80a3350 0x7f76f80a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7710106f10 0x7f771019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7710102790 0x7f7710198930 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 --2- 172.21.15.103:0/3626168788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7710101de0 0x7f77101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 >> 172.21.15.103:0/3626168788 conn(0x7f77100fd8c0 msgr2=0x7f77100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 shutdown_connections 2024-09-19T07:36:33.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:33.037+0000 7f7716ce6640 1 -- 172.21.15.103:0/3626168788 wait complete. 2024-09-19T07:36:33.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:33 smithi103 ceph-mon[31201]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 KiB/s wr, 19 op/s; 48 B/s, 0 objects/s recovering 2024-09-19T07:36:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:33 smithi134 ceph-mon[35414]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 KiB/s wr, 19 op/s; 48 B/s, 0 objects/s recovering 2024-09-19T07:36:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:33 smithi174 ceph-mon[35370]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 KiB/s wr, 19 op/s; 48 B/s, 0 objects/s recovering 2024-09-19T07:36:33.764 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-19T07:36:32.085809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639268,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":269,"num_read_kb":237,"num_write":410,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":337,"ondisk_log_size":337,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":343852,"kb_used_data":11236,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597972,"statfs":{"total":1151940427776,"available":1151588323328,"internally_reserved":0,"allocated":11505664,"data_stored":7220232,"data_compressed":177078,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42916,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":133,"num_read_kb":118,"num_write":99,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002874"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893903+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:36:17.893903+0000","last_peered":"2024-09-19T07:36:17.893903+0000","last_clean":"2024-09-19T07:36:17.893903+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:36:17.893903+0000","last_undegraded":"2024-09-19T07:36:17.893903+0000","last_fullsized":"2024-09-19T07:36:17.893903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895834+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:36:17.895834+0000","last_peered":"2024-09-19T07:36:17.895834+0000","last_clean":"2024-09-19T07:36:17.895834+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:36:17.895834+0000","last_undegraded":"2024-09-19T07:36:17.895834+0000","last_fullsized":"2024-09-19T07:36:17.895834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895895+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:36:17.895895+0000","last_peered":"2024-09-19T07:36:17.895895+0000","last_clean":"2024-09-19T07:36:17.895895+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:36:17.895895+0000","last_undegraded":"2024-09-19T07:36:17.895895+0000","last_fullsized":"2024-09-19T07:36:17.895895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896911+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:36:17.896911+0000","last_peered":"2024-09-19T07:36:17.896911+0000","last_clean":"2024-09-19T07:36:17.896911+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:36:17.896911+0000","last_undegraded":"2024-09-19T07:36:17.896911+0000","last_fullsized":"2024-09-19T07:36:17.896911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895313+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:36:17.895313+0000","last_peered":"2024-09-19T07:36:17.895313+0000","last_clean":"2024-09-19T07:36:17.895313+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:36:17.895313+0000","last_undegraded":"2024-09-19T07:36:17.895313+0000","last_fullsized":"2024-09-19T07:36:17.895313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897391+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:36:17.897391+0000","last_peered":"2024-09-19T07:36:17.897391+0000","last_clean":"2024-09-19T07:36:17.897391+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:36:17.897391+0000","last_undegraded":"2024-09-19T07:36:17.897391+0000","last_fullsized":"2024-09-19T07:36:17.897391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893437+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:36:17.893437+0000","last_peered":"2024-09-19T07:36:17.893437+0000","last_clean":"2024-09-19T07:36:17.893437+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:36:17.893437+0000","last_undegraded":"2024-09-19T07:36:17.893437+0000","last_fullsized":"2024-09-19T07:36:17.893437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891423+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:36:17.891423+0000","last_peered":"2024-09-19T07:36:17.891423+0000","last_clean":"2024-09-19T07:36:17.891423+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:36:17.891423+0000","last_undegraded":"2024-09-19T07:36:17.891423+0000","last_fullsized":"2024-09-19T07:36:17.891423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896616+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:36:17.896616+0000","last_peered":"2024-09-19T07:36:17.896616+0000","last_clean":"2024-09-19T07:36:17.896616+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:36:17.896616+0000","last_undegraded":"2024-09-19T07:36:17.896616+0000","last_fullsized":"2024-09-19T07:36:17.896616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891049+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:36:17.891049+0000","last_peered":"2024-09-19T07:36:17.891049+0000","last_clean":"2024-09-19T07:36:17.891049+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:36:17.891049+0000","last_undegraded":"2024-09-19T07:36:17.891049+0000","last_fullsized":"2024-09-19T07:36:17.891049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893904+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:36:17.893904+0000","last_peered":"2024-09-19T07:36:17.893904+0000","last_clean":"2024-09-19T07:36:17.893904+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:36:17.893904+0000","last_undegraded":"2024-09-19T07:36:17.893904+0000","last_fullsized":"2024-09-19T07:36:17.893904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.937191+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:36:17.937191+0000","last_peered":"2024-09-19T07:36:17.937191+0000","last_clean":"2024-09-19T07:36:17.937191+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:36:17.937191+0000","last_undegraded":"2024-09-19T07:36:17.937191+0000","last_fullsized":"2024-09-19T07:36:17.937191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889741+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:36:17.889741+0000","last_peered":"2024-09-19T07:36:17.889741+0000","last_clean":"2024-09-19T07:36:17.889741+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:36:17.889741+0000","last_undegraded":"2024-09-19T07:36:17.889741+0000","last_fullsized":"2024-09-19T07:36:17.889741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895614+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:36:17.895614+0000","last_peered":"2024-09-19T07:36:17.895614+0000","last_clean":"2024-09-19T07:36:17.895614+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:36:17.895614+0000","last_undegraded":"2024-09-19T07:36:17.895614+0000","last_fullsized":"2024-09-19T07:36:17.895614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895550+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:36:17.895550+0000","last_peered":"2024-09-19T07:36:17.895550+0000","last_clean":"2024-09-19T07:36:17.895550+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:36:17.895550+0000","last_undegraded":"2024-09-19T07:36:17.895550+0000","last_fullsized":"2024-09-19T07:36:17.895550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895680+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:36:17.895680+0000","last_peered":"2024-09-19T07:36:17.895680+0000","last_clean":"2024-09-19T07:36:17.895680+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:36:17.895680+0000","last_undegraded":"2024-09-19T07:36:17.895680+0000","last_fullsized":"2024-09-19T07:36:17.895680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893949+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:36:17.893949+0000","last_peered":"2024-09-19T07:36:17.893949+0000","last_clean":"2024-09-19T07:36:17.893949+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:36:17.893949+0000","last_undegraded":"2024-09-19T07:36:17.893949+0000","last_fullsized":"2024-09-19T07:36:17.893949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892102+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:36:17.892102+0000","last_peered":"2024-09-19T07:36:17.892102+0000","last_clean":"2024-09-19T07:36:17.892102+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:36:17.892102+0000","last_undegraded":"2024-09-19T07:36:17.892102+0000","last_fullsized":"2024-09-19T07:36:17.892102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892104+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:36:17.892104+0000","last_peered":"2024-09-19T07:36:17.892104+0000","last_clean":"2024-09-19T07:36:17.892104+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:36:17.892104+0000","last_undegraded":"2024-09-19T07:36:17.892104+0000","last_fullsized":"2024-09-19T07:36:17.892104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897489+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:36:17.897489+0000","last_peered":"2024-09-19T07:36:17.897489+0000","last_clean":"2024-09-19T07:36:17.897489+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:36:17.897489+0000","last_undegraded":"2024-09-19T07:36:17.897489+0000","last_fullsized":"2024-09-19T07:36:17.897489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889770+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:36:17.889770+0000","last_peered":"2024-09-19T07:36:17.889770+0000","last_clean":"2024-09-19T07:36:17.889770+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:36:17.889770+0000","last_undegraded":"2024-09-19T07:36:17.889770+0000","last_fullsized":"2024-09-19T07:36:17.889770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895916+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:36:17.895916+0000","last_peered":"2024-09-19T07:36:17.895916+0000","last_clean":"2024-09-19T07:36:17.895916+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:36:17.895916+0000","last_undegraded":"2024-09-19T07:36:17.895916+0000","last_fullsized":"2024-09-19T07:36:17.895916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891525+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:36:17.891525+0000","last_peered":"2024-09-19T07:36:17.891525+0000","last_clean":"2024-09-19T07:36:17.891525+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:36:17.891525+0000","last_undegraded":"2024-09-19T07:36:17.891525+0000","last_fullsized":"2024-09-19T07:36:17.891525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890710+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:36:17.890710+0000","last_peered":"2024-09-19T07:36:17.890710+0000","last_clean":"2024-09-19T07:36:17.890710+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:36:17.890710+0000","last_undegraded":"2024-09-19T07:36:17.890710+0000","last_fullsized":"2024-09-19T07:36:17.890710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891804+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:36:17.891804+0000","last_peered":"2024-09-19T07:36:17.891804+0000","last_clean":"2024-09-19T07:36:17.891804+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:36:17.891804+0000","last_undegraded":"2024-09-19T07:36:17.891804+0000","last_fullsized":"2024-09-19T07:36:17.891804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892254+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:36:17.892254+0000","last_peered":"2024-09-19T07:36:17.892254+0000","last_clean":"2024-09-19T07:36:17.892254+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:36:17.892254+0000","last_undegraded":"2024-09-19T07:36:17.892254+0000","last_fullsized":"2024-09-19T07:36:17.892254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893575+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:36:17.893575+0000","last_peered":"2024-09-19T07:36:17.893575+0000","last_clean":"2024-09-19T07:36:17.893575+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:36:17.893575+0000","last_undegraded":"2024-09-19T07:36:17.893575+0000","last_fullsized":"2024-09-19T07:36:17.893575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889691+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:36:17.889691+0000","last_peered":"2024-09-19T07:36:17.889691+0000","last_clean":"2024-09-19T07:36:17.889691+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:36:17.889691+0000","last_undegraded":"2024-09-19T07:36:17.889691+0000","last_fullsized":"2024-09-19T07:36:17.889691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895765+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:36:17.895765+0000","last_peered":"2024-09-19T07:36:17.895765+0000","last_clean":"2024-09-19T07:36:17.895765+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:36:17.895765+0000","last_undegraded":"2024-09-19T07:36:17.895765+0000","last_fullsized":"2024-09-19T07:36:17.895765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891796+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:36:17.891796+0000","last_peered":"2024-09-19T07:36:17.891796+0000","last_clean":"2024-09-19T07:36:17.891796+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:36:17.891796+0000","last_undegraded":"2024-09-19T07:36:17.891796+0000","last_fullsized":"2024-09-19T07:36:17.891796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892016+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:36:17.892016+0000","last_peered":"2024-09-19T07:36:17.892016+0000","last_clean":"2024-09-19T07:36:17.892016+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:36:17.892016+0000","last_undegraded":"2024-09-19T07:36:17.892016+0000","last_fullsized":"2024-09-19T07:36:17.892016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894219+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:36:17.894219+0000","last_peered":"2024-09-19T07:36:17.894219+0000","last_clean":"2024-09-19T07:36:17.894219+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:36:17.894219+0000","last_undegraded":"2024-09-19T07:36:17.894219+0000","last_fullsized":"2024-09-19T07:36:17.894219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895304+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:36:17.895304+0000","last_peered":"2024-09-19T07:36:17.895304+0000","last_clean":"2024-09-19T07:36:17.895304+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:36:17.895304+0000","last_undegraded":"2024-09-19T07:36:17.895304+0000","last_fullsized":"2024-09-19T07:36:17.895304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:27.927798+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:36:27.927798+0000","last_peered":"2024-09-19T07:36:27.927798+0000","last_clean":"2024-09-19T07:36:27.927798+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:36:27.927798+0000","last_undegraded":"2024-09-19T07:36:27.927798+0000","last_fullsized":"2024-09-19T07:36:27.927798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890814+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:36:17.890814+0000","last_peered":"2024-09-19T07:36:17.890814+0000","last_clean":"2024-09-19T07:36:17.890814+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:36:17.890814+0000","last_undegraded":"2024-09-19T07:36:17.890814+0000","last_fullsized":"2024-09-19T07:36:17.890814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890526+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:36:17.890526+0000","last_peered":"2024-09-19T07:36:17.890526+0000","last_clean":"2024-09-19T07:36:17.890526+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:36:17.890526+0000","last_undegraded":"2024-09-19T07:36:17.890526+0000","last_fullsized":"2024-09-19T07:36:17.890526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889543+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:36:17.889543+0000","last_peered":"2024-09-19T07:36:17.889543+0000","last_clean":"2024-09-19T07:36:17.889543+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:36:17.889543+0000","last_undegraded":"2024-09-19T07:36:17.889543+0000","last_fullsized":"2024-09-19T07:36:17.889543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891490+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:36:17.891490+0000","last_peered":"2024-09-19T07:36:17.891490+0000","last_clean":"2024-09-19T07:36:17.891490+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:36:17.891490+0000","last_undegraded":"2024-09-19T07:36:17.891490+0000","last_fullsized":"2024-09-19T07:36:17.891490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907827+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:36:18.907827+0000","last_peered":"2024-09-19T07:36:18.907827+0000","last_clean":"2024-09-19T07:36:18.907827+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:36:18.907827+0000","last_undegraded":"2024-09-19T07:36:18.907827+0000","last_fullsized":"2024-09-19T07:36:18.907827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892021+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:36:17.892021+0000","last_peered":"2024-09-19T07:36:17.892021+0000","last_clean":"2024-09-19T07:36:17.892021+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:36:17.892021+0000","last_undegraded":"2024-09-19T07:36:17.892021+0000","last_fullsized":"2024-09-19T07:36:17.892021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:27.927067+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:36:27.927067+0000","last_peered":"2024-09-19T07:36:27.927067+0000","last_clean":"2024-09-19T07:36:27.927067+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:36:27.927067+0000","last_undegraded":"2024-09-19T07:36:27.927067+0000","last_fullsized":"2024-09-19T07:36:27.927067+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896529+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:36:17.896529+0000","last_peered":"2024-09-19T07:36:17.896529+0000","last_clean":"2024-09-19T07:36:17.896529+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:36:17.896529+0000","last_undegraded":"2024-09-19T07:36:17.896529+0000","last_fullsized":"2024-09-19T07:36:17.896529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908690+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:36:18.908690+0000","last_peered":"2024-09-19T07:36:18.908690+0000","last_clean":"2024-09-19T07:36:18.908690+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:36:18.908690+0000","last_undegraded":"2024-09-19T07:36:18.908690+0000","last_fullsized":"2024-09-19T07:36:18.908690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896039+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:36:17.896039+0000","last_peered":"2024-09-19T07:36:17.896039+0000","last_clean":"2024-09-19T07:36:17.896039+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:36:17.896039+0000","last_undegraded":"2024-09-19T07:36:17.896039+0000","last_fullsized":"2024-09-19T07:36:17.896039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890888+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:36:17.890888+0000","last_peered":"2024-09-19T07:36:17.890888+0000","last_clean":"2024-09-19T07:36:17.890888+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:36:17.890888+0000","last_undegraded":"2024-09-19T07:36:17.890888+0000","last_fullsized":"2024-09-19T07:36:17.890888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895996+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:36:17.895996+0000","last_peered":"2024-09-19T07:36:17.895996+0000","last_clean":"2024-09-19T07:36:17.895996+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:36:17.895996+0000","last_undegraded":"2024-09-19T07:36:17.895996+0000","last_fullsized":"2024-09-19T07:36:17.895996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'23","reported_seq":68,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:27.928176+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:36:27.928176+0000","last_peered":"2024-09-19T07:36:27.928176+0000","last_clean":"2024-09-19T07:36:27.928176+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:36:27.928176+0000","last_undegraded":"2024-09-19T07:36:27.928176+0000","last_fullsized":"2024-09-19T07:36:27.928176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":51,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895733+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:36:17.895733+0000","last_peered":"2024-09-19T07:36:17.895733+0000","last_clean":"2024-09-19T07:36:17.895733+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:36:17.895733+0000","last_undegraded":"2024-09-19T07:36:17.895733+0000","last_fullsized":"2024-09-19T07:36:17.895733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913146+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:36:19.913146+0000","last_peered":"2024-09-19T07:36:19.913146+0000","last_clean":"2024-09-19T07:36:19.913146+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:36:19.913146+0000","last_undegraded":"2024-09-19T07:36:19.913146+0000","last_fullsized":"2024-09-19T07:36:19.913146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889534+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:36:17.889534+0000","last_peered":"2024-09-19T07:36:17.889534+0000","last_clean":"2024-09-19T07:36:17.889534+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:36:17.889534+0000","last_undegraded":"2024-09-19T07:36:17.889534+0000","last_fullsized":"2024-09-19T07:36:17.889534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.906778+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:36:18.906778+0000","last_peered":"2024-09-19T07:36:18.906778+0000","last_clean":"2024-09-19T07:36:18.906778+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:36:18.906778+0000","last_undegraded":"2024-09-19T07:36:18.906778+0000","last_fullsized":"2024-09-19T07:36:18.906778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893599+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:36:17.893599+0000","last_peered":"2024-09-19T07:36:17.893599+0000","last_clean":"2024-09-19T07:36:17.893599+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:36:17.893599+0000","last_undegraded":"2024-09-19T07:36:17.893599+0000","last_fullsized":"2024-09-19T07:36:17.893599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:28.928570+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:36:28.928570+0000","last_peered":"2024-09-19T07:36:28.928570+0000","last_clean":"2024-09-19T07:36:28.928570+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:36:28.928570+0000","last_undegraded":"2024-09-19T07:36:28.928570+0000","last_fullsized":"2024-09-19T07:36:28.928570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893370+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:36:17.893370+0000","last_peered":"2024-09-19T07:36:17.893370+0000","last_clean":"2024-09-19T07:36:17.893370+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:36:17.893370+0000","last_undegraded":"2024-09-19T07:36:17.893370+0000","last_fullsized":"2024-09-19T07:36:17.893370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907635+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:36:18.907635+0000","last_peered":"2024-09-19T07:36:18.907635+0000","last_clean":"2024-09-19T07:36:18.907635+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:36:18.907635+0000","last_undegraded":"2024-09-19T07:36:18.907635+0000","last_fullsized":"2024-09-19T07:36:18.907635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890552+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:36:17.890552+0000","last_peered":"2024-09-19T07:36:17.890552+0000","last_clean":"2024-09-19T07:36:17.890552+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:36:17.890552+0000","last_undegraded":"2024-09-19T07:36:17.890552+0000","last_fullsized":"2024-09-19T07:36:17.890552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895827+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:36:17.895827+0000","last_peered":"2024-09-19T07:36:17.895827+0000","last_clean":"2024-09-19T07:36:17.895827+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:36:17.895827+0000","last_undegraded":"2024-09-19T07:36:17.895827+0000","last_fullsized":"2024-09-19T07:36:17.895827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893694+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:36:17.893694+0000","last_peered":"2024-09-19T07:36:17.893694+0000","last_clean":"2024-09-19T07:36:17.893694+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:36:17.893694+0000","last_undegraded":"2024-09-19T07:36:17.893694+0000","last_fullsized":"2024-09-19T07:36:17.893694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908154+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:36:18.908154+0000","last_peered":"2024-09-19T07:36:18.908154+0000","last_clean":"2024-09-19T07:36:18.908154+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:36:18.908154+0000","last_undegraded":"2024-09-19T07:36:18.908154+0000","last_fullsized":"2024-09-19T07:36:18.908154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894155+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:36:17.894155+0000","last_peered":"2024-09-19T07:36:17.894155+0000","last_clean":"2024-09-19T07:36:17.894155+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:36:17.894155+0000","last_undegraded":"2024-09-19T07:36:17.894155+0000","last_fullsized":"2024-09-19T07:36:17.894155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907835+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:36:18.907835+0000","last_peered":"2024-09-19T07:36:18.907835+0000","last_clean":"2024-09-19T07:36:18.907835+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:36:18.907835+0000","last_undegraded":"2024-09-19T07:36:18.907835+0000","last_fullsized":"2024-09-19T07:36:18.907835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893835+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:36:17.893835+0000","last_peered":"2024-09-19T07:36:17.893835+0000","last_clean":"2024-09-19T07:36:17.893835+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:36:17.893835+0000","last_undegraded":"2024-09-19T07:36:17.893835+0000","last_fullsized":"2024-09-19T07:36:17.893835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:27.927608+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:36:27.927608+0000","last_peered":"2024-09-19T07:36:27.927608+0000","last_clean":"2024-09-19T07:36:27.927608+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:36:27.927608+0000","last_undegraded":"2024-09-19T07:36:27.927608+0000","last_fullsized":"2024-09-19T07:36:27.927608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894098+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:36:17.894098+0000","last_peered":"2024-09-19T07:36:17.894098+0000","last_clean":"2024-09-19T07:36:17.894098+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:36:17.894098+0000","last_undegraded":"2024-09-19T07:36:17.894098+0000","last_fullsized":"2024-09-19T07:36:17.894098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893546+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:36:17.893546+0000","last_peered":"2024-09-19T07:36:17.893546+0000","last_clean":"2024-09-19T07:36:17.893546+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:36:17.893546+0000","last_undegraded":"2024-09-19T07:36:17.893546+0000","last_fullsized":"2024-09-19T07:36:17.893546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913196+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:36:19.913196+0000","last_peered":"2024-09-19T07:36:19.913196+0000","last_clean":"2024-09-19T07:36:19.913196+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:36:19.913196+0000","last_undegraded":"2024-09-19T07:36:19.913196+0000","last_fullsized":"2024-09-19T07:36:19.913196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892747+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:36:17.892747+0000","last_peered":"2024-09-19T07:36:17.892747+0000","last_clean":"2024-09-19T07:36:17.892747+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:36:17.892747+0000","last_undegraded":"2024-09-19T07:36:17.892747+0000","last_fullsized":"2024-09-19T07:36:17.892747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893570+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:36:17.893570+0000","last_peered":"2024-09-19T07:36:17.893570+0000","last_clean":"2024-09-19T07:36:17.893570+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:36:17.893570+0000","last_undegraded":"2024-09-19T07:36:17.893570+0000","last_fullsized":"2024-09-19T07:36:17.893570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889475+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:36:17.889475+0000","last_peered":"2024-09-19T07:36:17.889475+0000","last_clean":"2024-09-19T07:36:17.889475+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:36:17.889475+0000","last_undegraded":"2024-09-19T07:36:17.889475+0000","last_fullsized":"2024-09-19T07:36:17.889475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:25.924825+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:36:25.924825+0000","last_peered":"2024-09-19T07:36:25.924825+0000","last_clean":"2024-09-19T07:36:25.924825+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:36:25.924825+0000","last_undegraded":"2024-09-19T07:36:25.924825+0000","last_fullsized":"2024-09-19T07:36:25.924825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890854+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:36:17.890854+0000","last_peered":"2024-09-19T07:36:17.890854+0000","last_clean":"2024-09-19T07:36:17.890854+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:36:17.890854+0000","last_undegraded":"2024-09-19T07:36:17.890854+0000","last_fullsized":"2024-09-19T07:36:17.890854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:29.929252+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:36:29.929252+0000","last_peered":"2024-09-19T07:36:29.929252+0000","last_clean":"2024-09-19T07:36:29.929252+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:36:29.929252+0000","last_undegraded":"2024-09-19T07:36:29.929252+0000","last_fullsized":"2024-09-19T07:36:29.929252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890537+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:36:17.890537+0000","last_peered":"2024-09-19T07:36:17.890537+0000","last_clean":"2024-09-19T07:36:17.890537+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:36:17.890537+0000","last_undegraded":"2024-09-19T07:36:17.890537+0000","last_fullsized":"2024-09-19T07:36:17.890537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892154+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:36:17.892154+0000","last_peered":"2024-09-19T07:36:17.892154+0000","last_clean":"2024-09-19T07:36:17.892154+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:36:17.892154+0000","last_undegraded":"2024-09-19T07:36:17.892154+0000","last_fullsized":"2024-09-19T07:36:17.892154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890794+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:36:17.890794+0000","last_peered":"2024-09-19T07:36:17.890794+0000","last_clean":"2024-09-19T07:36:17.890794+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:36:17.890794+0000","last_undegraded":"2024-09-19T07:36:17.890794+0000","last_fullsized":"2024-09-19T07:36:17.890794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.909170+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:36:18.909170+0000","last_peered":"2024-09-19T07:36:18.909170+0000","last_clean":"2024-09-19T07:36:18.909170+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:36:18.909170+0000","last_undegraded":"2024-09-19T07:36:18.909170+0000","last_fullsized":"2024-09-19T07:36:18.909170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889701+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:36:17.889701+0000","last_peered":"2024-09-19T07:36:17.889701+0000","last_clean":"2024-09-19T07:36:17.889701+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:36:17.889701+0000","last_undegraded":"2024-09-19T07:36:17.889701+0000","last_fullsized":"2024-09-19T07:36:17.889701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'10","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:23.851977+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:36:23.851977+0000","last_peered":"2024-09-19T07:36:23.851977+0000","last_clean":"2024-09-19T07:36:23.851977+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:36:23.851977+0000","last_undegraded":"2024-09-19T07:36:23.851977+0000","last_fullsized":"2024-09-19T07:36:23.851977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891890+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:36:17.891890+0000","last_peered":"2024-09-19T07:36:17.891890+0000","last_clean":"2024-09-19T07:36:17.891890+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:36:17.891890+0000","last_undegraded":"2024-09-19T07:36:17.891890+0000","last_fullsized":"2024-09-19T07:36:17.891890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907864+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:36:18.907864+0000","last_peered":"2024-09-19T07:36:18.907864+0000","last_clean":"2024-09-19T07:36:18.907864+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:36:18.907864+0000","last_undegraded":"2024-09-19T07:36:18.907864+0000","last_fullsized":"2024-09-19T07:36:18.907864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895903+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:36:17.895903+0000","last_peered":"2024-09-19T07:36:17.895903+0000","last_clean":"2024-09-19T07:36:17.895903+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:36:17.895903+0000","last_undegraded":"2024-09-19T07:36:17.895903+0000","last_fullsized":"2024-09-19T07:36:17.895903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895928+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:36:17.895928+0000","last_peered":"2024-09-19T07:36:17.895928+0000","last_clean":"2024-09-19T07:36:17.895928+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:36:17.895928+0000","last_undegraded":"2024-09-19T07:36:17.895928+0000","last_fullsized":"2024-09-19T07:36:17.895928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894935+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:36:17.894935+0000","last_peered":"2024-09-19T07:36:17.894935+0000","last_clean":"2024-09-19T07:36:17.894935+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:36:17.894935+0000","last_undegraded":"2024-09-19T07:36:17.894935+0000","last_fullsized":"2024-09-19T07:36:17.894935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907171+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:36:18.907171+0000","last_peered":"2024-09-19T07:36:18.907171+0000","last_clean":"2024-09-19T07:36:18.907171+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:36:18.907171+0000","last_undegraded":"2024-09-19T07:36:18.907171+0000","last_fullsized":"2024-09-19T07:36:18.907171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894276+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:36:17.894276+0000","last_peered":"2024-09-19T07:36:17.894276+0000","last_clean":"2024-09-19T07:36:17.894276+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:36:17.894276+0000","last_undegraded":"2024-09-19T07:36:17.894276+0000","last_fullsized":"2024-09-19T07:36:17.894276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.141159+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:36:18.141159+0000","last_peered":"2024-09-19T07:36:18.141159+0000","last_clean":"2024-09-19T07:36:18.141159+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:36:18.141159+0000","last_undegraded":"2024-09-19T07:36:18.141159+0000","last_fullsized":"2024-09-19T07:36:18.141159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891645+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:36:17.891645+0000","last_peered":"2024-09-19T07:36:17.891645+0000","last_clean":"2024-09-19T07:36:17.891645+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:36:17.891645+0000","last_undegraded":"2024-09-19T07:36:17.891645+0000","last_fullsized":"2024-09-19T07:36:17.891645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895305+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:36:17.895305+0000","last_peered":"2024-09-19T07:36:17.895305+0000","last_clean":"2024-09-19T07:36:17.895305+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:36:17.895305+0000","last_undegraded":"2024-09-19T07:36:17.895305+0000","last_fullsized":"2024-09-19T07:36:17.895305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893736+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:36:17.893736+0000","last_peered":"2024-09-19T07:36:17.893736+0000","last_clean":"2024-09-19T07:36:17.893736+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:36:17.893736+0000","last_undegraded":"2024-09-19T07:36:17.893736+0000","last_fullsized":"2024-09-19T07:36:17.893736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.936693+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:36:17.936693+0000","last_peered":"2024-09-19T07:36:17.936693+0000","last_clean":"2024-09-19T07:36:17.936693+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:36:17.936693+0000","last_undegraded":"2024-09-19T07:36:17.936693+0000","last_fullsized":"2024-09-19T07:36:17.936693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908619+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:36:18.908619+0000","last_peered":"2024-09-19T07:36:18.908619+0000","last_clean":"2024-09-19T07:36:18.908619+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:36:18.908619+0000","last_undegraded":"2024-09-19T07:36:18.908619+0000","last_fullsized":"2024-09-19T07:36:18.908619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895842+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:36:17.895842+0000","last_peered":"2024-09-19T07:36:17.895842+0000","last_clean":"2024-09-19T07:36:17.895842+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:36:17.895842+0000","last_undegraded":"2024-09-19T07:36:17.895842+0000","last_fullsized":"2024-09-19T07:36:17.895842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913109+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:36:19.913109+0000","last_peered":"2024-09-19T07:36:19.913109+0000","last_clean":"2024-09-19T07:36:19.913109+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:36:19.913109+0000","last_undegraded":"2024-09-19T07:36:19.913109+0000","last_fullsized":"2024-09-19T07:36:19.913109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895948+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:36:17.895948+0000","last_peered":"2024-09-19T07:36:17.895948+0000","last_clean":"2024-09-19T07:36:17.895948+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:36:17.895948+0000","last_undegraded":"2024-09-19T07:36:17.895948+0000","last_fullsized":"2024-09-19T07:36:17.895948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891880+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:36:17.891880+0000","last_peered":"2024-09-19T07:36:17.891880+0000","last_clean":"2024-09-19T07:36:17.891880+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:36:17.891880+0000","last_undegraded":"2024-09-19T07:36:17.891880+0000","last_fullsized":"2024-09-19T07:36:17.891880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891708+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:36:17.891708+0000","last_peered":"2024-09-19T07:36:17.891708+0000","last_clean":"2024-09-19T07:36:17.891708+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:36:17.891708+0000","last_undegraded":"2024-09-19T07:36:17.891708+0000","last_fullsized":"2024-09-19T07:36:17.891708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:25.924453+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:36:25.924453+0000","last_peered":"2024-09-19T07:36:25.924453+0000","last_clean":"2024-09-19T07:36:25.924453+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:36:25.924453+0000","last_undegraded":"2024-09-19T07:36:25.924453+0000","last_fullsized":"2024-09-19T07:36:25.924453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48666,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":133,"num_read_kb":118,"num_write":227,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":163074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247252,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":442143,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1074125,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441568,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":441162,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1034859,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441678,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1033751,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":921600,"data_stored":441539,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441110,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":492414,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":492866,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":443017,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":52420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:36:33.769 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:36:34.016 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:34.431 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:34 smithi103 ceph-mon[31201]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:36:34.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.509+0000 7f1212788640 1 -- 172.21.15.103:0/1782793943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c1027a0 msgr2=0x7f120c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.509+0000 7f1212788640 1 --2- 172.21.15.103:0/1782793943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c1027a0 0x7f120c1067f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009900 tx=0x7f11f402a410 comp rx=0 tx=0).stop 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- 172.21.15.103:0/1782793943 shutdown_connections 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 --2- 172.21.15.103:0/1782793943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c106f20 0x7f120c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 --2- 172.21.15.103:0/1782793943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c1027a0 0x7f120c1067f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 --2- 172.21.15.103:0/1782793943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f120c101df0 0x7f120c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- 172.21.15.103:0/1782793943 >> 172.21.15.103:0/1782793943 conn(0x7f120c0fd8f0 msgr2=0x7f120c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- 172.21.15.103:0/1782793943 shutdown_connections 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- 172.21.15.103:0/1782793943 wait complete. 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 Processor -- start 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- start start 2024-09-19T07:36:34.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c101df0 0x7f120c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:34.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f120c1027a0 0x7f120c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:34.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c106f20 0x7f120c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:34.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f120c111fa0 con 0x7f120c106f20 2024-09-19T07:36:34.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f120c111e20 con 0x7f120c101df0 2024-09-19T07:36:34.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.510+0000 7f1212788640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f120c112120 con 0x7f120c1027a0 2024-09-19T07:36:34.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c106f20 0x7f120c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c106f20 0x7f120c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43842/0 (socket says 172.21.15.103:43842) 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 -- 172.21.15.103:0/3094637187 learned_addr learned my addr 172.21.15.103:0/3094637187 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f120bfff640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c101df0 0x7f120c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1203fff640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f120c1027a0 0x7f120c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 -- 172.21.15.103:0/3094637187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f120c1027a0 msgr2=0x7f120c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f120c1027a0 0x7f120c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 -- 172.21.15.103:0/3094637187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c101df0 msgr2=0x7f120c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c101df0 0x7f120c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 -- 172.21.15.103:0/3094637187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c19d510 con 0x7f120c106f20 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1210d25640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c106f20 0x7f120c19ce30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f120400cce0 tx=0x7f120400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:34.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1209ffb640 1 -- 172.21.15.103:0/3094637187 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400f450 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c19d800 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1209ffb640 1 -- 172.21.15.103:0/3094637187 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120400fad0 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1209ffb640 1 -- 172.21.15.103:0/3094637187 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204018b60 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.511+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c1a50e0 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.512+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120c1021d0 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.514+0000 7f1209ffb640 1 -- 172.21.15.103:0/3094637187 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f120400fc80 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.514+0000 7f1209ffb640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f11f80a32d0 0x7f11f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.514+0000 7f120bfff640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f11f80a32d0 0x7f11f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.514+0000 7f1209ffb640 1 -- 172.21.15.103:0/3094637187 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f12040d0380 con 0x7f120c106f20 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.515+0000 7f120bfff640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f11f80a32d0 0x7f11f80a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009990 tx=0x7f11fc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:34.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.517+0000 7f1209ffb640 1 -- 172.21.15.103:0/3094637187 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1204014030 con 0x7f120c106f20 2024-09-19T07:36:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:34 smithi134 ceph-mon[35414]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:36:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:34 smithi174 ceph-mon[35370]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:36:34.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.681+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f120c062630 con 0x7f11f80a32d0 2024-09-19T07:36:34.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.688+0000 7f1209ffb640 1 -- 172.21.15.103:0/3094637187 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258999 (secure 0 0 0) 0x7f120c062630 con 0x7f11f80a32d0 2024-09-19T07:36:34.691 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:34.692 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:36:34.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f11f80a32d0 msgr2=0x7f11f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f11f80a32d0 0x7f11f80a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009990 tx=0x7f11fc009110 comp rx=0 tx=0).stop 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c106f20 msgr2=0x7f120c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c106f20 0x7f120c19ce30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f120400cce0 tx=0x7f120400a620 comp rx=0 tx=0).stop 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 shutdown_connections 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f11f80a32d0 0x7f11f80a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f120c106f20 0x7f120c19ce30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f120c1027a0 0x7f120c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 --2- 172.21.15.103:0/3094637187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c101df0 0x7f120c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 >> 172.21.15.103:0/3094637187 conn(0x7f120c0fd8f0 msgr2=0x7f120c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 shutdown_connections 2024-09-19T07:36:34.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:34.692+0000 7f1212788640 1 -- 172.21.15.103:0/3094637187 wait complete. 2024-09-19T07:36:35.456 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:35 smithi103 ceph-mon[31201]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 16 op/s; 48 B/s, 0 objects/s recovering 2024-09-19T07:36:35.486 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-19T07:36:34.086258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647806,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":300,"num_read_kb":271,"num_write":421,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":343860,"kb_used_data":11244,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597964,"statfs":{"total":1151940427776,"available":1151588315136,"internally_reserved":0,"allocated":11513856,"data_stored":7228770,"data_compressed":177078,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49516,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":145,"num_read_kb":136,"num_write":55,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002816"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893903+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:36:17.893903+0000","last_peered":"2024-09-19T07:36:17.893903+0000","last_clean":"2024-09-19T07:36:17.893903+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:36:17.893903+0000","last_undegraded":"2024-09-19T07:36:17.893903+0000","last_fullsized":"2024-09-19T07:36:17.893903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895834+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:36:17.895834+0000","last_peered":"2024-09-19T07:36:17.895834+0000","last_clean":"2024-09-19T07:36:17.895834+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:36:17.895834+0000","last_undegraded":"2024-09-19T07:36:17.895834+0000","last_fullsized":"2024-09-19T07:36:17.895834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895895+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:36:17.895895+0000","last_peered":"2024-09-19T07:36:17.895895+0000","last_clean":"2024-09-19T07:36:17.895895+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:36:17.895895+0000","last_undegraded":"2024-09-19T07:36:17.895895+0000","last_fullsized":"2024-09-19T07:36:17.895895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896911+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:36:17.896911+0000","last_peered":"2024-09-19T07:36:17.896911+0000","last_clean":"2024-09-19T07:36:17.896911+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:36:17.896911+0000","last_undegraded":"2024-09-19T07:36:17.896911+0000","last_fullsized":"2024-09-19T07:36:17.896911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895313+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:36:17.895313+0000","last_peered":"2024-09-19T07:36:17.895313+0000","last_clean":"2024-09-19T07:36:17.895313+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:36:17.895313+0000","last_undegraded":"2024-09-19T07:36:17.895313+0000","last_fullsized":"2024-09-19T07:36:17.895313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897391+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:36:17.897391+0000","last_peered":"2024-09-19T07:36:17.897391+0000","last_clean":"2024-09-19T07:36:17.897391+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:36:17.897391+0000","last_undegraded":"2024-09-19T07:36:17.897391+0000","last_fullsized":"2024-09-19T07:36:17.897391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893437+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:36:17.893437+0000","last_peered":"2024-09-19T07:36:17.893437+0000","last_clean":"2024-09-19T07:36:17.893437+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:36:17.893437+0000","last_undegraded":"2024-09-19T07:36:17.893437+0000","last_fullsized":"2024-09-19T07:36:17.893437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891423+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:36:17.891423+0000","last_peered":"2024-09-19T07:36:17.891423+0000","last_clean":"2024-09-19T07:36:17.891423+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:36:17.891423+0000","last_undegraded":"2024-09-19T07:36:17.891423+0000","last_fullsized":"2024-09-19T07:36:17.891423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896616+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:36:17.896616+0000","last_peered":"2024-09-19T07:36:17.896616+0000","last_clean":"2024-09-19T07:36:17.896616+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:36:17.896616+0000","last_undegraded":"2024-09-19T07:36:17.896616+0000","last_fullsized":"2024-09-19T07:36:17.896616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891049+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:36:17.891049+0000","last_peered":"2024-09-19T07:36:17.891049+0000","last_clean":"2024-09-19T07:36:17.891049+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:36:17.891049+0000","last_undegraded":"2024-09-19T07:36:17.891049+0000","last_fullsized":"2024-09-19T07:36:17.891049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893904+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:36:17.893904+0000","last_peered":"2024-09-19T07:36:17.893904+0000","last_clean":"2024-09-19T07:36:17.893904+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:36:17.893904+0000","last_undegraded":"2024-09-19T07:36:17.893904+0000","last_fullsized":"2024-09-19T07:36:17.893904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.937191+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:36:17.937191+0000","last_peered":"2024-09-19T07:36:17.937191+0000","last_clean":"2024-09-19T07:36:17.937191+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:36:17.937191+0000","last_undegraded":"2024-09-19T07:36:17.937191+0000","last_fullsized":"2024-09-19T07:36:17.937191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889741+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:36:17.889741+0000","last_peered":"2024-09-19T07:36:17.889741+0000","last_clean":"2024-09-19T07:36:17.889741+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:36:17.889741+0000","last_undegraded":"2024-09-19T07:36:17.889741+0000","last_fullsized":"2024-09-19T07:36:17.889741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895614+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:36:17.895614+0000","last_peered":"2024-09-19T07:36:17.895614+0000","last_clean":"2024-09-19T07:36:17.895614+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:36:17.895614+0000","last_undegraded":"2024-09-19T07:36:17.895614+0000","last_fullsized":"2024-09-19T07:36:17.895614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895550+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:36:17.895550+0000","last_peered":"2024-09-19T07:36:17.895550+0000","last_clean":"2024-09-19T07:36:17.895550+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:36:17.895550+0000","last_undegraded":"2024-09-19T07:36:17.895550+0000","last_fullsized":"2024-09-19T07:36:17.895550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895680+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:36:17.895680+0000","last_peered":"2024-09-19T07:36:17.895680+0000","last_clean":"2024-09-19T07:36:17.895680+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:36:17.895680+0000","last_undegraded":"2024-09-19T07:36:17.895680+0000","last_fullsized":"2024-09-19T07:36:17.895680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893949+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:36:17.893949+0000","last_peered":"2024-09-19T07:36:17.893949+0000","last_clean":"2024-09-19T07:36:17.893949+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:36:17.893949+0000","last_undegraded":"2024-09-19T07:36:17.893949+0000","last_fullsized":"2024-09-19T07:36:17.893949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892102+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:36:17.892102+0000","last_peered":"2024-09-19T07:36:17.892102+0000","last_clean":"2024-09-19T07:36:17.892102+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:36:17.892102+0000","last_undegraded":"2024-09-19T07:36:17.892102+0000","last_fullsized":"2024-09-19T07:36:17.892102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892104+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:36:17.892104+0000","last_peered":"2024-09-19T07:36:17.892104+0000","last_clean":"2024-09-19T07:36:17.892104+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:36:17.892104+0000","last_undegraded":"2024-09-19T07:36:17.892104+0000","last_fullsized":"2024-09-19T07:36:17.892104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897489+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:36:17.897489+0000","last_peered":"2024-09-19T07:36:17.897489+0000","last_clean":"2024-09-19T07:36:17.897489+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:36:17.897489+0000","last_undegraded":"2024-09-19T07:36:17.897489+0000","last_fullsized":"2024-09-19T07:36:17.897489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889770+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:36:17.889770+0000","last_peered":"2024-09-19T07:36:17.889770+0000","last_clean":"2024-09-19T07:36:17.889770+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:36:17.889770+0000","last_undegraded":"2024-09-19T07:36:17.889770+0000","last_fullsized":"2024-09-19T07:36:17.889770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895916+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:36:17.895916+0000","last_peered":"2024-09-19T07:36:17.895916+0000","last_clean":"2024-09-19T07:36:17.895916+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:36:17.895916+0000","last_undegraded":"2024-09-19T07:36:17.895916+0000","last_fullsized":"2024-09-19T07:36:17.895916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891525+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:36:17.891525+0000","last_peered":"2024-09-19T07:36:17.891525+0000","last_clean":"2024-09-19T07:36:17.891525+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:36:17.891525+0000","last_undegraded":"2024-09-19T07:36:17.891525+0000","last_fullsized":"2024-09-19T07:36:17.891525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890710+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:36:17.890710+0000","last_peered":"2024-09-19T07:36:17.890710+0000","last_clean":"2024-09-19T07:36:17.890710+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:36:17.890710+0000","last_undegraded":"2024-09-19T07:36:17.890710+0000","last_fullsized":"2024-09-19T07:36:17.890710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891804+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:36:17.891804+0000","last_peered":"2024-09-19T07:36:17.891804+0000","last_clean":"2024-09-19T07:36:17.891804+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:36:17.891804+0000","last_undegraded":"2024-09-19T07:36:17.891804+0000","last_fullsized":"2024-09-19T07:36:17.891804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892254+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:36:17.892254+0000","last_peered":"2024-09-19T07:36:17.892254+0000","last_clean":"2024-09-19T07:36:17.892254+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:36:17.892254+0000","last_undegraded":"2024-09-19T07:36:17.892254+0000","last_fullsized":"2024-09-19T07:36:17.892254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893575+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:36:17.893575+0000","last_peered":"2024-09-19T07:36:17.893575+0000","last_clean":"2024-09-19T07:36:17.893575+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:36:17.893575+0000","last_undegraded":"2024-09-19T07:36:17.893575+0000","last_fullsized":"2024-09-19T07:36:17.893575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889691+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:36:17.889691+0000","last_peered":"2024-09-19T07:36:17.889691+0000","last_clean":"2024-09-19T07:36:17.889691+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:36:17.889691+0000","last_undegraded":"2024-09-19T07:36:17.889691+0000","last_fullsized":"2024-09-19T07:36:17.889691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895765+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:36:17.895765+0000","last_peered":"2024-09-19T07:36:17.895765+0000","last_clean":"2024-09-19T07:36:17.895765+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:36:17.895765+0000","last_undegraded":"2024-09-19T07:36:17.895765+0000","last_fullsized":"2024-09-19T07:36:17.895765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891796+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:36:17.891796+0000","last_peered":"2024-09-19T07:36:17.891796+0000","last_clean":"2024-09-19T07:36:17.891796+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:36:17.891796+0000","last_undegraded":"2024-09-19T07:36:17.891796+0000","last_fullsized":"2024-09-19T07:36:17.891796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892016+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:36:17.892016+0000","last_peered":"2024-09-19T07:36:17.892016+0000","last_clean":"2024-09-19T07:36:17.892016+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:36:17.892016+0000","last_undegraded":"2024-09-19T07:36:17.892016+0000","last_fullsized":"2024-09-19T07:36:17.892016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894219+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:36:17.894219+0000","last_peered":"2024-09-19T07:36:17.894219+0000","last_clean":"2024-09-19T07:36:17.894219+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:36:17.894219+0000","last_undegraded":"2024-09-19T07:36:17.894219+0000","last_fullsized":"2024-09-19T07:36:17.894219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895304+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:36:17.895304+0000","last_peered":"2024-09-19T07:36:17.895304+0000","last_clean":"2024-09-19T07:36:17.895304+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:36:17.895304+0000","last_undegraded":"2024-09-19T07:36:17.895304+0000","last_fullsized":"2024-09-19T07:36:17.895304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:27.927798+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:36:27.927798+0000","last_peered":"2024-09-19T07:36:27.927798+0000","last_clean":"2024-09-19T07:36:27.927798+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:36:27.927798+0000","last_undegraded":"2024-09-19T07:36:27.927798+0000","last_fullsized":"2024-09-19T07:36:27.927798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890814+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:36:17.890814+0000","last_peered":"2024-09-19T07:36:17.890814+0000","last_clean":"2024-09-19T07:36:17.890814+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:36:17.890814+0000","last_undegraded":"2024-09-19T07:36:17.890814+0000","last_fullsized":"2024-09-19T07:36:17.890814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890526+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:36:17.890526+0000","last_peered":"2024-09-19T07:36:17.890526+0000","last_clean":"2024-09-19T07:36:17.890526+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:36:17.890526+0000","last_undegraded":"2024-09-19T07:36:17.890526+0000","last_fullsized":"2024-09-19T07:36:17.890526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889543+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:36:17.889543+0000","last_peered":"2024-09-19T07:36:17.889543+0000","last_clean":"2024-09-19T07:36:17.889543+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:36:17.889543+0000","last_undegraded":"2024-09-19T07:36:17.889543+0000","last_fullsized":"2024-09-19T07:36:17.889543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891490+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:36:17.891490+0000","last_peered":"2024-09-19T07:36:17.891490+0000","last_clean":"2024-09-19T07:36:17.891490+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:36:17.891490+0000","last_undegraded":"2024-09-19T07:36:17.891490+0000","last_fullsized":"2024-09-19T07:36:17.891490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907827+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:36:18.907827+0000","last_peered":"2024-09-19T07:36:18.907827+0000","last_clean":"2024-09-19T07:36:18.907827+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:36:18.907827+0000","last_undegraded":"2024-09-19T07:36:18.907827+0000","last_fullsized":"2024-09-19T07:36:18.907827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892021+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:36:17.892021+0000","last_peered":"2024-09-19T07:36:17.892021+0000","last_clean":"2024-09-19T07:36:17.892021+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:36:17.892021+0000","last_undegraded":"2024-09-19T07:36:17.892021+0000","last_fullsized":"2024-09-19T07:36:17.892021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:32.933449+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:36:32.933449+0000","last_peered":"2024-09-19T07:36:32.933449+0000","last_clean":"2024-09-19T07:36:32.933449+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:36:32.933449+0000","last_undegraded":"2024-09-19T07:36:32.933449+0000","last_fullsized":"2024-09-19T07:36:32.933449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896529+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:36:17.896529+0000","last_peered":"2024-09-19T07:36:17.896529+0000","last_clean":"2024-09-19T07:36:17.896529+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:36:17.896529+0000","last_undegraded":"2024-09-19T07:36:17.896529+0000","last_fullsized":"2024-09-19T07:36:17.896529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908690+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:36:18.908690+0000","last_peered":"2024-09-19T07:36:18.908690+0000","last_clean":"2024-09-19T07:36:18.908690+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:36:18.908690+0000","last_undegraded":"2024-09-19T07:36:18.908690+0000","last_fullsized":"2024-09-19T07:36:18.908690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896039+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:36:17.896039+0000","last_peered":"2024-09-19T07:36:17.896039+0000","last_clean":"2024-09-19T07:36:17.896039+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:36:17.896039+0000","last_undegraded":"2024-09-19T07:36:17.896039+0000","last_fullsized":"2024-09-19T07:36:17.896039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890888+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:36:17.890888+0000","last_peered":"2024-09-19T07:36:17.890888+0000","last_clean":"2024-09-19T07:36:17.890888+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:36:17.890888+0000","last_undegraded":"2024-09-19T07:36:17.890888+0000","last_fullsized":"2024-09-19T07:36:17.890888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895996+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:36:17.895996+0000","last_peered":"2024-09-19T07:36:17.895996+0000","last_clean":"2024-09-19T07:36:17.895996+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:36:17.895996+0000","last_undegraded":"2024-09-19T07:36:17.895996+0000","last_fullsized":"2024-09-19T07:36:17.895996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'27","reported_seq":83,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:32.933903+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:36:32.933903+0000","last_peered":"2024-09-19T07:36:32.933903+0000","last_clean":"2024-09-19T07:36:32.933903+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:36:32.933903+0000","last_undegraded":"2024-09-19T07:36:32.933903+0000","last_fullsized":"2024-09-19T07:36:32.933903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49603,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":65,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895733+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:36:17.895733+0000","last_peered":"2024-09-19T07:36:17.895733+0000","last_clean":"2024-09-19T07:36:17.895733+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:36:17.895733+0000","last_undegraded":"2024-09-19T07:36:17.895733+0000","last_fullsized":"2024-09-19T07:36:17.895733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913146+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:36:19.913146+0000","last_peered":"2024-09-19T07:36:19.913146+0000","last_clean":"2024-09-19T07:36:19.913146+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:36:19.913146+0000","last_undegraded":"2024-09-19T07:36:19.913146+0000","last_fullsized":"2024-09-19T07:36:19.913146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889534+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:36:17.889534+0000","last_peered":"2024-09-19T07:36:17.889534+0000","last_clean":"2024-09-19T07:36:17.889534+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:36:17.889534+0000","last_undegraded":"2024-09-19T07:36:17.889534+0000","last_fullsized":"2024-09-19T07:36:17.889534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.906778+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:36:18.906778+0000","last_peered":"2024-09-19T07:36:18.906778+0000","last_clean":"2024-09-19T07:36:18.906778+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:36:18.906778+0000","last_undegraded":"2024-09-19T07:36:18.906778+0000","last_fullsized":"2024-09-19T07:36:18.906778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893599+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:36:17.893599+0000","last_peered":"2024-09-19T07:36:17.893599+0000","last_clean":"2024-09-19T07:36:17.893599+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:36:17.893599+0000","last_undegraded":"2024-09-19T07:36:17.893599+0000","last_fullsized":"2024-09-19T07:36:17.893599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:28.928570+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:36:28.928570+0000","last_peered":"2024-09-19T07:36:28.928570+0000","last_clean":"2024-09-19T07:36:28.928570+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:36:28.928570+0000","last_undegraded":"2024-09-19T07:36:28.928570+0000","last_fullsized":"2024-09-19T07:36:28.928570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893370+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:36:17.893370+0000","last_peered":"2024-09-19T07:36:17.893370+0000","last_clean":"2024-09-19T07:36:17.893370+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:36:17.893370+0000","last_undegraded":"2024-09-19T07:36:17.893370+0000","last_fullsized":"2024-09-19T07:36:17.893370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907635+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:36:18.907635+0000","last_peered":"2024-09-19T07:36:18.907635+0000","last_clean":"2024-09-19T07:36:18.907635+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:36:18.907635+0000","last_undegraded":"2024-09-19T07:36:18.907635+0000","last_fullsized":"2024-09-19T07:36:18.907635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890552+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:36:17.890552+0000","last_peered":"2024-09-19T07:36:17.890552+0000","last_clean":"2024-09-19T07:36:17.890552+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:36:17.890552+0000","last_undegraded":"2024-09-19T07:36:17.890552+0000","last_fullsized":"2024-09-19T07:36:17.890552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895827+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:36:17.895827+0000","last_peered":"2024-09-19T07:36:17.895827+0000","last_clean":"2024-09-19T07:36:17.895827+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:36:17.895827+0000","last_undegraded":"2024-09-19T07:36:17.895827+0000","last_fullsized":"2024-09-19T07:36:17.895827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893694+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:36:17.893694+0000","last_peered":"2024-09-19T07:36:17.893694+0000","last_clean":"2024-09-19T07:36:17.893694+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:36:17.893694+0000","last_undegraded":"2024-09-19T07:36:17.893694+0000","last_fullsized":"2024-09-19T07:36:17.893694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908154+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:36:18.908154+0000","last_peered":"2024-09-19T07:36:18.908154+0000","last_clean":"2024-09-19T07:36:18.908154+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:36:18.908154+0000","last_undegraded":"2024-09-19T07:36:18.908154+0000","last_fullsized":"2024-09-19T07:36:18.908154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894155+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:36:17.894155+0000","last_peered":"2024-09-19T07:36:17.894155+0000","last_clean":"2024-09-19T07:36:17.894155+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:36:17.894155+0000","last_undegraded":"2024-09-19T07:36:17.894155+0000","last_fullsized":"2024-09-19T07:36:17.894155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907835+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:36:18.907835+0000","last_peered":"2024-09-19T07:36:18.907835+0000","last_clean":"2024-09-19T07:36:18.907835+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:36:18.907835+0000","last_undegraded":"2024-09-19T07:36:18.907835+0000","last_fullsized":"2024-09-19T07:36:18.907835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893835+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:36:17.893835+0000","last_peered":"2024-09-19T07:36:17.893835+0000","last_clean":"2024-09-19T07:36:17.893835+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:36:17.893835+0000","last_undegraded":"2024-09-19T07:36:17.893835+0000","last_fullsized":"2024-09-19T07:36:17.893835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:27.927608+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:36:27.927608+0000","last_peered":"2024-09-19T07:36:27.927608+0000","last_clean":"2024-09-19T07:36:27.927608+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:36:27.927608+0000","last_undegraded":"2024-09-19T07:36:27.927608+0000","last_fullsized":"2024-09-19T07:36:27.927608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894098+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:36:17.894098+0000","last_peered":"2024-09-19T07:36:17.894098+0000","last_clean":"2024-09-19T07:36:17.894098+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:36:17.894098+0000","last_undegraded":"2024-09-19T07:36:17.894098+0000","last_fullsized":"2024-09-19T07:36:17.894098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893546+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:36:17.893546+0000","last_peered":"2024-09-19T07:36:17.893546+0000","last_clean":"2024-09-19T07:36:17.893546+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:36:17.893546+0000","last_undegraded":"2024-09-19T07:36:17.893546+0000","last_fullsized":"2024-09-19T07:36:17.893546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913196+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:36:19.913196+0000","last_peered":"2024-09-19T07:36:19.913196+0000","last_clean":"2024-09-19T07:36:19.913196+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:36:19.913196+0000","last_undegraded":"2024-09-19T07:36:19.913196+0000","last_fullsized":"2024-09-19T07:36:19.913196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892747+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:36:17.892747+0000","last_peered":"2024-09-19T07:36:17.892747+0000","last_clean":"2024-09-19T07:36:17.892747+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:36:17.892747+0000","last_undegraded":"2024-09-19T07:36:17.892747+0000","last_fullsized":"2024-09-19T07:36:17.892747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893570+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:36:17.893570+0000","last_peered":"2024-09-19T07:36:17.893570+0000","last_clean":"2024-09-19T07:36:17.893570+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:36:17.893570+0000","last_undegraded":"2024-09-19T07:36:17.893570+0000","last_fullsized":"2024-09-19T07:36:17.893570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889475+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:36:17.889475+0000","last_peered":"2024-09-19T07:36:17.889475+0000","last_clean":"2024-09-19T07:36:17.889475+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:36:17.889475+0000","last_undegraded":"2024-09-19T07:36:17.889475+0000","last_fullsized":"2024-09-19T07:36:17.889475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:30.930485+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:36:30.930485+0000","last_peered":"2024-09-19T07:36:30.930485+0000","last_clean":"2024-09-19T07:36:30.930485+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:36:30.930485+0000","last_undegraded":"2024-09-19T07:36:30.930485+0000","last_fullsized":"2024-09-19T07:36:30.930485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890854+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:36:17.890854+0000","last_peered":"2024-09-19T07:36:17.890854+0000","last_clean":"2024-09-19T07:36:17.890854+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:36:17.890854+0000","last_undegraded":"2024-09-19T07:36:17.890854+0000","last_fullsized":"2024-09-19T07:36:17.890854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:29.929252+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:36:29.929252+0000","last_peered":"2024-09-19T07:36:29.929252+0000","last_clean":"2024-09-19T07:36:29.929252+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:36:29.929252+0000","last_undegraded":"2024-09-19T07:36:29.929252+0000","last_fullsized":"2024-09-19T07:36:29.929252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890537+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:36:17.890537+0000","last_peered":"2024-09-19T07:36:17.890537+0000","last_clean":"2024-09-19T07:36:17.890537+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:36:17.890537+0000","last_undegraded":"2024-09-19T07:36:17.890537+0000","last_fullsized":"2024-09-19T07:36:17.890537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892154+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:36:17.892154+0000","last_peered":"2024-09-19T07:36:17.892154+0000","last_clean":"2024-09-19T07:36:17.892154+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:36:17.892154+0000","last_undegraded":"2024-09-19T07:36:17.892154+0000","last_fullsized":"2024-09-19T07:36:17.892154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890794+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:36:17.890794+0000","last_peered":"2024-09-19T07:36:17.890794+0000","last_clean":"2024-09-19T07:36:17.890794+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:36:17.890794+0000","last_undegraded":"2024-09-19T07:36:17.890794+0000","last_fullsized":"2024-09-19T07:36:17.890794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.909170+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:36:18.909170+0000","last_peered":"2024-09-19T07:36:18.909170+0000","last_clean":"2024-09-19T07:36:18.909170+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:36:18.909170+0000","last_undegraded":"2024-09-19T07:36:18.909170+0000","last_fullsized":"2024-09-19T07:36:18.909170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889701+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:36:17.889701+0000","last_peered":"2024-09-19T07:36:17.889701+0000","last_clean":"2024-09-19T07:36:17.889701+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:36:17.889701+0000","last_undegraded":"2024-09-19T07:36:17.889701+0000","last_fullsized":"2024-09-19T07:36:17.889701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'13","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:30.853271+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:36:30.853271+0000","last_peered":"2024-09-19T07:36:30.853271+0000","last_clean":"2024-09-19T07:36:30.853271+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:36:30.853271+0000","last_undegraded":"2024-09-19T07:36:30.853271+0000","last_fullsized":"2024-09-19T07:36:30.853271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891890+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:36:17.891890+0000","last_peered":"2024-09-19T07:36:17.891890+0000","last_clean":"2024-09-19T07:36:17.891890+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:36:17.891890+0000","last_undegraded":"2024-09-19T07:36:17.891890+0000","last_fullsized":"2024-09-19T07:36:17.891890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907864+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:36:18.907864+0000","last_peered":"2024-09-19T07:36:18.907864+0000","last_clean":"2024-09-19T07:36:18.907864+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:36:18.907864+0000","last_undegraded":"2024-09-19T07:36:18.907864+0000","last_fullsized":"2024-09-19T07:36:18.907864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895903+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:36:17.895903+0000","last_peered":"2024-09-19T07:36:17.895903+0000","last_clean":"2024-09-19T07:36:17.895903+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:36:17.895903+0000","last_undegraded":"2024-09-19T07:36:17.895903+0000","last_fullsized":"2024-09-19T07:36:17.895903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895928+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:36:17.895928+0000","last_peered":"2024-09-19T07:36:17.895928+0000","last_clean":"2024-09-19T07:36:17.895928+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:36:17.895928+0000","last_undegraded":"2024-09-19T07:36:17.895928+0000","last_fullsized":"2024-09-19T07:36:17.895928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894935+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:36:17.894935+0000","last_peered":"2024-09-19T07:36:17.894935+0000","last_clean":"2024-09-19T07:36:17.894935+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:36:17.894935+0000","last_undegraded":"2024-09-19T07:36:17.894935+0000","last_fullsized":"2024-09-19T07:36:17.894935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907171+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:36:18.907171+0000","last_peered":"2024-09-19T07:36:18.907171+0000","last_clean":"2024-09-19T07:36:18.907171+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:36:18.907171+0000","last_undegraded":"2024-09-19T07:36:18.907171+0000","last_fullsized":"2024-09-19T07:36:18.907171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894276+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:36:17.894276+0000","last_peered":"2024-09-19T07:36:17.894276+0000","last_clean":"2024-09-19T07:36:17.894276+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:36:17.894276+0000","last_undegraded":"2024-09-19T07:36:17.894276+0000","last_fullsized":"2024-09-19T07:36:17.894276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.141159+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:36:18.141159+0000","last_peered":"2024-09-19T07:36:18.141159+0000","last_clean":"2024-09-19T07:36:18.141159+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:36:18.141159+0000","last_undegraded":"2024-09-19T07:36:18.141159+0000","last_fullsized":"2024-09-19T07:36:18.141159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891645+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:36:17.891645+0000","last_peered":"2024-09-19T07:36:17.891645+0000","last_clean":"2024-09-19T07:36:17.891645+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:36:17.891645+0000","last_undegraded":"2024-09-19T07:36:17.891645+0000","last_fullsized":"2024-09-19T07:36:17.891645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895305+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:36:17.895305+0000","last_peered":"2024-09-19T07:36:17.895305+0000","last_clean":"2024-09-19T07:36:17.895305+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:36:17.895305+0000","last_undegraded":"2024-09-19T07:36:17.895305+0000","last_fullsized":"2024-09-19T07:36:17.895305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893736+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:36:17.893736+0000","last_peered":"2024-09-19T07:36:17.893736+0000","last_clean":"2024-09-19T07:36:17.893736+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:36:17.893736+0000","last_undegraded":"2024-09-19T07:36:17.893736+0000","last_fullsized":"2024-09-19T07:36:17.893736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.936693+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:36:17.936693+0000","last_peered":"2024-09-19T07:36:17.936693+0000","last_clean":"2024-09-19T07:36:17.936693+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:36:17.936693+0000","last_undegraded":"2024-09-19T07:36:17.936693+0000","last_fullsized":"2024-09-19T07:36:17.936693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908619+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:36:18.908619+0000","last_peered":"2024-09-19T07:36:18.908619+0000","last_clean":"2024-09-19T07:36:18.908619+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:36:18.908619+0000","last_undegraded":"2024-09-19T07:36:18.908619+0000","last_fullsized":"2024-09-19T07:36:18.908619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895842+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:36:17.895842+0000","last_peered":"2024-09-19T07:36:17.895842+0000","last_clean":"2024-09-19T07:36:17.895842+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:36:17.895842+0000","last_undegraded":"2024-09-19T07:36:17.895842+0000","last_fullsized":"2024-09-19T07:36:17.895842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913109+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:36:19.913109+0000","last_peered":"2024-09-19T07:36:19.913109+0000","last_clean":"2024-09-19T07:36:19.913109+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:36:19.913109+0000","last_undegraded":"2024-09-19T07:36:19.913109+0000","last_fullsized":"2024-09-19T07:36:19.913109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895948+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:36:17.895948+0000","last_peered":"2024-09-19T07:36:17.895948+0000","last_clean":"2024-09-19T07:36:17.895948+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:36:17.895948+0000","last_undegraded":"2024-09-19T07:36:17.895948+0000","last_fullsized":"2024-09-19T07:36:17.895948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891880+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:36:17.891880+0000","last_peered":"2024-09-19T07:36:17.891880+0000","last_clean":"2024-09-19T07:36:17.891880+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:36:17.891880+0000","last_undegraded":"2024-09-19T07:36:17.891880+0000","last_fullsized":"2024-09-19T07:36:17.891880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891708+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:36:17.891708+0000","last_peered":"2024-09-19T07:36:17.891708+0000","last_clean":"2024-09-19T07:36:17.891708+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:36:17.891708+0000","last_undegraded":"2024-09-19T07:36:17.891708+0000","last_fullsized":"2024-09-19T07:36:17.891708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:30.929834+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:36:30.929834+0000","last_peered":"2024-09-19T07:36:30.929834+0000","last_clean":"2024-09-19T07:36:30.929834+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:36:30.929834+0000","last_undegraded":"2024-09-19T07:36:30.929834+0000","last_fullsized":"2024-09-19T07:36:30.929834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57204,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":164,"num_read_kb":152,"num_write":238,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":171612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247252,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514513,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":442143,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1082663,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906972,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441568,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":441162,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1034859,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528363,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441678,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1033751,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":921600,"data_stored":441539,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441110,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":492414,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":492866,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":443017,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":52420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":51732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:36:35.491 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T07:36:35.491 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T07:36:35.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T07:36:35.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:35 smithi134 ceph-mon[35414]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 16 op/s; 48 B/s, 0 objects/s recovering 2024-09-19T07:36:35.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:35 smithi174 ceph-mon[35370]: pgmap v293: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 16 op/s; 48 B/s, 0 objects/s recovering 2024-09-19T07:36:35.737 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:36.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 -- 172.21.15.103:0/1704959034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 msgr2=0x7fe58c107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:36.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 --2- 172.21.15.103:0/1704959034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c107420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe574009900 tx=0x7fe57402a410 comp rx=0 tx=0).stop 2024-09-19T07:36:36.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 -- 172.21.15.103:0/1704959034 shutdown_connections 2024-09-19T07:36:36.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 --2- 172.21.15.103:0/1704959034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c107960 0x7fe58c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 --2- 172.21.15.103:0/1704959034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c107420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 --2- 172.21.15.103:0/1704959034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe58c102700 0x7fe58c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 -- 172.21.15.103:0/1704959034 >> 172.21.15.103:0/1704959034 conn(0x7fe58c0fc280 msgr2=0x7fe58c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 -- 172.21.15.103:0/1704959034 shutdown_connections 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 -- 172.21.15.103:0/1704959034 wait complete. 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.208+0000 7fe5917bc640 1 Processor -- start 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 -- start start 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102700 0x7fe58c102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe58c107960 0x7fe58c100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:36.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c10d930 con 0x7fe58c105030 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c10d7b0 con 0x7fe58c107960 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c10dab0 con 0x7fe58c102700 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c1006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43864/0 (socket says 172.21.15.103:43864) 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 -- 172.21.15.103:0/691192902 learned_addr learned my addr 172.21.15.103:0/691192902 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58b7fe640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe58c107960 0x7fe58c100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58affd640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102700 0x7fe58c102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 -- 172.21.15.103:0/691192902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102700 msgr2=0x7fe58c102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102700 0x7fe58c102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 -- 172.21.15.103:0/691192902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe58c107960 msgr2=0x7fe58c100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:36.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe58c107960 0x7fe58c100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 -- 172.21.15.103:0/691192902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c1014b0 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58affd640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102700 0x7fe58c102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe58a7fc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c1006b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe574031610 tx=0x7fe5740043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe583fff640 1 -- 172.21.15.103:0/691192902 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57403f030 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe583fff640 1 -- 172.21.15.103:0/691192902 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe574003ce0 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c1a2180 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe583fff640 1 -- 172.21.15.103:0/691192902 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574035070 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.209+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c1a26b0 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.210+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c0fdd70 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.211+0000 7fe583fff640 1 -- 172.21.15.103:0/691192902 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe574042030 con 0x7fe58c105030 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.212+0000 7fe583fff640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5780a32d0 0x7fe5780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.212+0000 7fe58affd640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5780a32d0 0x7fe5780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:36.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.213+0000 7fe583fff640 1 -- 172.21.15.103:0/691192902 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe5740f0200 con 0x7fe58c105030 2024-09-19T07:36:36.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.213+0000 7fe58affd640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5780a32d0 0x7fe5780a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe57c007eb0 tx=0x7fe57c0073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:36.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.217+0000 7fe583fff640 1 -- 172.21.15.103:0/691192902 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5740bcdc0 con 0x7fe58c105030 2024-09-19T07:36:36.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.446+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe58c02d7c0 con 0x7fe58c105030 2024-09-19T07:36:36.447 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:36 smithi103 ceph-mon[31201]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:36:36.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.448+0000 7fe583fff640 1 -- 172.21.15.103:0/691192902 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe5740bc6c0 con 0x7fe58c105030 2024-09-19T07:36:36.449 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-09-19T07:36:36.449 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQCE1Otmf7OyGhAAbEONMO9alTwC5VAwr2NQQQ== 2024-09-19T07:36:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5780a32d0 msgr2=0x7fe5780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5780a32d0 0x7fe5780a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe57c007eb0 tx=0x7fe57c0073c0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 msgr2=0x7fe58c1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c1006b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe574031610 tx=0x7fe5740043b0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 shutdown_connections 2024-09-19T07:36:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5780a32d0 0x7fe5780a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe58c107960 0x7fe58c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe58c105030 0x7fe58c1006b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 --2- 172.21.15.103:0/691192902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102700 0x7fe58c102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:36.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.450+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 >> 172.21.15.103:0/691192902 conn(0x7fe58c0fc280 msgr2=0x7fe58c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:36.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.451+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 shutdown_connections 2024-09-19T07:36:36.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:36.451+0000 7fe5917bc640 1 -- 172.21.15.103:0/691192902 wait complete. 2024-09-19T07:36:36.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:36 smithi134 ceph-mon[35414]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:36:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:36 smithi174 ceph-mon[35370]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:36:37.076 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:36:37.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T07:36:37.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T07:36:37.112 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T07:36:37.112 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T07:36:37.113 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T07:36:37.399 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:37 smithi174 ceph-mon[35370]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T07:36:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:37 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/691192902' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T07:36:37.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:37 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/691192902' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T07:36:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:37 smithi103 ceph-mon[31201]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T07:36:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:37 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/691192902' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T07:36:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:37 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/691192902' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T07:36:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:37 smithi134 ceph-mon[35414]: pgmap v294: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-19T07:36:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:37 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/691192902' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T07:36:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:37 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/691192902' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T07:36:37.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 -- 172.21.15.103:0/812774718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 msgr2=0x7f046c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:37.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 --2- 172.21.15.103:0/812774718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c10bb50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0460009900 tx=0x7f046002a480 comp rx=0 tx=0).stop 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 -- 172.21.15.103:0/812774718 shutdown_connections 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 --2- 172.21.15.103:0/812774718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f046c10c170 0x7f046c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 --2- 172.21.15.103:0/812774718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c10bb50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 --2- 172.21.15.103:0/812774718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c102d40 0x7f046c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 -- 172.21.15.103:0/812774718 >> 172.21.15.103:0/812774718 conn(0x7f046c0fc7a0 msgr2=0x7f046c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 -- 172.21.15.103:0/812774718 shutdown_connections 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.932+0000 7f04711fc640 1 -- 172.21.15.103:0/812774718 wait complete. 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 Processor -- start 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 -- start start 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f046c102d40 0x7f046c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:37.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c10c170 0x7f046c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f046c111f70 con 0x7f046c103660 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f046c111df0 con 0x7f046c102d40 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f04711fc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f046c1120f0 con 0x7f046c10c170 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f046a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f046a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43878/0 (socket says 172.21.15.103:43878) 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.933+0000 7f046a59c640 1 -- 172.21.15.103:0/2432530619 learned_addr learned my addr 172.21.15.103:0/2432530619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046ad9d640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f046c102d40 0x7f046c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046b59e640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c10c170 0x7f046c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:37.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046a59c640 1 -- 172.21.15.103:0/2432530619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c10c170 msgr2=0x7f046c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046a59c640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c10c170 0x7f046c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046a59c640 1 -- 172.21.15.103:0/2432530619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f046c102d40 msgr2=0x7f046c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046a59c640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f046c102d40 0x7f046c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046a59c640 1 -- 172.21.15.103:0/2432530619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c19f620 con 0x7f046c103660 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046ad9d640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f046c102d40 0x7f046c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046b59e640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c10c170 0x7f046c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f046a59c640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c19abb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0460006010 tx=0x7f0460004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f0453fff640 1 -- 172.21.15.103:0/2432530619 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046003f030 con 0x7f046c103660 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c19f8b0 con 0x7f046c103660 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f0453fff640 1 -- 172.21.15.103:0/2432530619 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460004440 con 0x7f046c103660 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f0453fff640 1 -- 172.21.15.103:0/2432530619 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460035070 con 0x7f046c103660 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.934+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c1a71f0 con 0x7f046c103660 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.935+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046c06b070 con 0x7f046c103660 2024-09-19T07:36:37.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.936+0000 7f0453fff640 1 -- 172.21.15.103:0/2432530619 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0460042030 con 0x7f046c103660 2024-09-19T07:36:37.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.937+0000 7f0453fff640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04540a32d0 0x7f04540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:37.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.937+0000 7f046ad9d640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04540a32d0 0x7f04540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:37.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.937+0000 7f0453fff640 1 -- 172.21.15.103:0/2432530619 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f04600eff50 con 0x7f046c103660 2024-09-19T07:36:37.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.937+0000 7f046ad9d640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04540a32d0 0x7f04540a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f045c009c20 tx=0x7f045c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:37.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:37.941+0000 7f0453fff640 1 -- 172.21.15.103:0/2432530619 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04600bcb10 con 0x7f046c103660 2024-09-19T07:36:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.153+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f046c1a73f0 con 0x7f046c103660 2024-09-19T07:36:38.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.158+0000 7f0453fff640 1 -- 172.21.15.103:0/2432530619 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264057 (secure 0 0 0) 0x7f04600bc410 con 0x7f046c103660 2024-09-19T07:36:38.162 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:38.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.160+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04540a32d0 msgr2=0x7f04540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:38.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.160+0000 7f04711fc640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04540a32d0 0x7f04540a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f045c009c20 tx=0x7f045c005f90 comp rx=0 tx=0).stop 2024-09-19T07:36:38.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.160+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 msgr2=0x7f046c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.160+0000 7f04711fc640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c19abb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0460006010 tx=0x7f0460004040 comp rx=0 tx=0).stop 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 shutdown_connections 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04540a32d0 0x7f04540a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f046c10c170 0x7f046c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f046c103660 0x7f046c19abb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 --2- 172.21.15.103:0/2432530619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f046c102d40 0x7f046c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 >> 172.21.15.103:0/2432530619 conn(0x7f046c0fc7a0 msgr2=0x7f046c1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 shutdown_connections 2024-09-19T07:36:38.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:38.161+0000 7f04711fc640 1 -- 172.21.15.103:0/2432530619 wait complete. 2024-09-19T07:36:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:38 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2432530619' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T07:36:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:38 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2432530619' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T07:36:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:38 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2432530619' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T07:36:38.972 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2164654927},{"type":"v1","addr":"172.21.15.103:6801","nonce":2164654927}]},"active_addr":"172.21.15.103:6801/2164654927","active_change":"2024-09-19T07:29:16.000378+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.103:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2266926610}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":189968053}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":532109813}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":3492331771}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":3818938335}]}]} 2024-09-19T07:36:38.979 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T07:36:38.979 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T07:36:38.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:36:39.228 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:39.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:39 smithi103 ceph-mon[31201]: pgmap v295: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T07:36:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:39 smithi134 ceph-mon[35414]: pgmap v295: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T07:36:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:39 smithi174 ceph-mon[35370]: pgmap v295: 97 pgs: 97 active+clean; 633 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 -- 172.21.15.103:0/594846430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 msgr2=0x7fe89410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 --2- 172.21.15.103:0/594846430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe89410d950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe884009cd0 tx=0x7fe88402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 -- 172.21.15.103:0/594846430 shutdown_connections 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 --2- 172.21.15.103:0/594846430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe89410d950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 --2- 172.21.15.103:0/594846430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe894100a80 0x7fe894100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 --2- 172.21.15.103:0/594846430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe894106a80 0x7fe894106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 -- 172.21.15.103:0/594846430 >> 172.21.15.103:0/594846430 conn(0x7fe8940fc7a0 msgr2=0x7fe8940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 -- 172.21.15.103:0/594846430 shutdown_connections 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 -- 172.21.15.103:0/594846430 wait complete. 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.677+0000 7fe89b436640 1 Processor -- start 2024-09-19T07:36:39.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 -- start start 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe894100a80 0x7fe894198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe894198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe894106a80 0x7fe89419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110ec0 con 0x7fe894101420 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe894110d40 con 0x7fe894100a80 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe894111040 con 0x7fe894106a80 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe894198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe894198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40936/0 (socket says 172.21.15.103:40936) 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 -- 172.21.15.103:0/1149893795 learned_addr learned my addr 172.21.15.103:0/1149893795 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8991d2640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe894100a80 0x7fe894198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:39.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 -- 172.21.15.103:0/1149893795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe894106a80 msgr2=0x7fe89419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe894106a80 0x7fe89419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 -- 172.21.15.103:0/1149893795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe894100a80 msgr2=0x7fe894198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe894100a80 0x7fe894198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 -- 172.21.15.103:0/1149893795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89419d4f0 con 0x7fe894101420 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8989d1640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe894198a80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe88800b4a0 tx=0x7fe88800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8927fc640 1 -- 172.21.15.103:0/1149893795 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888014070 con 0x7fe894101420 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89419d7e0 con 0x7fe894101420 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8927fc640 1 -- 172.21.15.103:0/1149893795 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe888010af0 con 0x7fe894101420 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe8927fc640 1 -- 172.21.15.103:0/1149893795 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888017d10 con 0x7fe894101420 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.678+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8941a50c0 con 0x7fe894101420 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.679+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe894100ee0 con 0x7fe894101420 2024-09-19T07:36:39.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.681+0000 7fe8927fc640 1 -- 172.21.15.103:0/1149893795 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe888017070 con 0x7fe894101420 2024-09-19T07:36:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.681+0000 7fe8927fc640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe87c0a32d0 0x7fe87c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.681+0000 7fe8991d2640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe87c0a32d0 0x7fe87c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.681+0000 7fe8927fc640 1 -- 172.21.15.103:0/1149893795 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe8880cf1a0 con 0x7fe894101420 2024-09-19T07:36:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.682+0000 7fe8991d2640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe87c0a32d0 0x7fe87c0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005fd0 tx=0x7fe88c005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.684+0000 7fe8927fc640 1 -- 172.21.15.103:0/1149893795 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe88809be70 con 0x7fe894101420 2024-09-19T07:36:39.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.855+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8941998a0 con 0x7fe894101420 2024-09-19T07:36:39.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.857+0000 7fe8927fc640 1 -- 172.21.15.103:0/1149893795 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22818 (secure 0 0 0) 0x7fe88809b770 con 0x7fe894101420 2024-09-19T07:36:39.861 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:39.861 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:17.882336+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:36:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe87c0a32d0 msgr2=0x7fe87c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe87c0a32d0 0x7fe87c0a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005fd0 tx=0x7fe88c005e20 comp rx=0 tx=0).stop 2024-09-19T07:36:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 msgr2=0x7fe894198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe894198a80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe88800b4a0 tx=0x7fe88800b960 comp rx=0 tx=0).stop 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 shutdown_connections 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe87c0a32d0 0x7fe87c0a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe894106a80 0x7fe89419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe894101420 0x7fe894198a80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 --2- 172.21.15.103:0/1149893795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe894100a80 0x7fe894198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.859+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 >> 172.21.15.103:0/1149893795 conn(0x7fe8940fc7a0 msgr2=0x7fe89406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.860+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 shutdown_connections 2024-09-19T07:36:39.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:39.860+0000 7fe89b436640 1 -- 172.21.15.103:0/1149893795 wait complete. 2024-09-19T07:36:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:40 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1149893795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:40 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1149893795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:40.621 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T07:36:40.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:36:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:40 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1149893795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:40.878 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:41.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.376+0000 7f2c2d870640 1 -- 172.21.15.103:0/388541209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28102ef0 msgr2=0x7f2c2810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.376+0000 7f2c2d870640 1 --2- 172.21.15.103:0/388541209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28102ef0 0x7f2c2810da40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009900 tx=0x7f2c1802a480 comp rx=0 tx=0).stop 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 -- 172.21.15.103:0/388541209 shutdown_connections 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 --2- 172.21.15.103:0/388541209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28102ef0 0x7f2c2810da40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 --2- 172.21.15.103:0/388541209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c28102570 0x7f2c281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 --2- 172.21.15.103:0/388541209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c28100e20 0x7f2c28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 -- 172.21.15.103:0/388541209 >> 172.21.15.103:0/388541209 conn(0x7f2c280fc7d0 msgr2=0x7f2c280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 -- 172.21.15.103:0/388541209 shutdown_connections 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 -- 172.21.15.103:0/388541209 wait complete. 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.377+0000 7f2c2d870640 1 Processor -- start 2024-09-19T07:36:41.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2d870640 1 -- start start 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2d870640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28100e20 0x7f2c28197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2d870640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c28102570 0x7f2c281978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2d870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c28102ef0 0x7f2c2819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2d870640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28110f80 con 0x7f2c28100e20 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2d870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28110e00 con 0x7f2c28102ef0 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2d870640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28111100 con 0x7f2c28102570 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28100e20 0x7f2c28197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28100e20 0x7f2c28197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40966/0 (socket says 172.21.15.103:40966) 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 -- 172.21.15.103:0/3986673519 learned_addr learned my addr 172.21.15.103:0/3986673519 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c267fc640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c28102570 0x7f2c281978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:41.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c277fe640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c28102ef0 0x7f2c2819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 -- 172.21.15.103:0/3986673519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c28102570 msgr2=0x7f2c281978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c28102570 0x7f2c281978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 -- 172.21.15.103:0/3986673519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c28102ef0 msgr2=0x7f2c2819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c28102ef0 0x7f2c2819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 -- 172.21.15.103:0/3986673519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2819c330 con 0x7f2c28100e20 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c267fc640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c28102570 0x7f2c281978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c26ffd640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28100e20 0x7f2c28197380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000eab0 tx=0x7f2c2000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2c86e640 1 -- 172.21.15.103:0/3986673519 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000ce40 con 0x7f2c28100e20 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.378+0000 7f2c2c86e640 1 -- 172.21.15.103:0/3986673519 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c20011950 con 0x7f2c28100e20 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.379+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2819c620 con 0x7f2c28100e20 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.379+0000 7f2c2c86e640 1 -- 172.21.15.103:0/3986673519 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20010ae0 con 0x7f2c28100e20 2024-09-19T07:36:41.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.379+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c281a3f00 con 0x7f2c28100e20 2024-09-19T07:36:41.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.379+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c28101200 con 0x7f2c28100e20 2024-09-19T07:36:41.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.381+0000 7f2c2c86e640 1 -- 172.21.15.103:0/3986673519 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2c20011b00 con 0x7f2c28100e20 2024-09-19T07:36:41.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.382+0000 7f2c2c86e640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2c140a32d0 0x7f2c140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:41.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.382+0000 7f2c267fc640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2c140a32d0 0x7f2c140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:41.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.382+0000 7f2c2c86e640 1 -- 172.21.15.103:0/3986673519 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f2c200d0220 con 0x7f2c28100e20 2024-09-19T07:36:41.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.383+0000 7f2c267fc640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2c140a32d0 0x7f2c140a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c280fec10 tx=0x7f2c10005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:41.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.385+0000 7f2c2c86e640 1 -- 172.21.15.103:0/3986673519 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2c20016030 con 0x7f2c28100e20 2024-09-19T07:36:41.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:41 smithi103 ceph-mon[31201]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-19T07:36:41.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.560+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c281a40d0 con 0x7f2c28100e20 2024-09-19T07:36:41.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.561+0000 7f2c2c86e640 1 -- 172.21.15.103:0/3986673519 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22818 (secure 0 0 0) 0x7f2c2009cde0 con 0x7f2c28100e20 2024-09-19T07:36:41.562 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:36:41.563 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:17.882336+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:36:41.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2c140a32d0 msgr2=0x7f2c140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:41.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2c140a32d0 0x7f2c140a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c280fec10 tx=0x7f2c10005d30 comp rx=0 tx=0).stop 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28100e20 msgr2=0x7f2c28197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28100e20 0x7f2c28197380 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000eab0 tx=0x7f2c2000ef70 comp rx=0 tx=0).stop 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 shutdown_connections 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2c140a32d0 0x7f2c140a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c28102ef0 0x7f2c2819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.563+0000 7f2c2d870640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c28102570 0x7f2c281978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.564+0000 7f2c2d870640 1 --2- 172.21.15.103:0/3986673519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c28100e20 0x7f2c28197380 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.564+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 >> 172.21.15.103:0/3986673519 conn(0x7f2c280fc7d0 msgr2=0x7f2c28106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.564+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 shutdown_connections 2024-09-19T07:36:41.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:41.564+0000 7f2c2d870640 1 -- 172.21.15.103:0/3986673519 wait complete. 2024-09-19T07:36:41.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:41 smithi134 ceph-mon[35414]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-19T07:36:41.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:41 smithi174 ceph-mon[35370]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-19T07:36:42.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T07:36:42.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T07:36:42.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T07:36:42.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T07:36:42.289 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T07:36:42.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T07:36:42.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T07:36:42.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T07:36:42.292 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T07:36:42.292 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T07:36:42.292 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T07:36:42.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T07:36:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:42 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3986673519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:42 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3986673519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:42.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:42 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3986673519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:36:42.942 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:42.962 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:43.005 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:43.036 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:43.373 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:43 smithi103 ceph-mon[31201]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-19T07:36:43.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:43 smithi134 ceph-mon[35414]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-19T07:36:43.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:43 smithi174 ceph-mon[35370]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.749+0000 7f01f19a0640 1 -- 172.21.15.103:0/4059771127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 msgr2=0x7f01ec10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.749+0000 7f01f19a0640 1 --2- 172.21.15.103:0/4059771127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 0x7f01ec10e610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009ca0 tx=0x7f01e402a950 comp rx=0 tx=0).stop 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.749+0000 7f01f19a0640 1 -- 172.21.15.103:0/4059771127 shutdown_connections 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.749+0000 7f01f19a0640 1 --2- 172.21.15.103:0/4059771127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 0x7f01ec10e610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.749+0000 7f01f19a0640 1 --2- 172.21.15.103:0/4059771127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01ec103660 0x7f01ec10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.749+0000 7f01f19a0640 1 --2- 172.21.15.103:0/4059771127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec102d40 0x7f01ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.749+0000 7f01f19a0640 1 -- 172.21.15.103:0/4059771127 >> 172.21.15.103:0/4059771127 conn(0x7f01ec0fc7a0 msgr2=0x7f01ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 -- 172.21.15.103:0/4059771127 shutdown_connections 2024-09-19T07:36:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 -- 172.21.15.103:0/4059771127 wait complete. 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 Processor -- start 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 -- start start 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01ec102d40 0x7f01ec1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec103660 0x7f01ec198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 0x7f01ec19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec111fb0 con 0x7f01ec10c170 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec111e30 con 0x7f01ec102d40 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.750+0000 7f01f19a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec112130 con 0x7f01ec103660 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 0x7f01ec19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 0x7f01ec19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40992/0 (socket says 172.21.15.103:40992) 2024-09-19T07:36:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 -- 172.21.15.103:0/2141125356 learned_addr learned my addr 172.21.15.103:0/2141125356 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eaffd640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01ec102d40 0x7f01ec1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 -- 172.21.15.103:0/2141125356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec103660 msgr2=0x7f01ec198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01ea7fc640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec103660 0x7f01ec198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec103660 0x7f01ec198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 -- 172.21.15.103:0/2141125356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01ec102d40 msgr2=0x7f01ec1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01ec102d40 0x7f01ec1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 -- 172.21.15.103:0/2141125356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec19d490 con 0x7f01ec10c170 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01eb7fe640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 0x7f01ec19cdb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006010 tx=0x7f01e402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4034040 con 0x7f01ec10c170 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec19d720 con 0x7f01ec10c170 2024-09-19T07:36:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e4003e00 con 0x7f01ec10c170 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40456e0 con 0x7f01ec10c170 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.751+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec1a5060 con 0x7f01ec10c170 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.752+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01ec062900 con 0x7f01ec10c170 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.753+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f01e4037020 con 0x7f01ec10c170 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.754+0000 7f01f099e640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f01d40a32d0 0x7f01d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.754+0000 7f01eaffd640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f01d40a32d0 0x7f01d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.754+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f01e40f0bf0 con 0x7f01ec10c170 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.754+0000 7f01f099e640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] conn(0x7f01d40acf00 0x7f01d40af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.754+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 --> [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01e4033e50 con 0x7f01d40acf00 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.754+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f01e40f1020 con 0x7f01ec10c170 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.754+0000 7f01ea7fc640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] conn(0x7f01d40acf00 0x7f01d40af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.755+0000 7f01eaffd640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f01d40a32d0 0x7f01d40a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0058c0 tx=0x7f01dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.755+0000 7f01ea7fc640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] conn(0x7f01d40acf00 0x7f01d40af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.755+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== osd.3 v2:172.21.15.103:6810/2879777863 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01e4033e50 con 0x7f01d40acf00 2024-09-19T07:36:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.769+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 --> [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01ec062d80 con 0x7f01d40acf00 2024-09-19T07:36:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.769+0000 7f01f099e640 1 -- 172.21.15.103:0/2141125356 <== osd.3 v2:172.21.15.103:6810/2879777863 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f01ec062d80 con 0x7f01d40acf00 2024-09-19T07:36:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] conn(0x7f01d40acf00 msgr2=0x7f01d40af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] conn(0x7f01d40acf00 0x7f01d40af3a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f01d40a32d0 msgr2=0x7f01d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f01d40a32d0 0x7f01d40a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0058c0 tx=0x7f01dc00a2d0 comp rx=0 tx=0).stop 2024-09-19T07:36:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 msgr2=0x7f01ec19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 --2- 172.21.15.103:0/2141125356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ec10c170 0x7f01ec19cdb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006010 tx=0x7f01e402f710 comp rx=0 tx=0).stop 2024-09-19T07:36:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01eb7fe640 1 -- 172.21.15.103:0/2141125356 reap_dead start 2024-09-19T07:36:43.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 shutdown_connections 2024-09-19T07:36:43.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 >> 172.21.15.103:0/2141125356 conn(0x7f01ec0fc7a0 msgr2=0x7f01ec0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:43.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 shutdown_connections 2024-09-19T07:36:43.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:43.770+0000 7f01f19a0640 1 -- 172.21.15.103:0/2141125356 wait complete. 2024-09-19T07:36:44.002 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:44.002 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:44.002 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:44.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 -- 172.21.15.103:0/4158992844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c102ef0 msgr2=0x7f168c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 --2- 172.21.15.103:0/4158992844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c102ef0 0x7f168c10da40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1684009ca0 tx=0x7f168402a950 comp rx=0 tx=0).stop 2024-09-19T07:36:44.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 -- 172.21.15.103:0/4158992844 shutdown_connections 2024-09-19T07:36:44.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 --2- 172.21.15.103:0/4158992844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c102ef0 0x7f168c10da40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 --2- 172.21.15.103:0/4158992844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f168c102570 0x7f168c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 --2- 172.21.15.103:0/4158992844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f168c100e20 0x7f168c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 -- 172.21.15.103:0/4158992844 >> 172.21.15.103:0/4158992844 conn(0x7f168c0fc7d0 msgr2=0x7f168c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 -- 172.21.15.103:0/4158992844 shutdown_connections 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.738+0000 7f1690c8c640 1 -- 172.21.15.103:0/4158992844 wait complete. 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 Processor -- start 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 -- start start 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c100e20 0x7f168c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f168c102570 0x7f168c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f168c102ef0 0x7f168c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f168c110f80 con 0x7f168c100e20 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f168c110e00 con 0x7f168c102570 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f1690c8c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f168c111100 con 0x7f168c102ef0 2024-09-19T07:36:44.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f168a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c100e20 0x7f168c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f168a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c100e20 0x7f168c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41000/0 (socket says 172.21.15.103:41000) 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.739+0000 7f168a59c640 1 -- 172.21.15.103:0/1354292520 learned_addr learned my addr 172.21.15.103:0/1354292520 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f168ad9d640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f168c102ef0 0x7f168c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f1683fff640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f168c102570 0x7f168c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f168a59c640 1 -- 172.21.15.103:0/1354292520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f168c102ef0 msgr2=0x7f168c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f168a59c640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f168c102ef0 0x7f168c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f168a59c640 1 -- 172.21.15.103:0/1354292520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f168c102570 msgr2=0x7f168c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f168a59c640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f168c102570 0x7f168c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f168a59c640 1 -- 172.21.15.103:0/1354292520 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c19d460 con 0x7f168c100e20 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f1683fff640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f168c102570 0x7f168c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f168a59c640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c100e20 0x7f168c1984b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f167c00d950 tx=0x7f167c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c017030 con 0x7f168c100e20 2024-09-19T07:36:44.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f167c004020 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c19d750 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c0047a0 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.740+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c1a5030 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.741+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f168c062980 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.742+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f167c0041d0 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f16837fe640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16780a32d0 0x7f16780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f1683fff640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16780a32d0 0x7f16780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f167c0cfc90 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f16837fe640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] conn(0x7f16780acf00 0x7f16780af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 --> [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f167c01de50 con 0x7f16780acf00 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f167c0d00c0 con 0x7f168c100e20 2024-09-19T07:36:44.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f168ad9d640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] conn(0x7f16780acf00 0x7f16780af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f1683fff640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16780a32d0 0x7f16780a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f168c1051c0 tx=0x7f1674005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:44.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.743+0000 7f168ad9d640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] conn(0x7f16780acf00 0x7f16780af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:44.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.744+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== osd.6 v2:172.21.15.103:6818/3692913756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f167c01de50 con 0x7f16780acf00 2024-09-19T07:36:44.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.757+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 --> [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f168c062e00 con 0x7f16780acf00 2024-09-19T07:36:44.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f16837fe640 1 -- 172.21.15.103:0/1354292520 <== osd.6 v2:172.21.15.103:6818/3692913756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f168c062e00 con 0x7f16780acf00 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] conn(0x7f16780acf00 msgr2=0x7f16780af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] conn(0x7f16780acf00 0x7f16780af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16780a32d0 msgr2=0x7f16780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16780a32d0 0x7f16780a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f168c1051c0 tx=0x7f1674005cb0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c100e20 msgr2=0x7f168c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 --2- 172.21.15.103:0/1354292520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f168c100e20 0x7f168c1984b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f167c00d950 tx=0x7f167c00de10 comp rx=0 tx=0).stop 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f168ad9d640 1 -- 172.21.15.103:0/1354292520 reap_dead start 2024-09-19T07:36:44.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 shutdown_connections 2024-09-19T07:36:44.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 >> 172.21.15.103:0/1354292520 conn(0x7f168c0fc7d0 msgr2=0x7f168c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:44.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 shutdown_connections 2024-09-19T07:36:44.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.758+0000 7f1690c8c640 1 -- 172.21.15.103:0/1354292520 wait complete. 2024-09-19T07:36:44.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 -- 172.21.15.103:0/2257294733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740069f00 msgr2=0x7f4740100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 --2- 172.21.15.103:0/2257294733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740069f00 0x7f4740100480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4730009900 tx=0x7f473002a410 comp rx=0 tx=0).stop 2024-09-19T07:36:44.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 -- 172.21.15.103:0/2257294733 shutdown_connections 2024-09-19T07:36:44.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 --2- 172.21.15.103:0/2257294733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4740104e80 0x7f4740105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 --2- 172.21.15.103:0/2257294733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474006a370 0x7f47401009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 --2- 172.21.15.103:0/2257294733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740069f00 0x7f4740100480 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 -- 172.21.15.103:0/2257294733 >> 172.21.15.103:0/2257294733 conn(0x7f4740077210 msgr2=0x7f47400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 -- 172.21.15.103:0/2257294733 shutdown_connections 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.933+0000 7f47449e0640 1 -- 172.21.15.103:0/2257294733 wait complete. 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 Processor -- start 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 -- start start 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4740069f00 0x7f4740102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474006a370 0x7f4740103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740104e80 0x7f4740103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4740110eb0 con 0x7f4740104e80 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4740110d30 con 0x7f474006a370 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f47449e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4740111030 con 0x7f4740069f00 2024-09-19T07:36:44.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f473ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740104e80 0x7f4740103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f473ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740104e80 0x7f4740103560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41012/0 (socket says 172.21.15.103:41012) 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.934+0000 7f473ed9d640 1 -- 172.21.15.103:0/1516033182 learned_addr learned my addr 172.21.15.103:0/1516033182 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473dd9b640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474006a370 0x7f4740103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473e59c640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4740069f00 0x7f4740102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473ed9d640 1 -- 172.21.15.103:0/1516033182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4740069f00 msgr2=0x7f4740102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473ed9d640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4740069f00 0x7f4740102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473ed9d640 1 -- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474006a370 msgr2=0x7f4740103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473ed9d640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f474006a370 0x7f4740103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473ed9d640 1 -- 172.21.15.103:0/1516033182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47401a6990 con 0x7f4740104e80 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f473ed9d640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740104e80 0x7f4740103560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f473800b970 tx=0x7f473800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738019070 con 0x7f4740104e80 2024-09-19T07:36:44.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473800cb40 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738007650 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47401a6c80 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.935+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47401a7210 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.936+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47400fcf20 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.937+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f473801c070 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.938+0000 7f472f7fe640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f47280a3350 0x7f47280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.938+0000 7f473e59c640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f47280a3350 0x7f47280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.938+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f47380cf6a0 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.938+0000 7f472f7fe640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] conn(0x7f47280acf80 0x7f47280af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.938+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 --> [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47380047d0 con 0x7f47280acf80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.938+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f47380cfad0 con 0x7f4740104e80 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.938+0000 7f473e59c640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f47280a3350 0x7f47280a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4730005ab0 tx=0x7f4730005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:44.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.939+0000 7f473dd9b640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] conn(0x7f47280acf80 0x7f47280af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.939+0000 7f473dd9b640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] conn(0x7f47280acf80 0x7f47280af420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.940+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== osd.10 v2:172.21.15.134:6824/984759613 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47380047d0 con 0x7f47280acf80 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.953+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 --> [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4740101960 con 0x7f47280acf80 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f472f7fe640 1 -- 172.21.15.103:0/1516033182 <== osd.10 v2:172.21.15.134:6824/984759613 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4740101960 con 0x7f47280acf80 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] conn(0x7f47280acf80 msgr2=0x7f47280af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] conn(0x7f47280acf80 0x7f47280af420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f47280a3350 msgr2=0x7f47280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f47280a3350 0x7f47280a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4730005ab0 tx=0x7f4730005a20 comp rx=0 tx=0).stop 2024-09-19T07:36:44.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740104e80 msgr2=0x7f4740103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:44.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 --2- 172.21.15.103:0/1516033182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4740104e80 0x7f4740103560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f473800b970 tx=0x7f473800be30 comp rx=0 tx=0).stop 2024-09-19T07:36:44.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f473ed9d640 1 -- 172.21.15.103:0/1516033182 reap_dead start 2024-09-19T07:36:44.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 shutdown_connections 2024-09-19T07:36:44.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 >> 172.21.15.103:0/1516033182 conn(0x7f4740077210 msgr2=0x7f47400fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:44.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 shutdown_connections 2024-09-19T07:36:44.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:44.954+0000 7f47449e0640 1 -- 172.21.15.103:0/1516033182 wait complete. 2024-09-19T07:36:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:45 smithi134 ceph-mon[35414]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T07:36:45.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:45 smithi174 ceph-mon[35370]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T07:36:45.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:45 smithi103 ceph-mon[31201]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-09-19T07:36:45.746 INFO:teuthology.orchestra.run.smithi103.stdout:296352743450 2024-09-19T07:36:45.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T07:36:45.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 -- 172.21.15.103:0/1582002534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d0101df0 msgr2=0x7f16d01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:45.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 --2- 172.21.15.103:0/1582002534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d0101df0 0x7f16d01021d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009900 tx=0x7f16cc02a410 comp rx=0 tx=0).stop 2024-09-19T07:36:45.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 -- 172.21.15.103:0/1582002534 shutdown_connections 2024-09-19T07:36:45.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 --2- 172.21.15.103:0/1582002534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16d0106f20 0x7f16d010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:45.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 --2- 172.21.15.103:0/1582002534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d01027a0 0x7f16d01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 --2- 172.21.15.103:0/1582002534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d0101df0 0x7f16d01021d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 -- 172.21.15.103:0/1582002534 >> 172.21.15.103:0/1582002534 conn(0x7f16d00fd8f0 msgr2=0x7f16d00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 -- 172.21.15.103:0/1582002534 shutdown_connections 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.927+0000 7f16d7664640 1 -- 172.21.15.103:0/1582002534 wait complete. 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 Processor -- start 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 -- start start 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16d0101df0 0x7f16d0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d01027a0 0x7f16d01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0106f20 0x7f16d019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0111f80 con 0x7f16d01027a0 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0111e00 con 0x7f16d0106f20 2024-09-19T07:36:45.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d7664640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0112100 con 0x7f16d0101df0 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d01027a0 0x7f16d01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d01027a0 0x7f16d01989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41028/0 (socket says 172.21.15.103:41028) 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 -- 172.21.15.103:0/4091425294 learned_addr learned my addr 172.21.15.103:0/4091425294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d5400640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16d0101df0 0x7f16d0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d5c01640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0106f20 0x7f16d019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 -- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16d0101df0 msgr2=0x7f16d0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16d0101df0 0x7f16d0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 -- 172.21.15.103:0/4091425294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0106f20 msgr2=0x7f16d019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16d0106f20 0x7f16d019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:45.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.928+0000 7f16d4bff640 1 -- 172.21.15.103:0/4091425294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d019d410 con 0x7f16d01027a0 2024-09-19T07:36:45.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.929+0000 7f16d4bff640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d01027a0 0x7f16d01989a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f16c000d950 tx=0x7f16c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:45.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.929+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0017070 con 0x7f16d01027a0 2024-09-19T07:36:45.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.929+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c0009c30 con 0x7f16d01027a0 2024-09-19T07:36:45.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.929+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c000f6b0 con 0x7f16d01027a0 2024-09-19T07:36:45.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.929+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d019d700 con 0x7f16d01027a0 2024-09-19T07:36:45.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.929+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d0074420 con 0x7f16d01027a0 2024-09-19T07:36:45.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.930+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16d0062980 con 0x7f16d01027a0 2024-09-19T07:36:45.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.931+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f16c00109d0 con 0x7f16d01027a0 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16c67fc640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16bc0a32d0 0x7f16bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f16c00cf6c0 con 0x7f16d01027a0 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16d5400640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16bc0a32d0 0x7f16bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16c67fc640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] conn(0x7f16bc0acf00 0x7f16bc0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 --> [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16c0004470 con 0x7f16bc0acf00 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f16c00d6050 con 0x7f16d01027a0 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16d5400640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16bc0a32d0 0x7f16bc0a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009230 tx=0x7f16cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:45.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.932+0000 7f16d5c01640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] conn(0x7f16bc0acf00 0x7f16bc0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:45.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.933+0000 7f16d5c01640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] conn(0x7f16bc0acf00 0x7f16bc0af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:45.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.934+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== osd.5 v2:172.21.15.174:6808/2281357008 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16c0004470 con 0x7f16bc0acf00 2024-09-19T07:36:45.950 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:45.951 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:45.951 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:45.957 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:45.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.954+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 --> [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16d0062e00 con 0x7f16bc0acf00 2024-09-19T07:36:45.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.954+0000 7f16c67fc640 1 -- 172.21.15.103:0/4091425294 <== osd.5 v2:172.21.15.174:6808/2281357008 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f16d0062e00 con 0x7f16bc0acf00 2024-09-19T07:36:45.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.955+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] conn(0x7f16bc0acf00 msgr2=0x7f16bc0af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:45.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d7664640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] conn(0x7f16bc0acf00 0x7f16bc0af3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:45.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16bc0a32d0 msgr2=0x7f16bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:45.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d7664640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f16bc0a32d0 0x7f16bc0a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009230 tx=0x7f16cc0058d0 comp rx=0 tx=0).stop 2024-09-19T07:36:45.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d01027a0 msgr2=0x7f16d01989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:45.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d7664640 1 --2- 172.21.15.103:0/4091425294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16d01027a0 0x7f16d01989a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f16c000d950 tx=0x7f16c000de10 comp rx=0 tx=0).stop 2024-09-19T07:36:45.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d5c01640 1 -- 172.21.15.103:0/4091425294 reap_dead start 2024-09-19T07:36:45.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 shutdown_connections 2024-09-19T07:36:45.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.956+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 >> 172.21.15.103:0/4091425294 conn(0x7f16d00fd8f0 msgr2=0x7f16d00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:45.959 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:45.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.957+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 shutdown_connections 2024-09-19T07:36:45.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:45.958+0000 7f16d7664640 1 -- 172.21.15.103:0/4091425294 wait complete. 2024-09-19T07:36:47.058 INFO:teuthology.orchestra.run.smithi103.stdout:107374182460 2024-09-19T07:36:47.058 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T07:36:47.433 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:47 smithi103 ceph-mon[31201]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:47.459 INFO:teuthology.orchestra.run.smithi103.stdout:163208757299 2024-09-19T07:36:47.460 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T07:36:47.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:47 smithi134 ceph-mon[35414]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:47.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:47 smithi174 ceph-mon[35370]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:47.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.643+0000 7fd9ba25d640 1 -- 172.21.15.103:0/3352780488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4101de0 msgr2=0x7fd9b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:47.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.643+0000 7fd9ba25d640 1 --2- 172.21.15.103:0/3352780488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4101de0 0x7fd9b41021c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009900 tx=0x7fd9a402a410 comp rx=0 tx=0).stop 2024-09-19T07:36:47.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.643+0000 7fd9ba25d640 1 -- 172.21.15.103:0/3352780488 shutdown_connections 2024-09-19T07:36:47.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.643+0000 7fd9ba25d640 1 --2- 172.21.15.103:0/3352780488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4106f10 0x7fd9b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:47.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.643+0000 7fd9ba25d640 1 --2- 172.21.15.103:0/3352780488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b4102790 0x7fd9b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:47.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.643+0000 7fd9ba25d640 1 --2- 172.21.15.103:0/3352780488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4101de0 0x7fd9b41021c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.643+0000 7fd9ba25d640 1 -- 172.21.15.103:0/3352780488 >> 172.21.15.103:0/3352780488 conn(0x7fd9b40fd880 msgr2=0x7fd9b40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.644+0000 7fd9ba25d640 1 -- 172.21.15.103:0/3352780488 shutdown_connections 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.644+0000 7fd9ba25d640 1 -- 172.21.15.103:0/3352780488 wait complete. 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.644+0000 7fd9ba25d640 1 Processor -- start 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.644+0000 7fd9ba25d640 1 -- start start 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9ba25d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101de0 0x7fd9b4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9ba25d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b4102790 0x7fd9b4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9ba25d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4106f10 0x7fd9b419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9ba25d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4111f70 con 0x7fd9b4106f10 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9ba25d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4111df0 con 0x7fd9b4101de0 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9ba25d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41120f0 con 0x7fd9b4102790 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4106f10 0x7fd9b419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:47.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4106f10 0x7fd9b419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41044/0 (socket says 172.21.15.103:41044) 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 -- 172.21.15.103:0/1754916914 learned_addr learned my addr 172.21.15.103:0/1754916914 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b2ffd640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b4102790 0x7fd9b4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b37fe640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101de0 0x7fd9b4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 -- 172.21.15.103:0/1754916914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b4102790 msgr2=0x7fd9b4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b4102790 0x7fd9b4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 -- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101de0 msgr2=0x7fd9b4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101de0 0x7fd9b4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 -- 172.21.15.103:0/1754916914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b419d3c0 con 0x7fd9b4106f10 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b37fe640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b4101de0 0x7fd9b4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.645+0000 7fd9b3fff640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4106f10 0x7fd9b419cce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00eaa0 tx=0x7fd9ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.646+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00cdb0 con 0x7fd9b4106f10 2024-09-19T07:36:47.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.646+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac005020 con 0x7fd9b4106f10 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.646+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b419d6b0 con 0x7fd9b4106f10 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.646+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0116b0 con 0x7fd9b4106f10 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.646+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41a4f90 con 0x7fd9b4106f10 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.647+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd9b4062940 con 0x7fd9b4106f10 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.648+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd9ac005560 con 0x7fd9b4106f10 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b0ff9640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd99c0a3350 0x7fd99c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b37fe640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd99c0a3350 0x7fd99c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd9ac0d01b0 con 0x7fd9b4106f10 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b0ff9640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] conn(0x7fd99c0acf80 0x7fd99c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 --> [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9ac01ee50 con 0x7fd99c0acf80 2024-09-19T07:36:47.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac0d05e0 con 0x7fd9b4106f10 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b37fe640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd99c0a3350 0x7fd99c0a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4000c00 tx=0x7fd9a4032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.649+0000 7fd9b2ffd640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] conn(0x7fd99c0acf80 0x7fd99c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.650+0000 7fd9b2ffd640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] conn(0x7fd99c0acf80 0x7fd99c0af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.650+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== osd.7 v2:172.21.15.134:6816/2397823373 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9ac01ee50 con 0x7fd99c0acf80 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.664+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 --> [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd9b4062d40 con 0x7fd99c0acf80 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.664+0000 7fd9b0ff9640 1 -- 172.21.15.103:0/1754916914 <== osd.7 v2:172.21.15.134:6816/2397823373 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd9b4062d40 con 0x7fd99c0acf80 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] conn(0x7fd99c0acf80 msgr2=0x7fd99c0af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] conn(0x7fd99c0acf80 0x7fd99c0af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd99c0a3350 msgr2=0x7fd99c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd99c0a3350 0x7fd99c0a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4000c00 tx=0x7fd9a4032c70 comp rx=0 tx=0).stop 2024-09-19T07:36:47.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4106f10 msgr2=0x7fd9b419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:47.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 --2- 172.21.15.103:0/1754916914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4106f10 0x7fd9b419cce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00eaa0 tx=0x7fd9ac00ef60 comp rx=0 tx=0).stop 2024-09-19T07:36:47.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9b3fff640 1 -- 172.21.15.103:0/1754916914 reap_dead start 2024-09-19T07:36:47.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 shutdown_connections 2024-09-19T07:36:47.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 >> 172.21.15.103:0/1754916914 conn(0x7fd9b40fd880 msgr2=0x7fd9b40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:47.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 shutdown_connections 2024-09-19T07:36:47.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:47.665+0000 7fd9ba25d640 1 -- 172.21.15.103:0/1754916914 wait complete. 2024-09-19T07:36:48.787 INFO:teuthology.orchestra.run.smithi103.stdout:193273528366 2024-09-19T07:36:48.788 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 -- 172.21.15.103:0/1497313482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c102790 msgr2=0x7fe43c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 --2- 172.21.15.103:0/1497313482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c102790 0x7fe43c1067e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009900 tx=0x7fe42c02a410 comp rx=0 tx=0).stop 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 -- 172.21.15.103:0/1497313482 shutdown_connections 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 --2- 172.21.15.103:0/1497313482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe43c106f10 0x7fe43c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 --2- 172.21.15.103:0/1497313482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c102790 0x7fe43c1067e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 --2- 172.21.15.103:0/1497313482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe43c101de0 0x7fe43c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 -- 172.21.15.103:0/1497313482 >> 172.21.15.103:0/1497313482 conn(0x7fe43c0fd880 msgr2=0x7fe43c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 -- 172.21.15.103:0/1497313482 shutdown_connections 2024-09-19T07:36:48.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 -- 172.21.15.103:0/1497313482 wait complete. 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 Processor -- start 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.926+0000 7fe443659640 1 -- start start 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe443659640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe43c101de0 0x7fe43c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe443659640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe43c102790 0x7fe43c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe443659640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c106f10 0x7fe43c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe443659640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c111f70 con 0x7fe43c106f10 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe443659640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c111df0 con 0x7fe43c101de0 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe443659640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c1120f0 con 0x7fe43c102790 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c106f10 0x7fe43c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c106f10 0x7fe43c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42430/0 (socket says 172.21.15.103:42430) 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 -- 172.21.15.103:0/366592416 learned_addr learned my addr 172.21.15.103:0/366592416 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe440bf4640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe43c102790 0x7fe43c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:48.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe4413f5640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe43c101de0 0x7fe43c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 -- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe43c102790 msgr2=0x7fe43c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe43c102790 0x7fe43c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 -- 172.21.15.103:0/366592416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe43c101de0 msgr2=0x7fe43c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe43c101de0 0x7fe43c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 -- 172.21.15.103:0/366592416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c19d3c0 con 0x7fe43c106f10 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe441bf6640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c106f10 0x7fe43c19cce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe43400eaa0 tx=0x7fe43400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43400cdb0 con 0x7fe43c106f10 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.927+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c19d6b0 con 0x7fe43c106f10 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.928+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe434005020 con 0x7fe43c106f10 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.928+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340116d0 con 0x7fe43c106f10 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.928+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c1a4f90 con 0x7fe43c106f10 2024-09-19T07:36:48.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.928+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe43c062980 con 0x7fe43c106f10 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.930+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe4340054d0 con 0x7fe43c106f10 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.930+0000 7fe4327fc640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4280a32d0 0x7fe4280a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.931+0000 7fe4413f5640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4280a32d0 0x7fe4280a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.931+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe4340cff50 con 0x7fe43c106f10 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.931+0000 7fe4327fc640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] conn(0x7fe4280acf00 0x7fe4280af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.931+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 --> [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe434014ec0 con 0x7fe4280acf00 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.931+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe4340d0380 con 0x7fe43c106f10 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.931+0000 7fe4413f5640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4280a32d0 0x7fe4280a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe438005fd0 tx=0x7fe438005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.931+0000 7fe440bf4640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] conn(0x7fe4280acf00 0x7fe4280af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.932+0000 7fe440bf4640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] conn(0x7fe4280acf00 0x7fe4280af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:48.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.932+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== osd.2 v2:172.21.15.174:6800/1513938376 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe434014ec0 con 0x7fe4280acf00 2024-09-19T07:36:48.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.946+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 --> [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe43c062e30 con 0x7fe4280acf00 2024-09-19T07:36:48.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.946+0000 7fe4327fc640 1 -- 172.21.15.103:0/366592416 <== osd.2 v2:172.21.15.174:6800/1513938376 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe43c062e30 con 0x7fe4280acf00 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] conn(0x7fe4280acf00 msgr2=0x7fe4280af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] conn(0x7fe4280acf00 0x7fe4280af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4280a32d0 msgr2=0x7fe4280a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4280a32d0 0x7fe4280a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe438005fd0 tx=0x7fe438005850 comp rx=0 tx=0).stop 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c106f10 msgr2=0x7fe43c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 --2- 172.21.15.103:0/366592416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe43c106f10 0x7fe43c19cce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe43400eaa0 tx=0x7fe43400ef60 comp rx=0 tx=0).stop 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe441bf6640 1 -- 172.21.15.103:0/366592416 reap_dead start 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 shutdown_connections 2024-09-19T07:36:48.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 >> 172.21.15.103:0/366592416 conn(0x7fe43c0fd880 msgr2=0x7fe43c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 shutdown_connections 2024-09-19T07:36:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:48.947+0000 7fe443659640 1 -- 172.21.15.103:0/366592416 wait complete. 2024-09-19T07:36:49.500 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:49 smithi103 ceph-mon[31201]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:49.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:49 smithi134 ceph-mon[35414]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:49.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:49 smithi174 ceph-mon[35370]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:49.931 INFO:teuthology.orchestra.run.smithi103.stdout:77309411395 2024-09-19T07:36:49.931 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T07:36:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 -- 172.21.15.103:0/2237044942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 msgr2=0x7f7394101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 --2- 172.21.15.103:0/2237044942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 0x7f7394101200 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7390009900 tx=0x7f739002a480 comp rx=0 tx=0).stop 2024-09-19T07:36:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 -- 172.21.15.103:0/2237044942 shutdown_connections 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 --2- 172.21.15.103:0/2237044942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7394102ef0 0x7f739410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 --2- 172.21.15.103:0/2237044942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7394102570 0x7f73941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 --2- 172.21.15.103:0/2237044942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 0x7f7394101200 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 -- 172.21.15.103:0/2237044942 >> 172.21.15.103:0/2237044942 conn(0x7f73940fc7d0 msgr2=0x7f73940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.076+0000 7f739c38e640 1 -- 172.21.15.103:0/2237044942 shutdown_connections 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.077+0000 7f739c38e640 1 -- 172.21.15.103:0/2237044942 wait complete. 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.077+0000 7f739c38e640 1 Processor -- start 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.077+0000 7f739c38e640 1 -- start start 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739c38e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 0x7f73941973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739c38e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7394102570 0x7f7394197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739c38e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7394102ef0 0x7f739419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739c38e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7394110fa0 con 0x7f7394100e20 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739c38e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7394110e20 con 0x7f7394102ef0 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739c38e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7394111120 con 0x7f7394102570 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 0x7f73941973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 0x7f73941973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42460/0 (socket says 172.21.15.103:42460) 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 -- 172.21.15.103:0/1258865459 learned_addr learned my addr 172.21.15.103:0/1258865459 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a92b640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7394102ef0 0x7f739419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 -- 172.21.15.103:0/1258865459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7394102570 msgr2=0x7f7394197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7394102570 0x7f7394197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 -- 172.21.15.103:0/1258865459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7394102ef0 msgr2=0x7f739419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7394102ef0 0x7f739419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 -- 172.21.15.103:0/1258865459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739419c370 con 0x7f7394100e20 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a92b640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7394102ef0 0x7f739419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739a12a640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 0x7f73941973c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7390006010 tx=0x7f7390003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739003f070 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.078+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739419c600 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.079+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7390031e30 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.079+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390035030 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.079+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a3f40 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.080+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73941072a0 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.081+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7390042070 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.081+0000 7f738b7fe640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f73800a32d0 0x7f73800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.082+0000 7f7399929640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f73800a32d0 0x7f73800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.082+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f73900f04a0 con 0x7f7394100e20 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.082+0000 7f738b7fe640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] conn(0x7f73800acf00 0x7f73800af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.082+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 --> [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f739002fec0 con 0x7f73800acf00 2024-09-19T07:36:50.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.082+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f73900f7050 con 0x7f7394100e20 2024-09-19T07:36:50.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.082+0000 7f7399929640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f73800a32d0 0x7f73800a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73940fec10 tx=0x7f7384005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:50.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.082+0000 7f739a92b640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] conn(0x7f73800acf00 0x7f73800af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:50.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.083+0000 7f739a92b640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] conn(0x7f73800acf00 0x7f73800af3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:50.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.083+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== osd.11 v2:172.21.15.174:6824/1686831932 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f739002fec0 con 0x7f73800acf00 2024-09-19T07:36:50.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.097+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 --> [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f73941021b0 con 0x7f73800acf00 2024-09-19T07:36:50.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f738b7fe640 1 -- 172.21.15.103:0/1258865459 <== osd.11 v2:172.21.15.174:6824/1686831932 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f73941021b0 con 0x7f73800acf00 2024-09-19T07:36:50.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 >> [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] conn(0x7f73800acf00 msgr2=0x7f73800af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:50.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] conn(0x7f73800acf00 0x7f73800af3a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:50.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f73800a32d0 msgr2=0x7f73800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:50.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f73800a32d0 0x7f73800a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73940fec10 tx=0x7f7384005c70 comp rx=0 tx=0).stop 2024-09-19T07:36:50.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 msgr2=0x7f73941973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:50.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 --2- 172.21.15.103:0/1258865459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7394100e20 0x7f73941973c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7390006010 tx=0x7f7390003c80 comp rx=0 tx=0).stop 2024-09-19T07:36:50.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739a92b640 1 -- 172.21.15.103:0/1258865459 reap_dead start 2024-09-19T07:36:50.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 shutdown_connections 2024-09-19T07:36:50.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.098+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 >> 172.21.15.103:0/1258865459 conn(0x7f73940fc7d0 msgr2=0x7f7394106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:50.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.099+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 shutdown_connections 2024-09-19T07:36:50.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:50.099+0000 7f739c38e640 1 -- 172.21.15.103:0/1258865459 wait complete. 2024-09-19T07:36:50.760 INFO:teuthology.orchestra.run.smithi103.stdout:326417514517 2024-09-19T07:36:50.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T07:36:51.277 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:51 smithi103 ceph-mon[31201]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 -- 172.21.15.103:0/1343878701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe048101df0 msgr2=0x7fe0481021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 --2- 172.21.15.103:0/1343878701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe048101df0 0x7fe0481021d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe044009900 tx=0x7fe04402a410 comp rx=0 tx=0).stop 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 -- 172.21.15.103:0/1343878701 shutdown_connections 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 --2- 172.21.15.103:0/1343878701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe048106f20 0x7fe04810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 --2- 172.21.15.103:0/1343878701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0481027a0 0x7fe0481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 --2- 172.21.15.103:0/1343878701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe048101df0 0x7fe0481021d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 -- 172.21.15.103:0/1343878701 >> 172.21.15.103:0/1343878701 conn(0x7fe0480fd8b0 msgr2=0x7fe0480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 -- 172.21.15.103:0/1343878701 shutdown_connections 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.463+0000 7fe04f65d640 1 -- 172.21.15.103:0/1343878701 wait complete. 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 Processor -- start 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 -- start start 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048101df0 0x7fe048198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0481027a0 0x7fe048198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe048106f20 0x7fe04819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe048111f40 con 0x7fe0481027a0 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe048111dc0 con 0x7fe048101df0 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04f65d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0481120c0 con 0x7fe048106f20 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04cbf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0481027a0 0x7fe048198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04cbf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0481027a0 0x7fe048198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42476/0 (socket says 172.21.15.103:42476) 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04cbf8640 1 -- 172.21.15.103:0/701187326 learned_addr learned my addr 172.21.15.103:0/701187326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.464+0000 7fe04d3f9640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048101df0 0x7fe048198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04dbfa640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe048106f20 0x7fe04819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04cbf8640 1 -- 172.21.15.103:0/701187326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe048106f20 msgr2=0x7fe04819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04cbf8640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe048106f20 0x7fe04819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04cbf8640 1 -- 172.21.15.103:0/701187326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048101df0 msgr2=0x7fe048198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04cbf8640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048101df0 0x7fe048198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04cbf8640 1 -- 172.21.15.103:0/701187326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04819d3c0 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04cbf8640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0481027a0 0x7fe048198950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c920 tx=0x7fe03800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038007c70 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04819d6b0 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe038007e10 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038018890 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.465+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0481a4f90 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.466+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe048062940 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.467+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe038018a30 con 0x7fe0481027a0 2024-09-19T07:36:51.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe03e7fc640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0340a32d0 0x7fe0340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe04d3f9640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0340a32d0 0x7fe0340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe0380d0030 con 0x7fe0481027a0 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe03e7fc640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] conn(0x7fe0340acf00 0x7fe0340af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 --> [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0340afa70 con 0x7fe0340acf00 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe0380d0460 con 0x7fe0481027a0 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe04dbfa640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] conn(0x7fe0340acf00 0x7fe0340af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe04d3f9640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0340a32d0 0x7fe0340a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe044000c00 tx=0x7fe0440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.468+0000 7fe04dbfa640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] conn(0x7fe0340acf00 0x7fe0340af3a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.469+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== osd.0 v2:172.21.15.103:6802/81836251 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0340afa70 con 0x7fe0340acf00 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.482+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 --> [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe048062d40 con 0x7fe0340acf00 2024-09-19T07:36:51.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe03e7fc640 1 -- 172.21.15.103:0/701187326 <== osd.0 v2:172.21.15.103:6802/81836251 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe048062d40 con 0x7fe0340acf00 2024-09-19T07:36:51.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] conn(0x7fe0340acf00 msgr2=0x7fe0340af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] conn(0x7fe0340acf00 0x7fe0340af3a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0340a32d0 msgr2=0x7fe0340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0340a32d0 0x7fe0340a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe044000c00 tx=0x7fe0440058d0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0481027a0 msgr2=0x7fe048198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 --2- 172.21.15.103:0/701187326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0481027a0 0x7fe048198950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c920 tx=0x7fe03800cde0 comp rx=0 tx=0).stop 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04dbfa640 1 -- 172.21.15.103:0/701187326 reap_dead start 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 shutdown_connections 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 >> 172.21.15.103:0/701187326 conn(0x7fe0480fd8b0 msgr2=0x7fe0480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 shutdown_connections 2024-09-19T07:36:51.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:51.483+0000 7fe04f65d640 1 -- 172.21.15.103:0/701187326 wait complete. 2024-09-19T07:36:51.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:51 smithi134 ceph-mon[35414]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:51.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:51 smithi174 ceph-mon[35370]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-19T07:36:52.379 INFO:teuthology.orchestra.run.smithi103.stdout:214748364841 2024-09-19T07:36:52.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 -- 172.21.15.103:0/87973848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8102790 msgr2=0x7f7ad81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 --2- 172.21.15.103:0/87973848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8102790 0x7f7ad81067e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009900 tx=0x7f7ac402a410 comp rx=0 tx=0).stop 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 -- 172.21.15.103:0/87973848 shutdown_connections 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 --2- 172.21.15.103:0/87973848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad8106f10 0x7f7ad810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 --2- 172.21.15.103:0/87973848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8102790 0x7f7ad81067e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 --2- 172.21.15.103:0/87973848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ad8101de0 0x7f7ad81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 -- 172.21.15.103:0/87973848 >> 172.21.15.103:0/87973848 conn(0x7f7ad80fd8c0 msgr2=0x7f7ad80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 -- 172.21.15.103:0/87973848 shutdown_connections 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.832+0000 7f7adee21640 1 -- 172.21.15.103:0/87973848 wait complete. 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 Processor -- start 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 -- start start 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ad8101de0 0x7f7ad80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad8102790 0x7f7ad8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8106f10 0x7f7ad8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8111f90 con 0x7f7ad8106f10 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8111e10 con 0x7f7ad8102790 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adee21640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8112110 con 0x7f7ad8101de0 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8106f10 0x7f7ad8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8106f10 0x7f7ad8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42486/0 (socket says 172.21.15.103:42486) 2024-09-19T07:36:52.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 -- 172.21.15.103:0/15400815 learned_addr learned my addr 172.21.15.103:0/15400815 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7adcbbd640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ad8101de0 0x7f7ad80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 -- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ad8101de0 msgr2=0x7f7ad80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ad8101de0 0x7f7ad80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7ad7fff640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad8102790 0x7f7ad8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 -- 172.21.15.103:0/15400815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad8102790 msgr2=0x7f7ad8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad8102790 0x7f7ad8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 -- 172.21.15.103:0/15400815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad81a6950 con 0x7f7ad8106f10 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7ad7fff640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad8102790 0x7f7ad8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.833+0000 7f7add3be640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8106f10 0x7f7ad8074f70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000eaa0 tx=0x7f7ad000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.834+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000cdb0 con 0x7f7ad8106f10 2024-09-19T07:36:52.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.834+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad0005020 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.834+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad81a6c40 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.834+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00116b0 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.834+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad81a7180 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.835+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ad80628c0 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.836+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7ad0005560 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.836+0000 7f7ad5ffb640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7ac80a3350 0x7f7ac80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.836+0000 7f7adcbbd640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7ac80a3350 0x7f7ac80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.837+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7ad00cfd40 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.837+0000 7f7ad5ffb640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] conn(0x7f7ac80acf80 0x7f7ac80af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.837+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 --> [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ad0018ec0 con 0x7f7ac80acf80 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.837+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f7ad00d23e0 con 0x7f7ad8106f10 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.837+0000 7f7adcbbd640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7ac80a3350 0x7f7ac80a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00a810 tx=0x7f7acc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:52.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.837+0000 7f7ad7fff640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] conn(0x7f7ac80acf80 0x7f7ac80af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.837+0000 7f7ad7fff640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] conn(0x7f7ac80acf80 0x7f7ac80af420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.838+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== osd.8 v2:172.21.15.174:6816/477602146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ad0018ec0 con 0x7f7ac80acf80 2024-09-19T07:36:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.851+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 --> [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ad8062d40 con 0x7f7ac80acf80 2024-09-19T07:36:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7ad5ffb640 1 -- 172.21.15.103:0/15400815 <== osd.8 v2:172.21.15.174:6816/477602146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ad8062d40 con 0x7f7ac80acf80 2024-09-19T07:36:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] conn(0x7f7ac80acf80 msgr2=0x7f7ac80af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] conn(0x7f7ac80acf80 0x7f7ac80af420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:52.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7ac80a3350 msgr2=0x7f7ac80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7ac80a3350 0x7f7ac80a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00a810 tx=0x7f7acc008040 comp rx=0 tx=0).stop 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8106f10 msgr2=0x7f7ad8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 --2- 172.21.15.103:0/15400815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ad8106f10 0x7f7ad8074f70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000eaa0 tx=0x7f7ad000ef60 comp rx=0 tx=0).stop 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7add3be640 1 -- 172.21.15.103:0/15400815 reap_dead start 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 shutdown_connections 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 >> 172.21.15.103:0/15400815 conn(0x7f7ad80fd8c0 msgr2=0x7f7ad80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 shutdown_connections 2024-09-19T07:36:52.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:52.852+0000 7f7adee21640 1 -- 172.21.15.103:0/15400815 wait complete. 2024-09-19T07:36:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:53 smithi134 ceph-mon[35414]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T07:36:53.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:53 smithi103 ceph-mon[31201]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T07:36:53.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:53 smithi174 ceph-mon[35370]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T07:36:53.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.662+0000 7f79b47f2640 1 -- 172.21.15.103:0/776152959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac108300 msgr2=0x7f79ac10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:53.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.662+0000 7f79b47f2640 1 --2- 172.21.15.103:0/776152959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac108300 0x7f79ac10ec70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009cd0 tx=0x7f79a402a8d0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.662+0000 7f79b47f2640 1 -- 172.21.15.103:0/776152959 shutdown_connections 2024-09-19T07:36:53.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.662+0000 7f79b47f2640 1 --2- 172.21.15.103:0/776152959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac108300 0x7f79ac10ec70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.662+0000 7f79b47f2640 1 --2- 172.21.15.103:0/776152959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79ac103780 0x7f79ac107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.662+0000 7f79b47f2640 1 --2- 172.21.15.103:0/776152959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79ac102dd0 0x7f79ac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 -- 172.21.15.103:0/776152959 >> 172.21.15.103:0/776152959 conn(0x7f79ac0fc7d0 msgr2=0x7f79ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:53.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 -- 172.21.15.103:0/776152959 shutdown_connections 2024-09-19T07:36:53.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 -- 172.21.15.103:0/776152959 wait complete. 2024-09-19T07:36:53.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 Processor -- start 2024-09-19T07:36:53.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 -- start start 2024-09-19T07:36:53.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79ac102dd0 0x7f79ac198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:53.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac103780 0x7f79ac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:53.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79ac108300 0x7f79ac19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:53.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1120d0 con 0x7f79ac103780 2024-09-19T07:36:53.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac111f50 con 0x7f79ac102dd0 2024-09-19T07:36:53.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b47f2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac112250 con 0x7f79ac108300 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b1d8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac103780 0x7f79ac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b1d8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac103780 0x7f79ac198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42508/0 (socket says 172.21.15.103:42508) 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.663+0000 7f79b1d8d640 1 -- 172.21.15.103:0/839682114 learned_addr learned my addr 172.21.15.103:0/839682114 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b2d8f640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79ac108300 0x7f79ac19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b258e640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79ac102dd0 0x7f79ac198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b1d8d640 1 -- 172.21.15.103:0/839682114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79ac108300 msgr2=0x7f79ac19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b1d8d640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79ac108300 0x7f79ac19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b1d8d640 1 -- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79ac102dd0 msgr2=0x7f79ac198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b1d8d640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79ac102dd0 0x7f79ac198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b1d8d640 1 -- 172.21.15.103:0/839682114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac19d3f0 con 0x7f79ac103780 2024-09-19T07:36:53.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b1d8d640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac103780 0x7f79ac198980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c8c0 tx=0x7f799c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c007bb0 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c007d50 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac19d6e0 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c018830 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.664+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac1a4fc0 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.665+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79ac0fede0 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.666+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f799c0189d0 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79a37fe640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f79980a3350 0x7f79980a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79b258e640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f79980a3350 0x7f79980a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f799c0cfef0 con 0x7f79ac103780 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79a37fe640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] conn(0x7f79980acf80 0x7f79980af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 --> [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79980afaf0 con 0x7f79980acf80 2024-09-19T07:36:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f799c0d0320 con 0x7f79ac103780 2024-09-19T07:36:53.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79b258e640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f79980a3350 0x7f79980a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005e10 tx=0x7f79a8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:53.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.667+0000 7f79b2d8f640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] conn(0x7f79980acf80 0x7f79980af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:53.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.668+0000 7f79b2d8f640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] conn(0x7f79980acf80 0x7f79980af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:53.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.669+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== osd.1 v2:172.21.15.134:6800/2492227363 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79980afaf0 con 0x7f79980acf80 2024-09-19T07:36:53.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.682+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 --> [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79ac06a7b0 con 0x7f79980acf80 2024-09-19T07:36:53.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.682+0000 7f79a37fe640 1 -- 172.21.15.103:0/839682114 <== osd.1 v2:172.21.15.134:6800/2492227363 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f79ac06a7b0 con 0x7f79980acf80 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] conn(0x7f79980acf80 msgr2=0x7f79980af420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] conn(0x7f79980acf80 0x7f79980af420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f79980a3350 msgr2=0x7f79980a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f79980a3350 0x7f79980a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005e10 tx=0x7f79a8005da0 comp rx=0 tx=0).stop 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac103780 msgr2=0x7f79ac198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 --2- 172.21.15.103:0/839682114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79ac103780 0x7f79ac198980 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f799c00c8c0 tx=0x7f799c00cd80 comp rx=0 tx=0).stop 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b2d8f640 1 -- 172.21.15.103:0/839682114 reap_dead start 2024-09-19T07:36:53.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 shutdown_connections 2024-09-19T07:36:53.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 >> 172.21.15.103:0/839682114 conn(0x7f79ac0fc7d0 msgr2=0x7f79ac0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:53.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 shutdown_connections 2024-09-19T07:36:53.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:53.684+0000 7f79b47f2640 1 -- 172.21.15.103:0/839682114 wait complete. 2024-09-19T07:36:54.018 INFO:teuthology.orchestra.run.smithi103.stdout:244813135909 2024-09-19T07:36:54.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T07:36:54.431 INFO:teuthology.orchestra.run.smithi103.stdout:55834574921 2024-09-19T07:36:54.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T07:36:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:55.152 INFO:teuthology.orchestra.run.smithi103.stdout:34359738445 2024-09-19T07:36:55.153 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T07:36:55.370 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:55 smithi103 ceph-mon[31201]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T07:36:55.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:55 smithi134 ceph-mon[35414]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T07:36:55.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:55 smithi174 ceph-mon[35370]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-19T07:36:55.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 -- 172.21.15.103:0/3719277511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec106f20 msgr2=0x7ff5ec10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3719277511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec106f20 0x7ff5ec10eaa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009900 tx=0x7ff5dc02a410 comp rx=0 tx=0).stop 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 -- 172.21.15.103:0/3719277511 shutdown_connections 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3719277511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec106f20 0x7ff5ec10eaa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3719277511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ec1027a0 0x7ff5ec1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3719277511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec101df0 0x7ff5ec1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 -- 172.21.15.103:0/3719277511 >> 172.21.15.103:0/3719277511 conn(0x7ff5ec0fd8b0 msgr2=0x7ff5ec0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 -- 172.21.15.103:0/3719277511 shutdown_connections 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.755+0000 7ff5f4158640 1 -- 172.21.15.103:0/3719277511 wait complete. 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 Processor -- start 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 -- start start 2024-09-19T07:36:55.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ec101df0 0x7ff5ec198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec1027a0 0x7ff5ec198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec106f20 0x7ff5ec19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec112020 con 0x7ff5ec1027a0 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec111ea0 con 0x7ff5ec101df0 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f4158640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1121a0 con 0x7ff5ec106f20 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec1027a0 0x7ff5ec198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec1027a0 0x7ff5ec198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42524/0 (socket says 172.21.15.103:42524) 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 -- 172.21.15.103:0/3790481021 learned_addr learned my addr 172.21.15.103:0/3790481021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f1ef4640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ec101df0 0x7ff5ec198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f26f5640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec106f20 0x7ff5ec19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 -- 172.21.15.103:0/3790481021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec106f20 msgr2=0x7ff5ec19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec106f20 0x7ff5ec19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 -- 172.21.15.103:0/3790481021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ec101df0 msgr2=0x7ff5ec198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ec101df0 0x7ff5ec198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 -- 172.21.15.103:0/3790481021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec19d4d0 con 0x7ff5ec1027a0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.756+0000 7ff5f16f3640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec1027a0 0x7ff5ec198a60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000b4a0 tx=0x7ff5e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.757+0000 7ff5eaffd640 1 -- 172.21.15.103:0/3790481021 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0014070 con 0x7ff5ec1027a0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.757+0000 7ff5eaffd640 1 -- 172.21.15.103:0/3790481021 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e0010af0 con 0x7ff5ec1027a0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.757+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec19d7c0 con 0x7ff5ec1027a0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.757+0000 7ff5eaffd640 1 -- 172.21.15.103:0/3790481021 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0017d10 con 0x7ff5ec1027a0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.757+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1a50a0 con 0x7ff5ec1027a0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.758+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec102250 con 0x7ff5ec1027a0 2024-09-19T07:36:55.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.759+0000 7ff5eaffd640 1 -- 172.21.15.103:0/3790481021 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff5e0017070 con 0x7ff5ec1027a0 2024-09-19T07:36:55.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.759+0000 7ff5eaffd640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5d40a32d0 0x7ff5d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.760+0000 7ff5f1ef4640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5d40a32d0 0x7ff5d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.760+0000 7ff5eaffd640 1 -- 172.21.15.103:0/3790481021 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7ff5e00ced60 con 0x7ff5ec1027a0 2024-09-19T07:36:55.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.760+0000 7ff5f1ef4640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5d40a32d0 0x7ff5d40a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400a400 tx=0x7ff5e400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:55.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.764+0000 7ff5eaffd640 1 -- 172.21.15.103:0/3790481021 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5e009ba30 con 0x7ff5ec1027a0 2024-09-19T07:36:55.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.944+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff5ec109b70 con 0x7ff5ec1027a0 2024-09-19T07:36:55.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.944+0000 7ff5eaffd640 1 -- 172.21.15.103:0/3790481021 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff5e009b330 con 0x7ff5ec1027a0 2024-09-19T07:36:55.945 INFO:teuthology.orchestra.run.smithi103.stdout:296352743452 2024-09-19T07:36:55.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5d40a32d0 msgr2=0x7ff5d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5d40a32d0 0x7ff5d40a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400a400 tx=0x7ff5e400d040 comp rx=0 tx=0).stop 2024-09-19T07:36:55.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec1027a0 msgr2=0x7ff5ec198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec1027a0 0x7ff5ec198a60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000b4a0 tx=0x7ff5e000b960 comp rx=0 tx=0).stop 2024-09-19T07:36:55.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 shutdown_connections 2024-09-19T07:36:55.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5d40a32d0 0x7ff5d40a5790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec106f20 0x7ff5ec19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5ec1027a0 0x7ff5ec198a60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 --2- 172.21.15.103:0/3790481021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ec101df0 0x7ff5ec198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 >> 172.21.15.103:0/3790481021 conn(0x7ff5ec0fd8b0 msgr2=0x7ff5ec0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:55.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 shutdown_connections 2024-09-19T07:36:55.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.948+0000 7ff5f4158640 1 -- 172.21.15.103:0/3790481021 wait complete. 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 -- 172.21.15.103:0/1448524172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 msgr2=0x7f33d0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 --2- 172.21.15.103:0/1448524172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 0x7f33d0100ee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009930 tx=0x7f33c402a420 comp rx=0 tx=0).stop 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 -- 172.21.15.103:0/1448524172 shutdown_connections 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 --2- 172.21.15.103:0/1448524172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0101420 0x7f33d010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 --2- 172.21.15.103:0/1448524172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 0x7f33d0100ee0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 --2- 172.21.15.103:0/1448524172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d0106a80 0x7f33d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 -- 172.21.15.103:0/1448524172 >> 172.21.15.103:0/1448524172 conn(0x7f33d00fc7a0 msgr2=0x7f33d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 -- 172.21.15.103:0/1448524172 shutdown_connections 2024-09-19T07:36:55.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.982+0000 7f33d5490640 1 -- 172.21.15.103:0/1448524172 wait complete. 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 Processor -- start 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 -- start start 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 0x7f33d0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0101420 0x7f33d01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d0106a80 0x7f33d019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0110e80 con 0x7f33d0100a80 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0110d00 con 0x7f33d0106a80 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33d5490640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0111000 con 0x7f33d0101420 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 0x7f33d0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 0x7f33d0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42552/0 (socket says 172.21.15.103:42552) 2024-09-19T07:36:55.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 -- 172.21.15.103:0/1247151635 learned_addr learned my addr 172.21.15.103:0/1247151635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33cf7fe640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d0106a80 0x7f33d019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ce7fc640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0101420 0x7f33d01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 -- 172.21.15.103:0/1247151635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0101420 msgr2=0x7f33d01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0101420 0x7f33d01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 -- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d0106a80 msgr2=0x7f33d019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d0106a80 0x7f33d019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.983+0000 7f33ceffd640 1 -- 172.21.15.103:0/1247151635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d019d340 con 0x7f33d0100a80 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33ce7fc640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0101420 0x7f33d01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33cf7fe640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d0106a80 0x7f33d019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33ceffd640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 0x7f33d0198390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33c000d950 tx=0x7f33c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0017070 con 0x7f33d0100a80 2024-09-19T07:36:55.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d019d630 con 0x7f33d0100a80 2024-09-19T07:36:55.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c0009c30 con 0x7f33d0100a80 2024-09-19T07:36:55.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d01a4f10 con 0x7f33d0100a80 2024-09-19T07:36:55.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.984+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0020060 con 0x7f33d0100a80 2024-09-19T07:36:55.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.985+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33d0062900 con 0x7f33d0100a80 2024-09-19T07:36:55.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.986+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f33c00109d0 con 0x7f33d0100a80 2024-09-19T07:36:55.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33b7fff640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f33b80a32d0 0x7f33b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33ce7fc640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f33b80a32d0 0x7f33b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f33c00cfcc0 con 0x7f33d0100a80 2024-09-19T07:36:55.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33b7fff640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] conn(0x7f33b80acf00 0x7f33b80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:55.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 --> [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33b80afa70 con 0x7f33b80acf00 2024-09-19T07:36:55.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f33c00d00f0 con 0x7f33d0100a80 2024-09-19T07:36:55.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33ce7fc640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f33b80a32d0 0x7f33b80a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f33c4000c00 tx=0x7f33c4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:55.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.987+0000 7f33cf7fe640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] conn(0x7f33b80acf00 0x7f33b80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:55.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.988+0000 7f33cf7fe640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] conn(0x7f33b80acf00 0x7f33b80af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:55.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:55.988+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== osd.4 v2:172.21.15.134:6808/2364007931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33b80afa70 con 0x7f33b80acf00 2024-09-19T07:36:56.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 --> [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f33d0062d80 con 0x7f33b80acf00 2024-09-19T07:36:56.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33b7fff640 1 -- 172.21.15.103:0/1247151635 <== osd.4 v2:172.21.15.134:6808/2364007931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f33d0062d80 con 0x7f33b80acf00 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] conn(0x7f33b80acf00 msgr2=0x7f33b80af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] conn(0x7f33b80acf00 0x7f33b80af3a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f33b80a32d0 msgr2=0x7f33b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f33b80a32d0 0x7f33b80a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f33c4000c00 tx=0x7f33c4005a20 comp rx=0 tx=0).stop 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 msgr2=0x7f33d0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 --2- 172.21.15.103:0/1247151635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d0100a80 0x7f33d0198390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33c000d950 tx=0x7f33c000de10 comp rx=0 tx=0).stop 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33cf7fe640 1 -- 172.21.15.103:0/1247151635 reap_dead start 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 shutdown_connections 2024-09-19T07:36:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.002+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 >> 172.21.15.103:0/1247151635 conn(0x7f33d00fc7a0 msgr2=0x7f33d006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:56.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.003+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 shutdown_connections 2024-09-19T07:36:56.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:56.003+0000 7f33d5490640 1 -- 172.21.15.103:0/1247151635 wait complete. 2024-09-19T07:36:56.412 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:36:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:56 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3790481021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T07:36:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:56 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3790481021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T07:36:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:56 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3790481021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T07:36:56.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743450 got 296352743452 for osd.10 2024-09-19T07:36:56.844 DEBUG:teuthology.parallel:result is None 2024-09-19T07:36:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 -- 172.21.15.103:0/4177679239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34103660 msgr2=0x7fea3410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 --2- 172.21.15.103:0/4177679239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34103660 0x7fea3410bb50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fea24009900 tx=0x7fea2402a480 comp rx=0 tx=0).stop 2024-09-19T07:36:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 -- 172.21.15.103:0/4177679239 shutdown_connections 2024-09-19T07:36:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 --2- 172.21.15.103:0/4177679239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3410c170 0x7fea3410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 --2- 172.21.15.103:0/4177679239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34103660 0x7fea3410bb50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 --2- 172.21.15.103:0/4177679239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea34102d40 0x7fea34103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 -- 172.21.15.103:0/4177679239 >> 172.21.15.103:0/4177679239 conn(0x7fea340fc7a0 msgr2=0x7fea340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 -- 172.21.15.103:0/4177679239 shutdown_connections 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.016+0000 7fea3ba69640 1 -- 172.21.15.103:0/4177679239 wait complete. 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.017+0000 7fea3ba69640 1 Processor -- start 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.017+0000 7fea3ba69640 1 -- start start 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.017+0000 7fea3ba69640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34102d40 0x7fea34198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.017+0000 7fea3ba69640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea34103660 0x7fea34198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.017+0000 7fea3ba69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3410c170 0x7fea3419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea3ba69640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea34111f70 con 0x7fea34102d40 2024-09-19T07:36:57.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea3ba69640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea34111df0 con 0x7fea34103660 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea3ba69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea341120f0 con 0x7fea3410c170 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34102d40 0x7fea34198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34102d40 0x7fea34198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42560/0 (socket says 172.21.15.103:42560) 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 -- 172.21.15.103:0/205651254 learned_addr learned my addr 172.21.15.103:0/205651254 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 -- 172.21.15.103:0/205651254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3410c170 msgr2=0x7fea3419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3410c170 0x7fea3419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 -- 172.21.15.103:0/205651254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea34103660 msgr2=0x7fea34198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea34103660 0x7fea34198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 -- 172.21.15.103:0/205651254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3419d400 con 0x7fea34102d40 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea39805640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34102d40 0x7fea34198450 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fea3000e9f0 tx=0x7fea3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3000cd80 con 0x7fea34102d40 2024-09-19T07:36:57.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30005500 con 0x7fea34102d40 2024-09-19T07:36:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3419d6f0 con 0x7fea34102d40 2024-09-19T07:36:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30010690 con 0x7fea34102d40 2024-09-19T07:36:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.018+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea341a4fd0 con 0x7fea34102d40 2024-09-19T07:36:57.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.019+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea34062980 con 0x7fea34102d40 2024-09-19T07:36:57.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.021+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fea300046c0 con 0x7fea34102d40 2024-09-19T07:36:57.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea2affd640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea200a32d0 0x7fea200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fea300d0470 con 0x7fea34102d40 2024-09-19T07:36:57.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea39004640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea200a32d0 0x7fea200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:57.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea2affd640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] conn(0x7fea200acf00 0x7fea200af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 --> [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea30004470 con 0x7fea200acf00 2024-09-19T07:36:57.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fea300d08a0 con 0x7fea34102d40 2024-09-19T07:36:57.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea3a006640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] conn(0x7fea200acf00 0x7fea200af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:57.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.022+0000 7fea39004640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea200a32d0 0x7fea200a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea2402a9a0 tx=0x7fea24000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:57.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.023+0000 7fea3a006640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] conn(0x7fea200acf00 0x7fea200af3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:57.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.024+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== osd.9 v2:172.21.15.103:6826/1024309893 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea30004470 con 0x7fea200acf00 2024-09-19T07:36:57.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.041+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 --> [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea34062e00 con 0x7fea200acf00 2024-09-19T07:36:57.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.041+0000 7fea2affd640 1 -- 172.21.15.103:0/205651254 <== osd.9 v2:172.21.15.103:6826/1024309893 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea34062e00 con 0x7fea200acf00 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] conn(0x7fea200acf00 msgr2=0x7fea200af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] conn(0x7fea200acf00 0x7fea200af3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea200a32d0 msgr2=0x7fea200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fea200a32d0 0x7fea200a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea2402a9a0 tx=0x7fea24000bc0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34102d40 msgr2=0x7fea34198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 --2- 172.21.15.103:0/205651254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea34102d40 0x7fea34198450 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fea3000e9f0 tx=0x7fea3000edf0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3a006640 1 -- 172.21.15.103:0/205651254 reap_dead start 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 shutdown_connections 2024-09-19T07:36:57.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 >> 172.21.15.103:0/205651254 conn(0x7fea340fc7a0 msgr2=0x7fea340fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:57.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.042+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 shutdown_connections 2024-09-19T07:36:57.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.043+0000 7fea3ba69640 1 -- 172.21.15.103:0/205651254 wait complete. 2024-09-19T07:36:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:57 smithi103 ceph-mon[31201]: pgmap v304: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:36:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:57 smithi134 ceph-mon[35414]: pgmap v304: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:36:57.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:57 smithi174 ceph-mon[35370]: pgmap v304: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:36:57.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 -- 172.21.15.103:0/2028630758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34106f10 msgr2=0x7f9a3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/2028630758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34106f10 0x7f9a3410ea90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009ca0 tx=0x7f9a2c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 -- 172.21.15.103:0/2028630758 shutdown_connections 2024-09-19T07:36:57.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/2028630758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34106f10 0x7f9a3410ea90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/2028630758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a34102790 0x7f9a341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/2028630758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a34101de0 0x7f9a341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 -- 172.21.15.103:0/2028630758 >> 172.21.15.103:0/2028630758 conn(0x7f9a340fd8c0 msgr2=0x7f9a340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 -- 172.21.15.103:0/2028630758 shutdown_connections 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.580+0000 7f9a3be8a640 1 -- 172.21.15.103:0/2028630758 wait complete. 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 Processor -- start 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 -- start start 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34101de0 0x7f9a340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a34102790 0x7f9a34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a34106f10 0x7f9a34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34111f70 con 0x7f9a34101de0 2024-09-19T07:36:57.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a34111df0 con 0x7f9a34106f10 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3be8a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a341120f0 con 0x7f9a34102790 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34101de0 0x7f9a340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34101de0 0x7f9a340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42584/0 (socket says 172.21.15.103:42584) 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 -- 172.21.15.103:0/4155101633 learned_addr learned my addr 172.21.15.103:0/4155101633 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a3a427640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a34106f10 0x7f9a34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39425640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a34102790 0x7f9a34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 -- 172.21.15.103:0/4155101633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a34102790 msgr2=0x7f9a34074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a34102790 0x7f9a34074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 -- 172.21.15.103:0/4155101633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a34106f10 msgr2=0x7f9a34074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a34106f10 0x7f9a34074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stdout:77309411397 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 -- 172.21.15.103:0/4155101633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a341a68e0 con 0x7f9a34101de0 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.581+0000 7f9a39c26640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34101de0 0x7f9a340744f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c890 tx=0x7f9a3000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.582+0000 7f9a2affd640 1 -- 172.21.15.103:0/4155101633 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30004220 con 0x7f9a34101de0 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.582+0000 7f9a2affd640 1 -- 172.21.15.103:0/4155101633 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a300043c0 con 0x7f9a34101de0 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.582+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a341a6bd0 con 0x7f9a34101de0 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.582+0000 7f9a2affd640 1 -- 172.21.15.103:0/4155101633 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a300049e0 con 0x7f9a34101de0 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.582+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a341a7110 con 0x7f9a34101de0 2024-09-19T07:36:57.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.583+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a341021c0 con 0x7f9a34101de0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.584+0000 7f9a2affd640 1 -- 172.21.15.103:0/4155101633 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9a300079b0 con 0x7f9a34101de0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.584+0000 7f9a2affd640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9a200a32d0 0x7f9a200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.585+0000 7f9a39425640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9a200a32d0 0x7f9a200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.585+0000 7f9a2affd640 1 -- 172.21.15.103:0/4155101633 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f9a300cfd20 con 0x7f9a34101de0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.585+0000 7f9a39425640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9a200a32d0 0x7f9a200a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a24005fd0 tx=0x7f9a24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.588+0000 7f9a2affd640 1 -- 172.21.15.103:0/4155101633 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9a30014030 con 0x7f9a34101de0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.775+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9a34075a20 con 0x7f9a34101de0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.775+0000 7f9a2affd640 1 -- 172.21.15.103:0/4155101633 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a300bae20 con 0x7f9a34101de0 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9a200a32d0 msgr2=0x7f9a200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9a200a32d0 0x7f9a200a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a24005fd0 tx=0x7f9a24005850 comp rx=0 tx=0).stop 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34101de0 msgr2=0x7f9a340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:57.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34101de0 0x7f9a340744f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c890 tx=0x7f9a3000cd50 comp rx=0 tx=0).stop 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 shutdown_connections 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9a200a32d0 0x7f9a200a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a34106f10 0x7f9a34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a34102790 0x7f9a34074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 --2- 172.21.15.103:0/4155101633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a34101de0 0x7f9a340744f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 >> 172.21.15.103:0/4155101633 conn(0x7f9a340fd8c0 msgr2=0x7f9a340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 shutdown_connections 2024-09-19T07:36:57.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:57.778+0000 7f9a3be8a640 1 -- 172.21.15.103:0/4155101633 wait complete. 2024-09-19T07:36:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:58 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4155101633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T07:36:58.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:58 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4155101633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T07:36:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:58 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4155101633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T07:36:58.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.936+0000 7f930bfdc640 1 -- 172.21.15.103:0/4084402707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 msgr2=0x7f930410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.936+0000 7f930bfdc640 1 --2- 172.21.15.103:0/4084402707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f930410da40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009ca0 tx=0x7f92fc02a950 comp rx=0 tx=0).stop 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.936+0000 7f930bfdc640 1 -- 172.21.15.103:0/4084402707 shutdown_connections 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.936+0000 7f930bfdc640 1 --2- 172.21.15.103:0/4084402707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f930410da40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.936+0000 7f930bfdc640 1 --2- 172.21.15.103:0/4084402707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304102570 0x7f93041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.936+0000 7f930bfdc640 1 --2- 172.21.15.103:0/4084402707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9304100e20 0x7f9304101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.936+0000 7f930bfdc640 1 -- 172.21.15.103:0/4084402707 >> 172.21.15.103:0/4084402707 conn(0x7f93040fc7d0 msgr2=0x7f93040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.937+0000 7f930bfdc640 1 -- 172.21.15.103:0/4084402707 shutdown_connections 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.937+0000 7f930bfdc640 1 -- 172.21.15.103:0/4084402707 wait complete. 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.937+0000 7f930bfdc640 1 Processor -- start 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.937+0000 7f930bfdc640 1 -- start start 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930bfdc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9304100e20 0x7f93041992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:58.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930bfdc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304102570 0x7f9304199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930bfdc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f93041a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930bfdc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9304110f80 con 0x7f9304102ef0 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930bfdc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9304110e00 con 0x7f9304100e20 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930bfdc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9304111100 con 0x7f9304102570 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f93041a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f93041a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53308/0 (socket says 172.21.15.103:53308) 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 -- 172.21.15.103:0/2378904308 learned_addr learned my addr 172.21.15.103:0/2378904308 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f9309577640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304102570 0x7f9304199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f9309d78640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9304100e20 0x7f93041992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 -- 172.21.15.103:0/2378904308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304102570 msgr2=0x7f9304199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304102570 0x7f9304199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 -- 172.21.15.103:0/2378904308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9304100e20 msgr2=0x7f93041992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:58.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9304100e20 0x7f93041992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 -- 172.21.15.103:0/2378904308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93041a48b0 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f930a579640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f93041a4370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f92fc004af0 tx=0x7f92fc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.938+0000 7f92faffd640 1 -- 172.21.15.103:0/2378904308 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc034040 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.939+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93041a4a80 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.939+0000 7f92faffd640 1 -- 172.21.15.103:0/2378904308 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc030920 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.939+0000 7f92faffd640 1 -- 172.21.15.103:0/2378904308 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc004820 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.939+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93041a4f30 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.940+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9304101200 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.941+0000 7f92faffd640 1 -- 172.21.15.103:0/2378904308 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f92fc037020 con 0x7f9304102ef0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.942+0000 7f92faffd640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92f00a32d0 0x7f92f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.942+0000 7f9309d78640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92f00a32d0 0x7f92f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:58.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.942+0000 7f92faffd640 1 -- 172.21.15.103:0/2378904308 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f92fc0f08d0 con 0x7f9304102ef0 2024-09-19T07:36:58.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.942+0000 7f9309d78640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92f00a32d0 0x7f92f00a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9300009aa0 tx=0x7f9300005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:58.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.946+0000 7f92faffd640 1 -- 172.21.15.103:0/2378904308 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92fc0bd5a0 con 0x7f9304102ef0 2024-09-19T07:36:59.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 -- 172.21.15.103:0/2718600814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88100370 msgr2=0x7f3c88100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 --2- 172.21.15.103:0/2718600814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88100370 0x7f3c88100750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009930 tx=0x7f3c8402a420 comp rx=0 tx=0).stop 2024-09-19T07:36:59.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 -- 172.21.15.103:0/2718600814 shutdown_connections 2024-09-19T07:36:59.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 --2- 172.21.15.103:0/2718600814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c88106fc0 0x7f3c8810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 --2- 172.21.15.103:0/2718600814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c88100d20 0x7f3c88106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 --2- 172.21.15.103:0/2718600814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88100370 0x7f3c88100750 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 -- 172.21.15.103:0/2718600814 >> 172.21.15.103:0/2718600814 conn(0x7f3c88077210 msgr2=0x7f3c880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 -- 172.21.15.103:0/2718600814 shutdown_connections 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.825+0000 7f3c903c3640 1 -- 172.21.15.103:0/2718600814 wait complete. 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 Processor -- start 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 -- start start 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c88100370 0x7f3c881012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c88100d20 0x7f3c88101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88106fc0 0x7f3c88101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88111fa0 con 0x7f3c88106fc0 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88111e20 con 0x7f3c88100d20 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c903c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88112120 con 0x7f3c88100370 2024-09-19T07:36:59.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88106fc0 0x7f3c88101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88106fc0 0x7f3c88101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53276/0 (socket says 172.21.15.103:53276) 2024-09-19T07:36:59.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 -- 172.21.15.103:0/3635501587 learned_addr learned my addr 172.21.15.103:0/3635501587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:59.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e15f640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c88100370 0x7f3c881012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8d95e640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c88100d20 0x7f3c88101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.086 INFO:teuthology.orchestra.run.smithi103.stdout:107374182463 2024-09-19T07:36:59.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 -- 172.21.15.103:0/3635501587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c88100370 msgr2=0x7f3c881012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c88100370 0x7f3c881012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 -- 172.21.15.103:0/3635501587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c88100d20 msgr2=0x7f3c88101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c88100d20 0x7f3c88101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e960640 1 -- 172.21.15.103:0/3635501587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c881a68e0 con 0x7f3c88106fc0 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8e15f640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c88100370 0x7f3c881012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.826+0000 7f3c8d95e640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c88100d20 0x7f3c88101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.827+0000 7f3c8e960640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88106fc0 0x7f3c88101d70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000eaa0 tx=0x7f3c8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.827+0000 7f3c7f7fe640 1 -- 172.21.15.103:0/3635501587 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8000cdb0 con 0x7f3c88106fc0 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.827+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c881a6bd0 con 0x7f3c88106fc0 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.827+0000 7f3c7f7fe640 1 -- 172.21.15.103:0/3635501587 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c80005020 con 0x7f3c88106fc0 2024-09-19T07:36:59.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.827+0000 7f3c7f7fe640 1 -- 172.21.15.103:0/3635501587 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c800116d0 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.827+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c881a7160 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.828+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c88100750 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.829+0000 7f3c7f7fe640 1 -- 172.21.15.103:0/3635501587 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3c800054d0 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.830+0000 7f3c7f7fe640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3c740a32d0 0x7f3c740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.830+0000 7f3c8e15f640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3c740a32d0 0x7f3c740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.830+0000 7f3c7f7fe640 1 -- 172.21.15.103:0/3635501587 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f3c800cffa0 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.830+0000 7f3c8e15f640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3c740a32d0 0x7f3c740a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c84005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:58.833+0000 7f3c7f7fe640 1 -- 172.21.15.103:0/3635501587 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c80016030 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.044+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3c88109b70 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.045+0000 7f3c7f7fe640 1 -- 172.21.15.103:0/3635501587 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3c8009cc20 con 0x7f3c88106fc0 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3c740a32d0 msgr2=0x7f3c740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3c740a32d0 0x7f3c740a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c84005a20 comp rx=0 tx=0).stop 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88106fc0 msgr2=0x7f3c88101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88106fc0 0x7f3c88101d70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000eaa0 tx=0x7f3c8000ef60 comp rx=0 tx=0).stop 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 shutdown_connections 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f3c740a32d0 0x7f3c740a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88106fc0 0x7f3c88101d70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c88100d20 0x7f3c88101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 --2- 172.21.15.103:0/3635501587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c88100370 0x7f3c881012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 >> 172.21.15.103:0/3635501587 conn(0x7f3c88077210 msgr2=0x7f3c880fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 shutdown_connections 2024-09-19T07:36:59.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.047+0000 7f3c903c3640 1 -- 172.21.15.103:0/3635501587 wait complete. 2024-09-19T07:36:59.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.155+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f93041a5160 con 0x7f9304102ef0 2024-09-19T07:36:59.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.156+0000 7f92faffd640 1 -- 172.21.15.103:0/2378904308 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f92fc0bcea0 con 0x7f9304102ef0 2024-09-19T07:36:59.156 INFO:teuthology.orchestra.run.smithi103.stdout:163208757301 2024-09-19T07:36:59.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92f00a32d0 msgr2=0x7f92f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92f00a32d0 0x7f92f00a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9300009aa0 tx=0x7f9300005f90 comp rx=0 tx=0).stop 2024-09-19T07:36:59.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 msgr2=0x7f93041a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f93041a4370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f92fc004af0 tx=0x7f92fc02f5b0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 shutdown_connections 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f92f00a32d0 0x7f92f00a5790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9304102ef0 0x7f93041a4370 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304102570 0x7f9304199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 --2- 172.21.15.103:0/2378904308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9304100e20 0x7f93041992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 >> 172.21.15.103:0/2378904308 conn(0x7f93040fc7d0 msgr2=0x7f9304106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 shutdown_connections 2024-09-19T07:36:59.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.158+0000 7f930bfdc640 1 -- 172.21.15.103:0/2378904308 wait complete. 2024-09-19T07:36:59.399 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:59 smithi103 ceph-mon[31201]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T07:36:59.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:59 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3635501587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T07:36:59.400 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:36:59 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2378904308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T07:36:59.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.481+0000 7ff9e868a640 1 -- 172.21.15.103:0/2815613396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 msgr2=0x7ff9e0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 --2- 172.21.15.103:0/2815613396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e0101200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009900 tx=0x7ff9d402a480 comp rx=0 tx=0).stop 2024-09-19T07:36:59.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 -- 172.21.15.103:0/2815613396 shutdown_connections 2024-09-19T07:36:59.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 --2- 172.21.15.103:0/2815613396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9e0102ef0 0x7ff9e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 --2- 172.21.15.103:0/2815613396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9e0102570 0x7ff9e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 --2- 172.21.15.103:0/2815613396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e0101200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 -- 172.21.15.103:0/2815613396 >> 172.21.15.103:0/2815613396 conn(0x7ff9e00fc7d0 msgr2=0x7ff9e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:59.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 -- 172.21.15.103:0/2815613396 shutdown_connections 2024-09-19T07:36:59.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 -- 172.21.15.103:0/2815613396 wait complete. 2024-09-19T07:36:59.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 Processor -- start 2024-09-19T07:36:59.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.482+0000 7ff9e868a640 1 -- start start 2024-09-19T07:36:59.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e868a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e868a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9e0102570 0x7ff9e019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e868a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9e0102ef0 0x7ff9e019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e868a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0110fa0 con 0x7ff9e0100e20 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e868a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0110e20 con 0x7ff9e0102570 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e868a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0111120 con 0x7ff9e0102ef0 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53332/0 (socket says 172.21.15.103:53332) 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 -- 172.21.15.103:0/4033501223 learned_addr learned my addr 172.21.15.103:0/4033501223 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e5c25640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9e0102570 0x7ff9e019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6c27640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9e0102ef0 0x7ff9e019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 -- 172.21.15.103:0/4033501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9e0102ef0 msgr2=0x7ff9e019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9e0102ef0 0x7ff9e019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 -- 172.21.15.103:0/4033501223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9e0102570 msgr2=0x7ff9e019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9e0102570 0x7ff9e019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 -- 172.21.15.103:0/4033501223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e019f6c0 con 0x7ff9e0100e20 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.483+0000 7ff9e6426640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e019a710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006010 tx=0x7ff9d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.484+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4033501223 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d403f070 con 0x7ff9e0100e20 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.484+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e019f950 con 0x7ff9e0100e20 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.484+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4033501223 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4031c30 con 0x7ff9e0100e20 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.484+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4033501223 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4035030 con 0x7ff9e0100e20 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.484+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01a7290 con 0x7ff9e0100e20 2024-09-19T07:36:59.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.485+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e0101200 con 0x7ff9e0100e20 2024-09-19T07:36:59.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.487+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4033501223 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff9d4042070 con 0x7ff9e0100e20 2024-09-19T07:36:59.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.488+0000 7ff9df7fe640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9c80a32d0 0x7ff9c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:36:59.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.488+0000 7ff9e5c25640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9c80a32d0 0x7ff9c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:36:59.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.488+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4033501223 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7ff9d40f0850 con 0x7ff9e0100e20 2024-09-19T07:36:59.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.488+0000 7ff9e5c25640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9c80a32d0 0x7ff9c80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0104c00 tx=0x7ff9d000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:36:59.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.491+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4033501223 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff9d4031de0 con 0x7ff9e0100e20 2024-09-19T07:36:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:59 smithi134 ceph-mon[35414]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T07:36:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:59 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3635501587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T07:36:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:36:59 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2378904308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T07:36:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:59 smithi174 ceph-mon[35370]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T07:36:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:59 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3635501587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T07:36:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:36:59 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2378904308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T07:36:59.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.670+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff9e01a7460 con 0x7ff9e0100e20 2024-09-19T07:36:59.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.671+0000 7ff9df7fe640 1 -- 172.21.15.103:0/4033501223 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff9d40bd4d0 con 0x7ff9e0100e20 2024-09-19T07:36:59.671 INFO:teuthology.orchestra.run.smithi103.stdout:193273528369 2024-09-19T07:36:59.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9c80a32d0 msgr2=0x7ff9c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9c80a32d0 0x7ff9c80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0104c00 tx=0x7ff9d000d040 comp rx=0 tx=0).stop 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 msgr2=0x7ff9e019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e019a710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006010 tx=0x7ff9d4003c10 comp rx=0 tx=0).stop 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 shutdown_connections 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff9c80a32d0 0x7ff9c80a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9e0102ef0 0x7ff9e019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9e0102570 0x7ff9e019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 --2- 172.21.15.103:0/4033501223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9e0100e20 0x7ff9e019a710 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 >> 172.21.15.103:0/4033501223 conn(0x7ff9e00fc7d0 msgr2=0x7ff9e006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 shutdown_connections 2024-09-19T07:36:59.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:36:59.673+0000 7ff9e868a640 1 -- 172.21.15.103:0/4033501223 wait complete. 2024-09-19T07:37:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:00 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4033501223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T07:37:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:00 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4033501223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T07:37:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:00 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4033501223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T07:37:00.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757301 for osd.5 2024-09-19T07:37:00.776 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:00.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 -- 172.21.15.103:0/2213092359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc103740 msgr2=0x7fe0dc107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:00.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 --2- 172.21.15.103:0/2213092359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc103740 0x7fe0dc107b80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009930 tx=0x7fe0d002a420 comp rx=0 tx=0).stop 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 -- 172.21.15.103:0/2213092359 shutdown_connections 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 --2- 172.21.15.103:0/2213092359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0dc108290 0x7fe0dc10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 --2- 172.21.15.103:0/2213092359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc103740 0x7fe0dc107b80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 --2- 172.21.15.103:0/2213092359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0dc102d90 0x7fe0dc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 -- 172.21.15.103:0/2213092359 >> 172.21.15.103:0/2213092359 conn(0x7fe0dc0fc7d0 msgr2=0x7fe0dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 -- 172.21.15.103:0/2213092359 shutdown_connections 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.922+0000 7fe0e2702640 1 -- 172.21.15.103:0/2213092359 wait complete. 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 Processor -- start 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 -- start start 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc102d90 0x7fe0dc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0dc103740 0x7fe0dc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:00.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0dc108290 0x7fe0dc19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1120d0 con 0x7fe0dc102d90 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc111f50 con 0x7fe0dc103740 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.923+0000 7fe0e2702640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc112250 con 0x7fe0dc108290 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc102d90 0x7fe0dc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc102d90 0x7fe0dc198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53364/0 (socket says 172.21.15.103:53364) 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3397915110 learned_addr learned my addr 172.21.15.103:0/3397915110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0db7fe640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0dc103740 0x7fe0dc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0e0c9f640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0dc108290 0x7fe0dc19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3397915110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0dc108290 msgr2=0x7fe0dc19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0dc108290 0x7fe0dc19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3397915110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0dc103740 msgr2=0x7fe0dc198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:00.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0dc103740 0x7fe0dc198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 -- 172.21.15.103:0/3397915110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc19d5d0 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0e0c9f640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0dc108290 0x7fe0dc19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0dbfff640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc102d90 0x7fe0dc198620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c970 tx=0x7fe0cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0d97fa640 1 -- 172.21.15.103:0/3397915110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc007bb0 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0d97fa640 1 -- 172.21.15.103:0/3397915110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0cc007d50 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc19d8c0 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0d97fa640 1 -- 172.21.15.103:0/3397915110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc00f450 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.924+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a51a0 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.925+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc103170 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.927+0000 7fe0d97fa640 1 -- 172.21.15.103:0/3397915110 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe0cc004fe0 con 0x7fe0dc102d90 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.927+0000 7fe0d97fa640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0c40a32d0 0x7fe0c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.927+0000 7fe0db7fe640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0c40a32d0 0x7fe0c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:00.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.927+0000 7fe0d97fa640 1 -- 172.21.15.103:0/3397915110 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe0cc0cfd20 con 0x7fe0dc102d90 2024-09-19T07:37:00.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.928+0000 7fe0db7fe640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0c40a32d0 0x7fe0c40a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0006010 tx=0x7fe0d0031690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:00.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:00.931+0000 7fe0d97fa640 1 -- 172.21.15.103:0/3397915110 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe0cc014030 con 0x7fe0dc102d90 2024-09-19T07:37:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.104+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe0dc199980 con 0x7fe0dc102d90 2024-09-19T07:37:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.104+0000 7fe0d97fa640 1 -- 172.21.15.103:0/3397915110 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe0cc0bae20 con 0x7fe0dc102d90 2024-09-19T07:37:01.104 INFO:teuthology.orchestra.run.smithi103.stdout:214748364843 2024-09-19T07:37:01.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0c40a32d0 msgr2=0x7fe0c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:01.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0c40a32d0 0x7fe0c40a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0006010 tx=0x7fe0d0031690 comp rx=0 tx=0).stop 2024-09-19T07:37:01.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc102d90 msgr2=0x7fe0dc198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:01.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc102d90 0x7fe0dc198620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c970 tx=0x7fe0cc00ce30 comp rx=0 tx=0).stop 2024-09-19T07:37:01.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 shutdown_connections 2024-09-19T07:37:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe0c40a32d0 0x7fe0c40a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0dc108290 0x7fe0dc19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0dc103740 0x7fe0dc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 --2- 172.21.15.103:0/3397915110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0dc102d90 0x7fe0dc198620 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.106+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 >> 172.21.15.103:0/3397915110 conn(0x7fe0dc0fc7d0 msgr2=0x7fe0dc105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.107+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 shutdown_connections 2024-09-19T07:37:01.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:01.107+0000 7fe0e2702640 1 -- 172.21.15.103:0/3397915110 wait complete. 2024-09-19T07:37:01.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:01 smithi103 ceph-mon[31201]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T07:37:01.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:01 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3397915110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T07:37:01.437 INFO:teuthology.orchestra.run.smithi103.stdout:274877906977 2024-09-19T07:37:01.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T07:37:01.554 INFO:teuthology.orchestra.run.smithi103.stdout:141733920827 2024-09-19T07:37:01.554 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T07:37:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:01 smithi134 ceph-mon[35414]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T07:37:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:01 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3397915110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T07:37:01.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:01 smithi174 ceph-mon[35370]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-19T07:37:01.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:01 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3397915110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T07:37:01.955 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411397 for osd.2 2024-09-19T07:37:01.955 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:02.593 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:02.594 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:02.594 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:02.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182463 for osd.3 2024-09-19T07:37:02.629 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:03.018 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528366 got 193273528369 for osd.6 2024-09-19T07:37:03.019 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:03.024 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:03.082 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:03.520 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:03 smithi103 ceph-mon[31201]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:37:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:03 smithi134 ceph-mon[35414]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:37:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:03 smithi174 ceph-mon[35370]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:37:03.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.751+0000 7f78c8303640 1 -- 172.21.15.103:0/2118840884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 msgr2=0x7f78c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:03.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.751+0000 7f78c8303640 1 --2- 172.21.15.103:0/2118840884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01067e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009900 tx=0x7f78b002a410 comp rx=0 tx=0).stop 2024-09-19T07:37:03.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.751+0000 7f78c8303640 1 -- 172.21.15.103:0/2118840884 shutdown_connections 2024-09-19T07:37:03.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.751+0000 7f78c8303640 1 --2- 172.21.15.103:0/2118840884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78c0106f10 0x7f78c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.751+0000 7f78c8303640 1 --2- 172.21.15.103:0/2118840884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01067e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.751+0000 7f78c8303640 1 --2- 172.21.15.103:0/2118840884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0101de0 0x7f78c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.751+0000 7f78c8303640 1 -- 172.21.15.103:0/2118840884 >> 172.21.15.103:0/2118840884 conn(0x7f78c00fd8c0 msgr2=0x7f78c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 -- 172.21.15.103:0/2118840884 shutdown_connections 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 -- 172.21.15.103:0/2118840884 wait complete. 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 Processor -- start 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 -- start start 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0101de0 0x7f78c0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:03.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78c0106f10 0x7f78c019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0111f90 con 0x7f78c0102790 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0111e10 con 0x7f78c0101de0 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c8303640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0112110 con 0x7f78c0106f10 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c589e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c589e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53386/0 (socket says 172.21.15.103:53386) 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c589e640 1 -- 172.21.15.103:0/2618005981 learned_addr learned my addr 172.21.15.103:0/2618005981 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.752+0000 7f78c609f640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0101de0 0x7f78c0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c589e640 1 -- 172.21.15.103:0/2618005981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78c0106f10 msgr2=0x7f78c019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c68a0640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78c0106f10 0x7f78c019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:03.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c589e640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78c0106f10 0x7f78c019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c589e640 1 -- 172.21.15.103:0/2618005981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0101de0 msgr2=0x7f78c0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c589e640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0101de0 0x7f78c0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c589e640 1 -- 172.21.15.103:0/2618005981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c019d440 con 0x7f78c0102790 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c68a0640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78c0106f10 0x7f78c019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c589e640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01989d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f78b0005ab0 tx=0x7f78b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78b77fe640 1 -- 172.21.15.103:0/2618005981 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b003f030 con 0x7f78c0102790 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c019d6d0 con 0x7f78c0102790 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78b77fe640 1 -- 172.21.15.103:0/2618005981 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b0003ce0 con 0x7f78c0102790 2024-09-19T07:37:03.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78b77fe640 1 -- 172.21.15.103:0/2618005981 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0035070 con 0x7f78c0102790 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.753+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c01a5010 con 0x7f78c0102790 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.754+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c01021c0 con 0x7f78c0102790 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.755+0000 7f78b77fe640 1 -- 172.21.15.103:0/2618005981 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f78b0042030 con 0x7f78c0102790 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.756+0000 7f78b77fe640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f78ac0a32d0 0x7f78ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.756+0000 7f78c609f640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f78ac0a32d0 0x7f78ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.757+0000 7f78b77fe640 1 -- 172.21.15.103:0/2618005981 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f78b00f0680 con 0x7f78c0102790 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.757+0000 7f78c609f640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f78ac0a32d0 0x7f78ac0a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0058c0 tx=0x7f78bc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:03.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.760+0000 7f78b77fe640 1 -- 172.21.15.103:0/2618005981 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f78b00bd200 con 0x7f78c0102790 2024-09-19T07:37:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.937+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f78c010a180 con 0x7f78c0102790 2024-09-19T07:37:03.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.938+0000 7f78b77fe640 1 -- 172.21.15.103:0/2618005981 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f78b00bcb00 con 0x7f78c0102790 2024-09-19T07:37:03.938 INFO:teuthology.orchestra.run.smithi103.stdout:326417514519 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f78ac0a32d0 msgr2=0x7f78ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f78ac0a32d0 0x7f78ac0a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0058c0 tx=0x7f78bc00a2d0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 msgr2=0x7f78c01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01989d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f78b0005ab0 tx=0x7f78b00042f0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 shutdown_connections 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f78ac0a32d0 0x7f78ac0a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78c0106f10 0x7f78c019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78c0102790 0x7f78c01989d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 --2- 172.21.15.103:0/2618005981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c0101de0 0x7f78c0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 >> 172.21.15.103:0/2618005981 conn(0x7f78c00fd8c0 msgr2=0x7f78c00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 shutdown_connections 2024-09-19T07:37:03.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:03.940+0000 7f78c8303640 1 -- 172.21.15.103:0/2618005981 wait complete. 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 -- 172.21.15.103:0/1759243063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630105a90 msgr2=0x7fd630105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 --2- 172.21.15.103:0/1759243063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630105a90 0x7fd630105e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd624009900 tx=0x7fd62402a410 comp rx=0 tx=0).stop 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 -- 172.21.15.103:0/1759243063 shutdown_connections 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 --2- 172.21.15.103:0/1759243063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd630101bc0 0x7fd63010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 --2- 172.21.15.103:0/1759243063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd630101240 0x7fd630101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 --2- 172.21.15.103:0/1759243063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630105a90 0x7fd630105e70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 -- 172.21.15.103:0/1759243063 >> 172.21.15.103:0/1759243063 conn(0x7fd630077210 msgr2=0x7fd6300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 -- 172.21.15.103:0/1759243063 shutdown_connections 2024-09-19T07:37:04.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 -- 172.21.15.103:0/1759243063 wait complete. 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.351+0000 7fd637079640 1 Processor -- start 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd637079640 1 -- start start 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd637079640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630101240 0x7fd6300fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd637079640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd630101bc0 0x7fd6300fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd637079640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd630105a90 0x7fd6300fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd637079640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd630110f70 con 0x7fd630101240 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd637079640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd630110df0 con 0x7fd630101bc0 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd637079640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301110f0 con 0x7fd630105a90 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630101240 0x7fd6300fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630101240 0x7fd6300fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53406/0 (socket says 172.21.15.103:53406) 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 -- 172.21.15.103:0/2514007941 learned_addr learned my addr 172.21.15.103:0/2514007941 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:04.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd635616640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd630105a90 0x7fd6300fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 -- 172.21.15.103:0/2514007941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd630105a90 msgr2=0x7fd6300fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd630105a90 0x7fd6300fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 -- 172.21.15.103:0/2514007941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd630101bc0 msgr2=0x7fd6300fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd62ffff640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd630101bc0 0x7fd6300fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd630101bc0 0x7fd6300fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 -- 172.21.15.103:0/2514007941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6301a68e0 con 0x7fd630101240 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd635616640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd630105a90 0x7fd6300fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd634e15640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630101240 0x7fd6300fc690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd624006010 tx=0x7fd624003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd62dffb640 1 -- 172.21.15.103:0/2514007941 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62403f070 con 0x7fd630101240 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.352+0000 7fd62dffb640 1 -- 172.21.15.103:0/2514007941 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd624031c30 con 0x7fd630101240 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.353+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6301a6b70 con 0x7fd630101240 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.353+0000 7fd62dffb640 1 -- 172.21.15.103:0/2514007941 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624035030 con 0x7fd630101240 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.353+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301a70a0 con 0x7fd630101240 2024-09-19T07:37:04.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.354+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630101680 con 0x7fd630101240 2024-09-19T07:37:04.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.355+0000 7fd62dffb640 1 -- 172.21.15.103:0/2514007941 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd624042070 con 0x7fd630101240 2024-09-19T07:37:04.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.356+0000 7fd62dffb640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd6200a3350 0x7fd6200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.356+0000 7fd62ffff640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd6200a3350 0x7fd6200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.356+0000 7fd62dffb640 1 -- 172.21.15.103:0/2514007941 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd6240f0730 con 0x7fd630101240 2024-09-19T07:37:04.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.356+0000 7fd62ffff640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd6200a3350 0x7fd6200a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd61c005fd0 tx=0x7fd61c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:04.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.359+0000 7fd62dffb640 1 -- 172.21.15.103:0/2514007941 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd624031de0 con 0x7fd630101240 2024-09-19T07:37:04.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:04 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:04.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:04 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2618005981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T07:37:04.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.532+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd6301a72a0 con 0x7fd630101240 2024-09-19T07:37:04.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7fd62dffb640 1 -- 172.21.15.103:0/2514007941 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6240bd3b0 con 0x7fd630101240 2024-09-19T07:37:04.533 INFO:teuthology.orchestra.run.smithi103.stdout:34359738447 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd6200a3350 msgr2=0x7fd6200a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd6200a3350 0x7fd6200a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd61c005fd0 tx=0x7fd61c005850 comp rx=0 tx=0).stop 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630101240 msgr2=0x7fd6300fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630101240 0x7fd6300fc690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd624006010 tx=0x7fd624003c10 comp rx=0 tx=0).stop 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 shutdown_connections 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd6200a3350 0x7fd6200a5810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd630105a90 0x7fd6300fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd630101bc0 0x7fd6300fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 --2- 172.21.15.103:0/2514007941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd630101240 0x7fd6300fc690 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 >> 172.21.15.103:0/2514007941 conn(0x7fd630077210 msgr2=0x7fd6301002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:04.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 shutdown_connections 2024-09-19T07:37:04.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7fd637079640 1 -- 172.21.15.103:0/2514007941 wait complete. 2024-09-19T07:37:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:04 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:04 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2618005981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T07:37:04.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.531+0000 7f4cf8947640 1 -- 172.21.15.103:0/1352564239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 msgr2=0x7f4cf41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.531+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1352564239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf41067f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009900 tx=0x7f4ce802a410 comp rx=0 tx=0).stop 2024-09-19T07:37:04.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.531+0000 7f4cf8947640 1 -- 172.21.15.103:0/1352564239 shutdown_connections 2024-09-19T07:37:04.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.531+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1352564239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cf4106f20 0x7f4cf410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.531+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1352564239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf41067f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.531+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1352564239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf4101df0 0x7f4cf41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.531+0000 7f4cf8947640 1 -- 172.21.15.103:0/1352564239 >> 172.21.15.103:0/1352564239 conn(0x7f4cf40fd8f0 msgr2=0x7f4cf40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.532+0000 7f4cf8947640 1 -- 172.21.15.103:0/1352564239 shutdown_connections 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.532+0000 7f4cf8947640 1 -- 172.21.15.103:0/1352564239 wait complete. 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 Processor -- start 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 -- start start 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf4101df0 0x7f4cf419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cf4106f20 0x7f4cf419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4074e30 con 0x7f4cf41027a0 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4074cb0 con 0x7f4cf4106f20 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf8947640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4074fb0 con 0x7f4cf4101df0 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf419ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53412/0 (socket says 172.21.15.103:53412) 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 -- 172.21.15.103:0/1715038487 learned_addr learned my addr 172.21.15.103:0/1715038487 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf4101df0 0x7f4cf419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf2d9d640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cf4106f20 0x7f4cf419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 -- 172.21.15.103:0/1715038487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf4101df0 msgr2=0x7f4cf419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf4101df0 0x7f4cf419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 -- 172.21.15.103:0/1715038487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cf4106f20 msgr2=0x7f4cf419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cf4106f20 0x7f4cf419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf1d9b640 1 -- 172.21.15.103:0/1715038487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf419f5c0 con 0x7f4cf41027a0 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf2d9d640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cf4106f20 0x7f4cf419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.533+0000 7f4cf259c640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf4101df0 0x7f4cf419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:04.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.534+0000 7f4cf1d9b640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf419ad60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006010 tx=0x7f4ce80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.534+0000 7f4ce37fe640 1 -- 172.21.15.103:0/1715038487 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce803f030 con 0x7f4cf41027a0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.534+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf419f850 con 0x7f4cf41027a0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.534+0000 7f4ce37fe640 1 -- 172.21.15.103:0/1715038487 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce8003ce0 con 0x7f4cf41027a0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.534+0000 7f4ce37fe640 1 -- 172.21.15.103:0/1715038487 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8035070 con 0x7f4cf41027a0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.534+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf41a7190 con 0x7f4cf41027a0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.536+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf41021d0 con 0x7f4cf41027a0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.537+0000 7f4ce37fe640 1 -- 172.21.15.103:0/1715038487 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4ce8042030 con 0x7f4cf41027a0 2024-09-19T07:37:04.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.537+0000 7f4ce37fe640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f4cdc0a32d0 0x7f4cdc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:04.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.537+0000 7f4cf259c640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f4cdc0a32d0 0x7f4cdc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:04.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.537+0000 7f4ce37fe640 1 -- 172.21.15.103:0/1715038487 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f4ce80effa0 con 0x7f4cf41027a0 2024-09-19T07:37:04.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.538+0000 7f4cf259c640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f4cdc0a32d0 0x7f4cdc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400bfd0 tx=0x7f4ce400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:04.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.541+0000 7f4ce37fe640 1 -- 172.21.15.103:0/1715038487 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4ce80bc510 con 0x7f4cf41027a0 2024-09-19T07:37:04.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:04 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:04.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:04 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2618005981' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T07:37:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.715+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4cf41a7360 con 0x7f4cf41027a0 2024-09-19T07:37:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.715+0000 7f4ce37fe640 1 -- 172.21.15.103:0/1715038487 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4ce80c0810 con 0x7f4cf41027a0 2024-09-19T07:37:04.715 INFO:teuthology.orchestra.run.smithi103.stdout:141733920828 2024-09-19T07:37:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f4cdc0a32d0 msgr2=0x7f4cdc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f4cdc0a32d0 0x7f4cdc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400bfd0 tx=0x7f4ce400b390 comp rx=0 tx=0).stop 2024-09-19T07:37:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 msgr2=0x7f4cf419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf419ad60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006010 tx=0x7f4ce80042f0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 shutdown_connections 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f4cdc0a32d0 0x7f4cdc0a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cf4106f20 0x7f4cf419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cf41027a0 0x7f4cf419ad60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 --2- 172.21.15.103:0/1715038487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf4101df0 0x7f4cf419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.717+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 >> 172.21.15.103:0/1715038487 conn(0x7f4cf40fd8f0 msgr2=0x7f4cf40fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.718+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 shutdown_connections 2024-09-19T07:37:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:04.718+0000 7f4cf8947640 1 -- 172.21.15.103:0/1715038487 wait complete. 2024-09-19T07:37:05.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364841 got 214748364843 for osd.7 2024-09-19T07:37:05.042 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:05 smithi134 ceph-mon[35414]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T07:37:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:05 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2514007941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T07:37:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:05 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1715038487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T07:37:05.605 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:05 smithi103 ceph-mon[31201]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T07:37:05.606 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:05 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2514007941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T07:37:05.606 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:05 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1715038487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T07:37:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:05 smithi174 ceph-mon[35370]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-19T07:37:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:05 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2514007941' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T07:37:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:05 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1715038487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T07:37:05.935 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738447 for osd.0 2024-09-19T07:37:05.935 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:06.539 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:06.564 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.564 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.564 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.564 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:06 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920827 got 141733920828 for osd.4 2024-09-19T07:37:06.565 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:06 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:06 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:07.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 -- 172.21.15.103:0/3736188236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a010c170 msgr2=0x7f34a010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 --2- 172.21.15.103:0/3736188236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a010c170 0x7f34a010e610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3498009ca0 tx=0x7f349802a950 comp rx=0 tx=0).stop 2024-09-19T07:37:07.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 -- 172.21.15.103:0/3736188236 shutdown_connections 2024-09-19T07:37:07.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 --2- 172.21.15.103:0/3736188236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a010c170 0x7f34a010e610 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 --2- 172.21.15.103:0/3736188236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34a0103660 0x7f34a010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 --2- 172.21.15.103:0/3736188236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0102d40 0x7f34a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 -- 172.21.15.103:0/3736188236 >> 172.21.15.103:0/3736188236 conn(0x7f34a00fc7a0 msgr2=0x7f34a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 -- 172.21.15.103:0/3736188236 shutdown_connections 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.196+0000 7f34a80f5640 1 -- 172.21.15.103:0/3736188236 wait complete. 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 Processor -- start 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 -- start start 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a0102d40 0x7f34a0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0103660 0x7f34a0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34a010c170 0x7f34a019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111f70 con 0x7f34a0102d40 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111df0 con 0x7f34a0103660 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.197+0000 7f34a80f5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01120f0 con 0x7f34a010c170 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a0102d40 0x7f34a0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a0102d40 0x7f34a0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53432/0 (socket says 172.21.15.103:53432) 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 -- 172.21.15.103:0/1716333924 learned_addr learned my addr 172.21.15.103:0/1716333924 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5690640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0103660 0x7f34a0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 -- 172.21.15.103:0/1716333924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34a010c170 msgr2=0x7f34a019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a6692640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34a010c170 0x7f34a019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34a010c170 0x7f34a019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 -- 172.21.15.103:0/1716333924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0103660 msgr2=0x7f34a0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0103660 0x7f34a0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 -- 172.21.15.103:0/1716333924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a019d2e0 con 0x7f34a0102d40 2024-09-19T07:37:07.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5690640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0103660 0x7f34a0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a6692640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34a010c170 0x7f34a019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a5e91640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a0102d40 0x7f34a0198330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f349c00d950 tx=0x7f349c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f3496ffd640 1 -- 172.21.15.103:0/1716333924 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c017070 con 0x7f34a0102d40 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f3496ffd640 1 -- 172.21.15.103:0/1716333924 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349c009c30 con 0x7f34a0102d40 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a019d5d0 con 0x7f34a0102d40 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f3496ffd640 1 -- 172.21.15.103:0/1716333924 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c00f6b0 con 0x7f34a0102d40 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.198+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a4eb0 con 0x7f34a0102d40 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.199+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a0103120 con 0x7f34a0102d40 2024-09-19T07:37:07.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.200+0000 7f3496ffd640 1 -- 172.21.15.103:0/1716333924 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f349c0109d0 con 0x7f34a0102d40 2024-09-19T07:37:07.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.201+0000 7f3496ffd640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f348c0a32d0 0x7f348c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.201+0000 7f34a5690640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f348c0a32d0 0x7f348c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.201+0000 7f3496ffd640 1 -- 172.21.15.103:0/1716333924 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f349c0cf7a0 con 0x7f34a0102d40 2024-09-19T07:37:07.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.201+0000 7f34a5690640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f348c0a32d0 0x7f348c0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f34a00fecc0 tx=0x7f349000a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:07.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.206+0000 7f3496ffd640 1 -- 172.21.15.103:0/1716333924 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f349c009de0 con 0x7f34a0102d40 2024-09-19T07:37:07.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.377+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f34a01a5270 con 0x7f34a0102d40 2024-09-19T07:37:07.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.377+0000 7f3496ffd640 1 -- 172.21.15.103:0/1716333924 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f349c09c360 con 0x7f34a0102d40 2024-09-19T07:37:07.377 INFO:teuthology.orchestra.run.smithi103.stdout:55834574924 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f348c0a32d0 msgr2=0x7f348c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f348c0a32d0 0x7f348c0a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f34a00fecc0 tx=0x7f349000a5e0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a0102d40 msgr2=0x7f34a0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a0102d40 0x7f34a0198330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f349c00d950 tx=0x7f349c00de10 comp rx=0 tx=0).stop 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 shutdown_connections 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f348c0a32d0 0x7f348c0a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34a010c170 0x7f34a019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34a0103660 0x7f34a0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 --2- 172.21.15.103:0/1716333924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34a0102d40 0x7f34a0198330 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 >> 172.21.15.103:0/1716333924 conn(0x7f34a00fc7a0 msgr2=0x7f34a00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:07.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 shutdown_connections 2024-09-19T07:37:07.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.380+0000 7f34a80f5640 1 -- 172.21.15.103:0/1716333924 wait complete. 2024-09-19T07:37:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 -- 172.21.15.103:0/1978748219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 msgr2=0x7f659c117220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 --2- 172.21.15.103:0/1978748219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c117220 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6594009ca0 tx=0x7f659402a950 comp rx=0 tx=0).stop 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 -- 172.21.15.103:0/1978748219 shutdown_connections 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 --2- 172.21.15.103:0/1978748219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c117220 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 --2- 172.21.15.103:0/1978748219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f659c10a410 0x7f659c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 --2- 172.21.15.103:0/1978748219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c110470 0x7f659c110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 -- 172.21.15.103:0/1978748219 >> 172.21.15.103:0/1978748219 conn(0x7f659c071630 msgr2=0x7f659c073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 -- 172.21.15.103:0/1978748219 shutdown_connections 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.411+0000 7f65a3862640 1 -- 172.21.15.103:0/1978748219 wait complete. 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 Processor -- start 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 -- start start 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f659c10a410 0x7f659c1a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c1a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c110470 0x7f659c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f659c11a750 con 0x7f659c10ad90 2024-09-19T07:37:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f659c11a5d0 con 0x7f659c10a410 2024-09-19T07:37:07.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a3862640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f659c11a8d0 con 0x7f659c110470 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c1a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c1a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53448/0 (socket says 172.21.15.103:53448) 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 -- 172.21.15.103:0/1728161372 learned_addr learned my addr 172.21.15.103:0/1728161372 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a15fe640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f659c10a410 0x7f659c1a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a1dff640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c110470 0x7f659c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 -- 172.21.15.103:0/1728161372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c110470 msgr2=0x7f659c1a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c110470 0x7f659c1a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 -- 172.21.15.103:0/1728161372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f659c10a410 msgr2=0x7f659c1a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f659c10a410 0x7f659c1a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.412+0000 7f65a0dfd640 1 -- 172.21.15.103:0/1728161372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c1a1050 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.413+0000 7f65a0dfd640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c1a6ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c920 tx=0x7f658c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.413+0000 7f65927fc640 1 -- 172.21.15.103:0/1728161372 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c007c70 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.413+0000 7f65927fc640 1 -- 172.21.15.103:0/1728161372 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f658c007e10 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.413+0000 7f65927fc640 1 -- 172.21.15.103:0/1728161372 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658c018890 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.413+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c1a1340 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.413+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1ad630 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.414+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c10a850 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.415+0000 7f65927fc640 1 -- 172.21.15.103:0/1728161372 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f658c018a50 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.416+0000 7f65927fc640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65880a32d0 0x7f65880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.416+0000 7f65a15fe640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65880a32d0 0x7f65880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.416+0000 7f65927fc640 1 -- 172.21.15.103:0/1728161372 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f658c0cfdd0 con 0x7f659c10ad90 2024-09-19T07:37:07.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.416+0000 7f65a15fe640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65880a32d0 0x7f65880a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f659c1a1a60 tx=0x7f659800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:07.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.419+0000 7f65927fc640 1 -- 172.21.15.103:0/1728161372 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f658c014030 con 0x7f659c10ad90 2024-09-19T07:37:07.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:07 smithi103 ceph-mon[31201]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:07 smithi134 ceph-mon[35414]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:07.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.590+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f659c1a1e50 con 0x7f659c10ad90 2024-09-19T07:37:07.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.591+0000 7f65927fc640 1 -- 172.21.15.103:0/1728161372 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f659c1a1e50 con 0x7f659c10ad90 2024-09-19T07:37:07.592 INFO:teuthology.orchestra.run.smithi103.stdout:244813135911 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65880a32d0 msgr2=0x7f65880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65880a32d0 0x7f65880a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f659c1a1a60 tx=0x7f659800b390 comp rx=0 tx=0).stop 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 msgr2=0x7f659c1a6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c1a6ca0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c920 tx=0x7f658c00cde0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 shutdown_connections 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f65880a32d0 0x7f65880a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c110470 0x7f659c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f659c10ad90 0x7f659c1a6ca0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 --2- 172.21.15.103:0/1728161372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f659c10a410 0x7f659c1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:07.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 >> 172.21.15.103:0/1728161372 conn(0x7f659c071630 msgr2=0x7f659c072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:07.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 shutdown_connections 2024-09-19T07:37:07.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:07.593+0000 7f65a3862640 1 -- 172.21.15.103:0/1728161372 wait complete. 2024-09-19T07:37:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:07 smithi174 ceph-mon[35370]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 -- 172.21.15.103:0/1079111813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064102ef0 msgr2=0x7f406410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 --2- 172.21.15.103:0/1079111813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064102ef0 0x7f406410da40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f405c009ca0 tx=0x7f405c02a950 comp rx=0 tx=0).stop 2024-09-19T07:37:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 -- 172.21.15.103:0/1079111813 shutdown_connections 2024-09-19T07:37:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 --2- 172.21.15.103:0/1079111813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064102ef0 0x7f406410da40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 --2- 172.21.15.103:0/1079111813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4064102570 0x7f40641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 --2- 172.21.15.103:0/1079111813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064100e20 0x7f4064101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 -- 172.21.15.103:0/1079111813 >> 172.21.15.103:0/1079111813 conn(0x7f40640fc7d0 msgr2=0x7f40640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:08.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 -- 172.21.15.103:0/1079111813 shutdown_connections 2024-09-19T07:37:08.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 -- 172.21.15.103:0/1079111813 wait complete. 2024-09-19T07:37:08.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.020+0000 7f40694e7640 1 Processor -- start 2024-09-19T07:37:08.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40694e7640 1 -- start start 2024-09-19T07:37:08.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40694e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064100e20 0x7f4064199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:08.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40694e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064102570 0x7f40641997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:08.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40694e7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4064102ef0 0x7f40641a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:08.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40694e7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4064110f80 con 0x7f4064100e20 2024-09-19T07:37:08.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40694e7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4064110e00 con 0x7f4064102ef0 2024-09-19T07:37:08.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40694e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4064111100 con 0x7f4064102570 2024-09-19T07:37:08.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064100e20 0x7f4064199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:08.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064100e20 0x7f4064199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53478/0 (socket says 172.21.15.103:53478) 2024-09-19T07:37:08.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 -- 172.21.15.103:0/3511395183 learned_addr learned my addr 172.21.15.103:0/3511395183 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:08.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40637fe640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4064102ef0 0x7f40641a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:08.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 -- 172.21.15.103:0/3511395183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064102570 msgr2=0x7f40641997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:08.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40627fc640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064102570 0x7f40641997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:08.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064102570 0x7f40641997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 -- 172.21.15.103:0/3511395183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4064102ef0 msgr2=0x7f40641a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:08.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4064102ef0 0x7f40641a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 -- 172.21.15.103:0/3511395183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40641a4850 con 0x7f4064100e20 2024-09-19T07:37:08.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f40637fe640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4064102ef0 0x7f40641a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:08.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.021+0000 7f4062ffd640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064100e20 0x7f4064199290 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f405400c920 tx=0x7f405400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:08.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.022+0000 7f404bfff640 1 -- 172.21.15.103:0/3511395183 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054007c70 con 0x7f4064100e20 2024-09-19T07:37:08.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.022+0000 7f404bfff640 1 -- 172.21.15.103:0/3511395183 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4054007e10 con 0x7f4064100e20 2024-09-19T07:37:08.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.022+0000 7f404bfff640 1 -- 172.21.15.103:0/3511395183 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054018890 con 0x7f4064100e20 2024-09-19T07:37:08.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.022+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40641a4a20 con 0x7f4064100e20 2024-09-19T07:37:08.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.022+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40641a4f30 con 0x7f4064100e20 2024-09-19T07:37:08.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.023+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4064101200 con 0x7f4064100e20 2024-09-19T07:37:08.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.024+0000 7f404bfff640 1 -- 172.21.15.103:0/3511395183 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4054018a50 con 0x7f4064100e20 2024-09-19T07:37:08.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.025+0000 7f404bfff640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f404c0a32d0 0x7f404c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:08.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.025+0000 7f40627fc640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f404c0a32d0 0x7f404c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:08.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.025+0000 7f404bfff640 1 -- 172.21.15.103:0/3511395183 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f40540cfed0 con 0x7f4064100e20 2024-09-19T07:37:08.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.026+0000 7f40627fc640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f404c0a32d0 0x7f404c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4058005d80 tx=0x7f4058005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:08.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.028+0000 7f404bfff640 1 -- 172.21.15.103:0/3511395183 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4054014030 con 0x7f4064100e20 2024-09-19T07:37:08.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.204+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f406406a560 con 0x7f4064100e20 2024-09-19T07:37:08.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.205+0000 7f404bfff640 1 -- 172.21.15.103:0/3511395183 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f406406a560 con 0x7f4064100e20 2024-09-19T07:37:08.206 INFO:teuthology.orchestra.run.smithi103.stdout:274877906979 2024-09-19T07:37:08.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f404c0a32d0 msgr2=0x7f404c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:08.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f404c0a32d0 0x7f404c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4058005d80 tx=0x7f4058005c90 comp rx=0 tx=0).stop 2024-09-19T07:37:08.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064100e20 msgr2=0x7f4064199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064100e20 0x7f4064199290 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f405400c920 tx=0x7f405400cde0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 shutdown_connections 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f404c0a32d0 0x7f404c0a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4064102ef0 0x7f40641a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4064102570 0x7f40641997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 --2- 172.21.15.103:0/3511395183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064100e20 0x7f4064199290 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 >> 172.21.15.103:0/3511395183 conn(0x7f40640fc7d0 msgr2=0x7f4064106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 shutdown_connections 2024-09-19T07:37:08.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:08.207+0000 7f40694e7640 1 -- 172.21.15.103:0/3511395183 wait complete. 2024-09-19T07:37:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:08 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1716333924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T07:37:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:08 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1728161372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T07:37:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:08 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3511395183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T07:37:08.481 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135909 got 244813135911 for osd.8 2024-09-19T07:37:08.481 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:08 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1716333924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T07:37:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:08 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1728161372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T07:37:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:08 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3511395183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T07:37:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:08 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1716333924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T07:37:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:08 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1728161372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T07:37:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:08 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3511395183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T07:37:08.929 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514517 got 326417514519 for osd.11 2024-09-19T07:37:08.929 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:09.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574924 for osd.1 2024-09-19T07:37:09.342 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:09 smithi134 ceph-mon[35414]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:09 smithi174 ceph-mon[35370]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:09.668 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:09 smithi103 ceph-mon[31201]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:09.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906977 got 274877906979 for osd.9 2024-09-19T07:37:09.989 DEBUG:teuthology.parallel:result is None 2024-09-19T07:37:09.989 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T07:37:09.990 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:37:10.839 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 -- 172.21.15.103:0/2921027989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c100a80 msgr2=0x7feb7c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 --2- 172.21.15.103:0/2921027989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c100a80 0x7feb7c100ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feb64009930 tx=0x7feb6402a420 comp rx=0 tx=0).stop 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 -- 172.21.15.103:0/2921027989 shutdown_connections 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 --2- 172.21.15.103:0/2921027989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb7c101420 0x7feb7c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 --2- 172.21.15.103:0/2921027989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c100a80 0x7feb7c100ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 --2- 172.21.15.103:0/2921027989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb7c106a80 0x7feb7c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 -- 172.21.15.103:0/2921027989 >> 172.21.15.103:0/2921027989 conn(0x7feb7c0fc7a0 msgr2=0x7feb7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 -- 172.21.15.103:0/2921027989 shutdown_connections 2024-09-19T07:37:11.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.371+0000 7feb80cc9640 1 -- 172.21.15.103:0/2921027989 wait complete. 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 Processor -- start 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 -- start start 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb7c100a80 0x7feb7c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb7c101420 0x7feb7c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c106a80 0x7feb7c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c110f20 con 0x7feb7c106a80 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c110da0 con 0x7feb7c101420 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb80cc9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1110a0 con 0x7feb7c100a80 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb7ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c106a80 0x7feb7c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb7ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c106a80 0x7feb7c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56518/0 (socket says 172.21.15.103:56518) 2024-09-19T07:37:11.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb7ad9d640 1 -- 172.21.15.103:0/1030400713 learned_addr learned my addr 172.21.15.103:0/1030400713 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb7a59c640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb7c100a80 0x7feb7c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.372+0000 7feb79d9b640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb7c101420 0x7feb7c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb7ad9d640 1 -- 172.21.15.103:0/1030400713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb7c100a80 msgr2=0x7feb7c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb7ad9d640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb7c100a80 0x7feb7c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb7ad9d640 1 -- 172.21.15.103:0/1030400713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb7c101420 msgr2=0x7feb7c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb7ad9d640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb7c101420 0x7feb7c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb7ad9d640 1 -- 172.21.15.103:0/1030400713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c19d560 con 0x7feb7c106a80 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb7ad9d640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c106a80 0x7feb7c19ce80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009fd0 tx=0x7feb6c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb737fe640 1 -- 172.21.15.103:0/1030400713 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c011830 con 0x7feb7c106a80 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb737fe640 1 -- 172.21.15.103:0/1030400713 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6c016070 con 0x7feb7c106a80 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c19d850 con 0x7feb7c106a80 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb737fe640 1 -- 172.21.15.103:0/1030400713 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00f2e0 con 0x7feb7c106a80 2024-09-19T07:37:11.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.373+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c1a5130 con 0x7feb7c106a80 2024-09-19T07:37:11.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.374+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c100f60 con 0x7feb7c106a80 2024-09-19T07:37:11.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.375+0000 7feb737fe640 1 -- 172.21.15.103:0/1030400713 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7feb6c0119d0 con 0x7feb7c106a80 2024-09-19T07:37:11.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.376+0000 7feb737fe640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7feb680a32d0 0x7feb680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:11.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.376+0000 7feb7a59c640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7feb680a32d0 0x7feb680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:11.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.376+0000 7feb737fe640 1 -- 172.21.15.103:0/1030400713 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7feb6c0cef90 con 0x7feb7c106a80 2024-09-19T07:37:11.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.376+0000 7feb7a59c640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7feb680a32d0 0x7feb680a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb7400a400 tx=0x7feb7400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:11.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.380+0000 7feb737fe640 1 -- 172.21.15.103:0/1030400713 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb6c09bc60 con 0x7feb7c106a80 2024-09-19T07:37:11.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.567+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb7c10a090 con 0x7feb680a32d0 2024-09-19T07:37:11.568 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:11 smithi103 ceph-mon[31201]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:11 smithi134 ceph-mon[35414]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:11.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.579+0000 7feb737fe640 1 -- 172.21.15.103:0/1030400713 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259034 (secure 0 0 0) 0x7feb74013340 con 0x7feb680a32d0 2024-09-19T07:37:11.584 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:11.585 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:37:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7feb680a32d0 msgr2=0x7feb680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7feb680a32d0 0x7feb680a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb7400a400 tx=0x7feb7400d040 comp rx=0 tx=0).stop 2024-09-19T07:37:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c106a80 msgr2=0x7feb7c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c106a80 0x7feb7c19ce80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009fd0 tx=0x7feb6c00b8e0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 shutdown_connections 2024-09-19T07:37:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7feb680a32d0 0x7feb680a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb7c106a80 0x7feb7c19ce80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb7c101420 0x7feb7c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 --2- 172.21.15.103:0/1030400713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb7c100a80 0x7feb7c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 >> 172.21.15.103:0/1030400713 conn(0x7feb7c0fc7a0 msgr2=0x7feb7c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 shutdown_connections 2024-09-19T07:37:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:11.582+0000 7feb80cc9640 1 -- 172.21.15.103:0/1030400713 wait complete. 2024-09-19T07:37:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:11 smithi174 ceph-mon[35370]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:12.296 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-19T07:37:10.095474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660513,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":663,"num_read_kb":460,"num_write":428,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":343896,"kb_used_data":11280,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597928,"statfs":{"total":1151940427776,"available":1151588278272,"internally_reserved":0,"allocated":11550720,"data_stored":7266891,"data_compressed":181857,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002798"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893903+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:36:17.893903+0000","last_peered":"2024-09-19T07:36:17.893903+0000","last_clean":"2024-09-19T07:36:17.893903+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:36:17.893903+0000","last_undegraded":"2024-09-19T07:36:17.893903+0000","last_fullsized":"2024-09-19T07:36:17.893903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895834+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:36:17.895834+0000","last_peered":"2024-09-19T07:36:17.895834+0000","last_clean":"2024-09-19T07:36:17.895834+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:36:17.895834+0000","last_undegraded":"2024-09-19T07:36:17.895834+0000","last_fullsized":"2024-09-19T07:36:17.895834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895895+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:36:17.895895+0000","last_peered":"2024-09-19T07:36:17.895895+0000","last_clean":"2024-09-19T07:36:17.895895+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:36:17.895895+0000","last_undegraded":"2024-09-19T07:36:17.895895+0000","last_fullsized":"2024-09-19T07:36:17.895895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896911+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:36:17.896911+0000","last_peered":"2024-09-19T07:36:17.896911+0000","last_clean":"2024-09-19T07:36:17.896911+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:36:17.896911+0000","last_undegraded":"2024-09-19T07:36:17.896911+0000","last_fullsized":"2024-09-19T07:36:17.896911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895313+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:36:17.895313+0000","last_peered":"2024-09-19T07:36:17.895313+0000","last_clean":"2024-09-19T07:36:17.895313+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:36:17.895313+0000","last_undegraded":"2024-09-19T07:36:17.895313+0000","last_fullsized":"2024-09-19T07:36:17.895313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897391+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:36:17.897391+0000","last_peered":"2024-09-19T07:36:17.897391+0000","last_clean":"2024-09-19T07:36:17.897391+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:36:17.897391+0000","last_undegraded":"2024-09-19T07:36:17.897391+0000","last_fullsized":"2024-09-19T07:36:17.897391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893437+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:36:17.893437+0000","last_peered":"2024-09-19T07:36:17.893437+0000","last_clean":"2024-09-19T07:36:17.893437+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:36:17.893437+0000","last_undegraded":"2024-09-19T07:36:17.893437+0000","last_fullsized":"2024-09-19T07:36:17.893437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891423+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:36:17.891423+0000","last_peered":"2024-09-19T07:36:17.891423+0000","last_clean":"2024-09-19T07:36:17.891423+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:36:17.891423+0000","last_undegraded":"2024-09-19T07:36:17.891423+0000","last_fullsized":"2024-09-19T07:36:17.891423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896616+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:36:17.896616+0000","last_peered":"2024-09-19T07:36:17.896616+0000","last_clean":"2024-09-19T07:36:17.896616+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:36:17.896616+0000","last_undegraded":"2024-09-19T07:36:17.896616+0000","last_fullsized":"2024-09-19T07:36:17.896616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891049+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:36:17.891049+0000","last_peered":"2024-09-19T07:36:17.891049+0000","last_clean":"2024-09-19T07:36:17.891049+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:36:17.891049+0000","last_undegraded":"2024-09-19T07:36:17.891049+0000","last_fullsized":"2024-09-19T07:36:17.891049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893904+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:36:17.893904+0000","last_peered":"2024-09-19T07:36:17.893904+0000","last_clean":"2024-09-19T07:36:17.893904+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:36:17.893904+0000","last_undegraded":"2024-09-19T07:36:17.893904+0000","last_fullsized":"2024-09-19T07:36:17.893904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.937191+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:36:17.937191+0000","last_peered":"2024-09-19T07:36:17.937191+0000","last_clean":"2024-09-19T07:36:17.937191+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:36:17.937191+0000","last_undegraded":"2024-09-19T07:36:17.937191+0000","last_fullsized":"2024-09-19T07:36:17.937191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889741+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:36:17.889741+0000","last_peered":"2024-09-19T07:36:17.889741+0000","last_clean":"2024-09-19T07:36:17.889741+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:36:17.889741+0000","last_undegraded":"2024-09-19T07:36:17.889741+0000","last_fullsized":"2024-09-19T07:36:17.889741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895614+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:36:17.895614+0000","last_peered":"2024-09-19T07:36:17.895614+0000","last_clean":"2024-09-19T07:36:17.895614+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:36:17.895614+0000","last_undegraded":"2024-09-19T07:36:17.895614+0000","last_fullsized":"2024-09-19T07:36:17.895614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895550+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:36:17.895550+0000","last_peered":"2024-09-19T07:36:17.895550+0000","last_clean":"2024-09-19T07:36:17.895550+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:36:17.895550+0000","last_undegraded":"2024-09-19T07:36:17.895550+0000","last_fullsized":"2024-09-19T07:36:17.895550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895680+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:36:17.895680+0000","last_peered":"2024-09-19T07:36:17.895680+0000","last_clean":"2024-09-19T07:36:17.895680+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:36:17.895680+0000","last_undegraded":"2024-09-19T07:36:17.895680+0000","last_fullsized":"2024-09-19T07:36:17.895680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893949+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:36:17.893949+0000","last_peered":"2024-09-19T07:36:17.893949+0000","last_clean":"2024-09-19T07:36:17.893949+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:36:17.893949+0000","last_undegraded":"2024-09-19T07:36:17.893949+0000","last_fullsized":"2024-09-19T07:36:17.893949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892102+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:36:17.892102+0000","last_peered":"2024-09-19T07:36:17.892102+0000","last_clean":"2024-09-19T07:36:17.892102+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:36:17.892102+0000","last_undegraded":"2024-09-19T07:36:17.892102+0000","last_fullsized":"2024-09-19T07:36:17.892102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892104+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:36:17.892104+0000","last_peered":"2024-09-19T07:36:17.892104+0000","last_clean":"2024-09-19T07:36:17.892104+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:36:17.892104+0000","last_undegraded":"2024-09-19T07:36:17.892104+0000","last_fullsized":"2024-09-19T07:36:17.892104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897489+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:36:17.897489+0000","last_peered":"2024-09-19T07:36:17.897489+0000","last_clean":"2024-09-19T07:36:17.897489+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:36:17.897489+0000","last_undegraded":"2024-09-19T07:36:17.897489+0000","last_fullsized":"2024-09-19T07:36:17.897489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889770+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:36:17.889770+0000","last_peered":"2024-09-19T07:36:17.889770+0000","last_clean":"2024-09-19T07:36:17.889770+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:36:17.889770+0000","last_undegraded":"2024-09-19T07:36:17.889770+0000","last_fullsized":"2024-09-19T07:36:17.889770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895916+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:36:17.895916+0000","last_peered":"2024-09-19T07:36:17.895916+0000","last_clean":"2024-09-19T07:36:17.895916+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:36:17.895916+0000","last_undegraded":"2024-09-19T07:36:17.895916+0000","last_fullsized":"2024-09-19T07:36:17.895916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891525+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:36:17.891525+0000","last_peered":"2024-09-19T07:36:17.891525+0000","last_clean":"2024-09-19T07:36:17.891525+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:36:17.891525+0000","last_undegraded":"2024-09-19T07:36:17.891525+0000","last_fullsized":"2024-09-19T07:36:17.891525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890710+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:36:17.890710+0000","last_peered":"2024-09-19T07:36:17.890710+0000","last_clean":"2024-09-19T07:36:17.890710+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:36:17.890710+0000","last_undegraded":"2024-09-19T07:36:17.890710+0000","last_fullsized":"2024-09-19T07:36:17.890710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891804+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:36:17.891804+0000","last_peered":"2024-09-19T07:36:17.891804+0000","last_clean":"2024-09-19T07:36:17.891804+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:36:17.891804+0000","last_undegraded":"2024-09-19T07:36:17.891804+0000","last_fullsized":"2024-09-19T07:36:17.891804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892254+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:36:17.892254+0000","last_peered":"2024-09-19T07:36:17.892254+0000","last_clean":"2024-09-19T07:36:17.892254+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:36:17.892254+0000","last_undegraded":"2024-09-19T07:36:17.892254+0000","last_fullsized":"2024-09-19T07:36:17.892254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893575+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:36:17.893575+0000","last_peered":"2024-09-19T07:36:17.893575+0000","last_clean":"2024-09-19T07:36:17.893575+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:36:17.893575+0000","last_undegraded":"2024-09-19T07:36:17.893575+0000","last_fullsized":"2024-09-19T07:36:17.893575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889691+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:36:17.889691+0000","last_peered":"2024-09-19T07:36:17.889691+0000","last_clean":"2024-09-19T07:36:17.889691+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:36:17.889691+0000","last_undegraded":"2024-09-19T07:36:17.889691+0000","last_fullsized":"2024-09-19T07:36:17.889691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895765+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:36:17.895765+0000","last_peered":"2024-09-19T07:36:17.895765+0000","last_clean":"2024-09-19T07:36:17.895765+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:36:17.895765+0000","last_undegraded":"2024-09-19T07:36:17.895765+0000","last_fullsized":"2024-09-19T07:36:17.895765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891796+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:36:17.891796+0000","last_peered":"2024-09-19T07:36:17.891796+0000","last_clean":"2024-09-19T07:36:17.891796+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:36:17.891796+0000","last_undegraded":"2024-09-19T07:36:17.891796+0000","last_fullsized":"2024-09-19T07:36:17.891796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892016+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:36:17.892016+0000","last_peered":"2024-09-19T07:36:17.892016+0000","last_clean":"2024-09-19T07:36:17.892016+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:36:17.892016+0000","last_undegraded":"2024-09-19T07:36:17.892016+0000","last_fullsized":"2024-09-19T07:36:17.892016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894219+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:36:17.894219+0000","last_peered":"2024-09-19T07:36:17.894219+0000","last_clean":"2024-09-19T07:36:17.894219+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:36:17.894219+0000","last_undegraded":"2024-09-19T07:36:17.894219+0000","last_fullsized":"2024-09-19T07:36:17.894219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895304+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:36:17.895304+0000","last_peered":"2024-09-19T07:36:17.895304+0000","last_clean":"2024-09-19T07:36:17.895304+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:36:17.895304+0000","last_undegraded":"2024-09-19T07:36:17.895304+0000","last_fullsized":"2024-09-19T07:36:17.895304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.978446+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:37:07.978446+0000","last_peered":"2024-09-19T07:37:07.978446+0000","last_clean":"2024-09-19T07:37:07.978446+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:37:07.978446+0000","last_undegraded":"2024-09-19T07:37:07.978446+0000","last_fullsized":"2024-09-19T07:37:07.978446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890814+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:36:17.890814+0000","last_peered":"2024-09-19T07:36:17.890814+0000","last_clean":"2024-09-19T07:36:17.890814+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:36:17.890814+0000","last_undegraded":"2024-09-19T07:36:17.890814+0000","last_fullsized":"2024-09-19T07:36:17.890814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890526+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:36:17.890526+0000","last_peered":"2024-09-19T07:36:17.890526+0000","last_clean":"2024-09-19T07:36:17.890526+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:36:17.890526+0000","last_undegraded":"2024-09-19T07:36:17.890526+0000","last_fullsized":"2024-09-19T07:36:17.890526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889543+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:36:17.889543+0000","last_peered":"2024-09-19T07:36:17.889543+0000","last_clean":"2024-09-19T07:36:17.889543+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:36:17.889543+0000","last_undegraded":"2024-09-19T07:36:17.889543+0000","last_fullsized":"2024-09-19T07:36:17.889543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891490+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:36:17.891490+0000","last_peered":"2024-09-19T07:36:17.891490+0000","last_clean":"2024-09-19T07:36:17.891490+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:36:17.891490+0000","last_undegraded":"2024-09-19T07:36:17.891490+0000","last_fullsized":"2024-09-19T07:36:17.891490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907827+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:36:18.907827+0000","last_peered":"2024-09-19T07:36:18.907827+0000","last_clean":"2024-09-19T07:36:18.907827+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:36:18.907827+0000","last_undegraded":"2024-09-19T07:36:18.907827+0000","last_fullsized":"2024-09-19T07:36:18.907827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892021+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:36:17.892021+0000","last_peered":"2024-09-19T07:36:17.892021+0000","last_clean":"2024-09-19T07:36:17.892021+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:36:17.892021+0000","last_undegraded":"2024-09-19T07:36:17.892021+0000","last_fullsized":"2024-09-19T07:36:17.892021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":124,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.977797+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:37:07.977797+0000","last_peered":"2024-09-19T07:37:07.977797+0000","last_clean":"2024-09-19T07:37:07.977797+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:37:07.977797+0000","last_undegraded":"2024-09-19T07:37:07.977797+0000","last_fullsized":"2024-09-19T07:37:07.977797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896529+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:36:17.896529+0000","last_peered":"2024-09-19T07:36:17.896529+0000","last_clean":"2024-09-19T07:36:17.896529+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:36:17.896529+0000","last_undegraded":"2024-09-19T07:36:17.896529+0000","last_fullsized":"2024-09-19T07:36:17.896529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908690+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:36:18.908690+0000","last_peered":"2024-09-19T07:36:18.908690+0000","last_clean":"2024-09-19T07:36:18.908690+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:36:18.908690+0000","last_undegraded":"2024-09-19T07:36:18.908690+0000","last_fullsized":"2024-09-19T07:36:18.908690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896039+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:36:17.896039+0000","last_peered":"2024-09-19T07:36:17.896039+0000","last_clean":"2024-09-19T07:36:17.896039+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:36:17.896039+0000","last_undegraded":"2024-09-19T07:36:17.896039+0000","last_fullsized":"2024-09-19T07:36:17.896039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890888+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:36:17.890888+0000","last_peered":"2024-09-19T07:36:17.890888+0000","last_clean":"2024-09-19T07:36:17.890888+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:36:17.890888+0000","last_undegraded":"2024-09-19T07:36:17.890888+0000","last_fullsized":"2024-09-19T07:36:17.890888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895996+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:36:17.895996+0000","last_peered":"2024-09-19T07:36:17.895996+0000","last_clean":"2024-09-19T07:36:17.895996+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:36:17.895996+0000","last_undegraded":"2024-09-19T07:36:17.895996+0000","last_fullsized":"2024-09-19T07:36:17.895996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'32","reported_seq":161,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.977544+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:37:07.977544+0000","last_peered":"2024-09-19T07:37:07.977544+0000","last_clean":"2024-09-19T07:37:07.977544+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:37:07.977544+0000","last_undegraded":"2024-09-19T07:37:07.977544+0000","last_fullsized":"2024-09-19T07:37:07.977544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":114,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895733+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:36:17.895733+0000","last_peered":"2024-09-19T07:36:17.895733+0000","last_clean":"2024-09-19T07:36:17.895733+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:36:17.895733+0000","last_undegraded":"2024-09-19T07:36:17.895733+0000","last_fullsized":"2024-09-19T07:36:17.895733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913146+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:36:19.913146+0000","last_peered":"2024-09-19T07:36:19.913146+0000","last_clean":"2024-09-19T07:36:19.913146+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:36:19.913146+0000","last_undegraded":"2024-09-19T07:36:19.913146+0000","last_fullsized":"2024-09-19T07:36:19.913146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889534+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:36:17.889534+0000","last_peered":"2024-09-19T07:36:17.889534+0000","last_clean":"2024-09-19T07:36:17.889534+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:36:17.889534+0000","last_undegraded":"2024-09-19T07:36:17.889534+0000","last_fullsized":"2024-09-19T07:36:17.889534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.906778+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:36:18.906778+0000","last_peered":"2024-09-19T07:36:18.906778+0000","last_clean":"2024-09-19T07:36:18.906778+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:36:18.906778+0000","last_undegraded":"2024-09-19T07:36:18.906778+0000","last_fullsized":"2024-09-19T07:36:18.906778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893599+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:36:17.893599+0000","last_peered":"2024-09-19T07:36:17.893599+0000","last_clean":"2024-09-19T07:36:17.893599+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:36:17.893599+0000","last_undegraded":"2024-09-19T07:36:17.893599+0000","last_fullsized":"2024-09-19T07:36:17.893599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:03.973170+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:37:03.973170+0000","last_peered":"2024-09-19T07:37:03.973170+0000","last_clean":"2024-09-19T07:37:03.973170+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:37:03.973170+0000","last_undegraded":"2024-09-19T07:37:03.973170+0000","last_fullsized":"2024-09-19T07:37:03.973170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893370+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:36:17.893370+0000","last_peered":"2024-09-19T07:36:17.893370+0000","last_clean":"2024-09-19T07:36:17.893370+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:36:17.893370+0000","last_undegraded":"2024-09-19T07:36:17.893370+0000","last_fullsized":"2024-09-19T07:36:17.893370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907635+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:36:18.907635+0000","last_peered":"2024-09-19T07:36:18.907635+0000","last_clean":"2024-09-19T07:36:18.907635+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:36:18.907635+0000","last_undegraded":"2024-09-19T07:36:18.907635+0000","last_fullsized":"2024-09-19T07:36:18.907635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890552+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:36:17.890552+0000","last_peered":"2024-09-19T07:36:17.890552+0000","last_clean":"2024-09-19T07:36:17.890552+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:36:17.890552+0000","last_undegraded":"2024-09-19T07:36:17.890552+0000","last_fullsized":"2024-09-19T07:36:17.890552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895827+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:36:17.895827+0000","last_peered":"2024-09-19T07:36:17.895827+0000","last_clean":"2024-09-19T07:36:17.895827+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:36:17.895827+0000","last_undegraded":"2024-09-19T07:36:17.895827+0000","last_fullsized":"2024-09-19T07:36:17.895827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893694+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:36:17.893694+0000","last_peered":"2024-09-19T07:36:17.893694+0000","last_clean":"2024-09-19T07:36:17.893694+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:36:17.893694+0000","last_undegraded":"2024-09-19T07:36:17.893694+0000","last_fullsized":"2024-09-19T07:36:17.893694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908154+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:36:18.908154+0000","last_peered":"2024-09-19T07:36:18.908154+0000","last_clean":"2024-09-19T07:36:18.908154+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:36:18.908154+0000","last_undegraded":"2024-09-19T07:36:18.908154+0000","last_fullsized":"2024-09-19T07:36:18.908154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894155+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:36:17.894155+0000","last_peered":"2024-09-19T07:36:17.894155+0000","last_clean":"2024-09-19T07:36:17.894155+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:36:17.894155+0000","last_undegraded":"2024-09-19T07:36:17.894155+0000","last_fullsized":"2024-09-19T07:36:17.894155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907835+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:36:18.907835+0000","last_peered":"2024-09-19T07:36:18.907835+0000","last_clean":"2024-09-19T07:36:18.907835+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:36:18.907835+0000","last_undegraded":"2024-09-19T07:36:18.907835+0000","last_fullsized":"2024-09-19T07:36:18.907835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893835+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:36:17.893835+0000","last_peered":"2024-09-19T07:36:17.893835+0000","last_clean":"2024-09-19T07:36:17.893835+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:36:17.893835+0000","last_undegraded":"2024-09-19T07:36:17.893835+0000","last_fullsized":"2024-09-19T07:36:17.893835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":81,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.978259+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:37:07.978259+0000","last_peered":"2024-09-19T07:37:07.978259+0000","last_clean":"2024-09-19T07:37:07.978259+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:37:07.978259+0000","last_undegraded":"2024-09-19T07:37:07.978259+0000","last_fullsized":"2024-09-19T07:37:07.978259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894098+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:36:17.894098+0000","last_peered":"2024-09-19T07:36:17.894098+0000","last_clean":"2024-09-19T07:36:17.894098+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:36:17.894098+0000","last_undegraded":"2024-09-19T07:36:17.894098+0000","last_fullsized":"2024-09-19T07:36:17.894098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893546+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:36:17.893546+0000","last_peered":"2024-09-19T07:36:17.893546+0000","last_clean":"2024-09-19T07:36:17.893546+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:36:17.893546+0000","last_undegraded":"2024-09-19T07:36:17.893546+0000","last_fullsized":"2024-09-19T07:36:17.893546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913196+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:36:19.913196+0000","last_peered":"2024-09-19T07:36:19.913196+0000","last_clean":"2024-09-19T07:36:19.913196+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:36:19.913196+0000","last_undegraded":"2024-09-19T07:36:19.913196+0000","last_fullsized":"2024-09-19T07:36:19.913196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892747+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:36:17.892747+0000","last_peered":"2024-09-19T07:36:17.892747+0000","last_clean":"2024-09-19T07:36:17.892747+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:36:17.892747+0000","last_undegraded":"2024-09-19T07:36:17.892747+0000","last_fullsized":"2024-09-19T07:36:17.892747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893570+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:36:17.893570+0000","last_peered":"2024-09-19T07:36:17.893570+0000","last_clean":"2024-09-19T07:36:17.893570+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:36:17.893570+0000","last_undegraded":"2024-09-19T07:36:17.893570+0000","last_fullsized":"2024-09-19T07:36:17.893570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889475+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:36:17.889475+0000","last_peered":"2024-09-19T07:36:17.889475+0000","last_clean":"2024-09-19T07:36:17.889475+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:36:17.889475+0000","last_undegraded":"2024-09-19T07:36:17.889475+0000","last_fullsized":"2024-09-19T07:36:17.889475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":76,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:05.974744+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:37:05.974744+0000","last_peered":"2024-09-19T07:37:05.974744+0000","last_clean":"2024-09-19T07:37:05.974744+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:37:05.974744+0000","last_undegraded":"2024-09-19T07:37:05.974744+0000","last_fullsized":"2024-09-19T07:37:05.974744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890854+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:36:17.890854+0000","last_peered":"2024-09-19T07:36:17.890854+0000","last_clean":"2024-09-19T07:36:17.890854+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:36:17.890854+0000","last_undegraded":"2024-09-19T07:36:17.890854+0000","last_fullsized":"2024-09-19T07:36:17.890854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:04.973802+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:37:04.973802+0000","last_peered":"2024-09-19T07:37:04.973802+0000","last_clean":"2024-09-19T07:37:04.973802+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:37:04.973802+0000","last_undegraded":"2024-09-19T07:37:04.973802+0000","last_fullsized":"2024-09-19T07:37:04.973802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890537+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:36:17.890537+0000","last_peered":"2024-09-19T07:36:17.890537+0000","last_clean":"2024-09-19T07:36:17.890537+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:36:17.890537+0000","last_undegraded":"2024-09-19T07:36:17.890537+0000","last_fullsized":"2024-09-19T07:36:17.890537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892154+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:36:17.892154+0000","last_peered":"2024-09-19T07:36:17.892154+0000","last_clean":"2024-09-19T07:36:17.892154+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:36:17.892154+0000","last_undegraded":"2024-09-19T07:36:17.892154+0000","last_fullsized":"2024-09-19T07:36:17.892154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890794+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:36:17.890794+0000","last_peered":"2024-09-19T07:36:17.890794+0000","last_clean":"2024-09-19T07:36:17.890794+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:36:17.890794+0000","last_undegraded":"2024-09-19T07:36:17.890794+0000","last_fullsized":"2024-09-19T07:36:17.890794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.909170+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:36:18.909170+0000","last_peered":"2024-09-19T07:36:18.909170+0000","last_clean":"2024-09-19T07:36:18.909170+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:36:18.909170+0000","last_undegraded":"2024-09-19T07:36:18.909170+0000","last_fullsized":"2024-09-19T07:36:18.909170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889701+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:36:17.889701+0000","last_peered":"2024-09-19T07:36:17.889701+0000","last_clean":"2024-09-19T07:36:17.889701+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:36:17.889701+0000","last_undegraded":"2024-09-19T07:36:17.889701+0000","last_fullsized":"2024-09-19T07:36:17.889701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'14","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:31.853047+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:36:31.853047+0000","last_peered":"2024-09-19T07:36:31.853047+0000","last_clean":"2024-09-19T07:36:31.853047+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:36:31.853047+0000","last_undegraded":"2024-09-19T07:36:31.853047+0000","last_fullsized":"2024-09-19T07:36:31.853047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891890+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:36:17.891890+0000","last_peered":"2024-09-19T07:36:17.891890+0000","last_clean":"2024-09-19T07:36:17.891890+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:36:17.891890+0000","last_undegraded":"2024-09-19T07:36:17.891890+0000","last_fullsized":"2024-09-19T07:36:17.891890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907864+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:36:18.907864+0000","last_peered":"2024-09-19T07:36:18.907864+0000","last_clean":"2024-09-19T07:36:18.907864+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:36:18.907864+0000","last_undegraded":"2024-09-19T07:36:18.907864+0000","last_fullsized":"2024-09-19T07:36:18.907864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895903+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:36:17.895903+0000","last_peered":"2024-09-19T07:36:17.895903+0000","last_clean":"2024-09-19T07:36:17.895903+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:36:17.895903+0000","last_undegraded":"2024-09-19T07:36:17.895903+0000","last_fullsized":"2024-09-19T07:36:17.895903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895928+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:36:17.895928+0000","last_peered":"2024-09-19T07:36:17.895928+0000","last_clean":"2024-09-19T07:36:17.895928+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:36:17.895928+0000","last_undegraded":"2024-09-19T07:36:17.895928+0000","last_fullsized":"2024-09-19T07:36:17.895928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894935+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:36:17.894935+0000","last_peered":"2024-09-19T07:36:17.894935+0000","last_clean":"2024-09-19T07:36:17.894935+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:36:17.894935+0000","last_undegraded":"2024-09-19T07:36:17.894935+0000","last_fullsized":"2024-09-19T07:36:17.894935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907171+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:36:18.907171+0000","last_peered":"2024-09-19T07:36:18.907171+0000","last_clean":"2024-09-19T07:36:18.907171+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:36:18.907171+0000","last_undegraded":"2024-09-19T07:36:18.907171+0000","last_fullsized":"2024-09-19T07:36:18.907171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894276+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:36:17.894276+0000","last_peered":"2024-09-19T07:36:17.894276+0000","last_clean":"2024-09-19T07:36:17.894276+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:36:17.894276+0000","last_undegraded":"2024-09-19T07:36:17.894276+0000","last_fullsized":"2024-09-19T07:36:17.894276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.141159+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:36:18.141159+0000","last_peered":"2024-09-19T07:36:18.141159+0000","last_clean":"2024-09-19T07:36:18.141159+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:36:18.141159+0000","last_undegraded":"2024-09-19T07:36:18.141159+0000","last_fullsized":"2024-09-19T07:36:18.141159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891645+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:36:17.891645+0000","last_peered":"2024-09-19T07:36:17.891645+0000","last_clean":"2024-09-19T07:36:17.891645+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:36:17.891645+0000","last_undegraded":"2024-09-19T07:36:17.891645+0000","last_fullsized":"2024-09-19T07:36:17.891645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895305+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:36:17.895305+0000","last_peered":"2024-09-19T07:36:17.895305+0000","last_clean":"2024-09-19T07:36:17.895305+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:36:17.895305+0000","last_undegraded":"2024-09-19T07:36:17.895305+0000","last_fullsized":"2024-09-19T07:36:17.895305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893736+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:36:17.893736+0000","last_peered":"2024-09-19T07:36:17.893736+0000","last_clean":"2024-09-19T07:36:17.893736+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:36:17.893736+0000","last_undegraded":"2024-09-19T07:36:17.893736+0000","last_fullsized":"2024-09-19T07:36:17.893736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.936693+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:36:17.936693+0000","last_peered":"2024-09-19T07:36:17.936693+0000","last_clean":"2024-09-19T07:36:17.936693+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:36:17.936693+0000","last_undegraded":"2024-09-19T07:36:17.936693+0000","last_fullsized":"2024-09-19T07:36:17.936693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908619+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:36:18.908619+0000","last_peered":"2024-09-19T07:36:18.908619+0000","last_clean":"2024-09-19T07:36:18.908619+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:36:18.908619+0000","last_undegraded":"2024-09-19T07:36:18.908619+0000","last_fullsized":"2024-09-19T07:36:18.908619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895842+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:36:17.895842+0000","last_peered":"2024-09-19T07:36:17.895842+0000","last_clean":"2024-09-19T07:36:17.895842+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:36:17.895842+0000","last_undegraded":"2024-09-19T07:36:17.895842+0000","last_fullsized":"2024-09-19T07:36:17.895842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913109+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:36:19.913109+0000","last_peered":"2024-09-19T07:36:19.913109+0000","last_clean":"2024-09-19T07:36:19.913109+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:36:19.913109+0000","last_undegraded":"2024-09-19T07:36:19.913109+0000","last_fullsized":"2024-09-19T07:36:19.913109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895948+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:36:17.895948+0000","last_peered":"2024-09-19T07:36:17.895948+0000","last_clean":"2024-09-19T07:36:17.895948+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:36:17.895948+0000","last_undegraded":"2024-09-19T07:36:17.895948+0000","last_fullsized":"2024-09-19T07:36:17.895948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891880+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:36:17.891880+0000","last_peered":"2024-09-19T07:36:17.891880+0000","last_clean":"2024-09-19T07:36:17.891880+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:36:17.891880+0000","last_undegraded":"2024-09-19T07:36:17.891880+0000","last_fullsized":"2024-09-19T07:36:17.891880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891708+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:36:17.891708+0000","last_peered":"2024-09-19T07:36:17.891708+0000","last_clean":"2024-09-19T07:36:17.891708+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:36:17.891708+0000","last_undegraded":"2024-09-19T07:36:17.891708+0000","last_fullsized":"2024-09-19T07:36:17.891708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:05.974487+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:37:05.974487+0000","last_peered":"2024-09-19T07:37:05.974487+0000","last_clean":"2024-09-19T07:37:05.974487+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:37:05.974487+0000","last_undegraded":"2024-09-19T07:37:05.974487+0000","last_fullsized":"2024-09-19T07:37:05.974487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69911,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":527,"num_read_kb":341,"num_write":245,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209733,"data_compressed":4779,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247252,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":442143,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1095370,"data_compressed":43192,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441568,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":441162,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1034859,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528371,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441678,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1033751,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":921600,"data_stored":441539,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441110,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":505121,"data_compressed":7402,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":505573,"data_compressed":7402,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":443017,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65127,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64675,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64439,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:37:12.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:37:12.567 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:13.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 -- 172.21.15.103:0/297798274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18102ef0 msgr2=0x7fbd1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:13.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/297798274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18102ef0 0x7fbd1810da40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009ca0 tx=0x7fbd1002a950 comp rx=0 tx=0).stop 2024-09-19T07:37:13.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 -- 172.21.15.103:0/297798274 shutdown_connections 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/297798274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18102ef0 0x7fbd1810da40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/297798274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd18102570 0x7fbd181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/297798274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18100e20 0x7fbd18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 -- 172.21.15.103:0/297798274 >> 172.21.15.103:0/297798274 conn(0x7fbd180fc7d0 msgr2=0x7fbd180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 -- 172.21.15.103:0/297798274 shutdown_connections 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.037+0000 7fbd1dc95640 1 -- 172.21.15.103:0/297798274 wait complete. 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 Processor -- start 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 -- start start 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18100e20 0x7fbd181984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18102570 0x7fbd18198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:13.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd18102ef0 0x7fbd1819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18110f80 con 0x7fbd18100e20 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18110e00 con 0x7fbd18102570 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd1dc95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18111100 con 0x7fbd18102ef0 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18100e20 0x7fbd181984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18100e20 0x7fbd181984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56526/0 (socket says 172.21.15.103:56526) 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 -- 172.21.15.103:0/2044459409 learned_addr learned my addr 172.21.15.103:0/2044459409 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 -- 172.21.15.103:0/2044459409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd18102ef0 msgr2=0x7fbd1819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd18102ef0 0x7fbd1819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 -- 172.21.15.103:0/2044459409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18102570 msgr2=0x7fbd18198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:13.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18102570 0x7fbd18198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 -- 172.21.15.103:0/2044459409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1819d480 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd177fe640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18100e20 0x7fbd181984d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c920 tx=0x7fbd0800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.038+0000 7fbd157fa640 1 -- 172.21.15.103:0/2044459409 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08007c70 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.039+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1819d770 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.039+0000 7fbd157fa640 1 -- 172.21.15.103:0/2044459409 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd08007e10 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.039+0000 7fbd157fa640 1 -- 172.21.15.103:0/2044459409 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08018890 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.039+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a5050 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.039+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd18101200 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.041+0000 7fbd157fa640 1 -- 172.21.15.103:0/2044459409 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbd08018a50 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.041+0000 7fbd157fa640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd040a32d0 0x7fbd040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.042+0000 7fbd0ffff640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd040a32d0 0x7fbd040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.042+0000 7fbd157fa640 1 -- 172.21.15.103:0/2044459409 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fbd080cfc90 con 0x7fbd18100e20 2024-09-19T07:37:13.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.042+0000 7fbd0ffff640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd040a32d0 0x7fbd040a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd18105510 tx=0x7fbd00005b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:13.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.046+0000 7fbd157fa640 1 -- 172.21.15.103:0/2044459409 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbd080d5020 con 0x7fbd18100e20 2024-09-19T07:37:13.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.209+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd18062630 con 0x7fbd040a32d0 2024-09-19T07:37:13.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.215+0000 7fbd157fa640 1 -- 172.21.15.103:0/2044459409 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259034 (secure 0 0 0) 0x7fbd18062630 con 0x7fbd040a32d0 2024-09-19T07:37:13.218 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:13.218 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd040a32d0 msgr2=0x7fbd040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd040a32d0 0x7fbd040a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd18105510 tx=0x7fbd00005b60 comp rx=0 tx=0).stop 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18100e20 msgr2=0x7fbd181984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18100e20 0x7fbd181984d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c920 tx=0x7fbd0800cde0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 shutdown_connections 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbd040a32d0 0x7fbd040a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd18102ef0 0x7fbd1819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18102570 0x7fbd18198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 --2- 172.21.15.103:0/2044459409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd18100e20 0x7fbd181984d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.217+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 >> 172.21.15.103:0/2044459409 conn(0x7fbd180fc7d0 msgr2=0x7fbd1806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:13.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.218+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 shutdown_connections 2024-09-19T07:37:13.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:13.218+0000 7fbd1dc95640 1 -- 172.21.15.103:0/2044459409 wait complete. 2024-09-19T07:37:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:13 smithi103 ceph-mon[31201]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:37:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:13 smithi103 ceph-mon[31201]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:37:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:37:13.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:13 smithi103 ceph-mon[31201]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:13 smithi134 ceph-mon[35414]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:37:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:13 smithi134 ceph-mon[35414]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:37:13.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:37:13.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:13 smithi134 ceph-mon[35414]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:13 smithi174 ceph-mon[35370]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:37:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:13 smithi174 ceph-mon[35370]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:37:13.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:37:13.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:13 smithi174 ceph-mon[35370]: from='mgr.14150 172.21.15.103:0/4185978727' entity='mgr.x' 2024-09-19T07:37:13.851 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-19T07:37:12.096060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660513,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":673,"num_read_kb":460,"num_write":428,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":343896,"kb_used_data":11280,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597928,"statfs":{"total":1151940427776,"available":1151588278272,"internally_reserved":0,"allocated":11550720,"data_stored":7266891,"data_compressed":181857,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002987"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893903+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:36:17.893903+0000","last_peered":"2024-09-19T07:36:17.893903+0000","last_clean":"2024-09-19T07:36:17.893903+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:36:17.893903+0000","last_undegraded":"2024-09-19T07:36:17.893903+0000","last_fullsized":"2024-09-19T07:36:17.893903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895834+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:36:17.895834+0000","last_peered":"2024-09-19T07:36:17.895834+0000","last_clean":"2024-09-19T07:36:17.895834+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:36:17.895834+0000","last_undegraded":"2024-09-19T07:36:17.895834+0000","last_fullsized":"2024-09-19T07:36:17.895834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895895+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:36:17.895895+0000","last_peered":"2024-09-19T07:36:17.895895+0000","last_clean":"2024-09-19T07:36:17.895895+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:36:17.895895+0000","last_undegraded":"2024-09-19T07:36:17.895895+0000","last_fullsized":"2024-09-19T07:36:17.895895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896911+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:36:17.896911+0000","last_peered":"2024-09-19T07:36:17.896911+0000","last_clean":"2024-09-19T07:36:17.896911+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:36:17.896911+0000","last_undegraded":"2024-09-19T07:36:17.896911+0000","last_fullsized":"2024-09-19T07:36:17.896911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895313+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:36:17.895313+0000","last_peered":"2024-09-19T07:36:17.895313+0000","last_clean":"2024-09-19T07:36:17.895313+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:36:17.895313+0000","last_undegraded":"2024-09-19T07:36:17.895313+0000","last_fullsized":"2024-09-19T07:36:17.895313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897391+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:36:17.897391+0000","last_peered":"2024-09-19T07:36:17.897391+0000","last_clean":"2024-09-19T07:36:17.897391+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:36:17.897391+0000","last_undegraded":"2024-09-19T07:36:17.897391+0000","last_fullsized":"2024-09-19T07:36:17.897391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893437+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:36:17.893437+0000","last_peered":"2024-09-19T07:36:17.893437+0000","last_clean":"2024-09-19T07:36:17.893437+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:36:17.893437+0000","last_undegraded":"2024-09-19T07:36:17.893437+0000","last_fullsized":"2024-09-19T07:36:17.893437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891423+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:36:17.891423+0000","last_peered":"2024-09-19T07:36:17.891423+0000","last_clean":"2024-09-19T07:36:17.891423+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:36:17.891423+0000","last_undegraded":"2024-09-19T07:36:17.891423+0000","last_fullsized":"2024-09-19T07:36:17.891423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896616+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:36:17.896616+0000","last_peered":"2024-09-19T07:36:17.896616+0000","last_clean":"2024-09-19T07:36:17.896616+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:36:17.896616+0000","last_undegraded":"2024-09-19T07:36:17.896616+0000","last_fullsized":"2024-09-19T07:36:17.896616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891049+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:36:17.891049+0000","last_peered":"2024-09-19T07:36:17.891049+0000","last_clean":"2024-09-19T07:36:17.891049+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:36:17.891049+0000","last_undegraded":"2024-09-19T07:36:17.891049+0000","last_fullsized":"2024-09-19T07:36:17.891049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893904+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:36:17.893904+0000","last_peered":"2024-09-19T07:36:17.893904+0000","last_clean":"2024-09-19T07:36:17.893904+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:36:17.893904+0000","last_undegraded":"2024-09-19T07:36:17.893904+0000","last_fullsized":"2024-09-19T07:36:17.893904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.937191+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:36:17.937191+0000","last_peered":"2024-09-19T07:36:17.937191+0000","last_clean":"2024-09-19T07:36:17.937191+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:36:17.937191+0000","last_undegraded":"2024-09-19T07:36:17.937191+0000","last_fullsized":"2024-09-19T07:36:17.937191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889741+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:36:17.889741+0000","last_peered":"2024-09-19T07:36:17.889741+0000","last_clean":"2024-09-19T07:36:17.889741+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:36:17.889741+0000","last_undegraded":"2024-09-19T07:36:17.889741+0000","last_fullsized":"2024-09-19T07:36:17.889741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895614+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:36:17.895614+0000","last_peered":"2024-09-19T07:36:17.895614+0000","last_clean":"2024-09-19T07:36:17.895614+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:36:17.895614+0000","last_undegraded":"2024-09-19T07:36:17.895614+0000","last_fullsized":"2024-09-19T07:36:17.895614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895550+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:36:17.895550+0000","last_peered":"2024-09-19T07:36:17.895550+0000","last_clean":"2024-09-19T07:36:17.895550+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:36:17.895550+0000","last_undegraded":"2024-09-19T07:36:17.895550+0000","last_fullsized":"2024-09-19T07:36:17.895550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895680+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:36:17.895680+0000","last_peered":"2024-09-19T07:36:17.895680+0000","last_clean":"2024-09-19T07:36:17.895680+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:36:17.895680+0000","last_undegraded":"2024-09-19T07:36:17.895680+0000","last_fullsized":"2024-09-19T07:36:17.895680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893949+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:36:17.893949+0000","last_peered":"2024-09-19T07:36:17.893949+0000","last_clean":"2024-09-19T07:36:17.893949+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:36:17.893949+0000","last_undegraded":"2024-09-19T07:36:17.893949+0000","last_fullsized":"2024-09-19T07:36:17.893949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892102+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:36:17.892102+0000","last_peered":"2024-09-19T07:36:17.892102+0000","last_clean":"2024-09-19T07:36:17.892102+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:36:17.892102+0000","last_undegraded":"2024-09-19T07:36:17.892102+0000","last_fullsized":"2024-09-19T07:36:17.892102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892104+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:36:17.892104+0000","last_peered":"2024-09-19T07:36:17.892104+0000","last_clean":"2024-09-19T07:36:17.892104+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:36:17.892104+0000","last_undegraded":"2024-09-19T07:36:17.892104+0000","last_fullsized":"2024-09-19T07:36:17.892104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.897489+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:36:17.897489+0000","last_peered":"2024-09-19T07:36:17.897489+0000","last_clean":"2024-09-19T07:36:17.897489+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:36:17.897489+0000","last_undegraded":"2024-09-19T07:36:17.897489+0000","last_fullsized":"2024-09-19T07:36:17.897489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889770+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:36:17.889770+0000","last_peered":"2024-09-19T07:36:17.889770+0000","last_clean":"2024-09-19T07:36:17.889770+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:36:17.889770+0000","last_undegraded":"2024-09-19T07:36:17.889770+0000","last_fullsized":"2024-09-19T07:36:17.889770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895916+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:36:17.895916+0000","last_peered":"2024-09-19T07:36:17.895916+0000","last_clean":"2024-09-19T07:36:17.895916+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:36:17.895916+0000","last_undegraded":"2024-09-19T07:36:17.895916+0000","last_fullsized":"2024-09-19T07:36:17.895916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891525+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:36:17.891525+0000","last_peered":"2024-09-19T07:36:17.891525+0000","last_clean":"2024-09-19T07:36:17.891525+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:36:17.891525+0000","last_undegraded":"2024-09-19T07:36:17.891525+0000","last_fullsized":"2024-09-19T07:36:17.891525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890710+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:36:17.890710+0000","last_peered":"2024-09-19T07:36:17.890710+0000","last_clean":"2024-09-19T07:36:17.890710+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:36:17.890710+0000","last_undegraded":"2024-09-19T07:36:17.890710+0000","last_fullsized":"2024-09-19T07:36:17.890710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891804+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:36:17.891804+0000","last_peered":"2024-09-19T07:36:17.891804+0000","last_clean":"2024-09-19T07:36:17.891804+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:36:17.891804+0000","last_undegraded":"2024-09-19T07:36:17.891804+0000","last_fullsized":"2024-09-19T07:36:17.891804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892254+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:36:17.892254+0000","last_peered":"2024-09-19T07:36:17.892254+0000","last_clean":"2024-09-19T07:36:17.892254+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:36:17.892254+0000","last_undegraded":"2024-09-19T07:36:17.892254+0000","last_fullsized":"2024-09-19T07:36:17.892254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893575+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:36:17.893575+0000","last_peered":"2024-09-19T07:36:17.893575+0000","last_clean":"2024-09-19T07:36:17.893575+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:36:17.893575+0000","last_undegraded":"2024-09-19T07:36:17.893575+0000","last_fullsized":"2024-09-19T07:36:17.893575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889691+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:36:17.889691+0000","last_peered":"2024-09-19T07:36:17.889691+0000","last_clean":"2024-09-19T07:36:17.889691+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:36:17.889691+0000","last_undegraded":"2024-09-19T07:36:17.889691+0000","last_fullsized":"2024-09-19T07:36:17.889691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895765+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:36:17.895765+0000","last_peered":"2024-09-19T07:36:17.895765+0000","last_clean":"2024-09-19T07:36:17.895765+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:36:17.895765+0000","last_undegraded":"2024-09-19T07:36:17.895765+0000","last_fullsized":"2024-09-19T07:36:17.895765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891796+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:36:17.891796+0000","last_peered":"2024-09-19T07:36:17.891796+0000","last_clean":"2024-09-19T07:36:17.891796+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:36:17.891796+0000","last_undegraded":"2024-09-19T07:36:17.891796+0000","last_fullsized":"2024-09-19T07:36:17.891796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892016+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:36:17.892016+0000","last_peered":"2024-09-19T07:36:17.892016+0000","last_clean":"2024-09-19T07:36:17.892016+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:36:17.892016+0000","last_undegraded":"2024-09-19T07:36:17.892016+0000","last_fullsized":"2024-09-19T07:36:17.892016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894219+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:36:17.894219+0000","last_peered":"2024-09-19T07:36:17.894219+0000","last_clean":"2024-09-19T07:36:17.894219+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:36:17.894219+0000","last_undegraded":"2024-09-19T07:36:17.894219+0000","last_fullsized":"2024-09-19T07:36:17.894219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895304+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:36:17.895304+0000","last_peered":"2024-09-19T07:36:17.895304+0000","last_clean":"2024-09-19T07:36:17.895304+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:36:17.895304+0000","last_undegraded":"2024-09-19T07:36:17.895304+0000","last_fullsized":"2024-09-19T07:36:17.895304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.978446+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:37:07.978446+0000","last_peered":"2024-09-19T07:37:07.978446+0000","last_clean":"2024-09-19T07:37:07.978446+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:37:07.978446+0000","last_undegraded":"2024-09-19T07:37:07.978446+0000","last_fullsized":"2024-09-19T07:37:07.978446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890814+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:36:17.890814+0000","last_peered":"2024-09-19T07:36:17.890814+0000","last_clean":"2024-09-19T07:36:17.890814+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:36:17.890814+0000","last_undegraded":"2024-09-19T07:36:17.890814+0000","last_fullsized":"2024-09-19T07:36:17.890814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890526+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:36:17.890526+0000","last_peered":"2024-09-19T07:36:17.890526+0000","last_clean":"2024-09-19T07:36:17.890526+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:36:17.890526+0000","last_undegraded":"2024-09-19T07:36:17.890526+0000","last_fullsized":"2024-09-19T07:36:17.890526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889543+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:36:17.889543+0000","last_peered":"2024-09-19T07:36:17.889543+0000","last_clean":"2024-09-19T07:36:17.889543+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:36:17.889543+0000","last_undegraded":"2024-09-19T07:36:17.889543+0000","last_fullsized":"2024-09-19T07:36:17.889543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891490+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:36:17.891490+0000","last_peered":"2024-09-19T07:36:17.891490+0000","last_clean":"2024-09-19T07:36:17.891490+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:36:17.891490+0000","last_undegraded":"2024-09-19T07:36:17.891490+0000","last_fullsized":"2024-09-19T07:36:17.891490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907827+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:36:18.907827+0000","last_peered":"2024-09-19T07:36:18.907827+0000","last_clean":"2024-09-19T07:36:18.907827+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:36:18.907827+0000","last_undegraded":"2024-09-19T07:36:18.907827+0000","last_fullsized":"2024-09-19T07:36:18.907827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892021+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:36:17.892021+0000","last_peered":"2024-09-19T07:36:17.892021+0000","last_clean":"2024-09-19T07:36:17.892021+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:36:17.892021+0000","last_undegraded":"2024-09-19T07:36:17.892021+0000","last_fullsized":"2024-09-19T07:36:17.892021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":124,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.977797+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:37:07.977797+0000","last_peered":"2024-09-19T07:37:07.977797+0000","last_clean":"2024-09-19T07:37:07.977797+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:37:07.977797+0000","last_undegraded":"2024-09-19T07:37:07.977797+0000","last_fullsized":"2024-09-19T07:37:07.977797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896529+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:36:17.896529+0000","last_peered":"2024-09-19T07:36:17.896529+0000","last_clean":"2024-09-19T07:36:17.896529+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:36:17.896529+0000","last_undegraded":"2024-09-19T07:36:17.896529+0000","last_fullsized":"2024-09-19T07:36:17.896529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908690+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:36:18.908690+0000","last_peered":"2024-09-19T07:36:18.908690+0000","last_clean":"2024-09-19T07:36:18.908690+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:36:18.908690+0000","last_undegraded":"2024-09-19T07:36:18.908690+0000","last_fullsized":"2024-09-19T07:36:18.908690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.896039+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:36:17.896039+0000","last_peered":"2024-09-19T07:36:17.896039+0000","last_clean":"2024-09-19T07:36:17.896039+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:36:17.896039+0000","last_undegraded":"2024-09-19T07:36:17.896039+0000","last_fullsized":"2024-09-19T07:36:17.896039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890888+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:36:17.890888+0000","last_peered":"2024-09-19T07:36:17.890888+0000","last_clean":"2024-09-19T07:36:17.890888+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:36:17.890888+0000","last_undegraded":"2024-09-19T07:36:17.890888+0000","last_fullsized":"2024-09-19T07:36:17.890888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895996+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:36:17.895996+0000","last_peered":"2024-09-19T07:36:17.895996+0000","last_clean":"2024-09-19T07:36:17.895996+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:36:17.895996+0000","last_undegraded":"2024-09-19T07:36:17.895996+0000","last_fullsized":"2024-09-19T07:36:17.895996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'32","reported_seq":161,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.977544+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:37:07.977544+0000","last_peered":"2024-09-19T07:37:07.977544+0000","last_clean":"2024-09-19T07:37:07.977544+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:37:07.977544+0000","last_undegraded":"2024-09-19T07:37:07.977544+0000","last_fullsized":"2024-09-19T07:37:07.977544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":114,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895733+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:36:17.895733+0000","last_peered":"2024-09-19T07:36:17.895733+0000","last_clean":"2024-09-19T07:36:17.895733+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:36:17.895733+0000","last_undegraded":"2024-09-19T07:36:17.895733+0000","last_fullsized":"2024-09-19T07:36:17.895733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913146+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:36:19.913146+0000","last_peered":"2024-09-19T07:36:19.913146+0000","last_clean":"2024-09-19T07:36:19.913146+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:36:19.913146+0000","last_undegraded":"2024-09-19T07:36:19.913146+0000","last_fullsized":"2024-09-19T07:36:19.913146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889534+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:36:17.889534+0000","last_peered":"2024-09-19T07:36:17.889534+0000","last_clean":"2024-09-19T07:36:17.889534+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:36:17.889534+0000","last_undegraded":"2024-09-19T07:36:17.889534+0000","last_fullsized":"2024-09-19T07:36:17.889534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.906778+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:36:18.906778+0000","last_peered":"2024-09-19T07:36:18.906778+0000","last_clean":"2024-09-19T07:36:18.906778+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:36:18.906778+0000","last_undegraded":"2024-09-19T07:36:18.906778+0000","last_fullsized":"2024-09-19T07:36:18.906778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893599+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:36:17.893599+0000","last_peered":"2024-09-19T07:36:17.893599+0000","last_clean":"2024-09-19T07:36:17.893599+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:36:17.893599+0000","last_undegraded":"2024-09-19T07:36:17.893599+0000","last_fullsized":"2024-09-19T07:36:17.893599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:08.978813+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:37:08.978813+0000","last_peered":"2024-09-19T07:37:08.978813+0000","last_clean":"2024-09-19T07:37:08.978813+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:37:08.978813+0000","last_undegraded":"2024-09-19T07:37:08.978813+0000","last_fullsized":"2024-09-19T07:37:08.978813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893370+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:36:17.893370+0000","last_peered":"2024-09-19T07:36:17.893370+0000","last_clean":"2024-09-19T07:36:17.893370+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:36:17.893370+0000","last_undegraded":"2024-09-19T07:36:17.893370+0000","last_fullsized":"2024-09-19T07:36:17.893370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907635+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:36:18.907635+0000","last_peered":"2024-09-19T07:36:18.907635+0000","last_clean":"2024-09-19T07:36:18.907635+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:36:18.907635+0000","last_undegraded":"2024-09-19T07:36:18.907635+0000","last_fullsized":"2024-09-19T07:36:18.907635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890552+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:36:17.890552+0000","last_peered":"2024-09-19T07:36:17.890552+0000","last_clean":"2024-09-19T07:36:17.890552+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:36:17.890552+0000","last_undegraded":"2024-09-19T07:36:17.890552+0000","last_fullsized":"2024-09-19T07:36:17.890552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895827+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:36:17.895827+0000","last_peered":"2024-09-19T07:36:17.895827+0000","last_clean":"2024-09-19T07:36:17.895827+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:36:17.895827+0000","last_undegraded":"2024-09-19T07:36:17.895827+0000","last_fullsized":"2024-09-19T07:36:17.895827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893694+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:36:17.893694+0000","last_peered":"2024-09-19T07:36:17.893694+0000","last_clean":"2024-09-19T07:36:17.893694+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:36:17.893694+0000","last_undegraded":"2024-09-19T07:36:17.893694+0000","last_fullsized":"2024-09-19T07:36:17.893694+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908154+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:36:18.908154+0000","last_peered":"2024-09-19T07:36:18.908154+0000","last_clean":"2024-09-19T07:36:18.908154+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:36:18.908154+0000","last_undegraded":"2024-09-19T07:36:18.908154+0000","last_fullsized":"2024-09-19T07:36:18.908154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894155+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:36:17.894155+0000","last_peered":"2024-09-19T07:36:17.894155+0000","last_clean":"2024-09-19T07:36:17.894155+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:36:17.894155+0000","last_undegraded":"2024-09-19T07:36:17.894155+0000","last_fullsized":"2024-09-19T07:36:17.894155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907835+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:36:18.907835+0000","last_peered":"2024-09-19T07:36:18.907835+0000","last_clean":"2024-09-19T07:36:18.907835+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:36:18.907835+0000","last_undegraded":"2024-09-19T07:36:18.907835+0000","last_fullsized":"2024-09-19T07:36:18.907835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893835+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:36:17.893835+0000","last_peered":"2024-09-19T07:36:17.893835+0000","last_clean":"2024-09-19T07:36:17.893835+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:36:17.893835+0000","last_undegraded":"2024-09-19T07:36:17.893835+0000","last_fullsized":"2024-09-19T07:36:17.893835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":81,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:07.978259+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:37:07.978259+0000","last_peered":"2024-09-19T07:37:07.978259+0000","last_clean":"2024-09-19T07:37:07.978259+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:37:07.978259+0000","last_undegraded":"2024-09-19T07:37:07.978259+0000","last_fullsized":"2024-09-19T07:37:07.978259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894098+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:36:17.894098+0000","last_peered":"2024-09-19T07:36:17.894098+0000","last_clean":"2024-09-19T07:36:17.894098+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:36:17.894098+0000","last_undegraded":"2024-09-19T07:36:17.894098+0000","last_fullsized":"2024-09-19T07:36:17.894098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893546+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:36:17.893546+0000","last_peered":"2024-09-19T07:36:17.893546+0000","last_clean":"2024-09-19T07:36:17.893546+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:36:17.893546+0000","last_undegraded":"2024-09-19T07:36:17.893546+0000","last_fullsized":"2024-09-19T07:36:17.893546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913196+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:36:19.913196+0000","last_peered":"2024-09-19T07:36:19.913196+0000","last_clean":"2024-09-19T07:36:19.913196+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:36:19.913196+0000","last_undegraded":"2024-09-19T07:36:19.913196+0000","last_fullsized":"2024-09-19T07:36:19.913196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892747+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:36:17.892747+0000","last_peered":"2024-09-19T07:36:17.892747+0000","last_clean":"2024-09-19T07:36:17.892747+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:36:17.892747+0000","last_undegraded":"2024-09-19T07:36:17.892747+0000","last_fullsized":"2024-09-19T07:36:17.892747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893570+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:36:17.893570+0000","last_peered":"2024-09-19T07:36:17.893570+0000","last_clean":"2024-09-19T07:36:17.893570+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:36:17.893570+0000","last_undegraded":"2024-09-19T07:36:17.893570+0000","last_fullsized":"2024-09-19T07:36:17.893570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889475+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:36:17.889475+0000","last_peered":"2024-09-19T07:36:17.889475+0000","last_clean":"2024-09-19T07:36:17.889475+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:36:17.889475+0000","last_undegraded":"2024-09-19T07:36:17.889475+0000","last_fullsized":"2024-09-19T07:36:17.889475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":76,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:05.974744+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:37:05.974744+0000","last_peered":"2024-09-19T07:37:05.974744+0000","last_clean":"2024-09-19T07:37:05.974744+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:37:05.974744+0000","last_undegraded":"2024-09-19T07:37:05.974744+0000","last_fullsized":"2024-09-19T07:37:05.974744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890854+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:36:17.890854+0000","last_peered":"2024-09-19T07:36:17.890854+0000","last_clean":"2024-09-19T07:36:17.890854+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:36:17.890854+0000","last_undegraded":"2024-09-19T07:36:17.890854+0000","last_fullsized":"2024-09-19T07:36:17.890854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:09.979841+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:37:09.979841+0000","last_peered":"2024-09-19T07:37:09.979841+0000","last_clean":"2024-09-19T07:37:09.979841+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:37:09.979841+0000","last_undegraded":"2024-09-19T07:37:09.979841+0000","last_fullsized":"2024-09-19T07:37:09.979841+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890537+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:36:17.890537+0000","last_peered":"2024-09-19T07:36:17.890537+0000","last_clean":"2024-09-19T07:36:17.890537+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:36:17.890537+0000","last_undegraded":"2024-09-19T07:36:17.890537+0000","last_fullsized":"2024-09-19T07:36:17.890537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.892154+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:36:17.892154+0000","last_peered":"2024-09-19T07:36:17.892154+0000","last_clean":"2024-09-19T07:36:17.892154+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:36:17.892154+0000","last_undegraded":"2024-09-19T07:36:17.892154+0000","last_fullsized":"2024-09-19T07:36:17.892154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.890794+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:36:17.890794+0000","last_peered":"2024-09-19T07:36:17.890794+0000","last_clean":"2024-09-19T07:36:17.890794+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:36:17.890794+0000","last_undegraded":"2024-09-19T07:36:17.890794+0000","last_fullsized":"2024-09-19T07:36:17.890794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.909170+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:36:18.909170+0000","last_peered":"2024-09-19T07:36:18.909170+0000","last_clean":"2024-09-19T07:36:18.909170+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:36:18.909170+0000","last_undegraded":"2024-09-19T07:36:18.909170+0000","last_fullsized":"2024-09-19T07:36:18.909170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.889701+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:36:17.889701+0000","last_peered":"2024-09-19T07:36:17.889701+0000","last_clean":"2024-09-19T07:36:17.889701+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:36:17.889701+0000","last_undegraded":"2024-09-19T07:36:17.889701+0000","last_fullsized":"2024-09-19T07:36:17.889701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'14","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:31.853047+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:36:31.853047+0000","last_peered":"2024-09-19T07:36:31.853047+0000","last_clean":"2024-09-19T07:36:31.853047+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:36:31.853047+0000","last_undegraded":"2024-09-19T07:36:31.853047+0000","last_fullsized":"2024-09-19T07:36:31.853047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891890+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:36:17.891890+0000","last_peered":"2024-09-19T07:36:17.891890+0000","last_clean":"2024-09-19T07:36:17.891890+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:36:17.891890+0000","last_undegraded":"2024-09-19T07:36:17.891890+0000","last_fullsized":"2024-09-19T07:36:17.891890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907864+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:36:18.907864+0000","last_peered":"2024-09-19T07:36:18.907864+0000","last_clean":"2024-09-19T07:36:18.907864+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:36:18.907864+0000","last_undegraded":"2024-09-19T07:36:18.907864+0000","last_fullsized":"2024-09-19T07:36:18.907864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895903+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:36:17.895903+0000","last_peered":"2024-09-19T07:36:17.895903+0000","last_clean":"2024-09-19T07:36:17.895903+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:36:17.895903+0000","last_undegraded":"2024-09-19T07:36:17.895903+0000","last_fullsized":"2024-09-19T07:36:17.895903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895928+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:36:17.895928+0000","last_peered":"2024-09-19T07:36:17.895928+0000","last_clean":"2024-09-19T07:36:17.895928+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:36:17.895928+0000","last_undegraded":"2024-09-19T07:36:17.895928+0000","last_fullsized":"2024-09-19T07:36:17.895928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894935+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:36:17.894935+0000","last_peered":"2024-09-19T07:36:17.894935+0000","last_clean":"2024-09-19T07:36:17.894935+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:36:17.894935+0000","last_undegraded":"2024-09-19T07:36:17.894935+0000","last_fullsized":"2024-09-19T07:36:17.894935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.907171+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:36:18.907171+0000","last_peered":"2024-09-19T07:36:18.907171+0000","last_clean":"2024-09-19T07:36:18.907171+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:36:18.907171+0000","last_undegraded":"2024-09-19T07:36:18.907171+0000","last_fullsized":"2024-09-19T07:36:18.907171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.894276+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:36:17.894276+0000","last_peered":"2024-09-19T07:36:17.894276+0000","last_clean":"2024-09-19T07:36:17.894276+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:36:17.894276+0000","last_undegraded":"2024-09-19T07:36:17.894276+0000","last_fullsized":"2024-09-19T07:36:17.894276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.141159+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:36:18.141159+0000","last_peered":"2024-09-19T07:36:18.141159+0000","last_clean":"2024-09-19T07:36:18.141159+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:36:18.141159+0000","last_undegraded":"2024-09-19T07:36:18.141159+0000","last_fullsized":"2024-09-19T07:36:18.141159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891645+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:36:17.891645+0000","last_peered":"2024-09-19T07:36:17.891645+0000","last_clean":"2024-09-19T07:36:17.891645+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:36:17.891645+0000","last_undegraded":"2024-09-19T07:36:17.891645+0000","last_fullsized":"2024-09-19T07:36:17.891645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895305+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:36:17.895305+0000","last_peered":"2024-09-19T07:36:17.895305+0000","last_clean":"2024-09-19T07:36:17.895305+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:36:17.895305+0000","last_undegraded":"2024-09-19T07:36:17.895305+0000","last_fullsized":"2024-09-19T07:36:17.895305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.893736+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:36:17.893736+0000","last_peered":"2024-09-19T07:36:17.893736+0000","last_clean":"2024-09-19T07:36:17.893736+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:36:17.893736+0000","last_undegraded":"2024-09-19T07:36:17.893736+0000","last_fullsized":"2024-09-19T07:36:17.893736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.936693+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:36:17.936693+0000","last_peered":"2024-09-19T07:36:17.936693+0000","last_clean":"2024-09-19T07:36:17.936693+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:36:17.936693+0000","last_undegraded":"2024-09-19T07:36:17.936693+0000","last_fullsized":"2024-09-19T07:36:17.936693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:18.908619+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:36:18.908619+0000","last_peered":"2024-09-19T07:36:18.908619+0000","last_clean":"2024-09-19T07:36:18.908619+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:36:18.908619+0000","last_undegraded":"2024-09-19T07:36:18.908619+0000","last_fullsized":"2024-09-19T07:36:18.908619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895842+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:36:17.895842+0000","last_peered":"2024-09-19T07:36:17.895842+0000","last_clean":"2024-09-19T07:36:17.895842+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:36:17.895842+0000","last_undegraded":"2024-09-19T07:36:17.895842+0000","last_fullsized":"2024-09-19T07:36:17.895842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:19.913109+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:36:19.913109+0000","last_peered":"2024-09-19T07:36:19.913109+0000","last_clean":"2024-09-19T07:36:19.913109+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:36:19.913109+0000","last_undegraded":"2024-09-19T07:36:19.913109+0000","last_fullsized":"2024-09-19T07:36:19.913109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.895948+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:36:17.895948+0000","last_peered":"2024-09-19T07:36:17.895948+0000","last_clean":"2024-09-19T07:36:17.895948+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:36:17.895948+0000","last_undegraded":"2024-09-19T07:36:17.895948+0000","last_fullsized":"2024-09-19T07:36:17.895948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891880+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:36:17.891880+0000","last_peered":"2024-09-19T07:36:17.891880+0000","last_clean":"2024-09-19T07:36:17.891880+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:36:17.891880+0000","last_undegraded":"2024-09-19T07:36:17.891880+0000","last_fullsized":"2024-09-19T07:36:17.891880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:36:17.891708+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:36:17.891708+0000","last_peered":"2024-09-19T07:36:17.891708+0000","last_clean":"2024-09-19T07:36:17.891708+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:36:17.891708+0000","last_undegraded":"2024-09-19T07:36:17.891708+0000","last_fullsized":"2024-09-19T07:36:17.891708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T07:37:05.974487+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:37:05.974487+0000","last_peered":"2024-09-19T07:37:05.974487+0000","last_clean":"2024-09-19T07:37:05.974487+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:37:05.974487+0000","last_undegraded":"2024-09-19T07:37:05.974487+0000","last_fullsized":"2024-09-19T07:37:05.974487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69911,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":537,"num_read_kb":341,"num_write":245,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":209733,"data_compressed":4779,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247252,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":442143,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1095370,"data_compressed":43192,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441568,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":441162,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364846,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1034859,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528371,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441678,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1033751,"data_compressed":41599,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":921600,"data_stored":441539,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441110,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":505121,"data_compressed":7402,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":505573,"data_compressed":7402,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":443017,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34151,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65127,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64675,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64439,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:37:13.856 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T07:37:13.856 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T07:37:13.857 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T07:37:13.857 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T07:37:14.106 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:14.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:14 smithi103 ceph-mon[31201]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:37:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:14 smithi134 ceph-mon[35414]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:37:14.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 -- 172.21.15.103:0/2006348639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc102d40 msgr2=0x7fb1cc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 --2- 172.21.15.103:0/2006348639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc102d40 0x7fb1cc103120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc02a480 comp rx=0 tx=0).stop 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 -- 172.21.15.103:0/2006348639 shutdown_connections 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 --2- 172.21.15.103:0/2006348639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1cc10c170 0x7fb1cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 --2- 172.21.15.103:0/2006348639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1cc103660 0x7fb1cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 --2- 172.21.15.103:0/2006348639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc102d40 0x7fb1cc103120 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 -- 172.21.15.103:0/2006348639 >> 172.21.15.103:0/2006348639 conn(0x7fb1cc0fc7a0 msgr2=0x7fb1cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 -- 172.21.15.103:0/2006348639 shutdown_connections 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.571+0000 7fb1d1988640 1 -- 172.21.15.103:0/2006348639 wait complete. 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 Processor -- start 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 -- start start 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1cc102d40 0x7fb1cc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:14.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc103660 0x7fb1cc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1cc10c170 0x7fb1cc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc111f70 con 0x7fb1cc103660 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc111df0 con 0x7fb1cc10c170 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1d1988640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc1120f0 con 0x7fb1cc102d40 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc103660 0x7fb1cc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1c27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc103660 0x7fb1cc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56542/0 (socket says 172.21.15.103:56542) 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1c27fc640 1 -- 172.21.15.103:0/3259717311 learned_addr learned my addr 172.21.15.103:0/3259717311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1caffd640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1cc102d40 0x7fb1cc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.572+0000 7fb1cb7fe640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1cc10c170 0x7fb1cc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1c27fc640 1 -- 172.21.15.103:0/3259717311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1cc102d40 msgr2=0x7fb1cc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1c27fc640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1cc102d40 0x7fb1cc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1c27fc640 1 -- 172.21.15.103:0/3259717311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1cc10c170 msgr2=0x7fb1cc19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1c27fc640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1cc10c170 0x7fb1cc19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1c27fc640 1 -- 172.21.15.103:0/3259717311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc19d400 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1c27fc640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc103660 0x7fb1cc198990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400d740 tx=0x7fb1b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1d0986640 1 -- 172.21.15.103:0/3259717311 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4017070 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1d0986640 1 -- 172.21.15.103:0/3259717311 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b400de50 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1cc19d6f0 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1d0986640 1 -- 172.21.15.103:0/3259717311 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b400f620 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.573+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1cc1a4fd0 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.574+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1cc103120 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.575+0000 7fb1d0986640 1 -- 172.21.15.103:0/3259717311 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb1b4009c30 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.576+0000 7fb1d0986640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb1b80a3350 0x7fb1b80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.576+0000 7fb1caffd640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb1b80a3350 0x7fb1b80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.576+0000 7fb1d0986640 1 -- 172.21.15.103:0/3259717311 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fb1b40cf570 con 0x7fb1cc103660 2024-09-19T07:37:14.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.576+0000 7fb1caffd640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb1b80a3350 0x7fb1b80a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009230 tx=0x7fb1bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:14.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.580+0000 7fb1d0986640 1 -- 172.21.15.103:0/3259717311 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb1b409bae0 con 0x7fb1cc103660 2024-09-19T07:37:14.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:14 smithi174 ceph-mon[35370]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:37:14.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.784+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb1cc106c80 con 0x7fb1cc103660 2024-09-19T07:37:14.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.784+0000 7fb1d0986640 1 -- 172.21.15.103:0/3259717311 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb1b409fde0 con 0x7fb1cc103660 2024-09-19T07:37:14.785 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:14.786 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T07:37:14.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb1b80a3350 msgr2=0x7fb1b80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb1b80a3350 0x7fb1b80a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009230 tx=0x7fb1bc0058d0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc103660 msgr2=0x7fb1cc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc103660 0x7fb1cc198990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400d740 tx=0x7fb1b400dc00 comp rx=0 tx=0).stop 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 shutdown_connections 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fb1b80a3350 0x7fb1b80a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1cc10c170 0x7fb1cc19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1cc103660 0x7fb1cc198990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 --2- 172.21.15.103:0/3259717311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1cc102d40 0x7fb1cc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 >> 172.21.15.103:0/3259717311 conn(0x7fb1cc0fc7a0 msgr2=0x7fb1cc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 shutdown_connections 2024-09-19T07:37:14.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:14.787+0000 7fb1d1988640 1 -- 172.21.15.103:0/3259717311 wait complete. 2024-09-19T07:37:15.456 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:15 smithi103 ceph-mon[31201]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:15.456 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3259717311' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T07:37:15.489 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T07:37:15.489 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:15 smithi134 ceph-mon[35414]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3259717311' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T07:37:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:15 smithi174 ceph-mon[35370]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3259717311' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T07:37:15.739 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:16.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 -- 172.21.15.103:0/3681053600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 msgr2=0x7fcc58106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 --2- 172.21.15.103:0/3681053600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58106e50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009930 tx=0x7fcc4802a420 comp rx=0 tx=0).stop 2024-09-19T07:37:16.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 -- 172.21.15.103:0/3681053600 shutdown_connections 2024-09-19T07:37:16.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 --2- 172.21.15.103:0/3681053600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58101410 0x7fcc5810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 --2- 172.21.15.103:0/3681053600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc58100a70 0x7fcc58100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 --2- 172.21.15.103:0/3681053600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58106e50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 -- 172.21.15.103:0/3681053600 >> 172.21.15.103:0/3681053600 conn(0x7fcc580fc790 msgr2=0x7fcc580febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 -- 172.21.15.103:0/3681053600 shutdown_connections 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.172+0000 7fcc5da78640 1 -- 172.21.15.103:0/3681053600 wait complete. 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 Processor -- start 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 -- start start 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc58100a70 0x7fcc5819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58101410 0x7fcc581963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58110f10 con 0x7fcc58106a70 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58110d90 con 0x7fcc58101410 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc5da78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58111090 con 0x7fcc58100a70 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc577fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc58100a70 0x7fcc5819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56566/0 (socket says 172.21.15.103:56566) 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc57fff640 1 -- 172.21.15.103:0/2482383098 learned_addr learned my addr 172.21.15.103:0/2482383098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.173+0000 7fcc56ffd640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58101410 0x7fcc581963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc57fff640 1 -- 172.21.15.103:0/2482383098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc58100a70 msgr2=0x7fcc5819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc57fff640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc58100a70 0x7fcc5819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc57fff640 1 -- 172.21.15.103:0/2482383098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58101410 msgr2=0x7fcc581963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc57fff640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58101410 0x7fcc581963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc57fff640 1 -- 172.21.15.103:0/2482383098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc58196e60 con 0x7fcc58106a70 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc57fff640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58196920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b970 tx=0x7fcc5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc54ff9640 1 -- 172.21.15.103:0/2482383098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50019070 con 0x7fcc58106a70 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc54ff9640 1 -- 172.21.15.103:0/2482383098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc50014440 con 0x7fcc58106a70 2024-09-19T07:37:16.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc58197150 con 0x7fcc58106a70 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc54ff9640 1 -- 172.21.15.103:0/2482383098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc500075d0 con 0x7fcc58106a70 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.174+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581b3eb0 con 0x7fcc58106a70 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.175+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc581021b0 con 0x7fcc58106a70 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.176+0000 7fcc54ff9640 1 -- 172.21.15.103:0/2482383098 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fcc5001c070 con 0x7fcc58106a70 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.177+0000 7fcc54ff9640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcc400a32d0 0x7fcc400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.177+0000 7fcc577fe640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcc400a32d0 0x7fcc400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.177+0000 7fcc54ff9640 1 -- 172.21.15.103:0/2482383098 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fcc500cef10 con 0x7fcc58106a70 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.178+0000 7fcc577fe640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcc400a32d0 0x7fcc400a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009900 tx=0x7fcc480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:16.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.182+0000 7fcc54ff9640 1 -- 172.21.15.103:0/2482383098 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc5009b590 con 0x7fcc58106a70 2024-09-19T07:37:16.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.383+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc581b4140 con 0x7fcc58106a70 2024-09-19T07:37:16.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.384+0000 7fcc54ff9640 1 -- 172.21.15.103:0/2482383098 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11991 (secure 0 0 0) 0x7fcc5009f890 con 0x7fcc58106a70 2024-09-19T07:37:16.385 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:16.385 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T07:36:19:905223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:36:19.905198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:37:16.386 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T07:37:16.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcc400a32d0 msgr2=0x7fcc400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcc400a32d0 0x7fcc400a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009900 tx=0x7fcc480058d0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 msgr2=0x7fcc58196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58196920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b970 tx=0x7fcc5000bd70 comp rx=0 tx=0).stop 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 shutdown_connections 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fcc400a32d0 0x7fcc400a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc58106a70 0x7fcc58196920 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58101410 0x7fcc581963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 --2- 172.21.15.103:0/2482383098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc58100a70 0x7fcc5819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 >> 172.21.15.103:0/2482383098 conn(0x7fcc580fc790 msgr2=0x7fcc5806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:16.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 shutdown_connections 2024-09-19T07:37:16.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:16.386+0000 7fcc5da78640 1 -- 172.21.15.103:0/2482383098 wait complete. 2024-09-19T07:37:17.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:17.274 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:17.578 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:17 smithi103 ceph-mon[31201]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:17.578 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:17 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2482383098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:17.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:17 smithi134 ceph-mon[35414]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:17.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:17 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2482383098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:17 smithi174 ceph-mon[35370]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:17 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2482383098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:17.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 -- 172.21.15.103:0/35681616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468106f10 msgr2=0x7fe46810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 --2- 172.21.15.103:0/35681616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468106f10 0x7fe46810ea90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe460009ca0 tx=0x7fe46002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 -- 172.21.15.103:0/35681616 shutdown_connections 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 --2- 172.21.15.103:0/35681616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468106f10 0x7fe46810ea90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 --2- 172.21.15.103:0/35681616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe468102790 0x7fe4681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 --2- 172.21.15.103:0/35681616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe468101de0 0x7fe4681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 -- 172.21.15.103:0/35681616 >> 172.21.15.103:0/35681616 conn(0x7fe4680fd8c0 msgr2=0x7fe4680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 -- 172.21.15.103:0/35681616 shutdown_connections 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.741+0000 7fe46fd62640 1 -- 172.21.15.103:0/35681616 wait complete. 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 Processor -- start 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 -- start start 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468101de0 0x7fe4681983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe468102790 0x7fe468198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe468106f10 0x7fe46819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe468111f70 con 0x7fe468101de0 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe468111df0 con 0x7fe468106f10 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46fd62640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681120f0 con 0x7fe468102790 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468101de0 0x7fe4681983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468101de0 0x7fe4681983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56586/0 (socket says 172.21.15.103:56586) 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 -- 172.21.15.103:0/378156499 learned_addr learned my addr 172.21.15.103:0/378156499 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46d2fd640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe468102790 0x7fe468198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 -- 172.21.15.103:0/378156499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe468102790 msgr2=0x7fe468198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe468102790 0x7fe468198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 -- 172.21.15.103:0/378156499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe468106f10 msgr2=0x7fe46819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe468106f10 0x7fe46819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 -- 172.21.15.103:0/378156499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46819d3a0 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.742+0000 7fe46dafe640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468101de0 0x7fe4681983f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe45800d950 tx=0x7fe45800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.743+0000 7fe45effd640 1 -- 172.21.15.103:0/378156499 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458017030 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.743+0000 7fe45effd640 1 -- 172.21.15.103:0/378156499 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe458004020 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.743+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46819d690 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.743+0000 7fe45effd640 1 -- 172.21.15.103:0/378156499 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4580047a0 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.743+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681a4f70 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.744+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4681021c0 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.745+0000 7fe45effd640 1 -- 172.21.15.103:0/378156499 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe4580041d0 con 0x7fe468101de0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.746+0000 7fe45effd640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4540a32d0 0x7fe4540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.746+0000 7fe46d2fd640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4540a32d0 0x7fe4540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:17.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.746+0000 7fe45effd640 1 -- 172.21.15.103:0/378156499 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe4580cfb20 con 0x7fe468101de0 2024-09-19T07:37:17.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.746+0000 7fe46d2fd640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4540a32d0 0x7fe4540a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe4680fdce0 tx=0x7fe46400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:17.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.749+0000 7fe45effd640 1 -- 172.21.15.103:0/378156499 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe45809c6e0 con 0x7fe468101de0 2024-09-19T07:37:17.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.955+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe468199750 con 0x7fe468101de0 2024-09-19T07:37:17.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.955+0000 7fe45effd640 1 -- 172.21.15.103:0/378156499 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11991 (secure 0 0 0) 0x7fe45809bfe0 con 0x7fe468101de0 2024-09-19T07:37:17.956 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:17.957 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T07:36:19:905223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:36:19.905198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:37:17.957 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T07:37:17.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4540a32d0 msgr2=0x7fe4540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4540a32d0 0x7fe4540a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe4680fdce0 tx=0x7fe46400b390 comp rx=0 tx=0).stop 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468101de0 msgr2=0x7fe4681983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468101de0 0x7fe4681983f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe45800d950 tx=0x7fe45800de10 comp rx=0 tx=0).stop 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 shutdown_connections 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4540a32d0 0x7fe4540a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe468106f10 0x7fe46819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe468102790 0x7fe468198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 --2- 172.21.15.103:0/378156499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe468101de0 0x7fe4681983f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 >> 172.21.15.103:0/378156499 conn(0x7fe4680fd8c0 msgr2=0x7fe4680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 shutdown_connections 2024-09-19T07:37:17.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:17.958+0000 7fe46fd62640 1 -- 172.21.15.103:0/378156499 wait complete. 2024-09-19T07:37:18.568 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:18 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/378156499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:18 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/378156499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:18.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:37:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:18 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/378156499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:18.845 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 -- 172.21.15.103:0/3670061016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 msgr2=0x7ff51c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:19.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 --2- 172.21.15.103:0/3670061016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c1006d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff514009930 tx=0x7ff51402a420 comp rx=0 tx=0).stop 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 -- 172.21.15.103:0/3670061016 shutdown_connections 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 --2- 172.21.15.103:0/3670061016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c106f50 0x7ff51c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 --2- 172.21.15.103:0/3670061016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff51c100ca0 0x7ff51c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 --2- 172.21.15.103:0/3670061016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c1006d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 -- 172.21.15.103:0/3670061016 >> 172.21.15.103:0/3670061016 conn(0x7ff51c077210 msgr2=0x7ff51c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 -- 172.21.15.103:0/3670061016 shutdown_connections 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.294+0000 7ff522e92640 1 -- 172.21.15.103:0/3670061016 wait complete. 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 Processor -- start 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 -- start start 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:19.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c100ca0 0x7ff51c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff51c106f50 0x7ff51c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c112080 con 0x7ff51c1002f0 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c111f00 con 0x7ff51c106f50 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff522e92640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c112200 con 0x7ff51c100ca0 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40858/0 (socket says 172.21.15.103:40858) 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 -- 172.21.15.103:0/3074800503 learned_addr learned my addr 172.21.15.103:0/3074800503 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 -- 172.21.15.103:0/3074800503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c100ca0 msgr2=0x7ff51c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff51bfff640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c100ca0 0x7ff51c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c100ca0 0x7ff51c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 -- 172.21.15.103:0/3074800503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff51c106f50 msgr2=0x7ff51c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:19.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff51c106f50 0x7ff51c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 -- 172.21.15.103:0/3074800503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c19d5f0 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff51bfff640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c100ca0 0x7ff51c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.295+0000 7ff520c2e640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c198640 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff514009a50 tx=0x7ff514004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.296+0000 7ff519ffb640 1 -- 172.21.15.103:0/3074800503 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51403f030 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.296+0000 7ff519ffb640 1 -- 172.21.15.103:0/3074800503 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff514003ce0 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.296+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c19d880 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.296+0000 7ff519ffb640 1 -- 172.21.15.103:0/3074800503 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514035070 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.296+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1a51c0 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.297+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c100750 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.298+0000 7ff519ffb640 1 -- 172.21.15.103:0/3074800503 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff514042030 con 0x7ff51c1002f0 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.299+0000 7ff519ffb640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5080a3350 0x7ff5080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:19.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.299+0000 7ff51bfff640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5080a3350 0x7ff5080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.299+0000 7ff519ffb640 1 -- 172.21.15.103:0/3074800503 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7ff5140f04f0 con 0x7ff51c1002f0 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.299+0000 7ff51bfff640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5080a3350 0x7ff5080a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0ff8c0 tx=0x7ff50c0073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:19.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.302+0000 7ff519ffb640 1 -- 172.21.15.103:0/3074800503 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5140bd070 con 0x7ff51c1002f0 2024-09-19T07:37:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:19 smithi103 ceph-mon[31201]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:19.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.470+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff51c1999a0 con 0x7ff51c1002f0 2024-09-19T07:37:19.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.471+0000 7ff519ffb640 1 -- 172.21.15.103:0/3074800503 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22818 (secure 0 0 0) 0x7ff5140bc970 con 0x7ff51c1002f0 2024-09-19T07:37:19.499 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:19.499 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:17.882336+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5080a3350 msgr2=0x7ff5080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5080a3350 0x7ff5080a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0ff8c0 tx=0x7ff50c0073c0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 msgr2=0x7ff51c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c198640 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff514009a50 tx=0x7ff514004270 comp rx=0 tx=0).stop 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 shutdown_connections 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7ff5080a3350 0x7ff5080a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff51c106f50 0x7ff51c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c100ca0 0x7ff51c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 --2- 172.21.15.103:0/3074800503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff51c1002f0 0x7ff51c198640 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:19.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.473+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 >> 172.21.15.103:0/3074800503 conn(0x7ff51c077210 msgr2=0x7ff51c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:19.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.474+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 shutdown_connections 2024-09-19T07:37:19.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:19.474+0000 7ff522e92640 1 -- 172.21.15.103:0/3074800503 wait complete. 2024-09-19T07:37:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:19 smithi134 ceph-mon[35414]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:19 smithi174 ceph-mon[35370]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:20.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:20 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3074800503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:20 smithi103 ceph-mon[31201]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:20 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3074800503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:20 smithi134 ceph-mon[35414]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:20.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:20 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3074800503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:20.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:20 smithi174 ceph-mon[35370]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:20.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.919+0000 7f9f3e89e640 1 -- 172.21.15.103:0/897065614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38101bc0 msgr2=0x7f9f3810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:20.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.919+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/897065614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38101bc0 0x7f9f3810d910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009ca0 tx=0x7f9f3002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:20.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.919+0000 7f9f3e89e640 1 -- 172.21.15.103:0/897065614 shutdown_connections 2024-09-19T07:37:20.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.919+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/897065614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38101bc0 0x7f9f3810d910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:20.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.920+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/897065614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f38101240 0x7f9f38101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:20.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.920+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/897065614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f38105a90 0x7f9f38105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.920+0000 7f9f3e89e640 1 -- 172.21.15.103:0/897065614 >> 172.21.15.103:0/897065614 conn(0x7f9f38077210 msgr2=0x7f9f380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.920+0000 7f9f3e89e640 1 -- 172.21.15.103:0/897065614 shutdown_connections 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.920+0000 7f9f3e89e640 1 -- 172.21.15.103:0/897065614 wait complete. 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.920+0000 7f9f3e89e640 1 Processor -- start 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.920+0000 7f9f3e89e640 1 -- start start 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f38101240 0x7f9f3819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f38101bc0 0x7f9f3819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38105a90 0x7f9f3819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38110f70 con 0x7f9f38105a90 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38110df0 con 0x7f9f38101bc0 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381110f0 con 0x7f9f38101240 2024-09-19T07:37:20.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38105a90 0x7f9f3819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38105a90 0x7f9f3819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40880/0 (socket says 172.21.15.103:40880) 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 -- 172.21.15.103:0/1113531912 learned_addr learned my addr 172.21.15.103:0/1113531912 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f37fff640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f38101240 0x7f9f3819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f377fe640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f38101bc0 0x7f9f3819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 -- 172.21.15.103:0/1113531912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f38101240 msgr2=0x7f9f3819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f38101240 0x7f9f3819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 -- 172.21.15.103:0/1113531912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f38101bc0 msgr2=0x7f9f3819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f38101bc0 0x7f9f3819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 -- 172.21.15.103:0/1113531912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3819f5b0 con 0x7f9f38105a90 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3ce3b640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38105a90 0x7f9f3819eed0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006010 tx=0x7f9f30003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f357fa640 1 -- 172.21.15.103:0/1113531912 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30031650 con 0x7f9f38105a90 2024-09-19T07:37:20.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3819f840 con 0x7f9f38105a90 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f357fa640 1 -- 172.21.15.103:0/1113531912 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f300317f0 con 0x7f9f38105a90 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f357fa640 1 -- 172.21.15.103:0/1113531912 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f300309e0 con 0x7f9f38105a90 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.921+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f381a7180 con 0x7f9f38105a90 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.923+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f38101680 con 0x7f9f38105a90 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.924+0000 7f9f357fa640 1 -- 172.21.15.103:0/1113531912 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9f30030c00 con 0x7f9f38105a90 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.924+0000 7f9f357fa640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9f200a32d0 0x7f9f200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.924+0000 7f9f37fff640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9f200a32d0 0x7f9f200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.924+0000 7f9f357fa640 1 -- 172.21.15.103:0/1113531912 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f9f300f0e10 con 0x7f9f38105a90 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.925+0000 7f9f37fff640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9f200a32d0 0x7f9f200a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800bbe0 tx=0x7f9f2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:20.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:20.929+0000 7f9f357fa640 1 -- 172.21.15.103:0/1113531912 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9f300bdae0 con 0x7f9f38105a90 2024-09-19T07:37:21.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.131+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f380fdcd0 con 0x7f9f38105a90 2024-09-19T07:37:21.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.132+0000 7f9f357fa640 1 -- 172.21.15.103:0/1113531912 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11991 (secure 0 0 0) 0x7f9f300bd3e0 con 0x7f9f38105a90 2024-09-19T07:37:21.133 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:21.133 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T07:36:19:905223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:36:19.905198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:37:21.133 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T07:37:21.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9f200a32d0 msgr2=0x7f9f200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:21.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9f200a32d0 0x7f9f200a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800bbe0 tx=0x7f9f2800b390 comp rx=0 tx=0).stop 2024-09-19T07:37:21.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38105a90 msgr2=0x7f9f3819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:21.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38105a90 0x7f9f3819eed0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006010 tx=0x7f9f30003f10 comp rx=0 tx=0).stop 2024-09-19T07:37:21.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 shutdown_connections 2024-09-19T07:37:21.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f9f200a32d0 0x7f9f200a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:21.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f38105a90 0x7f9f3819eed0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:21.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f38101bc0 0x7f9f3819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:21.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 --2- 172.21.15.103:0/1113531912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f38101240 0x7f9f3819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:21.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 >> 172.21.15.103:0/1113531912 conn(0x7f9f38077210 msgr2=0x7f9f38104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:21.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 shutdown_connections 2024-09-19T07:37:21.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:21.134+0000 7f9f3e89e640 1 -- 172.21.15.103:0/1113531912 wait complete. 2024-09-19T07:37:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:21 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1113531912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:21 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1113531912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:21.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:21 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1113531912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:21.757 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T07:36:08.395229+0000', 'modified': '2024-09-19T07:36:19.905198+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24494, 'mds_1': 24466, 'mds_2': 24455, 'mds_3': 24454, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14568': {'gid': 14568, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/1660338093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1660338093}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1660338093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14580': {'gid': 14580, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/1521905484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 1521905484}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 1521905484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2440407987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2440407987}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2440407987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/1885230172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 1885230172}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 1885230172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/19326296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 19326296}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 19326296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6835/214062550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 214062550}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 214062550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/2527370860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2527370860}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2527370860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6837/3959604171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3959604171}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3959604171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/2514182394', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2514182394}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2514182394}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/3683561216', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3683561216}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3683561216}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24466, 24494, 24455, 24454, 24448]} 2024-09-19T07:37:21.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T07:37:21.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T07:37:22.005 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:22.417 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:22 smithi103 ceph-mon[31201]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:22.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 -- 172.21.15.103:0/2961531206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 msgr2=0x7fd12810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/2961531206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12810eaa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd120009ca0 tx=0x7fd12002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 -- 172.21.15.103:0/2961531206 shutdown_connections 2024-09-19T07:37:22.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/2961531206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12810eaa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/2961531206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1281027a0 0x7fd1281067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/2961531206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128101df0 0x7fd1281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 -- 172.21.15.103:0/2961531206 >> 172.21.15.103:0/2961531206 conn(0x7fd1280fd8f0 msgr2=0x7fd1280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 -- 172.21.15.103:0/2961531206 shutdown_connections 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.496+0000 7fd12e0b3640 1 -- 172.21.15.103:0/2961531206 wait complete. 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 Processor -- start 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 -- start start 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128101df0 0x7fd128198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1281027a0 0x7fd1281988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd128111fd0 con 0x7fd128106f20 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd128111e50 con 0x7fd1281027a0 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd12e0b3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd128112150 con 0x7fd128101df0 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd127fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd127fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40888/0 (socket says 172.21.15.103:40888) 2024-09-19T07:37:22.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd127fff640 1 -- 172.21.15.103:0/3841025293 learned_addr learned my addr 172.21.15.103:0/3841025293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd126ffd640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1281027a0 0x7fd1281988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.497+0000 7fd1277fe640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128101df0 0x7fd128198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd127fff640 1 -- 172.21.15.103:0/3841025293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128101df0 msgr2=0x7fd128198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd127fff640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128101df0 0x7fd128198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd127fff640 1 -- 172.21.15.103:0/3841025293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1281027a0 msgr2=0x7fd1281988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd127fff640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1281027a0 0x7fd1281988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd127fff640 1 -- 172.21.15.103:0/3841025293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12819d330 con 0x7fd128106f20 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd127fff640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12819cc50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd120003f10 tx=0x7fd120003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd124ff9640 1 -- 172.21.15.103:0/3841025293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120031650 con 0x7fd128106f20 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd124ff9640 1 -- 172.21.15.103:0/3841025293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1200317f0 con 0x7fd128106f20 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12819d5c0 con 0x7fd128106f20 2024-09-19T07:37:22.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd124ff9640 1 -- 172.21.15.103:0/3841025293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12002fa90 con 0x7fd128106f20 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.498+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a4f00 con 0x7fd128106f20 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.500+0000 7fd124ff9640 1 -- 172.21.15.103:0/3841025293 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fd12002fcb0 con 0x7fd128106f20 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 Processor -- start 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 -- start start 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 0x5607245bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd124ff9640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd1100a3350 0x7fd1100a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560724519090 con 0x5607245b29d0 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607244c0a70 con 0x5607245bdbb0 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd13048c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560724515050 con 0x5607245b32f0 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40892/0 (socket says 172.21.15.103:40892) 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 -- 172.21.15.103:0/892892675 learned_addr learned my addr 172.21.15.103:0/892892675 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd127fff640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd1100a3350 0x7fd1100a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd124ff9640 1 -- 172.21.15.103:0/3841025293 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd1200f0d50 con 0x7fd128106f20 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd1277fe640 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 -- 172.21.15.103:0/892892675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 msgr2=0x5607245bd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245bd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 -- 172.21.15.103:0/892892675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 msgr2=0x5607245bffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 0x5607245bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.501+0000 7fd126ffd640 1 -- 172.21.15.103:0/892892675 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607244cd130 con 0x5607245b29d0 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd126ffd640 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b2db0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00ac90 tx=0x7fd11c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22e7fc08cd3d73a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd127fff640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd1100a3350 0x7fd1100a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd1200ee840 tx=0x7fd1200095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd10dffb640 1 -- 172.21.15.103:0/892892675 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c012450 con 0x5607245b29d0 2024-09-19T07:37:22.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd10dffb640 1 -- 172.21.15.103:0/892892675 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd11c012ad0 con 0x5607245b29d0 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd10dffb640 1 -- 172.21.15.103:0/892892675 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00e960 con 0x5607245b29d0 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd13048c740 1 -- 172.21.15.103:0/892892675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 msgr2=0x5607245b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd13048c740 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b2db0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00ac90 tx=0x7fd11c0117b0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd13048c740 1 -- 172.21.15.103:0/892892675 shutdown_connections 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd13048c740 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 0x5607245bffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd13048c740 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245bd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.502+0000 7fd13048c740 1 --2- 172.21.15.103:0/892892675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b2db0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 -- 172.21.15.103:0/892892675 >> 172.21.15.103:0/892892675 conn(0x5607245aef60 msgr2=0x5607245af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 -- 172.21.15.103:0/892892675 shutdown_connections 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 -- 172.21.15.103:0/892892675 wait complete. 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 -- start start 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 Processor -- start 2024-09-19T07:37:22.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 0x5607245bad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607245c0780 con 0x5607245b29d0 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607244d3f20 con 0x5607245bdbb0 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607245c0900 con 0x5607245b32f0 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd13048c740 10 client.0 _ll_register_callbacks cb 0x56072449d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245bd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40908/0 (socket says 172.21.15.103:40908) 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.503+0000 7fd126ffd640 1 -- 172.21.15.103:0/4010961399 learned_addr learned my addr 172.21.15.103:0/4010961399 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd1277fe640 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd126ffd640 1 -- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 msgr2=0x5607245ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd126ffd640 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd126ffd640 1 -- 172.21.15.103:0/4010961399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 msgr2=0x5607245bad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd126ffd640 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 0x5607245bad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd126ffd640 1 -- 172.21.15.103:0/4010961399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56072428a2b0 con 0x5607245b29d0 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd126ffd640 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245bd4a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00a350 tx=0x7fd11c00e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c86eaa48929953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd106ffd640 1 -- 172.21.15.103:0/4010961399 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00eb10 con 0x5607245b29d0 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd106ffd640 1 -- 172.21.15.103:0/4010961399 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd11c00ecb0 con 0x5607245b29d0 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd11c012ad0 con 0x5607245b29d0 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd106ffd640 1 -- 172.21.15.103:0/4010961399 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd11c026470 con 0x5607245b29d0 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd13048c740 10 client.14937 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:22.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5607245cc0f0 con 0x5607245b29d0 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd106ffd640 1 -- 172.21.15.103:0/4010961399 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd11c012ad0 con 0x5607245b29d0 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.504+0000 7fd106ffd640 1 -- 172.21.15.103:0/4010961399 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd11c00f420 con 0x5607245b29d0 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd13048c740 10 client.14937 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd13048c740 10 client.14937 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd13048c740 20 client.14937 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd13048c740 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5607245ce520 0x5607245cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd13048c740 4 client.14937 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560724526600 con 0x5607245ce520 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd1277fe640 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5607245ce520 0x5607245cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd1277fe640 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5607245ce520 0x5607245cf990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb9d977f6017dd96 server_cookie=cec91bbd2f57203b in_seq=0 out_seq=0 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.505+0000 7fd106ffd640 10 client.14937 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd106ffd640 1 -- 172.21.15.103:0/4010961399 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560724526600 con 0x5607245ce520 2024-09-19T07:37:22.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd106ffd640 10 client.14937 handle_command_reply: tid=0 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 client.14937 shutdown 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 msgr2=0x5607245bd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245bd4a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00a350 tx=0x7fd11c00e410 comp rx=0 tx=0).stop 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 shutdown_connections 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5607245ce520 0x5607245cf990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245bdbb0 0x5607245bad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b32f0 0x5607245ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd106ffd640 0 client.14937 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 --2- 172.21.15.103:0/4010961399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245bd4a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 >> 172.21.15.103:0/4010961399 conn(0x5607245aef60 msgr2=0x5607245b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 shutdown_connections 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.506+0000 7fd13048c740 1 -- 172.21.15.103:0/4010961399 wait complete. 2024-09-19T07:37:22.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.507+0000 7fd13048c740 20 client.14937 trim_cache size 0 max 16384 2024-09-19T07:37:22.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 Processor -- start 2024-09-19T07:37:22.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 -- start start 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 0x5607245b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245bdbb0 0x5607245d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607245cdcd0 con 0x5607245b29d0 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607245ce3a0 con 0x5607245b32f0 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd13048c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607245cea70 con 0x5607245bdbb0 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40924/0 (socket says 172.21.15.103:40924) 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.524+0000 7fd126ffd640 1 -- 172.21.15.103:0/3543222387 learned_addr learned my addr 172.21.15.103:0/3543222387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd1277fe640 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 0x5607245b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd126ffd640 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245bdbb0 0x5607245d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd126ffd640 1 -- 172.21.15.103:0/3543222387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245bdbb0 msgr2=0x5607245d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd126ffd640 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245bdbb0 0x5607245d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd126ffd640 1 -- 172.21.15.103:0/3543222387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 msgr2=0x5607245b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd126ffd640 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 0x5607245b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd126ffd640 1 -- 172.21.15.103:0/3543222387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607245bd0c0 con 0x5607245b29d0 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd126ffd640 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b7a50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e5d0 tx=0x7fd11c00e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b38b09200be6fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd10d7fa640 1 -- 172.21.15.103:0/3543222387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00fc60 con 0x5607245b29d0 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd10d7fa640 1 -- 172.21.15.103:0/3543222387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd11c038070 con 0x5607245b29d0 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.525+0000 7fd10d7fa640 1 -- 172.21.15.103:0/3543222387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00ece0 con 0x5607245b29d0 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 -- 172.21.15.103:0/3543222387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 msgr2=0x5607245b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b7a50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e5d0 tx=0x7fd11c00e670 comp rx=0 tx=0).stop 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 -- 172.21.15.103:0/3543222387 shutdown_connections 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245bdbb0 0x5607245d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 0x5607245b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 --2- 172.21.15.103:0/3543222387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245b29d0 0x5607245b7a50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 -- 172.21.15.103:0/3543222387 >> 172.21.15.103:0/3543222387 conn(0x5607245aef60 msgr2=0x5607245b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 -- 172.21.15.103:0/3543222387 shutdown_connections 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 -- 172.21.15.103:0/3543222387 wait complete. 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 -- start start 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.526+0000 7fd13048c740 1 Processor -- start 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b29d0 0x5607245d7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 0x5607245d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245bdbb0 0x5607245d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607245cf810 con 0x5607245bdbb0 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607245cf140 con 0x5607245b32f0 2024-09-19T07:37:22.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607245cfee0 con 0x5607245b29d0 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 10 client.0 _ll_register_callbacks cb 0x7fd11c00ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245bdbb0 0x5607245d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245bdbb0 0x5607245d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40940/0 (socket says 172.21.15.103:40940) 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 -- 172.21.15.103:0/152414969 learned_addr learned my addr 172.21.15.103:0/152414969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b29d0 0x5607245d7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 -- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b29d0 msgr2=0x5607245d7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b29d0 0x5607245d7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 -- 172.21.15.103:0/152414969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 msgr2=0x5607245d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 0x5607245d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 -- 172.21.15.103:0/152414969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607245d19d0 con 0x5607245bdbb0 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd126ffd640 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245bdbb0 0x5607245d11a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e460 tx=0x7fd11c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8fd7f9f31cbd8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:22.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd106ffd640 1 -- 172.21.15.103:0/152414969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00a950 con 0x5607245bdbb0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd106ffd640 1 -- 172.21.15.103:0/152414969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fd11c00ea60 con 0x5607245bdbb0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.527+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd11c038070 con 0x5607245bdbb0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd106ffd640 1 -- 172.21.15.103:0/152414969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd11c00a630 con 0x5607245bdbb0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 10 client.14943 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5607245d6330 con 0x5607245bdbb0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd106ffd640 1 -- 172.21.15.103:0/152414969 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd11c038070 con 0x5607245bdbb0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd106ffd640 1 -- 172.21.15.103:0/152414969 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd11c031460 con 0x5607245bdbb0 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 10 client.14943 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 10 client.14943 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 20 client.14943 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56072463d040 0x56072463bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:22.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 4 client.14943 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "status"}) -- 0x560724520b10 con 0x56072463d040 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.528+0000 7fd1277fe640 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56072463d040 0x56072463bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd106ffd640 10 client.14943 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd1277fe640 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56072463d040 0x56072463bd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=165b948153778a85 server_cookie=db0c6f002e68c8d3 in_seq=0 out_seq=0 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd106ffd640 1 -- 172.21.15.103:0/152414969 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560724520b10 con 0x56072463d040 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd106ffd640 10 client.14943 handle_command_reply: tid=0 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 client.14943 shutdown 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245bdbb0 msgr2=0x5607245d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245bdbb0 0x5607245d11a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e460 tx=0x7fd11c0117b0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 shutdown_connections 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56072463d040 0x56072463bd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607245bdbb0 0x5607245d11a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607245b32f0 0x5607245d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 --2- 172.21.15.103:0/152414969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607245b29d0 0x5607245d7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd106ffd640 0 client.14943 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 >> 172.21.15.103:0/152414969 conn(0x5607245aef60 msgr2=0x5607245bbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 shutdown_connections 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 1 -- 172.21.15.103:0/152414969 wait complete. 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.529+0000 7fd13048c740 20 client.14943 trim_cache size 0 max 16384 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-19T07:37:22.539 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24448, 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 63.631125357000002, 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 100.238927112 2024-09-19T07:37:22.540 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:22.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd1100a3350 msgr2=0x7fd1100a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd1100a3350 0x7fd1100a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd1200ee840 tx=0x7fd1200095c0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 msgr2=0x7fd12819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:22.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12819cc50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd120003f10 tx=0x7fd120003f40 comp rx=0 tx=0).stop 2024-09-19T07:37:22.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 shutdown_connections 2024-09-19T07:37:22.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fd1100a3350 0x7fd1100a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd128106f20 0x7fd12819cc50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1281027a0 0x7fd1281988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 --2- 172.21.15.103:0/3841025293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128101df0 0x7fd128198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:22.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 >> 172.21.15.103:0/3841025293 conn(0x7fd1280fd8f0 msgr2=0x7fd1280ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:22.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 shutdown_connections 2024-09-19T07:37:22.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:22.530+0000 7fd12e0b3640 1 -- 172.21.15.103:0/3841025293 wait complete. 2024-09-19T07:37:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:22 smithi174 ceph-mon[35370]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:22 smithi134 ceph-mon[35414]: pgmap v317: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:23.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T07:37:23.525 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:24.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 -- 172.21.15.103:0/1815504524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc1027a0 msgr2=0x7f14bc1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 --2- 172.21.15.103:0/1815504524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc1027a0 0x7f14bc1067f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009900 tx=0x7f14b802a410 comp rx=0 tx=0).stop 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 -- 172.21.15.103:0/1815504524 shutdown_connections 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 --2- 172.21.15.103:0/1815504524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc106f20 0x7f14bc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 --2- 172.21.15.103:0/1815504524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc1027a0 0x7f14bc1067f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 --2- 172.21.15.103:0/1815504524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14bc101df0 0x7f14bc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 -- 172.21.15.103:0/1815504524 >> 172.21.15.103:0/1815504524 conn(0x7f14bc0fd8f0 msgr2=0x7f14bc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 -- 172.21.15.103:0/1815504524 shutdown_connections 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.007+0000 7f14c4458640 1 -- 172.21.15.103:0/1815504524 wait complete. 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 Processor -- start 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 -- start start 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc101df0 0x7f14bc198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14bc1027a0 0x7f14bc198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc106f20 0x7f14bc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc112020 con 0x7f14bc106f20 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc111ea0 con 0x7f14bc1027a0 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c4458640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1121a0 con 0x7f14bc101df0 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc106f20 0x7f14bc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc106f20 0x7f14bc19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40958/0 (socket says 172.21.15.103:40958) 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 -- 172.21.15.103:0/3251984269 learned_addr learned my addr 172.21.15.103:0/3251984269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c21f4640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc101df0 0x7f14bc198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c19f3640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14bc1027a0 0x7f14bc198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 -- 172.21.15.103:0/3251984269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc101df0 msgr2=0x7f14bc198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc101df0 0x7f14bc198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 -- 172.21.15.103:0/3251984269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14bc1027a0 msgr2=0x7f14bc198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14bc1027a0 0x7f14bc198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 -- 172.21.15.103:0/3251984269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc19d5f0 con 0x7f14bc106f20 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c19f3640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14bc1027a0 0x7f14bc198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14c29f5640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc106f20 0x7f14bc19cf10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b970 tx=0x7f14b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.008+0000 7f14af7fe640 1 -- 172.21.15.103:0/3251984269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4019070 con 0x7f14bc106f20 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.009+0000 7f14af7fe640 1 -- 172.21.15.103:0/3251984269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14b400cb40 con 0x7f14bc106f20 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.009+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc19d8e0 con 0x7f14bc106f20 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.009+0000 7f14af7fe640 1 -- 172.21.15.103:0/3251984269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4007650 con 0x7f14bc106f20 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.009+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc1a51c0 con 0x7f14bc106f20 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.011+0000 7f14af7fe640 1 -- 172.21.15.103:0/3251984269 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f14b401c070 con 0x7f14bc106f20 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.011+0000 7f14c6831740 1 Processor -- start 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.011+0000 7f14c6831740 1 -- start start 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c6831740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c014c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c6831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c023a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c6831740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c0263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c6831740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d8bf7b090 con 0x555d8c0148b0 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c6831740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555d8bf22a70 con 0x555d8c0151d0 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c6831740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d8bf77050 con 0x555d8c023fc0 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c014c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c014c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40970/0 (socket says 172.21.15.103:40970) 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 -- 172.21.15.103:0/2757136944 learned_addr learned my addr 172.21.15.103:0/2757136944 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14af7fe640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f14a80a32d0 0x7f14a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c0263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c21f4640 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c023a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c29f5640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f14a80a32d0 0x7f14a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 -- 172.21.15.103:0/2757136944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 msgr2=0x555d8c0263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c0263b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 -- 172.21.15.103:0/2757136944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 msgr2=0x555d8c023a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c023a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14af7fe640 1 -- 172.21.15.103:0/3251984269 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f14b40cefa0 con 0x7f14bc106f20 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 -- 172.21.15.103:0/2757136944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d8bf2f130 con 0x555d8c0148b0 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c19f3640 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c014c90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x555d8c012050 tx=0x7f14b8031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c26811e775ad49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14c29f5640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f14a80a32d0 0x7f14a80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f14b4097f40 tx=0x7f14b406e3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.012+0000 7f14acdea640 1 -- 172.21.15.103:0/2757136944 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8045070 con 0x555d8c0148b0 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14acdea640 1 -- 172.21.15.103:0/2757136944 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14b8005040 con 0x555d8c0148b0 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14acdea640 1 -- 172.21.15.103:0/2757136944 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8035050 con 0x555d8c0148b0 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 -- 172.21.15.103:0/2757136944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 msgr2=0x555d8c014c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c014c90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x555d8c012050 tx=0x7f14b8031440 comp rx=0 tx=0).stop 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 -- 172.21.15.103:0/2757136944 shutdown_connections 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c0263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c023a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 --2- 172.21.15.103:0/2757136944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c014c90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 -- 172.21.15.103:0/2757136944 >> 172.21.15.103:0/2757136944 conn(0x555d8c010f60 msgr2=0x555d8c011340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 -- 172.21.15.103:0/2757136944 shutdown_connections 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.013+0000 7f14c6831740 1 -- 172.21.15.103:0/2757136944 wait complete. 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 -- start start 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 Processor -- start 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c0238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c020c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c021140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d8c026b90 con 0x555d8c0148b0 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555d8bf35f20 con 0x555d8c0151d0 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d8c026d10 con 0x555d8c023fc0 2024-09-19T07:37:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c6831740 10 client.0 _ll_register_callbacks cb 0x555d8beff030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c0238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c0238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40984/0 (socket says 172.21.15.103:40984) 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 -- 172.21.15.103:0/3474010143 learned_addr learned my addr 172.21.15.103:0/3474010143 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c21f4640 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c020c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c021140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 -- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 msgr2=0x555d8c021140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c021140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 -- 172.21.15.103:0/3474010143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 msgr2=0x555d8c020c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c020c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.014+0000 7f14c19f3640 1 -- 172.21.15.103:0/3474010143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d8bcec2b0 con 0x555d8c0148b0 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c19f3640 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c0238b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14b8032f40 tx=0x7f14b8005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df27027f77916cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:24.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14a5ffb640 1 -- 172.21.15.103:0/3474010143 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b804f060 con 0x555d8c0148b0 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14a5ffb640 1 -- 172.21.15.103:0/3474010143 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f14b80054b0 con 0x555d8c0148b0 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b8005040 con 0x555d8c0148b0 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14a5ffb640 1 -- 172.21.15.103:0/3474010143 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14b8035070 con 0x555d8c0148b0 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 10 client.14955 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d8c032540 con 0x555d8c0148b0 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14a5ffb640 1 -- 172.21.15.103:0/3474010143 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14b8005040 con 0x555d8c0148b0 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14a5ffb640 1 -- 172.21.15.103:0/3474010143 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f14b8003850 con 0x555d8c0148b0 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 10 client.14955 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 10 client.14955 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 20 client.14955 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c033890 0x555d8c035a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 4 client.14955 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.015+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d8bf88600 con 0x555d8c033890 2024-09-19T07:37:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.016+0000 7f14c21f4640 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c033890 0x555d8c035a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.016+0000 7f14a5ffb640 10 client.14955 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.016+0000 7f14c21f4640 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c033890 0x555d8c035a60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b6351fa1da9f431 server_cookie=8ef41815bae454e4 in_seq=0 out_seq=0 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14a5ffb640 1 -- 172.21.15.103:0/3474010143 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d8bf88600 con 0x555d8c033890 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14a5ffb640 10 client.14955 handle_command_reply: tid=0 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 client.14955 shutdown 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 msgr2=0x555d8c0238b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c0238b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14b8032f40 tx=0x7f14b8005e60 comp rx=0 tx=0).stop 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 shutdown_connections 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c033890 0x555d8c035a60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c023fc0 0x555d8c021140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c020c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 --2- 172.21.15.103:0/3474010143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0148b0 0x555d8c0238b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14a5ffb640 0 client.14955 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 >> 172.21.15.103:0/3474010143 conn(0x555d8c010f60 msgr2=0x555d8c016cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 shutdown_connections 2024-09-19T07:37:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 1 -- 172.21.15.103:0/3474010143 wait complete. 2024-09-19T07:37:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.017+0000 7f14c6831740 20 client.14955 trim_cache size 0 max 16384 2024-09-19T07:37:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 Processor -- start 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 -- start start 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c0345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c01f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c023fc0 0x555d8c01fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d8c033da0 con 0x555d8c023fc0 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555d8c034470 con 0x555d8c0151d0 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c6831740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d8c034b40 con 0x555d8c0148b0 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c023fc0 0x555d8c01fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c023fc0 0x555d8c01fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40996/0 (socket says 172.21.15.103:40996) 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 -- 172.21.15.103:0/688576245 learned_addr learned my addr 172.21.15.103:0/688576245 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c21f4640 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c01f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c0345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 -- 172.21.15.103:0/688576245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 msgr2=0x555d8c0345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c0345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 -- 172.21.15.103:0/688576245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 msgr2=0x555d8c01f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c01f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.035+0000 7f14c19f3640 1 -- 172.21.15.103:0/688576245 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d8c011bf0 con 0x555d8c023fc0 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c19f3640 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c023fc0 0x555d8c01fa10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14b8004650 tx=0x7f14b8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c96484e0a7225b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14a7fff640 1 -- 172.21.15.103:0/688576245 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b804f070 con 0x555d8c023fc0 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14a7fff640 1 -- 172.21.15.103:0/688576245 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14b8041030 con 0x555d8c023fc0 2024-09-19T07:37:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14a7fff640 1 -- 172.21.15.103:0/688576245 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8059040 con 0x555d8c023fc0 2024-09-19T07:37:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 -- 172.21.15.103:0/688576245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c023fc0 msgr2=0x555d8c01fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c023fc0 0x555d8c01fa10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14b8004650 tx=0x7f14b8005a20 comp rx=0 tx=0).stop 2024-09-19T07:37:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 -- 172.21.15.103:0/688576245 shutdown_connections 2024-09-19T07:37:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c023fc0 0x555d8c01fa10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c0151d0 0x555d8c01f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 --2- 172.21.15.103:0/688576245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c0345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 -- 172.21.15.103:0/688576245 >> 172.21.15.103:0/688576245 conn(0x555d8c010f60 msgr2=0x555d8c00bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 -- 172.21.15.103:0/688576245 shutdown_connections 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.036+0000 7f14c6831740 1 -- 172.21.15.103:0/688576245 wait complete. 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 -- start start 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 Processor -- start 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c01d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0151d0 0x555d8c01d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c023fc0 0x555d8c03cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d8c0358e0 con 0x555d8c0151d0 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555d8c035210 con 0x555d8c023fc0 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d8c035fb0 con 0x555d8c0148b0 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c6831740 10 client.0 _ll_register_callbacks cb 0x7f14b8043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c21f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0151d0 0x555d8c01d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c21f4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0151d0 0x555d8c01d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41008/0 (socket says 172.21.15.103:41008) 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c21f4640 1 -- 172.21.15.103:0/565921573 learned_addr learned my addr 172.21.15.103:0/565921573 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c19f3640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c01d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c19f3640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c023fc0 0x555d8c03cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c21f4640 1 -- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 msgr2=0x555d8c01d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.037+0000 7f14c21f4640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c01d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c21f4640 1 -- 172.21.15.103:0/565921573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c023fc0 msgr2=0x555d8c03cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c21f4640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c023fc0 0x555d8c03cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c21f4640 1 -- 172.21.15.103:0/565921573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d8c03d100 con 0x555d8c0151d0 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c19f3640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c023fc0 0x555d8c03cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c21f4640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0151d0 0x555d8c01d770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x555d8c022240 tx=0x7f14b00069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e037a1ed47f4cda2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14a5ffb640 1 -- 172.21.15.103:0/565921573 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0015e00 con 0x555d8c0151d0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14a5ffb640 1 -- 172.21.15.103:0/565921573 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f14b00243e0 con 0x555d8c0151d0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b8041030 con 0x555d8c0151d0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14a5ffb640 1 -- 172.21.15.103:0/565921573 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14b00245e0 con 0x555d8c0151d0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 10 client.14961 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d8c03e400 con 0x555d8c0151d0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14a5ffb640 1 -- 172.21.15.103:0/565921573 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14b8041030 con 0x555d8c0151d0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14a5ffb640 1 -- 172.21.15.103:0/565921573 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f14b00259f0 con 0x555d8c0151d0 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 10 client.14961 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 10 client.14961 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 20 client.14961 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c0a30a0 0x555d8c0a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 4 client.14961 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.038+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "status"}) -- 0x555d8bf82b10 con 0x555d8c0a30a0 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c19f3640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c0a30a0 0x555d8c0a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c19f3640 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c0a30a0 0x555d8c0a1db0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51a19841c9a12369 server_cookie=f2fc6d41e8e42675 in_seq=0 out_seq=0 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14a5ffb640 10 client.14961 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14a5ffb640 1 -- 172.21.15.103:0/565921573 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x555d8bf82b10 con 0x555d8c0a30a0 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14a5ffb640 10 client.14961 handle_command_reply: tid=0 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 client.14961 shutdown 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0151d0 msgr2=0x555d8c01d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0151d0 0x555d8c01d770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x555d8c022240 tx=0x7f14b00069d0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 shutdown_connections 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555d8c0a30a0 0x555d8c0a1db0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d8c023fc0 0x555d8c03cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d8c0151d0 0x555d8c01d770 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 --2- 172.21.15.103:0/565921573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d8c0148b0 0x555d8c01d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 >> 172.21.15.103:0/565921573 conn(0x555d8c010f60 msgr2=0x555d8c01ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.039+0000 7f14a5ffb640 0 client.14961 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 shutdown_connections 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c6831740 1 -- 172.21.15.103:0/565921573 wait complete. 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c6831740 20 client.14961 trim_cache size 0 max 16384 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:24.047 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 66.283134094999994, 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 99.200310285 2024-09-19T07:37:24.048 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f14a80a32d0 msgr2=0x7f14a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f14a80a32d0 0x7f14a80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f14b4097f40 tx=0x7f14b406e3f0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc106f20 msgr2=0x7f14bc19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc106f20 0x7f14bc19cf10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b970 tx=0x7f14b400be30 comp rx=0 tx=0).stop 2024-09-19T07:37:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 shutdown_connections 2024-09-19T07:37:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f14a80a32d0 0x7f14a80a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14bc106f20 0x7f14bc19cf10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14bc1027a0 0x7f14bc198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 --2- 172.21.15.103:0/3251984269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc101df0 0x7f14bc198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 >> 172.21.15.103:0/3251984269 conn(0x7f14bc0fd8f0 msgr2=0x7f14bc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 shutdown_connections 2024-09-19T07:37:24.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:24.040+0000 7f14c4458640 1 -- 172.21.15.103:0/3251984269 wait complete. 2024-09-19T07:37:24.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T07:37:24.990 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:25.403 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:25 smithi103 ceph-mon[31201]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:25 smithi174 ceph-mon[35370]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 -- 172.21.15.103:0/1741166991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 msgr2=0x7fde28107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 --2- 172.21.15.103:0/1741166991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28107a50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009930 tx=0x7fde1c02a420 comp rx=0 tx=0).stop 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 -- 172.21.15.103:0/1741166991 shutdown_connections 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 --2- 172.21.15.103:0/1741166991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde28108160 0x7fde2810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 --2- 172.21.15.103:0/1741166991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28107a50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 --2- 172.21.15.103:0/1741166991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde281008e0 0x7fde28100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 -- 172.21.15.103:0/1741166991 >> 172.21.15.103:0/1741166991 conn(0x7fde28077210 msgr2=0x7fde28077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:25.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 -- 172.21.15.103:0/1741166991 shutdown_connections 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.483+0000 7fde2e013640 1 -- 172.21.15.103:0/1741166991 wait complete. 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 Processor -- start 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 -- start start 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde281008e0 0x7fde28198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28108160 0x7fde2819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde28111fb0 con 0x7fde28101290 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde28111e30 con 0x7fde28108160 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.484+0000 7fde2e013640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde28112130 con 0x7fde281008e0 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41020/0 (socket says 172.21.15.103:41020) 2024-09-19T07:37:25.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 -- 172.21.15.103:0/1447099689 learned_addr learned my addr 172.21.15.103:0/1447099689 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde277fe640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde281008e0 0x7fde28198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 -- 172.21.15.103:0/1447099689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde281008e0 msgr2=0x7fde28198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde281008e0 0x7fde28198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 -- 172.21.15.103:0/1447099689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28108160 msgr2=0x7fde2819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28108160 0x7fde2819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 -- 172.21.15.103:0/1447099689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2819d4c0 con 0x7fde28101290 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde26ffd640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28198a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005940 tx=0x7fde1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde24ff9640 1 -- 172.21.15.103:0/1447099689 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c03f030 con 0x7fde28101290 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde24ff9640 1 -- 172.21.15.103:0/1447099689 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1c003ce0 con 0x7fde28101290 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2819d750 con 0x7fde28101290 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde24ff9640 1 -- 172.21.15.103:0/1447099689 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c035070 con 0x7fde28101290 2024-09-19T07:37:25.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.485+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281a5090 con 0x7fde28101290 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde24ff9640 1 -- 172.21.15.103:0/1447099689 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fde1c042030 con 0x7fde28101290 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 Processor -- start 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 -- start start 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde24ff9640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde100a32d0 0x7fde100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6a9d0 0x55da2bd6adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6b2f0 0x55da2bd75670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd77fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da2bcd1090 con 0x55da2bd6b2f0 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da2bc78a70 con 0x55da2bd75bb0 2024-09-19T07:37:25.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde303ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da2bccd050 con 0x55da2bd6a9d0 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde27fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6a9d0 0x55da2bd6adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde277fe640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde100a32d0 0x7fde100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.488+0000 7fde26ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6b2f0 0x55da2bd75670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6a9d0 0x55da2bd6adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:58334/0 (socket says 172.21.15.103:58334) 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 -- 172.21.15.103:0/1542924840 learned_addr learned my addr 172.21.15.103:0/1542924840 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde24ff9640 1 -- 172.21.15.103:0/1447099689 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fde1c0efe90 con 0x7fde28101290 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd77fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 -- 172.21.15.103:0/1542924840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 msgr2=0x55da2bd77fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd77fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 -- 172.21.15.103:0/1542924840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6b2f0 msgr2=0x55da2bd75670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6b2f0 0x55da2bd75670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 -- 172.21.15.103:0/1542924840 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2bc85130 con 0x55da2bd6a9d0 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde277fe640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde100a32d0 0x7fde100a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fde1800bbe0 tx=0x7fde1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde27fff640 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6a9d0 0x55da2bd6adb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55da2bd682e0 tx=0x7fde200117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a784cea76240127 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde0dffb640 1 -- 172.21.15.103:0/1542924840 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20011310 con 0x55da2bd6a9d0 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde0dffb640 1 -- 172.21.15.103:0/1542924840 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde20009e90 con 0x55da2bd6a9d0 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.489+0000 7fde0dffb640 1 -- 172.21.15.103:0/1542924840 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2000fd70 con 0x55da2bd6a9d0 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 -- 172.21.15.103:0/1542924840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6a9d0 msgr2=0x55da2bd6adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6a9d0 0x55da2bd6adb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55da2bd682e0 tx=0x7fde200117d0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 -- 172.21.15.103:0/1542924840 shutdown_connections 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd77fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6b2f0 0x55da2bd75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 --2- 172.21.15.103:0/1542924840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6a9d0 0x55da2bd6adb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 -- 172.21.15.103:0/1542924840 >> 172.21.15.103:0/1542924840 conn(0x55da2bd66f60 msgr2=0x55da2bd67340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:25.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 -- 172.21.15.103:0/1542924840 shutdown_connections 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 -- 172.21.15.103:0/1542924840 wait complete. 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.490+0000 7fde303ec740 1 -- start start 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 1 Processor -- start 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd75420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd72cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd78780 con 0x55da2bd6a9d0 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da2bc8bf20 con 0x55da2bd75bb0 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd78900 con 0x55da2bd6b2f0 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde303ec740 10 client.0 _ll_register_callbacks cb 0x55da2bc55030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd75420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde26ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd75420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41050/0 (socket says 172.21.15.103:41050) 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde277fe640 1 -- 172.21.15.103:0/3305539015 learned_addr learned my addr 172.21.15.103:0/3305539015 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde27fff640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd72cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde26ffd640 1 -- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 msgr2=0x55da2bd72cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde26ffd640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd72cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde26ffd640 1 -- 172.21.15.103:0/3305539015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 msgr2=0x55da2bd75420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde26ffd640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd75420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde26ffd640 1 -- 172.21.15.103:0/3305539015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2ba422b0 con 0x55da2bd6b2f0 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde277fe640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd75420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde27fff640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd72cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde26ffd640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd72770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ed840 tx=0x7fde1c03b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5992e5a00ff8a2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.491+0000 7fde06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde06ffd640 1 -- 172.21.15.103:0/3305539015 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c049070 con 0x55da2bd6b2f0 2024-09-19T07:37:25.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde06ffd640 1 -- 172.21.15.103:0/3305539015 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fde1c0ba650 con 0x55da2bd6b2f0 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde20009e90 con 0x55da2bd6b2f0 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde06ffd640 1 -- 172.21.15.103:0/3305539015 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde1c0cf070 con 0x55da2bd6b2f0 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 10 client.24691 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da2bd78780 con 0x55da2bd6b2f0 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde06ffd640 1 -- 172.21.15.103:0/3305539015 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde20009e90 con 0x55da2bd6b2f0 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde06ffd640 1 -- 172.21.15.103:0/3305539015 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fde1c0b89c0 con 0x55da2bd6b2f0 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 10 client.24691 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 10 client.24691 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 20 client.24691 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bd86690 0x55da2bd87b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 4 client.24691 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.492+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da2bcde600 con 0x55da2bd86690 2024-09-19T07:37:25.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.493+0000 7fde277fe640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bd86690 0x55da2bd87b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.493+0000 7fde277fe640 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bd86690 0x55da2bd87b00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac404d96b24d2ee1 server_cookie=4cb0111c43a79b5a in_seq=0 out_seq=0 2024-09-19T07:37:25.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.493+0000 7fde06ffd640 10 client.24691 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde06ffd640 1 -- 172.21.15.103:0/3305539015 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da2bcde600 con 0x55da2bd86690 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde06ffd640 10 client.24691 handle_command_reply: tid=0 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 client.24691 shutdown 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 msgr2=0x55da2bd72770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd72770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0ed840 tx=0x7fde1c03b6b0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 shutdown_connections 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bd86690 0x55da2bd87b00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd72cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd72770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde06ffd640 0 client.24691 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:25.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 --2- 172.21.15.103:0/3305539015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd75420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 >> 172.21.15.103:0/3305539015 conn(0x55da2bd66f60 msgr2=0x55da2bd6cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:25.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 shutdown_connections 2024-09-19T07:37:25.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 1 -- 172.21.15.103:0/3305539015 wait complete. 2024-09-19T07:37:25.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.494+0000 7fde303ec740 20 client.24691 trim_cache size 0 max 16384 2024-09-19T07:37:25.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.511+0000 7fde303ec740 1 Processor -- start 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde303ec740 1 -- start start 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde303ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd72030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde303ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde303ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd85730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde303ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd85e40 con 0x55da2bd6a9d0 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde303ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd86510 con 0x55da2bd75bb0 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde303ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd86be0 con 0x55da2bd6b2f0 2024-09-19T07:37:25.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd72030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd72030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41052/0 (socket says 172.21.15.103:41052) 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 -- 172.21.15.103:0/264672935 learned_addr learned my addr 172.21.15.103:0/264672935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde26ffd640 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 -- 172.21.15.103:0/264672935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 msgr2=0x55da2bd851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd851f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 -- 172.21.15.103:0/264672935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 msgr2=0x55da2bd85730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd85730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 -- 172.21.15.103:0/264672935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2bd6b120 con 0x55da2bd6a9d0 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde277fe640 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd72030 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde18010860 tx=0x7fde18013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a87c1512366716 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.512+0000 7fde0d7fa640 1 -- 172.21.15.103:0/264672935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18011d00 con 0x55da2bd6a9d0 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde0d7fa640 1 -- 172.21.15.103:0/264672935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde18005500 con 0x55da2bd6a9d0 2024-09-19T07:37:25.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde0d7fa640 1 -- 172.21.15.103:0/264672935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18020de0 con 0x55da2bd6a9d0 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 -- 172.21.15.103:0/264672935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 msgr2=0x55da2bd72030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd72030 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde18010860 tx=0x7fde18013000 comp rx=0 tx=0).stop 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 -- 172.21.15.103:0/264672935 shutdown_connections 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd85730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 --2- 172.21.15.103:0/264672935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd72030 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 -- 172.21.15.103:0/264672935 >> 172.21.15.103:0/264672935 conn(0x55da2bd66f60 msgr2=0x55da2bd71060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 -- 172.21.15.103:0/264672935 shutdown_connections 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 -- 172.21.15.103:0/264672935 wait complete. 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.513+0000 7fde303ec740 1 -- start start 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 1 Processor -- start 2024-09-19T07:37:25.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd8bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd87980 con 0x55da2bd6a9d0 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd872b0 con 0x55da2bd75bb0 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da2bd88050 con 0x55da2bd6b2f0 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde303ec740 10 client.0 _ll_register_callbacks cb 0x7fde1c0b86a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd90210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41066/0 (socket says 172.21.15.103:41066) 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 -- 172.21.15.103:0/3899335187 learned_addr learned my addr 172.21.15.103:0/3899335187 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde26ffd640 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde27fff640 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd8bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 -- 172.21.15.103:0/3899335187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 msgr2=0x55da2bd8b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd8b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 -- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 msgr2=0x55da2bd8bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd8bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.514+0000 7fde277fe640 1 -- 172.21.15.103:0/3899335187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2bd8c010 con 0x55da2bd6a9d0 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde277fe640 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd90210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fde1802a040 tx=0x7fde180056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04a2e730b399bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde06ffd640 1 -- 172.21.15.103:0/3899335187 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18035020 con 0x55da2bd6a9d0 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde06ffd640 1 -- 172.21.15.103:0/3899335187 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fde1802e040 con 0x55da2bd6a9d0 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde18005500 con 0x55da2bd6a9d0 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde06ffd640 1 -- 172.21.15.103:0/3899335187 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde18020b60 con 0x55da2bd6a9d0 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 10 client.14979 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:25.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55da2bdf0a90 con 0x55da2bd6a9d0 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde06ffd640 1 -- 172.21.15.103:0/3899335187 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde18005500 con 0x55da2bd6a9d0 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde06ffd640 1 -- 172.21.15.103:0/3899335187 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fde180313f0 con 0x55da2bd6a9d0 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 10 client.14979 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 10 client.14979 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 20 client.14979 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bdf5370 0x55da2bdf4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 4 client.14979 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.515+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "status"}) -- 0x55da2bcd8b10 con 0x55da2bdf5370 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde26ffd640 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bdf5370 0x55da2bdf4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde06ffd640 10 client.14979 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde26ffd640 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bdf5370 0x55da2bdf4080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ac3c0756813262a server_cookie=4322c289b199e3b5 in_seq=0 out_seq=0 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde06ffd640 1 -- 172.21.15.103:0/3899335187 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55da2bcd8b10 con 0x55da2bdf5370 2024-09-19T07:37:25.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde06ffd640 10 client.14979 handle_command_reply: tid=0 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 client.14979 shutdown 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 msgr2=0x55da2bd90210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd90210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fde1802a040 tx=0x7fde180056b0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 shutdown_connections 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55da2bdf5370 0x55da2bdf4080 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da2bd75bb0 0x55da2bd8bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da2bd6b2f0 0x55da2bd8b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde06ffd640 0 client.14979 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 --2- 172.21.15.103:0/3899335187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55da2bd6a9d0 0x55da2bd90210 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.516+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 >> 172.21.15.103:0/3899335187 conn(0x55da2bd66f60 msgr2=0x55da2bd70ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:25.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 shutdown_connections 2024-09-19T07:37:25.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde303ec740 1 -- 172.21.15.103:0/3899335187 wait complete. 2024-09-19T07:37:25.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde303ec740 20 client.14979 trim_cache size 0 max 16384 2024-09-19T07:37:25.525 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:25.525 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:25.525 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-19T07:37:25.525 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24455, 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 68.766569580999999, 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 98.054035575 2024-09-19T07:37:25.526 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:25.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde100a32d0 msgr2=0x7fde100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde100a32d0 0x7fde100a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fde1800bbe0 tx=0x7fde1800b390 comp rx=0 tx=0).stop 2024-09-19T07:37:25.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 msgr2=0x7fde28198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:25.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28198a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005940 tx=0x7fde1c0042f0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 shutdown_connections 2024-09-19T07:37:25.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fde100a32d0 0x7fde100a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde28108160 0x7fde2819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde28101290 0x7fde28198a50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 --2- 172.21.15.103:0/1447099689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde281008e0 0x7fde28198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:25.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 >> 172.21.15.103:0/1447099689 conn(0x7fde28077210 msgr2=0x7fde28105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:25.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 shutdown_connections 2024-09-19T07:37:25.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:25.517+0000 7fde2e013640 1 -- 172.21.15.103:0/1447099689 wait complete. 2024-09-19T07:37:25.532 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:25 smithi134 ceph-mon[35414]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:26.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T07:37:26.408 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:26.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 -- 172.21.15.103:0/2003995878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74100a60 msgr2=0x7fdf74100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 --2- 172.21.15.103:0/2003995878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74100a60 0x7fdf74100ec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009930 tx=0x7fdf6802a420 comp rx=0 tx=0).stop 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 -- 172.21.15.103:0/2003995878 shutdown_connections 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 --2- 172.21.15.103:0/2003995878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf74101400 0x7fdf7410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 --2- 172.21.15.103:0/2003995878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74100a60 0x7fdf74100ec0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 --2- 172.21.15.103:0/2003995878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf74106a60 0x7fdf74106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 -- 172.21.15.103:0/2003995878 >> 172.21.15.103:0/2003995878 conn(0x7fdf740fc780 msgr2=0x7fdf740feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 -- 172.21.15.103:0/2003995878 shutdown_connections 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.901+0000 7fdf78ece640 1 -- 172.21.15.103:0/2003995878 wait complete. 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 Processor -- start 2024-09-19T07:37:26.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 -- start start 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf74100a60 0x7fdf74199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74101400 0x7fdf74199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf74106a60 0x7fdf741a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74110e60 con 0x7fdf74101400 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74110ce0 con 0x7fdf74100a60 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf78ece640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74110fe0 con 0x7fdf74106a60 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74101400 0x7fdf74199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74101400 0x7fdf74199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41076/0 (socket says 172.21.15.103:41076) 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 -- 172.21.15.103:0/3880272178 learned_addr learned my addr 172.21.15.103:0/3880272178 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf74106a60 0x7fdf741a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf7259c640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf74100a60 0x7fdf74199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 -- 172.21.15.103:0/3880272178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf74106a60 msgr2=0x7fdf741a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf74106a60 0x7fdf741a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 -- 172.21.15.103:0/3880272178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf74100a60 msgr2=0x7fdf74199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf74100a60 0x7fdf74199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 -- 172.21.15.103:0/3880272178 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf741a48d0 con 0x7fdf74101400 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.902+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74101400 0x7fdf74199850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009230 tx=0x7fdf68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.903+0000 7fdf637fe640 1 -- 172.21.15.103:0/3880272178 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6803f030 con 0x7fdf74101400 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.903+0000 7fdf637fe640 1 -- 172.21.15.103:0/3880272178 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf68003ad0 con 0x7fdf74101400 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.903+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf741a4aa0 con 0x7fdf74101400 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.903+0000 7fdf637fe640 1 -- 172.21.15.103:0/3880272178 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68035070 con 0x7fdf74101400 2024-09-19T07:37:26.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.903+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a4f50 con 0x7fdf74101400 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.905+0000 7fdf637fe640 1 -- 172.21.15.103:0/3880272178 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdf68042030 con 0x7fdf74101400 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.905+0000 7fdf7b2a7740 1 Processor -- start 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.905+0000 7fdf7b2a7740 1 -- start start 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829af5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829af62f0 0x562829b00670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829b00bb0 0x562829b02fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562829a5c090 con 0x562829af59d0 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562829a03a70 con 0x562829b00bb0 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562829a58050 con 0x562829af62f0 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf637fe640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829af5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829af5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41092/0 (socket says 172.21.15.103:41092) 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 -- 172.21.15.103:0/3421033314 learned_addr learned my addr 172.21.15.103:0/3421033314 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7259c640 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829af62f0 0x562829b00670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829b00bb0 0x562829b02fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf637fe640 1 -- 172.21.15.103:0/3880272178 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdf680efe80 con 0x7fdf74101400 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 -- 172.21.15.103:0/3421033314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829af62f0 msgr2=0x562829b00670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829af62f0 0x562829b00670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 -- 172.21.15.103:0/3421033314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829b00bb0 msgr2=0x562829b02fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829b00bb0 0x562829b02fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 -- 172.21.15.103:0/3421033314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562829a10130 con 0x562829af59d0 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf7259c640 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829af62f0 0x562829b00670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:26.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829b00bb0 0x562829b02fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829af5db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c012a30 tx=0x7fdf6c012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44226bf8cf34abd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c010800 tx=0x7fdf6c017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.906+0000 7fdf60ff9640 1 -- 172.21.15.103:0/3421033314 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c024070 con 0x562829af59d0 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf60ff9640 1 -- 172.21.15.103:0/3421033314 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf6c00ba00 con 0x562829af59d0 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf60ff9640 1 -- 172.21.15.103:0/3421033314 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c01f7b0 con 0x562829af59d0 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/3421033314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 msgr2=0x562829af5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829af5db0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c012a30 tx=0x7fdf6c012ef0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/3421033314 shutdown_connections 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829b00bb0 0x562829b02fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829af62f0 0x562829b00670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/3421033314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829af5db0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/3421033314 >> 172.21.15.103:0/3421033314 conn(0x562829af1f60 msgr2=0x562829af2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/3421033314 shutdown_connections 2024-09-19T07:37:26.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.907+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/3421033314 wait complete. 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 -- start start 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 Processor -- start 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829b004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829afdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562829b03780 con 0x562829af62f0 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562829a16f20 con 0x562829af59d0 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562829b03900 con 0x562829b00bb0 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7b2a7740 10 client.0 _ll_register_callbacks cb 0x5628299e0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41106/0 (socket says 172.21.15.103:41106) 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 -- 172.21.15.103:0/2953277755 learned_addr learned my addr 172.21.15.103:0/2953277755 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:26.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf7259c640 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829b004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829afdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 -- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 msgr2=0x562829afdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829afdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 -- 172.21.15.103:0/2953277755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 msgr2=0x562829b004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829b004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.908+0000 7fdf71d9b640 1 -- 172.21.15.103:0/2953277755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628297cd2b0 con 0x562829af62f0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afd830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009900 tx=0x7fdf6803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a49acac4ca99293 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf59ffb640 1 -- 172.21.15.103:0/2953277755 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68049040 con 0x562829af62f0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf59ffb640 1 -- 172.21.15.103:0/2953277755 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdf680eda80 con 0x562829af62f0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf6c00ba00 con 0x562829af62f0 2024-09-19T07:37:26.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf59ffb640 1 -- 172.21.15.103:0/2953277755 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf680bc3f0 con 0x562829af62f0 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 10 client.14991 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562829b0f130 con 0x562829af62f0 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf59ffb640 1 -- 172.21.15.103:0/2953277755 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf6c00ba00 con 0x562829af62f0 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf59ffb640 1 -- 172.21.15.103:0/2953277755 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdf680330d0 con 0x562829af62f0 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 10 client.14991 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 10 client.14991 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 20 client.14991 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b11770 0x562829b12be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 4 client.14991 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.909+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562829a69600 con 0x562829b11770 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.910+0000 7fdf7259c640 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b11770 0x562829b12be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.910+0000 7fdf59ffb640 10 client.14991 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.910+0000 7fdf7259c640 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b11770 0x562829b12be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33f23e6a79f742de server_cookie=fd6499ed39dcdd35 in_seq=0 out_seq=0 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf59ffb640 1 -- 172.21.15.103:0/2953277755 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562829a69600 con 0x562829b11770 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf59ffb640 10 client.14991 handle_command_reply: tid=0 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 client.14991 shutdown 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 msgr2=0x562829afd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afd830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009900 tx=0x7fdf6803b000 comp rx=0 tx=0).stop 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 shutdown_connections 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b11770 0x562829b12be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829afdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afd830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/2953277755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829b004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 >> 172.21.15.103:0/2953277755 conn(0x562829af1f60 msgr2=0x562829af7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf59ffb640 0 client.14991 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:26.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 shutdown_connections 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/2953277755 wait complete. 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.911+0000 7fdf7b2a7740 20 client.14991 trim_cache size 0 max 16384 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.928+0000 7fdf7b2a7740 1 Processor -- start 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7b2a7740 1 -- start start 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829b110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af62f0 0x562829b12510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829af6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562829b10f20 con 0x562829af59d0 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562829b115f0 con 0x562829af62f0 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562829b11cc0 con 0x562829b00bb0 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829b110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829b110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41110/0 (socket says 172.21.15.103:41110) 2024-09-19T07:37:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 -- 172.21.15.103:0/711640521 learned_addr learned my addr 172.21.15.103:0/711640521 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 -- 172.21.15.103:0/711640521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 msgr2=0x562829af6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829af6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 -- 172.21.15.103:0/711640521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af62f0 msgr2=0x562829b12510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af62f0 0x562829b12510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af62f0 0x562829b12510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 -- 172.21.15.103:0/711640521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562829af57e0 con 0x562829af59d0 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af62f0 0x562829b12510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf7259c640 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829b110a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdf64013360 tx=0x7fdf64015af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec23a7a2812aca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf5bfff640 1 -- 172.21.15.103:0/711640521 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64015390 con 0x562829af59d0 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf5bfff640 1 -- 172.21.15.103:0/711640521 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf64003e50 con 0x562829af59d0 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.929+0000 7fdf5bfff640 1 -- 172.21.15.103:0/711640521 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64004050 con 0x562829af59d0 2024-09-19T07:37:26.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/711640521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 msgr2=0x562829b110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829b110a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdf64013360 tx=0x7fdf64015af0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/711640521 shutdown_connections 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829af6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af62f0 0x562829b12510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/711640521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af59d0 0x562829b110a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/711640521 >> 172.21.15.103:0/711640521 conn(0x562829af1f60 msgr2=0x562829af2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/711640521 shutdown_connections 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/711640521 wait complete. 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 -- start start 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.930+0000 7fdf7b2a7740 1 Processor -- start 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829afac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829b1a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562829b12a60 con 0x562829af62f0 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562829b12390 con 0x562829af59d0 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562829b13130 con 0x562829b00bb0 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 10 client.0 _ll_register_callbacks cb 0x7fdf68009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41126/0 (socket says 172.21.15.103:41126) 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 -- 172.21.15.103:0/1242902980 learned_addr learned my addr 172.21.15.103:0/1242902980 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7259c640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829afac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf72d9d640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829b1a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 -- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 msgr2=0x562829b1a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829b1a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 -- 172.21.15.103:0/1242902980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 msgr2=0x562829afac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829afac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 -- 172.21.15.103:0/1242902980 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562829b1a810 con 0x562829af62f0 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7259c640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829afac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf71d9b640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afb1d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdf680b8850 tx=0x7fdf68005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0683d9ad6ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf59ffb640 1 -- 172.21.15.103:0/1242902980 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68049050 con 0x562829af62f0 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf59ffb640 1 -- 172.21.15.103:0/1242902980 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fdf680eda80 con 0x562829af62f0 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf64003e50 con 0x562829af62f0 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf59ffb640 1 -- 172.21.15.103:0/1242902980 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf680bbf30 con 0x562829af62f0 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 10 client.14997 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:26.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.931+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562829b7f680 con 0x562829af62f0 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf59ffb640 1 -- 172.21.15.103:0/1242902980 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf64003e50 con 0x562829af62f0 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf59ffb640 1 -- 172.21.15.103:0/1242902980 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdf680bc650 con 0x562829af62f0 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7b2a7740 10 client.14997 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7b2a7740 10 client.14997 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7b2a7740 20 client.14997 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b83ff0 0x562829b82d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7b2a7740 4 client.14997 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 71.190064798999998, 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 96.984531328000003 2024-09-19T07:37:26.958 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "status"}) -- 0x562829a63b10 con 0x562829b83ff0 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7259c640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b83ff0 0x562829b82d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf7259c640 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b83ff0 0x562829b82d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adb1fd5e4a1d582d server_cookie=4de1d27111f7fe26 in_seq=0 out_seq=0 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.932+0000 7fdf59ffb640 10 client.14997 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf59ffb640 1 -- 172.21.15.103:0/1242902980 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562829a63b10 con 0x562829b83ff0 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf59ffb640 10 client.14997 handle_command_reply: tid=0 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 client.14997 shutdown 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 msgr2=0x562829afb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afb1d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdf680b8850 tx=0x7fdf68005c00 comp rx=0 tx=0).stop 2024-09-19T07:37:26.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 shutdown_connections 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x562829b83ff0 0x562829b82d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562829b00bb0 0x562829b1a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562829af62f0 0x562829afb1d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf59ffb640 0 client.14997 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 --2- 172.21.15.103:0/1242902980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562829af59d0 0x562829afac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 >> 172.21.15.103:0/1242902980 conn(0x562829af1f60 msgr2=0x562829af7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 shutdown_connections 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 1 -- 172.21.15.103:0/1242902980 wait complete. 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf7b2a7740 20 client.14997 trim_cache size 0 max 16384 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fdf5c0a3350 msgr2=0x7fdf5c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c010800 tx=0x7fdf6c017000 comp rx=0 tx=0).stop 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74101400 msgr2=0x7fdf74199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74101400 0x7fdf74199850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009230 tx=0x7fdf68004270 comp rx=0 tx=0).stop 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 shutdown_connections 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fdf5c0a3350 0x7fdf5c0a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf74106a60 0x7fdf741a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf74101400 0x7fdf74199850 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 --2- 172.21.15.103:0/3880272178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf74100a60 0x7fdf74199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 >> 172.21.15.103:0/3880272178 conn(0x7fdf740fc780 msgr2=0x7fdf740fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 shutdown_connections 2024-09-19T07:37:26.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:26.933+0000 7fdf78ece640 1 -- 172.21.15.103:0/3880272178 wait complete. 2024-09-19T07:37:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:27 smithi174 ceph-mon[35370]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:27.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:27 smithi103 ceph-mon[31201]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:27 smithi134 ceph-mon[35414]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:27.630 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T07:37:27.878 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:28.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 -- 172.21.15.103:0/1690258687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100d20 msgr2=0x7f88e8106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 --2- 172.21.15.103:0/1690258687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100d20 0x7f88e8106860 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009930 tx=0x7f88dc02a420 comp rx=0 tx=0).stop 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 -- 172.21.15.103:0/1690258687 shutdown_connections 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 --2- 172.21.15.103:0/1690258687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88e8106fc0 0x7f88e810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 --2- 172.21.15.103:0/1690258687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100d20 0x7f88e8106860 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 --2- 172.21.15.103:0/1690258687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88e8100370 0x7f88e8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 -- 172.21.15.103:0/1690258687 >> 172.21.15.103:0/1690258687 conn(0x7f88e8077210 msgr2=0x7f88e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 -- 172.21.15.103:0/1690258687 shutdown_connections 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.360+0000 7f88ee480640 1 -- 172.21.15.103:0/1690258687 wait complete. 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 Processor -- start 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 -- start start 2024-09-19T07:37:28.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100370 0x7f88e81012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88e8100d20 0x7f88e8101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88e8106fc0 0x7f88e8101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8111fa0 con 0x7f88e8100370 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8111e20 con 0x7f88e8106fc0 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88ee480640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8112120 con 0x7f88e8100d20 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100370 0x7f88e81012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100370 0x7f88e81012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36398/0 (socket says 172.21.15.103:36398) 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 -- 172.21.15.103:0/2587963275 learned_addr learned my addr 172.21.15.103:0/2587963275 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e77fe640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88e8100d20 0x7f88e8101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88eca1d640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88e8106fc0 0x7f88e8101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 -- 172.21.15.103:0/2587963275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88e8100d20 msgr2=0x7f88e8101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88e8100d20 0x7f88e8101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 -- 172.21.15.103:0/2587963275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88e8106fc0 msgr2=0x7f88e8101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88e8106fc0 0x7f88e8101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e7fff640 1 -- 172.21.15.103:0/2587963275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e81a6920 con 0x7f88e8100370 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.361+0000 7f88e77fe640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88e8100d20 0x7f88e8101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.362+0000 7f88e7fff640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100370 0x7f88e81012f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d950 tx=0x7f88d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.362+0000 7f88e57fa640 1 -- 172.21.15.103:0/2587963275 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8017070 con 0x7f88e8100370 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.362+0000 7f88e57fa640 1 -- 172.21.15.103:0/2587963275 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d8009c30 con 0x7f88e8100370 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.362+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e81a6c10 con 0x7f88e8100370 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.362+0000 7f88e57fa640 1 -- 172.21.15.103:0/2587963275 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d800f6b0 con 0x7f88e8100370 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.362+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81a7150 con 0x7f88e8100370 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.364+0000 7f88e57fa640 1 -- 172.21.15.103:0/2587963275 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f88d80109d0 con 0x7f88e8100370 2024-09-19T07:37:28.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 Processor -- start 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 -- start start 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e57fa640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f88d00a32d0 0x7f88d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d89d0 0x55a5be1d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e57fa640 1 -- 172.21.15.103:0/2587963275 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f88d80cf7a0 con 0x7f88e8100370 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5be13f090 con 0x55a5be1d92f0 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88eca1d640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f88d00a32d0 0x7f88d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5be0e6a70 con 0x55a5be1e3bb0 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88f0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5be13b050 con 0x55a5be1d89d0 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36404/0 (socket says 172.21.15.103:36404) 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 -- 172.21.15.103:0/3689428505 learned_addr learned my addr 172.21.15.103:0/3689428505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e7fff640 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d89d0 0x55a5be1d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88eca1d640 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 -- 172.21.15.103:0/3689428505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d89d0 msgr2=0x55a5be1d8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d89d0 0x55a5be1d8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 -- 172.21.15.103:0/3689428505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 msgr2=0x55a5be1e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88e77fe640 1 -- 172.21.15.103:0/3689428505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5be0f3130 con 0x55a5be1d92f0 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.365+0000 7f88eca1d640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f88d00a32d0 0x7f88d00a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88e00125e0 tx=0x7f88e00129e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.366+0000 7f88e77fe640 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e3670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f88dc006010 tx=0x7f88dc009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec52a02517d8fe3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.366+0000 7f88ce7fc640 1 -- 172.21.15.103:0/3689428505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc03f070 con 0x55a5be1d92f0 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.366+0000 7f88ce7fc640 1 -- 172.21.15.103:0/3689428505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88dc004030 con 0x55a5be1d92f0 2024-09-19T07:37:28.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.366+0000 7f88ce7fc640 1 -- 172.21.15.103:0/3689428505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc035070 con 0x55a5be1d92f0 2024-09-19T07:37:28.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.366+0000 7f88f0859740 1 -- 172.21.15.103:0/3689428505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 msgr2=0x55a5be1e3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.366+0000 7f88f0859740 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e3670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f88dc006010 tx=0x7f88dc009680 comp rx=0 tx=0).stop 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- 172.21.15.103:0/3689428505 shutdown_connections 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e3670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 --2- 172.21.15.103:0/3689428505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d89d0 0x55a5be1d8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- 172.21.15.103:0/3689428505 >> 172.21.15.103:0/3689428505 conn(0x55a5be1d4f60 msgr2=0x55a5be1d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- 172.21.15.103:0/3689428505 shutdown_connections 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- 172.21.15.103:0/3689428505 wait complete. 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- start start 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 Processor -- start 2024-09-19T07:37:28.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1e3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1e6780 con 0x55a5be1d92f0 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5be0f9f20 con 0x55a5be1d89d0 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1e6900 con 0x55a5be1e3bb0 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.367+0000 7f88f0859740 10 client.0 _ll_register_callbacks cb 0x55a5be0c3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36412/0 (socket says 172.21.15.103:36412) 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 -- 172.21.15.103:0/4180780236 learned_addr learned my addr 172.21.15.103:0/4180780236 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e7fff640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88eca1d640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 -- 172.21.15.103:0/4180780236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 msgr2=0x55a5be1e0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 -- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 msgr2=0x55a5be1e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1e3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 -- 172.21.15.103:0/4180780236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5bdeb02b0 con 0x55a5be1d92f0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88eca1d640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88e77fe640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e0650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17af0d0a6c467e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88c77fe640 1 -- 172.21.15.103:0/4180780236 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc04b020 con 0x55a5be1d92f0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88c77fe640 1 -- 172.21.15.103:0/4180780236 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f88dc0315b0 con 0x55a5be1d92f0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88dc004030 con 0x55a5be1d92f0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88c77fe640 1 -- 172.21.15.103:0/4180780236 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88dc035070 con 0x55a5be1d92f0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88f0859740 10 client.15009 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.368+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5be1f1f90 con 0x55a5be1d92f0 2024-09-19T07:37:28.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88c77fe640 1 -- 172.21.15.103:0/4180780236 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88dc004030 con 0x55a5be1d92f0 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88c77fe640 1 -- 172.21.15.103:0/4180780236 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f88dc0046c0 con 0x55a5be1d92f0 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88f0859740 10 client.15009 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88f0859740 10 client.15009 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88f0859740 20 client.15009 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88f0859740 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be1f41c0 0x55a5be1f5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88f0859740 4 client.15009 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5be14c600 con 0x55a5be1f41c0 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.369+0000 7f88e7fff640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be1f41c0 0x55a5be1f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.370+0000 7f88c77fe640 10 client.15009 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.370+0000 7f88e7fff640 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be1f41c0 0x55a5be1f5630 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd917e623fe4789 server_cookie=bbbc91c8431e8b38 in_seq=0 out_seq=0 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.370+0000 7f88c77fe640 1 -- 172.21.15.103:0/4180780236 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5be14c600 con 0x55a5be1f41c0 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.370+0000 7f88c77fe640 10 client.15009 handle_command_reply: tid=0 2024-09-19T07:37:28.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.370+0000 7f88f0859740 1 client.15009 shutdown 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.370+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 msgr2=0x55a5be1e0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.370+0000 7f88f0859740 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e0650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc009260 comp rx=0 tx=0).stop 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 shutdown_connections 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be1f41c0 0x55a5be1f5630 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1e0650 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88c77fe640 0 client.15009 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 --2- 172.21.15.103:0/4180780236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 >> 172.21.15.103:0/4180780236 conn(0x55a5be1d4f60 msgr2=0x55a5be1dac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 shutdown_connections 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 1 -- 172.21.15.103:0/4180780236 wait complete. 2024-09-19T07:37:28.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.371+0000 7f88f0859740 20 client.15009 trim_cache size 0 max 16384 2024-09-19T07:37:28.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 Processor -- start 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 -- start start 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1f5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1f3970 con 0x55a5be1d92f0 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1f4040 con 0x55a5be1d89d0 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88f0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1f4710 con 0x55a5be1e3bb0 2024-09-19T07:37:28.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36414/0 (socket says 172.21.15.103:36414) 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88e77fe640 1 -- 172.21.15.103:0/1504692950 learned_addr learned my addr 172.21.15.103:0/1504692950 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88e7fff640 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.388+0000 7f88eca1d640 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88e77fe640 1 -- 172.21.15.103:0/1504692950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 msgr2=0x55a5be1e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88e77fe640 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88e77fe640 1 -- 172.21.15.103:0/1504692950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 msgr2=0x55a5be1f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88e77fe640 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88e77fe640 1 -- 172.21.15.103:0/1504692950 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5be1d4d90 con 0x55a5be1d92f0 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88e77fe640 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1cfe10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88dc004c80 tx=0x7f88dc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f1e1d4d4a35d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88cdffb640 1 -- 172.21.15.103:0/1504692950 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc04b040 con 0x55a5be1d92f0 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88cdffb640 1 -- 172.21.15.103:0/1504692950 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88dc032de0 con 0x55a5be1d92f0 2024-09-19T07:37:28.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88cdffb640 1 -- 172.21.15.103:0/1504692950 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc03f040 con 0x55a5be1d92f0 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88f0859740 1 -- 172.21.15.103:0/1504692950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 msgr2=0x55a5be1cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88f0859740 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1cfe10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88dc004c80 tx=0x7f88dc005fd0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88f0859740 1 -- 172.21.15.103:0/1504692950 shutdown_connections 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88f0859740 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1e3bb0 0x55a5be1e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88f0859740 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d92f0 0x55a5be1cfe10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88f0859740 1 --2- 172.21.15.103:0/1504692950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1d89d0 0x55a5be1f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.389+0000 7f88f0859740 1 -- 172.21.15.103:0/1504692950 >> 172.21.15.103:0/1504692950 conn(0x55a5be1d4f60 msgr2=0x55a5be1e1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 -- 172.21.15.103:0/1504692950 shutdown_connections 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 -- 172.21.15.103:0/1504692950 wait complete. 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 -- start start 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 Processor -- start 2024-09-19T07:37:28.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d89d0 0x55a5be1fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d92f0 0x55a5be1dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1f54b0 con 0x55a5be1d89d0 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1f4de0 con 0x55a5be1e3bb0 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5be1f5b80 con 0x55a5be1d92f0 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88f0859740 10 client.0 _ll_register_callbacks cb 0x7f88dc032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d89d0 0x55a5be1fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d89d0 0x55a5be1fdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36416/0 (socket says 172.21.15.103:36416) 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.390+0000 7f88e7fff640 1 -- 172.21.15.103:0/3722215799 learned_addr learned my addr 172.21.15.103:0/3722215799 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88eca1d640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e77fe640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d92f0 0x55a5be1dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e7fff640 1 -- 172.21.15.103:0/3722215799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d92f0 msgr2=0x55a5be1dadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e7fff640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d92f0 0x55a5be1dadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e7fff640 1 -- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 msgr2=0x55a5be1db2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e7fff640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1db2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e7fff640 1 -- 172.21.15.103:0/3722215799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5be1dba00 con 0x55a5be1d89d0 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e77fe640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d92f0 0x55a5be1dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88e7fff640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d89d0 0x55a5be1fdf00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f88d80980c0 tx=0x7f88d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de312bf7e3ba1217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88c77fe640 1 -- 172.21.15.103:0/3722215799 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d80b4e60 con 0x55a5be1d89d0 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88c77fe640 1 -- 172.21.15.103:0/3722215799 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f88d80982d0 con 0x55a5be1d89d0 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88dc032de0 con 0x55a5be1d89d0 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88c77fe640 1 -- 172.21.15.103:0/3722215799 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88d8020040 con 0x55a5be1d89d0 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88f0859740 10 client.15015 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:28.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a5be25e8d0 con 0x55a5be1d89d0 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.391+0000 7f88c77fe640 1 -- 172.21.15.103:0/3722215799 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88dc032de0 con 0x55a5be1d89d0 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88c77fe640 1 -- 172.21.15.103:0/3722215799 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f88d809abb0 con 0x55a5be1d89d0 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88f0859740 10 client.15015 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88f0859740 10 client.15015 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88f0859740 20 client.15015 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88f0859740 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be2633e0 0x55a5be2620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88f0859740 4 client.15015 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "status"}) -- 0x55a5be146b10 con 0x55a5be2633e0 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88e77fe640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be2633e0 0x55a5be2620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88e77fe640 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be2633e0 0x55a5be2620f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79bd0ab26d203437 server_cookie=8024d220622d6c3c in_seq=0 out_seq=0 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.392+0000 7f88c77fe640 10 client.15015 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88c77fe640 1 -- 172.21.15.103:0/3722215799 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a5be146b10 con 0x55a5be2633e0 2024-09-19T07:37:28.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88c77fe640 10 client.15015 handle_command_reply: tid=0 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 client.15015 shutdown 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d89d0 msgr2=0x55a5be1fdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d89d0 0x55a5be1fdf00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f88d80980c0 tx=0x7f88d8000bc0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 shutdown_connections 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55a5be2633e0 0x55a5be2620f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a5be1e3bb0 0x55a5be1db2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88c77fe640 0 client.15015 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5be1d92f0 0x55a5be1dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 --2- 172.21.15.103:0/3722215799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5be1d89d0 0x55a5be1fdf00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 >> 172.21.15.103:0/3722215799 conn(0x55a5be1d4f60 msgr2=0x55a5be1e06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 shutdown_connections 2024-09-19T07:37:28.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 1 -- 172.21.15.103:0/3722215799 wait complete. 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88f0859740 20 client.15015 trim_cache size 0 max 16384 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24494, 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:28.403 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 79.548797468000004, 2024-09-19T07:37:28.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:28.404 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:28.404 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:28.404 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 89.885316017999997 2024-09-19T07:37:28.404 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f88d00a32d0 msgr2=0x7f88d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88ee480640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f88d00a32d0 0x7f88d00a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88e00125e0 tx=0x7f88e00129e0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100370 msgr2=0x7f88e81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.393+0000 7f88ee480640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100370 0x7f88e81012f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d950 tx=0x7f88d800de10 comp rx=0 tx=0).stop 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 shutdown_connections 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f88d00a32d0 0x7f88d00a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88e8106fc0 0x7f88e8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88e8100d20 0x7f88e8101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 --2- 172.21.15.103:0/2587963275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88e8100370 0x7f88e81012f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:28.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 >> 172.21.15.103:0/2587963275 conn(0x7f88e8077210 msgr2=0x7f88e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:28.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 shutdown_connections 2024-09-19T07:37:28.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:28.394+0000 7f88ee480640 1 -- 172.21.15.103:0/2587963275 wait complete. 2024-09-19T07:37:29.215 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T07:37:29.216 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T07:37:29.226 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:37:29.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T07:37:29.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:29 smithi174 ceph-mon[35370]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:29 smithi103 ceph-mon[31201]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:29.476 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:29 smithi134 ceph-mon[35414]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 -- 172.21.15.103:0/2606642926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12981026a0 msgr2=0x7f1298104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 --2- 172.21.15.103:0/2606642926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12981026a0 0x7f1298104a90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1294009900 tx=0x7f129402a410 comp rx=0 tx=0).stop 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 -- 172.21.15.103:0/2606642926 shutdown_connections 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 --2- 172.21.15.103:0/2606642926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1298107900 0x7f1298109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 --2- 172.21.15.103:0/2606642926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1298104fd0 0x7f12981073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 --2- 172.21.15.103:0/2606642926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12981026a0 0x7f1298104a90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 -- 172.21.15.103:0/2606642926 >> 172.21.15.103:0/2606642926 conn(0x7f12980fc2a0 msgr2=0x7f12980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 -- 172.21.15.103:0/2606642926 shutdown_connections 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.931+0000 7f129fc0e640 1 -- 172.21.15.103:0/2606642926 wait complete. 2024-09-19T07:37:29.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 Processor -- start 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 -- start start 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12981026a0 0x7f1298102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1298104fd0 0x7f1298100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1298107900 0x7f1298100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f129810d940 con 0x7f1298107900 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f129810d7c0 con 0x7f1298104fd0 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129fc0e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f129810dac0 con 0x7f12981026a0 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1298107900 0x7f1298100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1298107900 0x7f1298100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36426/0 (socket says 172.21.15.103:36426) 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 -- 172.21.15.103:0/3165228766 learned_addr learned my addr 172.21.15.103:0/3165228766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129d9aa640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12981026a0 0x7f1298102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:29.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129d1a9640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1298104fd0 0x7f1298100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 -- 172.21.15.103:0/3165228766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12981026a0 msgr2=0x7f1298102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12981026a0 0x7f1298102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 -- 172.21.15.103:0/3165228766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1298104fd0 msgr2=0x7f1298100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1298104fd0 0x7f1298100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 -- 172.21.15.103:0/3165228766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1298101450 con 0x7f1298107900 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129d1a9640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1298104fd0 0x7f1298100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.932+0000 7f129e1ab640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1298107900 0x7f1298100b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f129000b970 tx=0x7f129000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.933+0000 7f128effd640 1 -- 172.21.15.103:0/3165228766 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290019070 con 0x7f1298107900 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.933+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12981a2130 con 0x7f1298107900 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.933+0000 7f128effd640 1 -- 172.21.15.103:0/3165228766 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129000cb40 con 0x7f1298107900 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.933+0000 7f128effd640 1 -- 172.21.15.103:0/3165228766 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290007650 con 0x7f1298107900 2024-09-19T07:37:29.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.933+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981a26c0 con 0x7f1298107900 2024-09-19T07:37:29.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.934+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12980fdd90 con 0x7f1298107900 2024-09-19T07:37:29.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.935+0000 7f128effd640 1 -- 172.21.15.103:0/3165228766 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f129001c070 con 0x7f1298107900 2024-09-19T07:37:29.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.935+0000 7f128effd640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f12840a32d0 0x7f12840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:29.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.935+0000 7f129d9aa640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f12840a32d0 0x7f12840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:29.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.935+0000 7f128effd640 1 -- 172.21.15.103:0/3165228766 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f12900cef30 con 0x7f1298107900 2024-09-19T07:37:29.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.936+0000 7f129d9aa640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f12840a32d0 0x7f12840a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12940314e0 tx=0x7f1294000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:29.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:29.940+0000 7f128effd640 1 -- 172.21.15.103:0/3165228766 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f129009b5b0 con 0x7f1298107900 2024-09-19T07:37:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.122+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1298062630 con 0x7f12840a32d0 2024-09-19T07:37:30.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.124+0000 7f128effd640 1 -- 172.21.15.103:0/3165228766 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1298062630 con 0x7f12840a32d0 2024-09-19T07:37:30.124 INFO:teuthology.orchestra.run.smithi103.stdout:Backend: cephadm 2024-09-19T07:37:30.125 INFO:teuthology.orchestra.run.smithi103.stdout:Available: Yes 2024-09-19T07:37:30.125 INFO:teuthology.orchestra.run.smithi103.stdout:Paused: No 2024-09-19T07:37:30.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f12840a32d0 msgr2=0x7f12840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f12840a32d0 0x7f12840a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12940314e0 tx=0x7f1294000bc0 comp rx=0 tx=0).stop 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1298107900 msgr2=0x7f1298100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1298107900 0x7f1298100b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f129000b970 tx=0x7f129000be30 comp rx=0 tx=0).stop 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 shutdown_connections 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f12840a32d0 0x7f12840a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1298107900 0x7f1298100b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1298104fd0 0x7f1298100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 --2- 172.21.15.103:0/3165228766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12981026a0 0x7f1298102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 >> 172.21.15.103:0/3165228766 conn(0x7f12980fc2a0 msgr2=0x7f1298103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 shutdown_connections 2024-09-19T07:37:30.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:30.126+0000 7f129fc0e640 1 -- 172.21.15.103:0/3165228766 wait complete. 2024-09-19T07:37:30.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T07:37:31.082 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:31 smithi174 ceph-mon[35370]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:31 smithi174 ceph-mon[35370]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:31.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:31 smithi103 ceph-mon[31201]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:31.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:31 smithi103 ceph-mon[31201]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:31.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 -- 172.21.15.103:0/1835021139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 msgr2=0x7f069c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:31.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 --2- 172.21.15.103:0/1835021139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c102150 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0684009900 tx=0x7f068402a410 comp rx=0 tx=0).stop 2024-09-19T07:37:31.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 -- 172.21.15.103:0/1835021139 shutdown_connections 2024-09-19T07:37:31.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 --2- 172.21.15.103:0/1835021139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c102690 0x7f069c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 --2- 172.21.15.103:0/1835021139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c102150 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 --2- 172.21.15.103:0/1835021139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f069c100ad0 0x7f069c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 -- 172.21.15.103:0/1835021139 >> 172.21.15.103:0/1835021139 conn(0x7f069c0fc280 msgr2=0x7f069c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 -- 172.21.15.103:0/1835021139 shutdown_connections 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 -- 172.21.15.103:0/1835021139 wait complete. 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 Processor -- start 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.532+0000 7f06a09ea640 1 -- start start 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f06a09ea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c100ad0 0x7f069c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f06a09ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f06a09ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f069c102690 0x7f069c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f06a09ea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f069c10c7b0 con 0x7f069c101cd0 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f06a09ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f069c10c630 con 0x7f069c102690 2024-09-19T07:37:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f06a09ea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f069c10c930 con 0x7f069c100ad0 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36438/0 (socket says 172.21.15.103:36438) 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 -- 172.21.15.103:0/3510517345 learned_addr learned my addr 172.21.15.103:0/3510517345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f069ad9d640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f069c102690 0x7f069c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f069a59c640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c100ad0 0x7f069c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 -- 172.21.15.103:0/3510517345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c100ad0 msgr2=0x7f069c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c100ad0 0x7f069c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 -- 172.21.15.103:0/3510517345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f069c102690 msgr2=0x7f069c19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f069c102690 0x7f069c19f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 -- 172.21.15.103:0/3510517345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c19fbe0 con 0x7f069c101cd0 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f069a59c640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c100ad0 0x7f069c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:31.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.533+0000 7f0699d9b640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c19ccb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0684009230 tx=0x7f0684004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.534+0000 7f06937fe640 1 -- 172.21.15.103:0/3510517345 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068403f030 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.534+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c19fe10 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.534+0000 7f06937fe640 1 -- 172.21.15.103:0/3510517345 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0684003ce0 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.534+0000 7f06937fe640 1 -- 172.21.15.103:0/3510517345 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684035070 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.534+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c1a02f0 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.535+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069c107ed0 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.536+0000 7f06937fe640 1 -- 172.21.15.103:0/3510517345 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f0684042030 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.536+0000 7f06937fe640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f06880a32d0 0x7f06880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.537+0000 7f069a59c640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f06880a32d0 0x7f06880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.537+0000 7f06937fe640 1 -- 172.21.15.103:0/3510517345 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f06840f05b0 con 0x7f069c101cd0 2024-09-19T07:37:31.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.537+0000 7f069a59c640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f06880a32d0 0x7f06880a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f069c101b30 tx=0x7f069400a800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:31.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.540+0000 7f06937fe640 1 -- 172.21.15.103:0/3510517345 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f06840bd170 con 0x7f069c101cd0 2024-09-19T07:37:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:31 smithi134 ceph-mon[35414]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:31.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:31 smithi134 ceph-mon[35414]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:31.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.717+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f069c062630 con 0x7f06880a32d0 2024-09-19T07:37:31.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.731+0000 7f06937fe640 1 -- 172.21.15.103:0/3510517345 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f069c062630 con 0x7f06880a32d0 2024-09-19T07:37:31.732 INFO:teuthology.orchestra.run.smithi103.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T07:37:31.732 INFO:teuthology.orchestra.run.smithi103.stdout:mds.a smithi103 running (96s) 19s ago 95s 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 0af034840cb8 2024-09-19T07:37:31.732 INFO:teuthology.orchestra.run.smithi103.stdout:mds.b smithi134 running (93s) 26s ago 93s 26.3M - 19.1.1-361-g2f599a46 d2baa855c067 a76ebc26f058 2024-09-19T07:37:31.732 INFO:teuthology.orchestra.run.smithi103.stdout:mds.c smithi174 running (107s) 26s ago 106s 19.8M - 19.1.1-361-g2f599a46 d2baa855c067 b916fbfb0045 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.d smithi103 running (114s) 19s ago 114s 14.3M - 19.1.1-361-g2f599a46 d2baa855c067 a0d1b727f2f2 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.e smithi134 running (112s) 26s ago 111s 20.5M - 19.1.1-361-g2f599a46 d2baa855c067 e3df714ada3e 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.f smithi174 running (109s) 26s ago 109s 18.1M - 19.1.1-361-g2f599a46 d2baa855c067 01da294647bd 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.g smithi103 running (99s) 19s ago 99s 16.3M - 19.1.1-361-g2f599a46 d2baa855c067 0e48ee93f058 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.h smithi134 running (104s) 26s ago 104s 20.2M - 19.1.1-361-g2f599a46 d2baa855c067 68564d7e5b73 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.i smithi174 running (102s) 26s ago 101s 18.0M - 19.1.1-361-g2f599a46 d2baa855c067 afc78d30af92 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.j smithi103 running (118s) 19s ago 118s 15.9M - 19.1.1-361-g2f599a46 d2baa855c067 3d9a484773c4 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.k smithi134 running (2m) 26s ago 2m 19.1M - 19.1.1-361-g2f599a46 d2baa855c067 25a6eb472a19 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mds.l smithi174 running (2m) 26s ago 2m 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 fabe4f01144f 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.x smithi103 *:9283,8765 running (8m) 19s ago 8m 509M - 19.1.1-361-g2f599a46 d2baa855c067 a2584dd22f2b 2024-09-19T07:37:31.733 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.y smithi134 *:8443,8765 running (7m) 26s ago 7m 441M - 19.1.1-361-g2f599a46 d2baa855c067 e8078c7d2cfd 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.z smithi174 *:8443,8765 running (7m) 26s ago 7m 443M - 19.1.1-361-g2f599a46 d2baa855c067 15ac4ad71a81 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:mon.a smithi103 running (8m) 19s ago 9m 52.2M 2048M 19.1.1-361-g2f599a46 d2baa855c067 4589aa5fd5c9 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:mon.b smithi134 running (7m) 26s ago 7m 41.8M 2048M 19.1.1-361-g2f599a46 d2baa855c067 8cb15dd05f85 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:mon.c smithi174 running (7m) 26s ago 7m 39.1M 2048M 19.1.1-361-g2f599a46 d2baa855c067 aa0db4862662 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 smithi103 running (7m) 19s ago 7m 44.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 d3adbdcc2834 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 smithi134 running (6m) 26s ago 6m 46.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 6f5682f49e88 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 smithi174 running (6m) 26s ago 6m 47.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 8bb3e274bf43 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 smithi103 running (5m) 19s ago 5m 43.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 4f58e3de90bf 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 smithi134 running (5m) 26s ago 5m 52.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 abaf9ff3e1d0 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 smithi174 running (4m) 26s ago 4m 42.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 cc251e5c3aec 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 smithi103 running (4m) 19s ago 4m 41.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 87659a438582 2024-09-19T07:37:31.734 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 smithi134 running (4m) 26s ago 4m 46.6M 4272M 19.1.1-361-g2f599a46 d2baa855c067 989b72fa0d32 2024-09-19T07:37:31.735 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 smithi174 running (3m) 26s ago 3m 53.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 75e322fed024 2024-09-19T07:37:31.735 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 smithi103 running (3m) 19s ago 3m 44.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 c9628ac5a5b9 2024-09-19T07:37:31.735 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 smithi134 running (2m) 26s ago 2m 45.0M 4272M 19.1.1-361-g2f599a46 d2baa855c067 8abbb173f7ef 2024-09-19T07:37:31.735 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 smithi174 running (2m) 26s ago 2m 41.4M 4272M 19.1.1-361-g2f599a46 d2baa855c067 84e693903cfb 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.733+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f06880a32d0 msgr2=0x7f06880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.733+0000 7f06a09ea640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f06880a32d0 0x7f06880a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f069c101b30 tx=0x7f069400a800 comp rx=0 tx=0).stop 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.733+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 msgr2=0x7f069c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.733+0000 7f06a09ea640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c19ccb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0684009230 tx=0x7f0684004430 comp rx=0 tx=0).stop 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 shutdown_connections 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f06880a32d0 0x7f06880a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f069c102690 0x7f069c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f069c101cd0 0x7f069c19ccb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 --2- 172.21.15.103:0/3510517345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c100ad0 0x7f069c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:31.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 >> 172.21.15.103:0/3510517345 conn(0x7f069c0fc280 msgr2=0x7f069c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:31.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 shutdown_connections 2024-09-19T07:37:31.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:31.734+0000 7f06a09ea640 1 -- 172.21.15.103:0/3510517345 wait complete. 2024-09-19T07:37:32.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T07:37:32.666 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:33.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 -- 172.21.15.103:0/3541111268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301037c0 msgr2=0x7f293010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:33.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 --2- 172.21.15.103:0/3541111268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301037c0 0x7f293010a050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ca0 tx=0x7f292802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 -- 172.21.15.103:0/3541111268 shutdown_connections 2024-09-19T07:37:33.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 --2- 172.21.15.103:0/3541111268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29301037c0 0x7f293010a050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 --2- 172.21.15.103:0/3541111268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2930102e00 0x7f2930103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 --2- 172.21.15.103:0/3541111268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2930101c00 0x7f2930102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 -- 172.21.15.103:0/3541111268 >> 172.21.15.103:0/3541111268 conn(0x7f29300fd390 msgr2=0x7f29300ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 -- 172.21.15.103:0/3541111268 shutdown_connections 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 -- 172.21.15.103:0/3541111268 wait complete. 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 Processor -- start 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.145+0000 7f2937903640 1 -- start start 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2930101c00 0x7f2930075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930102e00 0x7f29300745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29301037c0 0x7f2930074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f293010d980 con 0x7f2930102e00 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f293010d800 con 0x7f29301037c0 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f293010db00 con 0x7f2930101c00 2024-09-19T07:37:33.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930102e00 0x7f29300745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930102e00 0x7f29300745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36448/0 (socket says 172.21.15.103:36448) 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 -- 172.21.15.103:0/2783923497 learned_addr learned my addr 172.21.15.103:0/2783923497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 -- 172.21.15.103:0/2783923497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2930101c00 msgr2=0x7f2930075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2930101c00 0x7f2930075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2935ea0640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29301037c0 0x7f2930074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 -- 172.21.15.103:0/2783923497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29301037c0 msgr2=0x7f2930074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29301037c0 0x7f2930074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 -- 172.21.15.103:0/2783923497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29300753e0 con 0x7f2930102e00 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2934e9e640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930102e00 0x7f29300745e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f292000b4a0 tx=0x7f292000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f29267fc640 1 -- 172.21.15.103:0/2783923497 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920014070 con 0x7f2930102e00 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f29267fc640 1 -- 172.21.15.103:0/2783923497 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2920010af0 con 0x7f2930102e00 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29301a21c0 con 0x7f2930102e00 2024-09-19T07:37:33.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f29267fc640 1 -- 172.21.15.103:0/2783923497 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920017d10 con 0x7f2930102e00 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.146+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301a2750 con 0x7f2930102e00 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.147+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2930109050 con 0x7f2930102e00 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.148+0000 7f29267fc640 1 -- 172.21.15.103:0/2783923497 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2920017070 con 0x7f2930102e00 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.149+0000 7f29267fc640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f291c0a32d0 0x7f291c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.149+0000 7f293569f640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f291c0a32d0 0x7f291c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.149+0000 7f29267fc640 1 -- 172.21.15.103:0/2783923497 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f29200ceda0 con 0x7f2930102e00 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.150+0000 7f293569f640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f291c0a32d0 0x7f291c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2930102c60 tx=0x7f292c006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:33.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.153+0000 7f29267fc640 1 -- 172.21.15.103:0/2783923497 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f292009ba70 con 0x7f2930102e00 2024-09-19T07:37:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.330+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2930062630 con 0x7f291c0a32d0 2024-09-19T07:37:33.331 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:33 smithi103 ceph-mon[31201]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:33.331 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:33 smithi103 ceph-mon[31201]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:33.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.337+0000 7f29267fc640 1 -- 172.21.15.103:0/2783923497 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2930062630 con 0x7f291c0a32d0 2024-09-19T07:37:33.338 INFO:teuthology.orchestra.run.smithi103.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T07:37:33.339 INFO:teuthology.orchestra.run.smithi103.stdout:mds.all 12/12 28s ago 2m smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-19T07:37:33.339 INFO:teuthology.orchestra.run.smithi103.stdout:mgr 3/3 28s ago 7m smithi103=x;smithi134=y;smithi174=z;count:3 2024-09-19T07:37:33.339 INFO:teuthology.orchestra.run.smithi103.stdout:mon 3/3 28s ago 7m smithi103:172.21.15.103=a;smithi134:172.21.15.134=b;smithi174:172.21.15.174=c;count:3 2024-09-19T07:37:33.339 INFO:teuthology.orchestra.run.smithi103.stdout:osd 12 28s ago - 2024-09-19T07:37:33.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f291c0a32d0 msgr2=0x7f291c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f291c0a32d0 0x7f291c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2930102c60 tx=0x7f292c006c90 comp rx=0 tx=0).stop 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930102e00 msgr2=0x7f29300745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930102e00 0x7f29300745e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f292000b4a0 tx=0x7f292000b960 comp rx=0 tx=0).stop 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 shutdown_connections 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f291c0a32d0 0x7f291c0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29301037c0 0x7f2930074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2930102e00 0x7f29300745e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 --2- 172.21.15.103:0/2783923497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2930101c00 0x7f2930075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.339+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 >> 172.21.15.103:0/2783923497 conn(0x7f29300fd390 msgr2=0x7f2930106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.340+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 shutdown_connections 2024-09-19T07:37:33.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:33.340+0000 7f2937903640 1 -- 172.21.15.103:0/2783923497 wait complete. 2024-09-19T07:37:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:33 smithi174 ceph-mon[35370]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:33.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:33 smithi174 ceph-mon[35370]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:33 smithi134 ceph-mon[35414]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:33 smithi134 ceph-mon[35414]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:33.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T07:37:34.225 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:34.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 -- 172.21.15.103:0/3721664521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107900 msgr2=0x7f953c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:34.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 --2- 172.21.15.103:0/3721664521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107900 0x7f953c109d50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9534009ca0 tx=0x7f953402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 -- 172.21.15.103:0/3721664521 shutdown_connections 2024-09-19T07:37:34.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 --2- 172.21.15.103:0/3721664521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c107900 0x7f953c109d50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 --2- 172.21.15.103:0/3721664521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f953c104fd0 0x7f953c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 --2- 172.21.15.103:0/3721664521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f953c1026a0 0x7f953c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 -- 172.21.15.103:0/3721664521 >> 172.21.15.103:0/3721664521 conn(0x7f953c0fc2a0 msgr2=0x7f953c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 -- 172.21.15.103:0/3721664521 shutdown_connections 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.736+0000 7f95424a8640 1 -- 172.21.15.103:0/3721664521 wait complete. 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 Processor -- start 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 -- start start 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f953c1026a0 0x7f953c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c104fd0 0x7f953c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f953c107900 0x7f953c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:34.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f953c10d8c0 con 0x7f953c104fd0 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f953c10d740 con 0x7f953c107900 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f95424a8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f953c10da40 con 0x7f953c1026a0 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c104fd0 0x7f953c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c104fd0 0x7f953c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36476/0 (socket says 172.21.15.103:36476) 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 -- 172.21.15.103:0/339193276 learned_addr learned my addr 172.21.15.103:0/339193276 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953bfff640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f953c1026a0 0x7f953c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 -- 172.21.15.103:0/339193276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f953c1026a0 msgr2=0x7f953c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f953c1026a0 0x7f953c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f9540a45640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f953c107900 0x7f953c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 -- 172.21.15.103:0/339193276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f953c107900 msgr2=0x7f953c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:34.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f953c107900 0x7f953c19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.737+0000 7f953b7fe640 1 -- 172.21.15.103:0/339193276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c19fa90 con 0x7f953c104fd0 2024-09-19T07:37:34.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.738+0000 7f953b7fe640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c104fd0 0x7f953c1983a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f953000c970 tx=0x7f953000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:34.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.738+0000 7f95397fa640 1 -- 172.21.15.103:0/339193276 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530007bb0 con 0x7f953c104fd0 2024-09-19T07:37:34.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.738+0000 7f95397fa640 1 -- 172.21.15.103:0/339193276 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530007d50 con 0x7f953c104fd0 2024-09-19T07:37:34.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.738+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953c19fd80 con 0x7f953c104fd0 2024-09-19T07:37:34.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.738+0000 7f95397fa640 1 -- 172.21.15.103:0/339193276 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95300187a0 con 0x7f953c104fd0 2024-09-19T07:37:34.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.738+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953c1a0210 con 0x7f953c104fd0 2024-09-19T07:37:34.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.739+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953c0fdd90 con 0x7f953c104fd0 2024-09-19T07:37:34.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.740+0000 7f95397fa640 1 -- 172.21.15.103:0/339193276 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f953000f450 con 0x7f953c104fd0 2024-09-19T07:37:34.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.740+0000 7f95397fa640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f95240a3300 0x7f95240a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:34.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.741+0000 7f953bfff640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f95240a3300 0x7f95240a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:34.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.741+0000 7f95397fa640 1 -- 172.21.15.103:0/339193276 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f95300cff40 con 0x7f953c104fd0 2024-09-19T07:37:34.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.741+0000 7f953bfff640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f95240a3300 0x7f95240a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f952c005e10 tx=0x7f952c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:34.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.744+0000 7f95397fa640 1 -- 172.21.15.103:0/339193276 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9530014030 con 0x7f953c104fd0 2024-09-19T07:37:34.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.919+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f953c062630 con 0x7f95240a3300 2024-09-19T07:37:34.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.920+0000 7f95397fa640 1 -- 172.21.15.103:0/339193276 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f953c062630 con 0x7f95240a3300 2024-09-19T07:37:34.921 INFO:teuthology.orchestra.run.smithi103.stdout:HOST ADDR LABELS STATUS 2024-09-19T07:37:34.922 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 172.21.15.103 2024-09-19T07:37:34.922 INFO:teuthology.orchestra.run.smithi103.stdout:smithi134 172.21.15.134 2024-09-19T07:37:34.922 INFO:teuthology.orchestra.run.smithi103.stdout:smithi174 172.21.15.174 2024-09-19T07:37:34.922 INFO:teuthology.orchestra.run.smithi103.stdout:3 hosts in cluster 2024-09-19T07:37:34.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.922+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f95240a3300 msgr2=0x7f95240a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:34.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.922+0000 7f95424a8640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f95240a3300 0x7f95240a57c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f952c005e10 tx=0x7f952c005da0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.922+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c104fd0 msgr2=0x7f953c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.922+0000 7f95424a8640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c104fd0 0x7f953c1983a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f953000c970 tx=0x7f953000ce30 comp rx=0 tx=0).stop 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 shutdown_connections 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f95240a3300 0x7f95240a57c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f953c107900 0x7f953c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f953c104fd0 0x7f953c1983a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 --2- 172.21.15.103:0/339193276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f953c1026a0 0x7f953c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 >> 172.21.15.103:0/339193276 conn(0x7f953c0fc2a0 msgr2=0x7f953c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 shutdown_connections 2024-09-19T07:37:34.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:34.923+0000 7f95424a8640 1 -- 172.21.15.103:0/339193276 wait complete. 2024-09-19T07:37:35.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:35 smithi103 ceph-mon[31201]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:35.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:35 smithi103 ceph-mon[31201]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:35.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:35 smithi134 ceph-mon[35414]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:35.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:35 smithi134 ceph-mon[35414]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:35.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T07:37:35.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:35 smithi174 ceph-mon[35370]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:35.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:35 smithi174 ceph-mon[35370]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:35.856 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:36.256 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:36 smithi103 ceph-mon[31201]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:36.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 -- 172.21.15.103:0/466747250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc101d10 msgr2=0x7f04fc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 --2- 172.21.15.103:0/466747250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc101d10 0x7f04fc102190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a410 comp rx=0 tx=0).stop 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 -- 172.21.15.103:0/466747250 shutdown_connections 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 --2- 172.21.15.103:0/466747250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04fc1026d0 0x7f04fc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 --2- 172.21.15.103:0/466747250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc101d10 0x7f04fc102190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 --2- 172.21.15.103:0/466747250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04fc100b10 0x7f04fc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 -- 172.21.15.103:0/466747250 >> 172.21.15.103:0/466747250 conn(0x7f04fc0fc2a0 msgr2=0x7f04fc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 -- 172.21.15.103:0/466747250 shutdown_connections 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.337+0000 7f0501cb9640 1 -- 172.21.15.103:0/466747250 wait complete. 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 Processor -- start 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 -- start start 2024-09-19T07:37:36.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc100b10 0x7f04fc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04fc101d10 0x7f04fc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04fc1026d0 0x7f04fc19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d920 con 0x7f04fc100b10 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10d7a0 con 0x7f04fc101d10 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f0501cb9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc10daa0 con 0x7f04fc1026d0 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc100b10 0x7f04fc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc100b10 0x7f04fc197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36502/0 (socket says 172.21.15.103:36502) 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f04fb7fe640 1 -- 172.21.15.103:0/3669373267 learned_addr learned my addr 172.21.15.103:0/3669373267 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f04faffd640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04fc101d10 0x7f04fc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.338+0000 7f04fbfff640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04fc1026d0 0x7f04fc19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04fb7fe640 1 -- 172.21.15.103:0/3669373267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04fc1026d0 msgr2=0x7f04fc19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:36.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04fb7fe640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04fc1026d0 0x7f04fc19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04fb7fe640 1 -- 172.21.15.103:0/3669373267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04fc101d10 msgr2=0x7f04fc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04fb7fe640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04fc101d10 0x7f04fc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04fb7fe640 1 -- 172.21.15.103:0/3669373267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc19fb70 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04fb7fe640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc100b10 0x7f04fc197f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00b450 tx=0x7f04ec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04f8ff9640 1 -- 172.21.15.103:0/3669373267 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec015020 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04f8ff9640 1 -- 172.21.15.103:0/3669373267 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ec011d70 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc19fe60 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f04f8ff9640 1 -- 172.21.15.103:0/3669373267 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00f340 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.339+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1a02e0 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.340+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc107ee0 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.341+0000 7f04f8ff9640 1 -- 172.21.15.103:0/3669373267 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f04ec018020 con 0x7f04fc100b10 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.342+0000 7f04f8ff9640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04e40a3300 0x7f04e40a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:36.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.342+0000 7f04faffd640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04e40a3300 0x7f04e40a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:36.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.342+0000 7f04f8ff9640 1 -- 172.21.15.103:0/3669373267 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f04ec0cf380 con 0x7f04fc100b10 2024-09-19T07:37:36.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.342+0000 7f04faffd640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04e40a3300 0x7f04e40a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04f00314e0 tx=0x7f04f0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:36.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.346+0000 7f04f8ff9640 1 -- 172.21.15.103:0/3669373267 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f04ec01f020 con 0x7f04fc100b10 2024-09-19T07:37:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:36 smithi174 ceph-mon[35370]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:36.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.521+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 --> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f04fc062630 con 0x7f04e40a3300 2024-09-19T07:37:36.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.523+0000 7f04f8ff9640 1 -- 172.21.15.103:0/3669373267 <== mgr.14150 v2:172.21.15.103:6800/2164654927 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f04fc062630 con 0x7f04e40a3300 2024-09-19T07:37:36.524 INFO:teuthology.orchestra.run.smithi103.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T07:37:36.524 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T07:37:36.524 INFO:teuthology.orchestra.run.smithi103.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T07:37:36.524 INFO:teuthology.orchestra.run.smithi103.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T07:37:36.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04e40a3300 msgr2=0x7f04e40a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04e40a3300 0x7f04e40a57c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04f00314e0 tx=0x7f04f0000bc0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc100b10 msgr2=0x7f04fc197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc100b10 0x7f04fc197f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00b450 tx=0x7f04ec00b910 comp rx=0 tx=0).stop 2024-09-19T07:37:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 shutdown_connections 2024-09-19T07:37:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f04e40a3300 0x7f04e40a57c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04fc1026d0 0x7f04fc19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04fc101d10 0x7f04fc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 --2- 172.21.15.103:0/3669373267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04fc100b10 0x7f04fc197f40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 >> 172.21.15.103:0/3669373267 conn(0x7f04fc0fc2a0 msgr2=0x7f04fc1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 shutdown_connections 2024-09-19T07:37:36.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:36.525+0000 7f0501cb9640 1 -- 172.21.15.103:0/3669373267 wait complete. 2024-09-19T07:37:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:36 smithi134 ceph-mon[35414]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:37.176 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T07:37:37.189 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:37:37.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T07:37:37.438 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:37.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:37 smithi103 ceph-mon[31201]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:37.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:37 smithi134 ceph-mon[35414]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:37.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:37 smithi174 ceph-mon[35370]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 -- 172.21.15.103:0/2862520798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 msgr2=0x7faa24100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 --2- 172.21.15.103:0/2862520798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24100f10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faa18009900 tx=0x7faa1802a410 comp rx=0 tx=0).stop 2024-09-19T07:37:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 -- 172.21.15.103:0/2862520798 shutdown_connections 2024-09-19T07:37:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 --2- 172.21.15.103:0/2862520798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa241026d0 0x7faa24108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 --2- 172.21.15.103:0/2862520798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa24101d10 0x7faa24102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 --2- 172.21.15.103:0/2862520798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24100f10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 -- 172.21.15.103:0/2862520798 >> 172.21.15.103:0/2862520798 conn(0x7faa240fc2a0 msgr2=0x7faa240fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 -- 172.21.15.103:0/2862520798 shutdown_connections 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.886+0000 7faa2abf2640 1 -- 172.21.15.103:0/2862520798 wait complete. 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 Processor -- start 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 -- start start 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa24101d10 0x7faa241984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa241026d0 0x7faa2419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa2410d900 con 0x7faa24100b10 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa2410d780 con 0x7faa24101d10 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2abf2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa2410da80 con 0x7faa241026d0 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36520/0 (socket says 172.21.15.103:36520) 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 -- 172.21.15.103:0/1183510630 learned_addr learned my addr 172.21.15.103:0/1183510630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 -- 172.21.15.103:0/1183510630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa241026d0 msgr2=0x7faa2419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2918f640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa241026d0 0x7faa2419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa241026d0 0x7faa2419f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 -- 172.21.15.103:0/1183510630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa24101d10 msgr2=0x7faa241984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa24101d10 0x7faa241984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 -- 172.21.15.103:0/1183510630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2419fbb0 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.887+0000 7faa2898e640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24197f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faa18031610 tx=0x7faa180043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.888+0000 7faa21ffb640 1 -- 172.21.15.103:0/1183510630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1803f030 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.888+0000 7faa21ffb640 1 -- 172.21.15.103:0/1183510630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa18003ce0 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.888+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2419fe40 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.888+0000 7faa21ffb640 1 -- 172.21.15.103:0/1183510630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18035070 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.888+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa241a0310 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.889+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa24107f10 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.890+0000 7faa21ffb640 1 -- 172.21.15.103:0/1183510630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7faa18042030 con 0x7faa24100b10 2024-09-19T07:37:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.890+0000 7faa21ffb640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa140a3300 0x7faa140a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.890+0000 7faa23fff640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa140a3300 0x7faa140a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.890+0000 7faa21ffb640 1 -- 172.21.15.103:0/1183510630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7faa180f01e0 con 0x7faa24100b10 2024-09-19T07:37:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.891+0000 7faa23fff640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa140a3300 0x7faa140a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa241994a0 tx=0x7faa10005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:37.894+0000 7faa21ffb640 1 -- 172.21.15.103:0/1183510630 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faa180bcda0 con 0x7faa24100b10 2024-09-19T07:37:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.099+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7faa2402d9f0 con 0x7faa24100b10 2024-09-19T07:37:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.099+0000 7faa21ffb640 1 -- 172.21.15.103:0/1183510630 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3140 (secure 0 0 0) 0x7faa180bc6a0 con 0x7faa24100b10 2024-09-19T07:37:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:e26 2024-09-19T07:37:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:btime 2024-09-19T07:36:19:905223+0000 2024-09-19T07:37:38.100 INFO:teuthology.orchestra.run.smithi103.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:legacy client fscid: 1 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:Filesystem 'cephfs' (1) 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:fs_name cephfs 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 26 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-19T07:36:08.395229+0000 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-19T07:36:19.905198+0000 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:tableserver 0 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:root 0 2024-09-19T07:37:38.101 INFO:teuthology.orchestra.run.smithi103.stdout:session_timeout 300 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:session_autoclose 300 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:max_file_size 1099511627776 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:max_xattr_size 65536 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:required_client_features {} 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure 0 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure_osd_epoch 0 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:max_mds 5 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:in 0,1,2,3,4 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:up {0=24494,1=24466,2=24455,3=24454,4=24448} 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:failed 2024-09-19T07:37:38.102 INFO:teuthology.orchestra.run.smithi103.stdout:damaged 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:stopped 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:data_pools [3] 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:metadata_pool 2 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:inline_data disabled 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:balancer 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:bal_rank_mask -1 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:standby_count_wanted 0 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:qdb_cluster leader: 24494 members: 24466,24494,24455,24454,24448 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b{0:24494} state up:active seq 4 addr [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.e{0:24431} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/19326296,v1:172.21.15.134:6835/19326296] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.103 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.i{1:24466} state up:active seq 8 addr [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.l{1:24424} state up:standby-replay seq 1 addr [v2:172.21.15.174:6832/1885230172,v1:172.21.15.174:6833/1885230172] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.h{2:24455} state up:active seq 9 addr [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.k{2:24410} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/2440407987,v1:172.21.15.134:6833/2440407987] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c{3:24454} state up:active seq 10 addr [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a{3:14580} state up:standby-replay seq 1 addr [v2:172.21.15.103:6840/1521905484,v1:172.21.15.103:6841/1521905484] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.f{4:24448} state up:active seq 11 addr [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.104 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.g{4:14568} state up:standby-replay seq 1 addr [v2:172.21.15.103:6838/1660338093,v1:172.21.15.103:6839/1660338093] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.105 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:38.105 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:Standby daemons: 2024-09-19T07:37:38.105 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.j{-1:14520} state up:standby seq 1 addr [v2:172.21.15.103:6834/2425532131,v1:172.21.15.103:6835/2425532131] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.105 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d{-1:14532} state up:standby seq 1 addr [v2:172.21.15.103:6836/1711280456,v1:172.21.15.103:6837/1711280456] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T07:37:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T07:37:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa140a3300 msgr2=0x7faa140a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa140a3300 0x7faa140a57c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa241994a0 tx=0x7faa10005bf0 comp rx=0 tx=0).stop 2024-09-19T07:37:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 msgr2=0x7faa24197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24197f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faa18031610 tx=0x7faa180043b0 comp rx=0 tx=0).stop 2024-09-19T07:37:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 shutdown_connections 2024-09-19T07:37:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7faa140a3300 0x7faa140a57c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:38.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa241026d0 0x7faa2419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:38.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa24101d10 0x7faa241984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:38.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 --2- 172.21.15.103:0/1183510630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa24100b10 0x7faa24197f80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:38.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 >> 172.21.15.103:0/1183510630 conn(0x7faa240fc2a0 msgr2=0x7faa241050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:38.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 shutdown_connections 2024-09-19T07:37:38.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:38.102+0000 7faa2abf2640 1 -- 172.21.15.103:0/1183510630 wait complete. 2024-09-19T07:37:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:38 smithi103 ceph-mon[31201]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:38 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1183510630' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T07:37:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:38 smithi134 ceph-mon[35414]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:38 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1183510630' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T07:37:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:38 smithi174 ceph-mon[35370]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:37:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:38 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1183510630' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T07:37:38.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T07:37:38.992 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:39.437 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:39 smithi103 ceph-mon[31201]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 -- 172.21.15.103:0/671140413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 msgr2=0x7fbac4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 --2- 172.21.15.103:0/671140413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac4108f60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009ca0 tx=0x7fbabc02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 -- 172.21.15.103:0/671140413 shutdown_connections 2024-09-19T07:37:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 --2- 172.21.15.103:0/671140413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac4108f60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 --2- 172.21.15.103:0/671140413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac4101d10 0x7fbac4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 --2- 172.21.15.103:0/671140413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac4100b10 0x7fbac4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 -- 172.21.15.103:0/671140413 >> 172.21.15.103:0/671140413 conn(0x7fbac40fc2a0 msgr2=0x7fbac40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 -- 172.21.15.103:0/671140413 shutdown_connections 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.436+0000 7fbaca795640 1 -- 172.21.15.103:0/671140413 wait complete. 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 Processor -- start 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 -- start start 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac4100b10 0x7fbac4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac4101d10 0x7fbac41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbac410d8c0 con 0x7fbac41026d0 2024-09-19T07:37:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbac410d740 con 0x7fbac4101d10 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbaca795640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbac410da40 con 0x7fbac4100b10 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbac8d32640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbac8d32640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac419f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58664/0 (socket says 172.21.15.103:58664) 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbac8d32640 1 -- 172.21.15.103:0/1056050903 learned_addr learned my addr 172.21.15.103:0/1056050903 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbac3fff640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac4100b10 0x7fbac4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.437+0000 7fbac37fe640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac4101d10 0x7fbac41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac8d32640 1 -- 172.21.15.103:0/1056050903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac4100b10 msgr2=0x7fbac4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac8d32640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac4100b10 0x7fbac4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac8d32640 1 -- 172.21.15.103:0/1056050903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac4101d10 msgr2=0x7fbac41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac8d32640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac4101d10 0x7fbac41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac8d32640 1 -- 172.21.15.103:0/1056050903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac419faa0 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac8d32640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac419f430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0098d0 tx=0x7fbabc02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac17fa640 1 -- 172.21.15.103:0/1056050903 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc02fc30 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac419fd30 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac17fa640 1 -- 172.21.15.103:0/1056050903 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbabc003e50 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbac17fa640 1 -- 172.21.15.103:0/1056050903 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0305c0 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.438+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac41a0210 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.439+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac4107ec0 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.440+0000 7fbac17fa640 1 -- 172.21.15.103:0/1056050903 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbabc004b40 con 0x7fbac41026d0 2024-09-19T07:37:39.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.441+0000 7fbac17fa640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbaac0a3300 0x7fbaac0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.441+0000 7fbac3fff640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbaac0a3300 0x7fbaac0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.441+0000 7fbac17fa640 1 -- 172.21.15.103:0/1056050903 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fbabc0f0e20 con 0x7fbac41026d0 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.441+0000 7fbac3fff640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbaac0a3300 0x7fbaac0a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbac4101b70 tx=0x7fbab400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:39.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.444+0000 7fbac17fa640 1 -- 172.21.15.103:0/1056050903 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbabc0bdab0 con 0x7fbac41026d0 2024-09-19T07:37:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:39 smithi134 ceph-mon[35414]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:39.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.613+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fbac4062e90 con 0x7fbac41026d0 2024-09-19T07:37:39.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.614+0000 7fbac17fa640 1 -- 172.21.15.103:0/1056050903 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5332 (secure 0 0 0) 0x7fbabc0bd3b0 con 0x7fbac41026d0 2024-09-19T07:37:39.615 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 84 2024-09-19T07:37:39.615 INFO:teuthology.orchestra.run.smithi103.stdout:fsid 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:37:39.615 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-19T07:28:30.723617+0000 2024-09-19T07:37:39.615 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-19T07:36:17.882336+0000 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:crush_version 26 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:full_ratio 0.95 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:backfillfull_ratio 0.9 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:nearfull_ratio 0.85 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:require_min_compat_client luminous 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:min_compat_client luminous 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:require_osd_release squid 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:stretch_mode_enabled false 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T07:37:39.616 INFO:teuthology.orchestra.run.smithi103.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:max_osd 12 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6802/81836251,v1:172.21.15.103:6803/81836251] [v2:172.21.15.103:6804/81836251,v1:172.21.15.103:6805/81836251] exists,up d25f6411-3656-482d-bd89-a94dd43726b9 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/2492227363,v1:172.21.15.134:6801/2492227363] [v2:172.21.15.134:6802/2492227363,v1:172.21.15.134:6803/2492227363] exists,up f51a85b8-e79c-496f-a662-59b58fdb0f58 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/1513938376,v1:172.21.15.174:6801/1513938376] [v2:172.21.15.174:6802/1513938376,v1:172.21.15.174:6803/1513938376] exists,up f82ba336-4618-4e9c-825e-d51abe0ab8c5 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6810/2879777863,v1:172.21.15.103:6811/2879777863] [v2:172.21.15.103:6812/2879777863,v1:172.21.15.103:6813/2879777863] exists,up 0b1b00f7-68c9-4888-81f7-2f2544a97c0d 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 up in weight 1 up_from 33 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/2364007931,v1:172.21.15.134:6809/2364007931] [v2:172.21.15.134:6810/2364007931,v1:172.21.15.134:6811/2364007931] exists,up 2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 up in weight 1 up_from 38 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/2281357008,v1:172.21.15.174:6809/2281357008] [v2:172.21.15.174:6810/2281357008,v1:172.21.15.174:6811/2281357008] exists,up 14cf312f-9b50-4214-8f5d-2b4d68af438f 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 up in weight 1 up_from 45 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6818/3692913756,v1:172.21.15.103:6819/3692913756] [v2:172.21.15.103:6820/3692913756,v1:172.21.15.103:6821/3692913756] exists,up eaf8bd47-9f95-4e26-b8e0-387cb89c1de3 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 up in weight 1 up_from 50 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/2397823373,v1:172.21.15.134:6817/2397823373] [v2:172.21.15.134:6818/2397823373,v1:172.21.15.134:6819/2397823373] exists,up c5ac551b-6bfa-45bb-a12c-7684b5d30079 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 up in weight 1 up_from 57 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/477602146,v1:172.21.15.174:6817/477602146] [v2:172.21.15.174:6818/477602146,v1:172.21.15.174:6819/477602146] exists,up 3aa078e4-5626-4b10-8727-a060b52ffb58 2024-09-19T07:37:39.617 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 up in weight 1 up_from 64 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6826/1024309893,v1:172.21.15.103:6827/1024309893] [v2:172.21.15.103:6828/1024309893,v1:172.21.15.103:6829/1024309893] exists,up e9f63e40-aa84-48be-b99b-2f16545d058f 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 up in weight 1 up_from 69 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/984759613,v1:172.21.15.134:6825/984759613] [v2:172.21.15.134:6826/984759613,v1:172.21.15.134:6827/984759613] exists,up 60776c91-f34f-499f-825d-43691743cd2d 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 up in weight 1 up_from 76 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/1686831932,v1:172.21.15.174:6825/1686831932] [v2:172.21.15.174:6826/1686831932,v1:172.21.15.174:6827/1686831932] exists,up d731972a-f6ca-4e9d-b1a7-9b1b82f0e381 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.a [2,11] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.e [2,4] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.10 [8,11] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.16 [2,4] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.1f [8,1] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.1d [5,9] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.2c [4,9] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.3a [4,8] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.3c [4,9] 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/2002728097 expires 2024-09-20T07:29:16.000257+0000 2024-09-19T07:37:39.618 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1595075196 expires 2024-09-20T07:29:16.000257+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1436062068 expires 2024-09-20T07:29:16.000257+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/315623738 expires 2024-09-20T07:29:16.000257+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/891568998 expires 2024-09-20T07:28:52.493301+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/2002728097 expires 2024-09-20T07:29:16.000257+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3372836762 expires 2024-09-20T07:28:52.493301+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/1746275589 expires 2024-09-20T07:28:52.493301+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/1746275589 expires 2024-09-20T07:28:52.493301+0000 2024-09-19T07:37:39.619 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3706845955 expires 2024-09-20T07:28:52.493301+0000 2024-09-19T07:37:39.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbaac0a3300 msgr2=0x7fbaac0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:39.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbaac0a3300 0x7fbaac0a57c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbac4101b70 tx=0x7fbab400a2d0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 msgr2=0x7fbac419f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac419f430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0098d0 tx=0x7fbabc02f840 comp rx=0 tx=0).stop 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 shutdown_connections 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbaac0a3300 0x7fbaac0a57c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbac41026d0 0x7fbac419f430 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac4101d10 0x7fbac41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.616+0000 7fbaca795640 1 --2- 172.21.15.103:0/1056050903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac4100b10 0x7fbac4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.617+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 >> 172.21.15.103:0/1056050903 conn(0x7fbac40fc2a0 msgr2=0x7fbac4105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.617+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 shutdown_connections 2024-09-19T07:37:39.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:39.617+0000 7fbaca795640 1 -- 172.21.15.103:0/1056050903 wait complete. 2024-09-19T07:37:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:39 smithi174 ceph-mon[35370]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-19T07:37:40.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:40 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1056050903' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T07:37:40.405 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T07:37:40.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:40 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1056050903' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T07:37:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:40 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1056050903' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T07:37:40.675 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/4275994025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 msgr2=0x7fa5e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/4275994025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e01029b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009900 tx=0x7fa5d402a480 comp rx=0 tx=0).stop 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/4275994025 shutdown_connections 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/4275994025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0102ef0 0x7fa5e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/4275994025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e01029b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/4275994025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e0100e20 0x7fa5e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/4275994025 >> 172.21.15.103:0/4275994025 conn(0x7fa5e00fc7d0 msgr2=0x7fa5e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/4275994025 shutdown_connections 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.150+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/4275994025 wait complete. 2024-09-19T07:37:41.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 Processor -- start 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 -- start start 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0100e20 0x7fa5e0197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e0102ef0 0x7fa5e019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0110f80 con 0x7fa5e0102570 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0110e00 con 0x7fa5e0100e20 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.151+0000 7fa5e4d3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0111100 con 0x7fa5e0102ef0 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:41.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e0197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58686/0 (socket says 172.21.15.103:58686) 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 -- 172.21.15.103:0/1974223351 learned_addr learned my addr 172.21.15.103:0/1974223351 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ded9d640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e0102ef0 0x7fa5e019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5de59c640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0100e20 0x7fa5e0197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 -- 172.21.15.103:0/1974223351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e0102ef0 msgr2=0x7fa5e019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e0102ef0 0x7fa5e019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 -- 172.21.15.103:0/1974223351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0100e20 msgr2=0x7fa5e0197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0100e20 0x7fa5e0197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 -- 172.21.15.103:0/1974223351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e019c3d0 con 0x7fa5e0102570 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5ddd9b640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e0197960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006010 tx=0x7fa5d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5cf7fe640 1 -- 172.21.15.103:0/1974223351 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d403f070 con 0x7fa5e0102570 2024-09-19T07:37:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5cf7fe640 1 -- 172.21.15.103:0/1974223351 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d4031c30 con 0x7fa5e0102570 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e019c660 con 0x7fa5e0102570 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5cf7fe640 1 -- 172.21.15.103:0/1974223351 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4035030 con 0x7fa5e0102570 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.152+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01a3fa0 con 0x7fa5e0102570 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.153+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e01021b0 con 0x7fa5e0102570 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.155+0000 7fa5cf7fe640 1 -- 172.21.15.103:0/1974223351 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa5d4042070 con 0x7fa5e0102570 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.155+0000 7fa5cf7fe640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa5c80a3350 0x7fa5c80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.156+0000 7fa5de59c640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa5c80a3350 0x7fa5c80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.156+0000 7fa5cf7fe640 1 -- 172.21.15.103:0/1974223351 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fa5d40f06f0 con 0x7fa5e0102570 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.156+0000 7fa5de59c640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa5c80a3350 0x7fa5c80a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8005fd0 tx=0x7fa5d8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:41.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.159+0000 7fa5cf7fe640 1 -- 172.21.15.103:0/1974223351 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5d4031de0 con 0x7fa5e0102570 2024-09-19T07:37:41.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.361+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e0109090 con 0x7fa5e0102570 2024-09-19T07:37:41.363 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:41 smithi103 ceph-mon[31201]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:41.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.362+0000 7fa5cf7fe640 1 -- 172.21.15.103:0/1974223351 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11991 (secure 0 0 0) 0x7fa5d40bd370 con 0x7fa5e0102570 2024-09-19T07:37:41.364 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:41.365 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T07:36:19:905223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:36:19.905198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:37:41.367 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T07:37:41.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa5c80a3350 msgr2=0x7fa5c80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:41.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa5c80a3350 0x7fa5c80a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8005fd0 tx=0x7fa5d8005850 comp rx=0 tx=0).stop 2024-09-19T07:37:41.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 msgr2=0x7fa5e0197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:41.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e0197960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006010 tx=0x7fa5d4003c10 comp rx=0 tx=0).stop 2024-09-19T07:37:41.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 shutdown_connections 2024-09-19T07:37:41.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fa5c80a3350 0x7fa5c80a5810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e0102ef0 0x7fa5e019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e0102570 0x7fa5e0197960 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 --2- 172.21.15.103:0/1974223351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e0100e20 0x7fa5e0197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:41.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 >> 172.21.15.103:0/1974223351 conn(0x7fa5e00fc7d0 msgr2=0x7fa5e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 shutdown_connections 2024-09-19T07:37:41.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:41.365+0000 7fa5e4d3a640 1 -- 172.21.15.103:0/1974223351 wait complete. 2024-09-19T07:37:41.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:41 smithi134 ceph-mon[35414]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:41.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:41 smithi174 ceph-mon[35370]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-19T07:37:42.009 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:42.260 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:42.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:42 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1974223351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:42 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1974223351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:42 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1974223351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 -- 172.21.15.103:0/1833378132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534102790 msgr2=0x7fe5341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 --2- 172.21.15.103:0/1833378132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534102790 0x7fe5341067e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe524009900 tx=0x7fe52402a410 comp rx=0 tx=0).stop 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 -- 172.21.15.103:0/1833378132 shutdown_connections 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 --2- 172.21.15.103:0/1833378132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe534106f10 0x7fe53410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 --2- 172.21.15.103:0/1833378132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534102790 0x7fe5341067e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 --2- 172.21.15.103:0/1833378132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe534101de0 0x7fe5341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 -- 172.21.15.103:0/1833378132 >> 172.21.15.103:0/1833378132 conn(0x7fe5340fd8c0 msgr2=0x7fe5340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 -- 172.21.15.103:0/1833378132 shutdown_connections 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.725+0000 7fe53c217640 1 -- 172.21.15.103:0/1833378132 wait complete. 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 Processor -- start 2024-09-19T07:37:42.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 -- start start 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe534101de0 0x7fe5340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe534102790 0x7fe534074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534106f10 0x7fe534074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe534111f70 con 0x7fe534106f10 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe534111df0 con 0x7fe534101de0 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53c217640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341120f0 con 0x7fe534102790 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534106f10 0x7fe534074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534106f10 0x7fe534074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58714/0 (socket says 172.21.15.103:58714) 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 -- 172.21.15.103:0/1195533455 learned_addr learned my addr 172.21.15.103:0/1195533455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe5397b2640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe534102790 0x7fe534074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe539fb3640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe534101de0 0x7fe5340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 -- 172.21.15.103:0/1195533455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe534102790 msgr2=0x7fe534074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe534102790 0x7fe534074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 -- 172.21.15.103:0/1195533455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe534101de0 msgr2=0x7fe5340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe534101de0 0x7fe5340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe53a7b4640 1 -- 172.21.15.103:0/1195533455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341a6890 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.726+0000 7fe539fb3640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe534101de0 0x7fe5340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.727+0000 7fe53a7b4640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534106f10 0x7fe534074f70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00cce0 tx=0x7fe52c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.727+0000 7fe52affd640 1 -- 172.21.15.103:0/1195533455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00f450 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.727+0000 7fe52affd640 1 -- 172.21.15.103:0/1195533455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52c00fad0 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.727+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341a6b80 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.727+0000 7fe52affd640 1 -- 172.21.15.103:0/1195533455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c018c10 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.727+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341a7040 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.728+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5341021c0 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.730+0000 7fe52affd640 1 -- 172.21.15.103:0/1195533455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe52c00f5f0 con 0x7fe534106f10 2024-09-19T07:37:42.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.730+0000 7fe52affd640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5200a32d0 0x7fe5200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:42.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.731+0000 7fe539fb3640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5200a32d0 0x7fe5200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:42.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.731+0000 7fe52affd640 1 -- 172.21.15.103:0/1195533455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe52c0d0160 con 0x7fe534106f10 2024-09-19T07:37:42.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.731+0000 7fe539fb3640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5200a32d0 0x7fe5200a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe5300058c0 tx=0x7fe530005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:42.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.735+0000 7fe52affd640 1 -- 172.21.15.103:0/1195533455 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe52c014030 con 0x7fe534106f10 2024-09-19T07:37:42.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.937+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe53402d7c0 con 0x7fe534106f10 2024-09-19T07:37:42.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.938+0000 7fe52affd640 1 -- 172.21.15.103:0/1195533455 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11991 (secure 0 0 0) 0x7fe52c09cd70 con 0x7fe534106f10 2024-09-19T07:37:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:42.938 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-19T07:36:19:905223+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:36:19.905198+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:37:42.939 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-19T07:37:42.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5200a32d0 msgr2=0x7fe5200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5200a32d0 0x7fe5200a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe5300058c0 tx=0x7fe530005850 comp rx=0 tx=0).stop 2024-09-19T07:37:42.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534106f10 msgr2=0x7fe534074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:42.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534106f10 0x7fe534074f70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00cce0 tx=0x7fe52c00a620 comp rx=0 tx=0).stop 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 shutdown_connections 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe5200a32d0 0x7fe5200a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe534106f10 0x7fe534074f70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe534102790 0x7fe534074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 --2- 172.21.15.103:0/1195533455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe534101de0 0x7fe5340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.940+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 >> 172.21.15.103:0/1195533455 conn(0x7fe5340fd8c0 msgr2=0x7fe5340fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.941+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 shutdown_connections 2024-09-19T07:37:42.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:42.941+0000 7fe53c217640 1 -- 172.21.15.103:0/1195533455 wait complete. 2024-09-19T07:37:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:43 smithi103 ceph-mon[31201]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:43.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:43 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1195533455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:43.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:43 smithi134 ceph-mon[35414]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:43.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:43 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1195533455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:43.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:43 smithi174 ceph-mon[35370]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:43.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:43 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1195533455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:37:43.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:44.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 -- 172.21.15.103:0/774443250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 msgr2=0x7fbb8410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:44.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/774443250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb8410ea90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009ca0 tx=0x7fbb7c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 -- 172.21.15.103:0/774443250 shutdown_connections 2024-09-19T07:37:44.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/774443250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb8410ea90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/774443250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb84102790 0x7fbb841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/774443250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb84101de0 0x7fbb841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 -- 172.21.15.103:0/774443250 >> 172.21.15.103:0/774443250 conn(0x7fbb840fd8e0 msgr2=0x7fbb840ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 -- 172.21.15.103:0/774443250 shutdown_connections 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.459+0000 7fbb89d2b640 1 -- 172.21.15.103:0/774443250 wait complete. 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 Processor -- start 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 -- start start 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb84101de0 0x7fbb840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:44.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb84102790 0x7fbb84074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb84074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84111fb0 con 0x7fbb84106f10 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84111e30 con 0x7fbb84102790 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb89d2b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112130 con 0x7fbb84101de0 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb84074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb84074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58740/0 (socket says 172.21.15.103:58740) 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb83fff640 1 -- 172.21.15.103:0/1587581448 learned_addr learned my addr 172.21.15.103:0/1587581448 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:44.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.460+0000 7fbb837fe640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb84101de0 0x7fbb840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb7bfff640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb84102790 0x7fbb84074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb83fff640 1 -- 172.21.15.103:0/1587581448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb84101de0 msgr2=0x7fbb840744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb83fff640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb84101de0 0x7fbb840744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb83fff640 1 -- 172.21.15.103:0/1587581448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb84102790 msgr2=0x7fbb84074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb83fff640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb84102790 0x7fbb84074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb83fff640 1 -- 172.21.15.103:0/1587581448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb841a6920 con 0x7fbb84106f10 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb7bfff640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb84102790 0x7fbb84074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb83fff640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb84074f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c031a20 tx=0x7fbb7c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb817fa640 1 -- 172.21.15.103:0/1587581448 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c003c30 con 0x7fbb84106f10 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb841a6bb0 con 0x7fbb84106f10 2024-09-19T07:37:44.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb817fa640 1 -- 172.21.15.103:0/1587581448 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c003dd0 con 0x7fbb84106f10 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb817fa640 1 -- 172.21.15.103:0/1587581448 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c02fcf0 con 0x7fbb84106f10 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.461+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb841a70f0 con 0x7fbb84106f10 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.462+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb841021c0 con 0x7fbb84106f10 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.463+0000 7fbb817fa640 1 -- 172.21.15.103:0/1587581448 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbb7c004b40 con 0x7fbb84106f10 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.464+0000 7fbb817fa640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbb700a32d0 0x7fbb700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.464+0000 7fbb837fe640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbb700a32d0 0x7fbb700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.464+0000 7fbb817fa640 1 -- 172.21.15.103:0/1587581448 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fbb7c0f0e50 con 0x7fbb84106f10 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.464+0000 7fbb837fe640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbb700a32d0 0x7fbb700a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:44.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.467+0000 7fbb817fa640 1 -- 172.21.15.103:0/1587581448 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb7c0bdbb0 con 0x7fbb84106f10 2024-09-19T07:37:44.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.634+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb84075a20 con 0x7fbb84106f10 2024-09-19T07:37:44.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.636+0000 7fbb817fa640 1 -- 172.21.15.103:0/1587581448 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22818 (secure 0 0 0) 0x7fbb84075a20 con 0x7fbb84106f10 2024-09-19T07:37:44.636 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:44.636 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:17.882336+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:37:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbb700a32d0 msgr2=0x7fbb700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbb700a32d0 0x7fbb700a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb7400b390 comp rx=0 tx=0).stop 2024-09-19T07:37:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 msgr2=0x7fbb84074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb84074f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c031a20 tx=0x7fbb7c031a50 comp rx=0 tx=0).stop 2024-09-19T07:37:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 shutdown_connections 2024-09-19T07:37:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fbb700a32d0 0x7fbb700a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb84106f10 0x7fbb84074f70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb84102790 0x7fbb84074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 --2- 172.21.15.103:0/1587581448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb84101de0 0x7fbb840744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 >> 172.21.15.103:0/1587581448 conn(0x7fbb840fd8e0 msgr2=0x7fbb840fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 shutdown_connections 2024-09-19T07:37:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:44.639+0000 7fbb89d2b640 1 -- 172.21.15.103:0/1587581448 wait complete. 2024-09-19T07:37:45.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T07:36:08.395229+0000', 'modified': '2024-09-19T07:36:19.905198+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24494, 'mds_1': 24466, 'mds_2': 24455, 'mds_3': 24454, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14568': {'gid': 14568, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/1660338093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1660338093}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1660338093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14580': {'gid': 14580, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/1521905484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 1521905484}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 1521905484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2440407987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2440407987}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2440407987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/1885230172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 1885230172}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 1885230172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/19326296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 19326296}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 19326296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6835/214062550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 214062550}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 214062550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/2527370860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2527370860}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2527370860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6837/3959604171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3959604171}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3959604171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/2514182394', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2514182394}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2514182394}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/3683561216', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3683561216}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3683561216}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24466, 24494, 24455, 24454, 24448]} 2024-09-19T07:37:45.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T07:37:45.264 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T07:37:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:45 smithi103 ceph-mon[31201]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:45.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:45 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1587581448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:45.511 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:45 smithi134 ceph-mon[35414]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:45 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1587581448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:45.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:45 smithi174 ceph-mon[35370]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-09-19T07:37:45.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:45 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1587581448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 -- 172.21.15.103:0/106576659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988102790 msgr2=0x7f99881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 --2- 172.21.15.103:0/106576659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988102790 0x7f99881067e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f997c009900 tx=0x7f997c02a410 comp rx=0 tx=0).stop 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 -- 172.21.15.103:0/106576659 shutdown_connections 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 --2- 172.21.15.103:0/106576659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9988106f10 0x7f998810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 --2- 172.21.15.103:0/106576659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988102790 0x7f99881067e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 --2- 172.21.15.103:0/106576659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9988101de0 0x7f99881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 -- 172.21.15.103:0/106576659 >> 172.21.15.103:0/106576659 conn(0x7f99880fd8c0 msgr2=0x7f99880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 -- 172.21.15.103:0/106576659 shutdown_connections 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.011+0000 7f998e9af640 1 -- 172.21.15.103:0/106576659 wait complete. 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 Processor -- start 2024-09-19T07:37:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 -- start start 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9988101de0 0x7f99880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9988102790 0x7f9988074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988106f10 0x7f9988074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9988111f70 con 0x7f9988106f10 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9988111df0 con 0x7f9988101de0 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998e9af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99881120f0 con 0x7f9988102790 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998cf4c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988106f10 0x7f9988074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998cf4c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988106f10 0x7f9988074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58750/0 (socket says 172.21.15.103:58750) 2024-09-19T07:37:46.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.012+0000 7f998cf4c640 1 -- 172.21.15.103:0/394472726 learned_addr learned my addr 172.21.15.103:0/394472726 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f9987fff640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9988101de0 0x7f99880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f99877fe640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9988102790 0x7f9988074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998cf4c640 1 -- 172.21.15.103:0/394472726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9988102790 msgr2=0x7f9988074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998cf4c640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9988102790 0x7f9988074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998cf4c640 1 -- 172.21.15.103:0/394472726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9988101de0 msgr2=0x7f99880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998cf4c640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9988101de0 0x7f99880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998cf4c640 1 -- 172.21.15.103:0/394472726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99881a68f0 con 0x7f9988106f10 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998cf4c640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988106f10 0x7f9988074f70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f998000eaa0 tx=0x7f998000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f99857fa640 1 -- 172.21.15.103:0/394472726 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998000cdb0 con 0x7f9988106f10 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f99857fa640 1 -- 172.21.15.103:0/394472726 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9980005020 con 0x7f9988106f10 2024-09-19T07:37:46.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99881a6be0 con 0x7f9988106f10 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f99857fa640 1 -- 172.21.15.103:0/394472726 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800116b0 con 0x7f9988106f10 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.013+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881a7120 con 0x7f9988106f10 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.015+0000 7f99857fa640 1 -- 172.21.15.103:0/394472726 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9980005560 con 0x7f9988106f10 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 Processor -- start 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 -- start start 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d302cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99857fa640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99700a32d0 0x7f99700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d3037670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2f93090 con 0x55c4d302c9d0 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2f3aa70 con 0x55c4d302d2f0 2024-09-19T07:37:46.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9990d88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2f8f050 con 0x55c4d3037bb0 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d302cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d302cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58766/0 (socket says 172.21.15.103:58766) 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 -- 172.21.15.103:0/1970887694 learned_addr learned my addr 172.21.15.103:0/1970887694 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f9987fff640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99700a32d0 0x7f99700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99857fa640 1 -- 172.21.15.103:0/394472726 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f99800cfc80 con 0x7f9988106f10 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f998cf4c640 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 -- 172.21.15.103:0/1970887694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 msgr2=0x55c4d3039fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3039fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 -- 172.21.15.103:0/1970887694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 msgr2=0x55c4d3037670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d3037670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 -- 172.21.15.103:0/1970887694 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d2f47130 con 0x55c4d302c9d0 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.016+0000 7f99877fe640 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d302cdb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f997c005940 tx=0x7f997c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0965992fb4edfd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9987fff640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99700a32d0 0x7f99700a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f997800bbe0 tx=0x7f997800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f996e7fc640 1 -- 172.21.15.103:0/1970887694 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c035050 con 0x55c4d302c9d0 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f996e7fc640 1 -- 172.21.15.103:0/1970887694 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c004030 con 0x55c4d302c9d0 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f996e7fc640 1 -- 172.21.15.103:0/1970887694 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c005640 con 0x55c4d302c9d0 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 -- 172.21.15.103:0/1970887694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 msgr2=0x55c4d302cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d302cdb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f997c005940 tx=0x7f997c005e70 comp rx=0 tx=0).stop 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 -- 172.21.15.103:0/1970887694 shutdown_connections 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3039fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d3037670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 --2- 172.21.15.103:0/1970887694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d302cdb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 -- 172.21.15.103:0/1970887694 >> 172.21.15.103:0/1970887694 conn(0x55c4d3028f60 msgr2=0x55c4d3029340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 -- 172.21.15.103:0/1970887694 shutdown_connections 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.017+0000 7f9990d88740 1 -- 172.21.15.103:0/1970887694 wait complete. 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 -- start start 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 Processor -- start 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d3037460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d30347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3034cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4d303a780 con 0x55c4d302c9d0 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2f4df20 con 0x55c4d302d2f0 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4d303a900 con 0x55c4d3037bb0 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9990d88740 10 client.0 _ll_register_callbacks cb 0x55c4d2f17030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9987fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d3037460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9987fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d3037460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58770/0 (socket says 172.21.15.103:58770) 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f9987fff640 1 -- 172.21.15.103:0/3128070959 learned_addr learned my addr 172.21.15.103:0/3128070959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f99877fe640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d30347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.018+0000 7f998cf4c640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3034cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9987fff640 1 -- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 msgr2=0x55c4d3034cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9987fff640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3034cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9987fff640 1 -- 172.21.15.103:0/3128070959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 msgr2=0x55c4d30347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9987fff640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d30347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9987fff640 1 -- 172.21.15.103:0/3128070959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d2d042b0 con 0x55c4d302c9d0 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f99877fe640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d30347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f998cf4c640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3034cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9987fff640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d3037460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f99780116c0 tx=0x7f9978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7393eb411a260296 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f99677fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f99677fe640 1 -- 172.21.15.103:0/3128070959 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997801e8f0 con 0x55c4d302c9d0 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f99677fe640 1 -- 172.21.15.103:0/3128070959 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f997801ea90 con 0x55c4d302c9d0 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f997c004030 con 0x55c4d302c9d0 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f99677fe640 1 -- 172.21.15.103:0/3128070959 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f997801ec90 con 0x55c4d302c9d0 2024-09-19T07:37:46.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9990d88740 10 client.15087 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c4d30460a0 con 0x55c4d302c9d0 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f99677fe640 1 -- 172.21.15.103:0/3128070959 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c004030 con 0x55c4d302c9d0 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.019+0000 7f99677fe640 1 -- 172.21.15.103:0/3128070959 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f9978032520 con 0x55c4d302c9d0 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f9990d88740 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f9990d88740 10 client.15087 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f9990d88740 20 client.15087 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f9990d88740 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d3048770 0x55c4d3049be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f9990d88740 4 client.15087 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d2fa0600 con 0x55c4d3048770 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f99877fe640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d3048770 0x55c4d3049be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f99877fe640 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d3048770 0x55c4d3049be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=865fe5faea6a40ef server_cookie=1a93f480e4e30d34 in_seq=0 out_seq=0 2024-09-19T07:37:46.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.020+0000 7f99677fe640 10 client.15087 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:46.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f99677fe640 1 -- 172.21.15.103:0/3128070959 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4d2fa0600 con 0x55c4d3048770 2024-09-19T07:37:46.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f99677fe640 10 client.15087 handle_command_reply: tid=0 2024-09-19T07:37:46.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 client.15087 shutdown 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 msgr2=0x55c4d3037460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d3037460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f99780116c0 tx=0x7f9978004730 comp rx=0 tx=0).stop 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 shutdown_connections 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d3048770 0x55c4d3049be0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d3037bb0 0x55c4d3034cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d302d2f0 0x55c4d30347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 --2- 172.21.15.103:0/3128070959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302c9d0 0x55c4d3037460 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 >> 172.21.15.103:0/3128070959 conn(0x55c4d3028f60 msgr2=0x55c4d302ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f99677fe640 0 client.15087 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 shutdown_connections 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.021+0000 7f9990d88740 1 -- 172.21.15.103:0/3128070959 wait complete. 2024-09-19T07:37:46.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.022+0000 7f9990d88740 20 client.15087 trim_cache size 0 max 16384 2024-09-19T07:37:46.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.039+0000 7f9990d88740 1 Processor -- start 2024-09-19T07:37:46.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.039+0000 7f9990d88740 1 -- start start 2024-09-19T07:37:46.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.039+0000 7f9990d88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3049510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.039+0000 7f9990d88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d3051ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.039+0000 7f9990d88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d3052400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f9990d88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4d3047f20 con 0x55c4d302d2f0 2024-09-19T07:37:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f9990d88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4d30485f0 con 0x55c4d3037bb0 2024-09-19T07:37:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f9990d88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4d3048cc0 con 0x55c4d302c9d0 2024-09-19T07:37:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d3051ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d3051ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58784/0 (socket says 172.21.15.103:58784) 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 -- 172.21.15.103:0/3772823200 learned_addr learned my addr 172.21.15.103:0/3772823200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f998cf4c640 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d3052400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f9987fff640 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3049510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 -- 172.21.15.103:0/3772823200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 msgr2=0x55c4d3049510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3049510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 -- 172.21.15.103:0/3772823200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 msgr2=0x55c4d3052400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d3052400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 -- 172.21.15.103:0/3772823200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d3037080 con 0x55c4d302d2f0 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f99877fe640 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d3051ec0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f997c02a930 tx=0x7f997c005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491d85973de6343e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f996dffb640 1 -- 172.21.15.103:0/3772823200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c035040 con 0x55c4d302d2f0 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f996dffb640 1 -- 172.21.15.103:0/3772823200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c031a90 con 0x55c4d302d2f0 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.040+0000 7f996dffb640 1 -- 172.21.15.103:0/3772823200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c004bc0 con 0x55c4d302d2f0 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 -- 172.21.15.103:0/3772823200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 msgr2=0x55c4d3051ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d3051ec0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f997c02a930 tx=0x7f997c005260 comp rx=0 tx=0).stop 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 -- 172.21.15.103:0/3772823200 shutdown_connections 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d3052400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d3051ec0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 --2- 172.21.15.103:0/3772823200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3049510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 -- 172.21.15.103:0/3772823200 >> 172.21.15.103:0/3772823200 conn(0x55c4d3028f60 msgr2=0x55c4d3033080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 -- 172.21.15.103:0/3772823200 shutdown_connections 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 -- 172.21.15.103:0/3772823200 wait complete. 2024-09-19T07:37:46.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.041+0000 7f9990d88740 1 -- start start 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 1 Processor -- start 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3052410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d304c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d304ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4d3049a60 con 0x55c4d302d2f0 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4d3049390 con 0x55c4d3037bb0 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4d304a130 con 0x55c4d302c9d0 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9990d88740 10 client.0 _ll_register_callbacks cb 0x7f9978010c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d304c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d304c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58790/0 (socket says 172.21.15.103:58790) 2024-09-19T07:37:46.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 -- 172.21.15.103:0/2859716002 learned_addr learned my addr 172.21.15.103:0/2859716002 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f9987fff640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3052410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f998cf4c640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d304ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 -- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 msgr2=0x55c4d3052410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3052410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 -- 172.21.15.103:0/2859716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 msgr2=0x55c4d304ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d304ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 -- 172.21.15.103:0/2859716002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d304d2b0 con 0x55c4d302d2f0 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f998cf4c640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d304ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99877fe640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d304c780 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f997c0096c0 tx=0x7f997c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706d0918cb2e27cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.042+0000 7f99677fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f99677fe640 1 -- 172.21.15.103:0/2859716002 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c035070 con 0x55c4d302d2f0 2024-09-19T07:37:46.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f99677fe640 1 -- 172.21.15.103:0/2859716002 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f997c0315b0 con 0x55c4d302d2f0 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f997c031a90 con 0x55c4d302d2f0 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f99677fe640 1 -- 172.21.15.103:0/2859716002 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f997c031e40 con 0x55c4d302d2f0 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 10 client.15093 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c4d30b1c70 con 0x55c4d302d2f0 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f99677fe640 1 -- 172.21.15.103:0/2859716002 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c031a90 con 0x55c4d302d2f0 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f99677fe640 1 -- 172.21.15.103:0/2859716002 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f997c058750 con 0x55c4d302d2f0 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:46.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 10 client.15093 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 20 client.15093 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d30b6130 0x55c4d30b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 4 client.15093 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "status"}) -- 0x55c4d2f9ab10 con 0x55c4d30b6130 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.043+0000 7f9987fff640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d30b6130 0x55c4d30b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9987fff640 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d30b6130 0x55c4d30b4e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80cad3aa04f37ccb server_cookie=1c58de912b95294a in_seq=0 out_seq=0 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f99677fe640 10 client.15093 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f99677fe640 1 -- 172.21.15.103:0/2859716002 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c4d2f9ab10 con 0x55c4d30b6130 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f99677fe640 10 client.15093 handle_command_reply: tid=0 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 client.15093 shutdown 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 msgr2=0x55c4d304c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d304c780 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f997c0096c0 tx=0x7f997c02a410 comp rx=0 tx=0).stop 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 shutdown_connections 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55c4d30b6130 0x55c4d30b4e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f99677fe640 0 client.15093 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4d3037bb0 0x55c4d304ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4d302d2f0 0x55c4d304c780 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 --2- 172.21.15.103:0/2859716002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4d302c9d0 0x55c4d3052410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 >> 172.21.15.103:0/2859716002 conn(0x55c4d3028f60 msgr2=0x55c4d30334a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 shutdown_connections 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.044+0000 7f9990d88740 1 -- 172.21.15.103:0/2859716002 wait complete. 2024-09-19T07:37:46.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f9990d88740 20 client.15093 trim_cache size 0 max 16384 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24448, 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 87.145280670999995, 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 123.75308405299999 2024-09-19T07:37:46.055 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:46.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99700a32d0 msgr2=0x7f99700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99700a32d0 0x7f99700a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f997800bbe0 tx=0x7f997800b390 comp rx=0 tx=0).stop 2024-09-19T07:37:46.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988106f10 msgr2=0x7f9988074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:46.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988106f10 0x7f9988074f70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f998000eaa0 tx=0x7f998000ef60 comp rx=0 tx=0).stop 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 shutdown_connections 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99700a32d0 0x7f99700a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9988106f10 0x7f9988074f70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9988102790 0x7f9988074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 --2- 172.21.15.103:0/394472726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9988101de0 0x7f99880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 >> 172.21.15.103:0/394472726 conn(0x7f99880fd8c0 msgr2=0x7f99880fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 shutdown_connections 2024-09-19T07:37:46.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:46.045+0000 7f998e9af640 1 -- 172.21.15.103:0/394472726 wait complete. 2024-09-19T07:37:46.839 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T07:37:47.087 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:47.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:47 smithi103 ceph-mon[31201]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:47.457 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:47 smithi103 ceph-mon[31201]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T07:37:47.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.537+0000 7f7d7d784640 1 -- 172.21.15.103:0/2903666900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d781036e0 msgr2=0x7f7d7810bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.537+0000 7f7d7d784640 1 --2- 172.21.15.103:0/2903666900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d781036e0 0x7f7d7810bbd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009ca0 tx=0x7f7d6c02a950 comp rx=0 tx=0).stop 2024-09-19T07:37:47.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 -- 172.21.15.103:0/2903666900 shutdown_connections 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 --2- 172.21.15.103:0/2903666900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d7810c160 0x7f7d7810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 --2- 172.21.15.103:0/2903666900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d781036e0 0x7f7d7810bbd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 --2- 172.21.15.103:0/2903666900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d78102dc0 0x7f7d781031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 -- 172.21.15.103:0/2903666900 >> 172.21.15.103:0/2903666900 conn(0x7f7d780fc780 msgr2=0x7f7d780feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 -- 172.21.15.103:0/2903666900 shutdown_connections 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 -- 172.21.15.103:0/2903666900 wait complete. 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 Processor -- start 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.538+0000 7f7d7d784640 1 -- start start 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d78102dc0 0x7f7d78198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d781036e0 0x7f7d781989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d7810c160 0x7f7d7819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78111f50 con 0x7f7d7810c160 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78111dd0 con 0x7f7d781036e0 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781120d0 con 0x7f7d78102dc0 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d7810c160 0x7f7d7819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d7810c160 0x7f7d7819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58806/0 (socket says 172.21.15.103:58806) 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 -- 172.21.15.103:0/1023728552 learned_addr learned my addr 172.21.15.103:0/1023728552 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d78102dc0 0x7f7d78198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d781036e0 0x7f7d781989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 -- 172.21.15.103:0/1023728552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d78102dc0 msgr2=0x7f7d78198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d78102dc0 0x7f7d78198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 -- 172.21.15.103:0/1023728552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d781036e0 msgr2=0x7f7d781989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d781036e0 0x7f7d781989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 -- 172.21.15.103:0/1023728552 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7819d420 con 0x7f7d7810c160 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d78102dc0 0x7f7d78198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d777fe640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d7810c160 0x7f7d7819cd40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800d750 tx=0x7f7d6800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d5ffff640 1 -- 172.21.15.103:0/1023728552 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68017070 con 0x7f7d7810c160 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7819d710 con 0x7f7d7810c160 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d5ffff640 1 -- 172.21.15.103:0/1023728552 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d68005500 con 0x7f7d7810c160 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d5ffff640 1 -- 172.21.15.103:0/1023728552 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68004d90 con 0x7f7d7810c160 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.539+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d781a4ff0 con 0x7f7d7810c160 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d5ffff640 1 -- 172.21.15.103:0/1023728552 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7d68004f30 con 0x7f7d7810c160 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 Processor -- start 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 -- start start 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c405db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4062f0 0x562a2c410670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c410bb0 0x562a2c412fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562a2c36c090 con 0x562a2c4062f0 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a2c313a70 con 0x562a2c4059d0 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562a2c368050 con 0x562a2c410bb0 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d5ffff640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7d600a3240 0x7f7d600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d76ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4062f0 0x562a2c410670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d76ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4062f0 0x562a2c410670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58814/0 (socket says 172.21.15.103:58814) 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d76ffd640 1 -- 172.21.15.103:0/3808110771 learned_addr learned my addr 172.21.15.103:0/3808110771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d777fe640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7d600a3240 0x7f7d600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.542+0000 7f7d767fc640 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c405db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d767fc640 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c410bb0 0x562a2c412fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d5ffff640 1 -- 172.21.15.103:0/1023728552 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7d680cf330 con 0x7f7d7810c160 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d76ffd640 1 -- 172.21.15.103:0/3808110771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c410bb0 msgr2=0x562a2c412fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c410bb0 0x562a2c412fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d76ffd640 1 -- 172.21.15.103:0/3808110771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 msgr2=0x562a2c405db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c405db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d76ffd640 1 -- 172.21.15.103:0/3808110771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2c320130 con 0x562a2c4062f0 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d767fc640 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c405db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4062f0 0x562a2c410670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009c50 tx=0x7f7d7000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d596d04c6a32c731 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d777fe640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7d600a3240 0x7f7d600a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d6809b8e0 tx=0x7f7d6806e7d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d5d7fa640 1 -- 172.21.15.103:0/3808110771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70016070 con 0x562a2c4062f0 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d5d7fa640 1 -- 172.21.15.103:0/3808110771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d70011950 con 0x562a2c4062f0 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.543+0000 7f7d5d7fa640 1 -- 172.21.15.103:0/3808110771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700108d0 con 0x562a2c4062f0 2024-09-19T07:37:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3808110771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4062f0 msgr2=0x562a2c410670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4062f0 0x562a2c410670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009c50 tx=0x7f7d7000e6c0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3808110771 shutdown_connections 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c410bb0 0x562a2c412fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4062f0 0x562a2c410670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3808110771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c405db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3808110771 >> 172.21.15.103:0/3808110771 conn(0x562a2c401f60 msgr2=0x562a2c402340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3808110771 shutdown_connections 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3808110771 wait complete. 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- start start 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 Processor -- start 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4059d0 0x562a2c4103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4062f0 0x562a2c40d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c40dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562a2c413780 con 0x562a2c410bb0 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a2c326f20 con 0x562a2c4062f0 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562a2c413900 con 0x562a2c4059d0 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.544+0000 7f7d7fb5d740 10 client.0 _ll_register_callbacks cb 0x562a2c2f0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c40dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c40dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58824/0 (socket says 172.21.15.103:58824) 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 -- 172.21.15.103:0/1592899363 learned_addr learned my addr 172.21.15.103:0/1592899363 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4062f0 0x562a2c40d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4059d0 0x562a2c4103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 -- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4059d0 msgr2=0x562a2c4103c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4059d0 0x562a2c4103c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 -- 172.21.15.103:0/1592899363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4062f0 msgr2=0x562a2c40d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4062f0 0x562a2c40d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 -- 172.21.15.103:0/1592899363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a2c0dd2b0 con 0x562a2c410bb0 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c40dc50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004af0 tx=0x7f7d6c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91c0de2bd3aac38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1592899363 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c045040 con 0x562a2c410bb0 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1592899363 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7d6c003d80 con 0x562a2c410bb0 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d70011950 con 0x562a2c410bb0 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1592899363 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d6c051050 con 0x562a2c410bb0 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d7fb5d740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:47.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.545+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a2c41f000 con 0x562a2c410bb0 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1592899363 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d70011950 con 0x562a2c410bb0 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1592899363 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7d6c0396b0 con 0x562a2c410bb0 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d7fb5d740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d7fb5d740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d7fb5d740 20 client.15105 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c4215c0 0x562a2c422a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d7fb5d740 4 client.15105 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a2c379600 con 0x562a2c4215c0 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.546+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c4215c0 0x562a2c422a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.547+0000 7f7d5aff5640 10 client.15105 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.547+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c4215c0 0x562a2c422a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83dc92cd7ce3eab4 server_cookie=8b2f23135a957873 in_seq=0 out_seq=0 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.547+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1592899363 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a2c379600 con 0x562a2c4215c0 2024-09-19T07:37:47.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.547+0000 7f7d5aff5640 10 client.15105 handle_command_reply: tid=0 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.547+0000 7f7d7fb5d740 1 client.15105 shutdown 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.547+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 msgr2=0x562a2c40dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c40dc50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004af0 tx=0x7f7d6c005c30 comp rx=0 tx=0).stop 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 shutdown_connections 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c4215c0 0x562a2c422a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c40dc50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4062f0 0x562a2c40d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1592899363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4059d0 0x562a2c4103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 >> 172.21.15.103:0/1592899363 conn(0x562a2c401f60 msgr2=0x562a2c407c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d5aff5640 0 client.15105 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 shutdown_connections 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1592899363 wait complete. 2024-09-19T07:37:47.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.548+0000 7f7d7fb5d740 20 client.15105 trim_cache size 0 max 16384 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.565+0000 7f7d7fb5d740 1 Processor -- start 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.565+0000 7f7d7fb5d740 1 -- start start 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.565+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4059d0 0x562a2c421c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c40cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c410bb0 0x562a2c40a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562a2c420d70 con 0x562a2c4059d0 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a2c421440 con 0x562a2c410bb0 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562a2c421b10 con 0x562a2c4062f0 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4059d0 0x562a2c421c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4059d0 0x562a2c421c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58840/0 (socket says 172.21.15.103:58840) 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 -- 172.21.15.103:0/3395190985 learned_addr learned my addr 172.21.15.103:0/3395190985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c40cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c410bb0 0x562a2c40a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 -- 172.21.15.103:0/3395190985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 msgr2=0x562a2c40cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c40cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 -- 172.21.15.103:0/3395190985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c410bb0 msgr2=0x562a2c40a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c410bb0 0x562a2c40a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 -- 172.21.15.103:0/3395190985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2c40bc60 con 0x562a2c4059d0 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c40cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d767fc640 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4059d0 0x562a2c421c90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004ac0 tx=0x7f7d6c039fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82a9776e7d56dae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d5cff9640 1 -- 172.21.15.103:0/3395190985 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c034040 con 0x562a2c4059d0 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d5cff9640 1 -- 172.21.15.103:0/3395190985 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6c03e040 con 0x562a2c4059d0 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.566+0000 7f7d5cff9640 1 -- 172.21.15.103:0/3395190985 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c030af0 con 0x562a2c4059d0 2024-09-19T07:37:47.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3395190985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4059d0 msgr2=0x562a2c421c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4059d0 0x562a2c421c90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c004ac0 tx=0x7f7d6c039fb0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3395190985 shutdown_connections 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c410bb0 0x562a2c40a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c40cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/3395190985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c4059d0 0x562a2c421c90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3395190985 >> 172.21.15.103:0/3395190985 conn(0x562a2c401f60 msgr2=0x562a2c402340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3395190985 shutdown_connections 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/3395190985 wait complete. 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 -- start start 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.567+0000 7f7d7fb5d740 1 Processor -- start 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c4240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c424620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c489cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562a2c4228b0 con 0x562a2c410bb0 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a2c4221e0 con 0x562a2c4059d0 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562a2c422f80 con 0x562a2c4062f0 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 10 client.0 _ll_register_callbacks cb 0x7f7d6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c489cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c489cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58852/0 (socket says 172.21.15.103:58852) 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 -- 172.21.15.103:0/1242413794 learned_addr learned my addr 172.21.15.103:0/1242413794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 -- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 msgr2=0x562a2c424620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c424620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c424620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 -- 172.21.15.103:0/1242413794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 msgr2=0x562a2c4240e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c4240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 -- 172.21.15.103:0/1242413794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a2c48a200 con 0x562a2c410bb0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d767fc640 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c489cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c02fde0 tx=0x7f7d6c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc903a1bef70826a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1242413794 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c034040 con 0x562a2c410bb0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1242413794 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7d6c039450 con 0x562a2c410bb0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6c03e040 con 0x562a2c410bb0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.568+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1242413794 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d6c039650 con 0x562a2c410bb0 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 10 client.15111 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:47.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a2c48b350 con 0x562a2c410bb0 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1242413794 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6c03e040 con 0x562a2c410bb0 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1242413794 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7d6c056640 con 0x562a2c410bb0 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 10 client.15111 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 20 client.15111 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c48fc10 0x562a2c48e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 4 client.15111 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "status"}) -- 0x562a2c373b10 con 0x562a2c48fc10 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.569+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c48fc10 0x562a2c48e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:47.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d76ffd640 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c48fc10 0x562a2c48e920 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=87375a98695e8fd3 server_cookie=452446260c630681 in_seq=0 out_seq=0 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d5aff5640 10 client.15111 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d5aff5640 1 -- 172.21.15.103:0/1242413794 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x562a2c373b10 con 0x562a2c48fc10 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d5aff5640 10 client.15111 handle_command_reply: tid=0 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 client.15111 shutdown 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 msgr2=0x562a2c489cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c489cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c02fde0 tx=0x7f7d6c009940 comp rx=0 tx=0).stop 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 shutdown_connections 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x562a2c48fc10 0x562a2c48e920 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562a2c410bb0 0x562a2c489cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d5aff5640 0 client.15111 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:37:47.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a2c4062f0 0x562a2c424620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 --2- 172.21.15.103:0/1242413794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a2c4059d0 0x562a2c4240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 >> 172.21.15.103:0/1242413794 conn(0x562a2c401f60 msgr2=0x562a2c4007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 shutdown_connections 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:47.594 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 89.812649399999998, 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 122.72981329 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 1 -- 172.21.15.103:0/1242413794 wait complete. 2024-09-19T07:37:47.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.570+0000 7f7d7fb5d740 20 client.15111 trim_cache size 0 max 16384 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7d600a3240 msgr2=0x7f7d600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7d600a3240 0x7f7d600a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d6809b8e0 tx=0x7f7d6806e7d0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d7810c160 msgr2=0x7f7d7819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d7810c160 0x7f7d7819cd40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800d750 tx=0x7f7d6800dc10 comp rx=0 tx=0).stop 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 shutdown_connections 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7d600a3240 0x7f7d600a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d7810c160 0x7f7d7819cd40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d781036e0 0x7f7d781989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 --2- 172.21.15.103:0/1023728552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d78102dc0 0x7f7d78198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 >> 172.21.15.103:0/1023728552 conn(0x7f7d780fc780 msgr2=0x7f7d780fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 shutdown_connections 2024-09-19T07:37:47.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:47.571+0000 7f7d7d784640 1 -- 172.21.15.103:0/1023728552 wait complete. 2024-09-19T07:37:47.597 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:47 smithi134 ceph-mon[35414]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:47.597 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:47 smithi134 ceph-mon[35414]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T07:37:47.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:47 smithi174 ceph-mon[35370]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-19T07:37:47.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:47 smithi174 ceph-mon[35370]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T07:37:48.273 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T07:37:48.519 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 -- 172.21.15.103:0/671219766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 msgr2=0x7f4438100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 --2- 172.21.15.103:0/671219766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f4438100f00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4428009ca0 tx=0x7f442802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 -- 172.21.15.103:0/671219766 shutdown_connections 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 --2- 172.21.15.103:0/671219766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443810e500 0x7f4438110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 --2- 172.21.15.103:0/671219766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4438101440 0x7f443810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 --2- 172.21.15.103:0/671219766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f4438100f00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 -- 172.21.15.103:0/671219766 >> 172.21.15.103:0/671219766 conn(0x7f44380fc7d0 msgr2=0x7f44380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 -- 172.21.15.103:0/671219766 shutdown_connections 2024-09-19T07:37:49.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 -- 172.21.15.103:0/671219766 wait complete. 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.982+0000 7f443dcf7640 1 Processor -- start 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f443dcf7640 1 -- start start 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f443dcf7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f44381a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f443dcf7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4438101440 0x7f44381a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f443dcf7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443810e500 0x7f443819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f443dcf7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4438114370 con 0x7f4438100b20 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f443dcf7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44381141f0 con 0x7f4438101440 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f443dcf7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44381144f0 con 0x7f443810e500 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f44381a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f44381a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33254/0 (socket says 172.21.15.103:33254) 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 -- 172.21.15.103:0/4286488154 learned_addr learned my addr 172.21.15.103:0/4286488154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f4437fff640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443810e500 0x7f443819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f4436ffd640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4438101440 0x7f44381a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 -- 172.21.15.103:0/4286488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443810e500 msgr2=0x7f443819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443810e500 0x7f443819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 -- 172.21.15.103:0/4286488154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4438101440 msgr2=0x7f44381a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4438101440 0x7f44381a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 -- 172.21.15.103:0/4286488154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443819af40 con 0x7f4438100b20 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f4437fff640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443810e500 0x7f443819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.983+0000 7f44377fe640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f44381a0600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44280097b0 tx=0x7f4428031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.984+0000 7f4434ff9640 1 -- 172.21.15.103:0/4286488154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442802fba0 con 0x7f4438100b20 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.984+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443819b1d0 con 0x7f4438100b20 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.984+0000 7f4434ff9640 1 -- 172.21.15.103:0/4286488154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442802fd40 con 0x7f4438100b20 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.984+0000 7f4434ff9640 1 -- 172.21.15.103:0/4286488154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442803b930 con 0x7f4438100b20 2024-09-19T07:37:49.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.984+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44381a7580 con 0x7f4438100b20 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f4434ff9640 1 -- 172.21.15.103:0/4286488154 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f4428030cc0 con 0x7f4438100b20 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 Processor -- start 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 -- start start 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f4434ff9640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44240a3240 0x7f44240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abb7321090 con 0x55abb73c5bb0 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55abb72c8a70 con 0x55abb73bb2f0 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.986+0000 7f44400d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abb731d050 con 0x55abb73ba9d0 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33260/0 (socket says 172.21.15.103:33260) 2024-09-19T07:37:49.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 -- 172.21.15.103:0/3826650031 learned_addr learned my addr 172.21.15.103:0/3826650031 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f44377fe640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44240a3240 0x7f44240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4434ff9640 1 -- 172.21.15.103:0/4286488154 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f44280f1100 con 0x7f4438100b20 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4436ffd640 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 -- 172.21.15.103:0/3826650031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 msgr2=0x55abb73badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73badb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 -- 172.21.15.103:0/3826650031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 msgr2=0x55abb73c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 -- 172.21.15.103:0/3826650031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb72d5130 con 0x55abb73c5bb0 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f44377fe640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44240a3240 0x7f44240a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4428006010 tx=0x7f442803b400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f4437fff640 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73c7fa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4430016500 tx=0x7f443000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517ac14e5e68665e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f442dffb640 1 -- 172.21.15.103:0/3826650031 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430018430 con 0x55abb73c5bb0 2024-09-19T07:37:49.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f442dffb640 1 -- 172.21.15.103:0/3826650031 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4430018ab0 con 0x55abb73c5bb0 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.987+0000 7f442dffb640 1 -- 172.21.15.103:0/3826650031 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430017840 con 0x55abb73c5bb0 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 -- 172.21.15.103:0/3826650031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 msgr2=0x55abb73c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73c7fa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4430016500 tx=0x7f443000a2d0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 -- 172.21.15.103:0/3826650031 shutdown_connections 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73c7fa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 --2- 172.21.15.103:0/3826650031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 -- 172.21.15.103:0/3826650031 >> 172.21.15.103:0/3826650031 conn(0x55abb73b6f60 msgr2=0x55abb73b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 -- 172.21.15.103:0/3826650031 shutdown_connections 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 -- 172.21.15.103:0/3826650031 wait complete. 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.988+0000 7f44400d0740 1 -- start start 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 1 Processor -- start 2024-09-19T07:37:49.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73ba9d0 0x55abb73c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73bb2f0 0x55abb73c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 0x55abb73c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abb73c8780 con 0x55abb73ba9d0 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55abb72dbf20 con 0x55abb73c5bb0 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abb73c8900 con 0x55abb73bb2f0 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f44400d0740 10 client.0 _ll_register_callbacks cb 0x55abb72a5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73ba9d0 0x55abb73c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73ba9d0 0x55abb73c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33268/0 (socket says 172.21.15.103:33268) 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 -- 172.21.15.103:0/622191183 learned_addr learned my addr 172.21.15.103:0/622191183 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4436ffd640 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73bb2f0 0x55abb73c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 -- 172.21.15.103:0/622191183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73bb2f0 msgr2=0x55abb73c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73bb2f0 0x55abb73c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 -- 172.21.15.103:0/622191183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 msgr2=0x55abb73c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 0x55abb73c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 -- 172.21.15.103:0/622191183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb70922b0 con 0x55abb73ba9d0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4437fff640 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73ba9d0 0x55abb73c53f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f443001a040 tx=0x7f443000a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83fcb2c276beeac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4416ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4416ffd640 1 -- 172.21.15.103:0/622191183 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430025020 con 0x55abb73ba9d0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.989+0000 7f4416ffd640 1 -- 172.21.15.103:0/622191183 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f4430018840 con 0x55abb73ba9d0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4430018ab0 con 0x55abb73ba9d0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f4416ffd640 1 -- 172.21.15.103:0/622191183 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44300059c0 con 0x55abb73ba9d0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abb73d4070 con 0x55abb73ba9d0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f4416ffd640 1 -- 172.21.15.103:0/622191183 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4430018ab0 con 0x55abb73ba9d0 2024-09-19T07:37:49.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f4416ffd640 1 -- 172.21.15.103:0/622191183 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4430012050 con 0x55abb73ba9d0 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 10 client.15123 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 20 client.15123 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb73d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 4 client.15123 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abb732e600 con 0x55abb73d62c0 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.990+0000 7f4436ffd640 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb73d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.991+0000 7f4436ffd640 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb73d7730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5313abf994b1be6f server_cookie=3acb77ca9d1e414a in_seq=0 out_seq=0 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.991+0000 7f4416ffd640 10 client.15123 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.991+0000 7f4416ffd640 1 -- 172.21.15.103:0/622191183 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abb732e600 con 0x55abb73d62c0 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.991+0000 7f4416ffd640 10 client.15123 handle_command_reply: tid=0 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 client.15123 shutdown 2024-09-19T07:37:49.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73ba9d0 msgr2=0x55abb73c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73ba9d0 0x55abb73c53f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f443001a040 tx=0x7f443000a1b0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 shutdown_connections 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb73d7730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 0x55abb73c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73bb2f0 0x55abb73c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f4416ffd640 0 client.15123 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 --2- 172.21.15.103:0/622191183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73ba9d0 0x55abb73c53f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 >> 172.21.15.103:0/622191183 conn(0x55abb73b6f60 msgr2=0x55abb73bcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 shutdown_connections 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 1 -- 172.21.15.103:0/622191183 wait complete. 2024-09-19T07:37:49.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:48.992+0000 7f44400d0740 20 client.15123 trim_cache size 0 max 16384 2024-09-19T07:37:49.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.009+0000 7f44400d0740 1 Processor -- start 2024-09-19T07:37:49.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.009+0000 7f44400d0740 1 -- start start 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f44400d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f44400d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f44400d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f44400d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abb73d5a70 con 0x55abb73c5bb0 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f44400d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55abb73d6140 con 0x55abb73bb2f0 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f44400d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abb73d6810 con 0x55abb73ba9d0 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33272/0 (socket says 172.21.15.103:33272) 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 -- 172.21.15.103:0/3739994524 learned_addr learned my addr 172.21.15.103:0/3739994524 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4436ffd640 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 -- 172.21.15.103:0/3739994524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 msgr2=0x55abb73d7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73d7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 -- 172.21.15.103:0/3739994524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 msgr2=0x55abb73c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 -- 172.21.15.103:0/3739994524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb73bb120 con 0x55abb73c5bb0 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f4437fff640 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73b6000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4430006e10 tx=0x7f4430006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1616eb401d3d55a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f442d7fa640 1 -- 172.21.15.103:0/3739994524 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430025070 con 0x55abb73c5bb0 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f442d7fa640 1 -- 172.21.15.103:0/3739994524 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4430021460 con 0x55abb73c5bb0 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.010+0000 7f442d7fa640 1 -- 172.21.15.103:0/3739994524 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300177f0 con 0x55abb73c5bb0 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 -- 172.21.15.103:0/3739994524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 msgr2=0x55abb73b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73b6000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4430006e10 tx=0x7f4430006da0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 -- 172.21.15.103:0/3739994524 shutdown_connections 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73c5bb0 0x55abb73b6000 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73bb2f0 0x55abb73c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 --2- 172.21.15.103:0/3739994524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73d7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 -- 172.21.15.103:0/3739994524 >> 172.21.15.103:0/3739994524 conn(0x55abb73b6f60 msgr2=0x55abb73b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 -- 172.21.15.103:0/3739994524 shutdown_connections 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 -- 172.21.15.103:0/3739994524 wait complete. 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.011+0000 7f44400d0740 1 -- start start 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 1 Processor -- start 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73bb2f0 0x55abb73db480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 0x55abb743e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abb73d75b0 con 0x55abb73bb2f0 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55abb73d6ee0 con 0x55abb73c5bb0 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abb73d7c80 con 0x55abb73ba9d0 2024-09-19T07:37:49.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f44400d0740 10 client.0 _ll_register_callbacks cb 0x7f442000b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73bb2f0 0x55abb73db480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73bb2f0 0x55abb73db480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33284/0 (socket says 172.21.15.103:33284) 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 -- 172.21.15.103:0/4253306327 learned_addr learned my addr 172.21.15.103:0/4253306327 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4437fff640 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 -- 172.21.15.103:0/4253306327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 msgr2=0x55abb73daf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73daf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 -- 172.21.15.103:0/4253306327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 msgr2=0x55abb743e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 0x55abb743e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.012+0000 7f4436ffd640 1 -- 172.21.15.103:0/4253306327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb743efb0 con 0x55abb73bb2f0 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f4436ffd640 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73bb2f0 0x55abb73db480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55abb73b7dc0 tx=0x7f44200157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e84801fe00296d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f4416ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f4416ffd640 1 -- 172.21.15.103:0/4253306327 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420025430 con 0x55abb73bb2f0 2024-09-19T07:37:49.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4430021460 con 0x55abb73bb2f0 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f4416ffd640 1 -- 172.21.15.103:0/4253306327 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f44200255d0 con 0x55abb73bb2f0 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f4416ffd640 1 -- 172.21.15.103:0/4253306327 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44200257d0 con 0x55abb73bb2f0 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 10 client.15129 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abb7440240 con 0x55abb73bb2f0 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f4416ffd640 1 -- 172.21.15.103:0/4253306327 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4430021460 con 0x55abb73bb2f0 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f4416ffd640 1 -- 172.21.15.103:0/4253306327 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4420030a60 con 0x55abb73bb2f0 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 10 client.15129 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 10 client.15129 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 20 client.15129 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb7443b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 4 client.15129 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:49.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.013+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "status"}) -- 0x55abb7328b10 con 0x55abb73d62c0 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f4437fff640 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb7443b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f4437fff640 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb7443b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=201ad9ef0548f051 server_cookie=9a36940b018ad5e8 in_seq=0 out_seq=0 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f4416ffd640 10 client.15129 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f4416ffd640 1 -- 172.21.15.103:0/4253306327 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55abb7328b10 con 0x55abb73d62c0 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f4416ffd640 10 client.15129 handle_command_reply: tid=0 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f44400d0740 1 client.15129 shutdown 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73bb2f0 msgr2=0x55abb73db480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.014+0000 7f44400d0740 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73bb2f0 0x55abb73db480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55abb73b7dc0 tx=0x7f44200157f0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 shutdown_connections 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55abb73d62c0 0x55abb7443b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abb73c5bb0 0x55abb743e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abb73bb2f0 0x55abb73db480 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 --2- 172.21.15.103:0/4253306327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abb73ba9d0 0x55abb73daf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f4416ffd640 0 client.15129 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 >> 172.21.15.103:0/4253306327 conn(0x55abb73b6f60 msgr2=0x55abb73c1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 shutdown_connections 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 1 -- 172.21.15.103:0/4253306327 wait complete. 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f44400d0740 20 client.15129 trim_cache size 0 max 16384 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-19T07:37:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24455, 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 92.263646352999999, 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 121.551097986 2024-09-19T07:37:49.023 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44240a3240 msgr2=0x7f44240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44240a3240 0x7f44240a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4428006010 tx=0x7f442803b400 comp rx=0 tx=0).stop 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 msgr2=0x7f44381a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f44381a0600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44280097b0 tx=0x7f4428031b10 comp rx=0 tx=0).stop 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 shutdown_connections 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f44240a3240 0x7f44240a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443810e500 0x7f443819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4438101440 0x7f44381a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 --2- 172.21.15.103:0/4286488154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4438100b20 0x7f44381a0600 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:49.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 >> 172.21.15.103:0/4286488154 conn(0x7f44380fc7d0 msgr2=0x7f44380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:49.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 shutdown_connections 2024-09-19T07:37:49.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:49.015+0000 7f443dcf7640 1 -- 172.21.15.103:0/4286488154 wait complete. 2024-09-19T07:37:49.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:49 smithi103 ceph-mon[31201]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:37:49.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:49 smithi134 ceph-mon[35414]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:37:49.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:49 smithi174 ceph-mon[35370]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-19T07:37:49.707 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T07:37:49.955 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:50.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 -- 172.21.15.103:0/4201878416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984101450 msgr2=0x7f8984105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 --2- 172.21.15.103:0/4201878416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984101450 0x7f8984105920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8974009900 tx=0x7f897402a470 comp rx=0 tx=0).stop 2024-09-19T07:37:50.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 -- 172.21.15.103:0/4201878416 shutdown_connections 2024-09-19T07:37:50.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 --2- 172.21.15.103:0/4201878416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8984105f40 0x7f89841063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 --2- 172.21.15.103:0/4201878416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984101450 0x7f8984105920 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 --2- 172.21.15.103:0/4201878416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8984100b30 0x7f8984100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 -- 172.21.15.103:0/4201878416 >> 172.21.15.103:0/4201878416 conn(0x7f89840fc7a0 msgr2=0x7f89840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 -- 172.21.15.103:0/4201878416 shutdown_connections 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.473+0000 7f898b74d640 1 -- 172.21.15.103:0/4201878416 wait complete. 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 Processor -- start 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 -- start start 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984100b30 0x7f89841a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8984101450 0x7f89841a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8984105f40 0x7f898419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8984118560 con 0x7f8984100b30 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89841183e0 con 0x7f8984105f40 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f898b74d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89841186e0 con 0x7f8984101450 2024-09-19T07:37:50.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f89894e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984100b30 0x7f89841a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f89894e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984100b30 0x7f89841a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33290/0 (socket says 172.21.15.103:33290) 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f89894e9640 1 -- 172.21.15.103:0/170123439 learned_addr learned my addr 172.21.15.103:0/170123439 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.474+0000 7f8989cea640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8984105f40 0x7f898419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f89894e9640 1 -- 172.21.15.103:0/170123439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8984101450 msgr2=0x7f89841a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f8988ce8640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8984101450 0x7f89841a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f89894e9640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8984101450 0x7f89841a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f89894e9640 1 -- 172.21.15.103:0/170123439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8984105f40 msgr2=0x7f898419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f89894e9640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8984105f40 0x7f898419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f89894e9640 1 -- 172.21.15.103:0/170123439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898419efe0 con 0x7f8984100b30 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f8989cea640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8984105f40 0x7f898419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:50.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f89894e9640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984100b30 0x7f89841a46f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f898000c920 tx=0x7f898000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f897a7fc640 1 -- 172.21.15.103:0/170123439 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980007c70 con 0x7f8984100b30 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f897a7fc640 1 -- 172.21.15.103:0/170123439 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8980007e10 con 0x7f8984100b30 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898419f2d0 con 0x7f8984100b30 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f897a7fc640 1 -- 172.21.15.103:0/170123439 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980018890 con 0x7f8984100b30 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.475+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89841ab5c0 con 0x7f8984100b30 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.477+0000 7f897a7fc640 1 -- 172.21.15.103:0/170123439 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8980018a70 con 0x7f8984100b30 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 Processor -- start 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 -- start start 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f897a7fc640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89700a3240 0x7f89700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd22e090 con 0x55f3bd2c79d0 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd1d5a70 con 0x55f3bd2c82f0 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f897a7fc640 1 -- 172.21.15.103:0/170123439 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f89800cfb10 con 0x7f8984100b30 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f898db26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd22a050 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f8989cea640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89700a3240 0x7f89700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33306/0 (socket says 172.21.15.103:33306) 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 -- 172.21.15.103:0/3013994077 learned_addr learned my addr 172.21.15.103:0/3013994077 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f8988ce8640 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f8989cea640 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 -- 172.21.15.103:0/3013994077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2d2bb0 msgr2=0x55f3bd2d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2d4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 -- 172.21.15.103:0/3013994077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c82f0 msgr2=0x55f3bd2d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2d2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f8989cea640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89700a3240 0x7f89700a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f897c0092f0 tx=0x7f897c010740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.478+0000 7f89894e9640 1 -- 172.21.15.103:0/3013994077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3bd1e2130 con 0x55f3bd2c79d0 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.479+0000 7f8988ce8640 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.479+0000 7f89894e9640 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2c7db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f898009be30 tx=0x7f8980000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6377b93b080b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.479+0000 7f896f7fe640 1 -- 172.21.15.103:0/3013994077 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800b5070 con 0x55f3bd2c79d0 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.479+0000 7f896f7fe640 1 -- 172.21.15.103:0/3013994077 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89800a4da0 con 0x55f3bd2c79d0 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.479+0000 7f896f7fe640 1 -- 172.21.15.103:0/3013994077 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800987a0 con 0x55f3bd2c79d0 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- 172.21.15.103:0/3013994077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 msgr2=0x55f3bd2c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2c7db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f898009be30 tx=0x7f8980000bc0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- 172.21.15.103:0/3013994077 shutdown_connections 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2d4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 --2- 172.21.15.103:0/3013994077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2c7db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- 172.21.15.103:0/3013994077 >> 172.21.15.103:0/3013994077 conn(0x55f3bd2c3f60 msgr2=0x55f3bd2c4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- 172.21.15.103:0/3013994077 shutdown_connections 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- 172.21.15.103:0/3013994077 wait complete. 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- start start 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 Processor -- start 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2d5780 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd1e8f20 con 0x55f3bd2c79d0 2024-09-19T07:37:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2d5900 con 0x55f3bd2c82f0 2024-09-19T07:37:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.480+0000 7f898db26740 10 client.0 _ll_register_callbacks cb 0x55f3bd1b2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33320/0 (socket says 172.21.15.103:33320) 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 -- 172.21.15.103:0/192100071 learned_addr learned my addr 172.21.15.103:0/192100071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f89894e9640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8988ce8640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 -- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 msgr2=0x55f3bd2cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 -- 172.21.15.103:0/192100071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 msgr2=0x55f3bd2d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 -- 172.21.15.103:0/192100071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3bcf9f2b0 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8988ce8640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f8989cea640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfd70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f897c013860 tx=0x7f897c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f518e3b615bbc227 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f896cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f896cff9640 1 -- 172.21.15.103:0/192100071 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c023070 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f896cff9640 1 -- 172.21.15.103:0/192100071 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f897c011b60 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89800a4da0 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f896cff9640 1 -- 172.21.15.103:0/192100071 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f897c011d60 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f898db26740 10 client.15141 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3bd2e1200 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.481+0000 7f896cff9640 1 -- 172.21.15.103:0/192100071 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89800a4da0 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f896cff9640 1 -- 172.21.15.103:0/192100071 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f897c00f040 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f898db26740 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f898db26740 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f898db26740 20 client.15141 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f898db26740 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd2e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f898db26740 4 client.15141 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3bd23b600 con 0x55f3bd2e37f0 2024-09-19T07:37:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.482+0000 7f89894e9640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd2e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f89894e9640 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd2e4c60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9e730b30ac37f37 server_cookie=88dec7539067b093 in_seq=0 out_seq=0 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f896cff9640 10 client.15141 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f896cff9640 1 -- 172.21.15.103:0/192100071 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3bd23b600 con 0x55f3bd2e37f0 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f896cff9640 10 client.15141 handle_command_reply: tid=0 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 client.15141 shutdown 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 msgr2=0x55f3bd2cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfd70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f897c013860 tx=0x7f897c004ab0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 shutdown_connections 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd2e4c60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfd70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f896cff9640 0 client.15141 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 --2- 172.21.15.103:0/192100071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.483+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 >> 172.21.15.103:0/192100071 conn(0x55f3bd2c3f60 msgr2=0x55f3bd2c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.484+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 shutdown_connections 2024-09-19T07:37:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.484+0000 7f898db26740 1 -- 172.21.15.103:0/192100071 wait complete. 2024-09-19T07:37:50.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.484+0000 7f898db26740 20 client.15141 trim_cache size 0 max 16384 2024-09-19T07:37:50.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.501+0000 7f898db26740 1 Processor -- start 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.501+0000 7f898db26740 1 -- start start 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f898db26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f898db26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f898db26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f898db26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2e2fa0 con 0x55f3bd2c79d0 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f898db26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2e3670 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f898db26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2e3d40 con 0x55f3bd2c82f0 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33336/0 (socket says 172.21.15.103:33336) 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 -- 172.21.15.103:0/3238430739 learned_addr learned my addr 172.21.15.103:0/3238430739 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f8989cea640 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f8988ce8640 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 -- 172.21.15.103:0/3238430739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 msgr2=0x55f3bd2cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 -- 172.21.15.103:0/3238430739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2d2bb0 msgr2=0x55f3bd2cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 -- 172.21.15.103:0/3238430739 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3bd2c7800 con 0x55f3bd2c79d0 2024-09-19T07:37:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f8989cea640 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f8988ce8640 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f89894e9640 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e4590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f898009c0c0 tx=0x7f898000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252621c6404ebea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f896effd640 1 -- 172.21.15.103:0/3238430739 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800b5040 con 0x55f3bd2c79d0 2024-09-19T07:37:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f896effd640 1 -- 172.21.15.103:0/3238430739 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8980018d60 con 0x55f3bd2c79d0 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.502+0000 7f896effd640 1 -- 172.21.15.103:0/3238430739 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800bc040 con 0x55f3bd2c79d0 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 -- 172.21.15.103:0/3238430739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 msgr2=0x55f3bd2e4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e4590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f898009c0c0 tx=0x7f898000fbd0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 -- 172.21.15.103:0/3238430739 shutdown_connections 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd2cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 --2- 172.21.15.103:0/3238430739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e4590 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 -- 172.21.15.103:0/3238430739 >> 172.21.15.103:0/3238430739 conn(0x55f3bd2c3f60 msgr2=0x55f3bd2ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 -- 172.21.15.103:0/3238430739 shutdown_connections 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 -- 172.21.15.103:0/3238430739 wait complete. 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 -- start start 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.503+0000 7f898db26740 1 Processor -- start 2024-09-19T07:37:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f898db26740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f898db26740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2e6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f898db26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd34bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f898db26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2e4ae0 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f898db26740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2e4410 con 0x55f3bd2c79d0 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f898db26740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3bd2e51b0 con 0x55f3bd2c82f0 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f898db26740 10 client.0 _ll_register_callbacks cb 0x7f897c009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd34bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd34bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33350/0 (socket says 172.21.15.103:33350) 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 -- 172.21.15.103:0/4135996099 learned_addr learned my addr 172.21.15.103:0/4135996099 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f89894e9640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8988ce8640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 -- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 msgr2=0x55f3bd2e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2e6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 -- 172.21.15.103:0/4135996099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 msgr2=0x55f3bd2e6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f8989cea640 1 -- 172.21.15.103:0/4135996099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3bd34c4b0 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.504+0000 7f89894e9640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f8989cea640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd34bef0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f897c01f980 tx=0x7f897c01fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6f275e0f4a4f37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f896cff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f896cff9640 1 -- 172.21.15.103:0/4135996099 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c023070 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f896cff9640 1 -- 172.21.15.103:0/4135996099 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f897c01f610 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8980018d60 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f896cff9640 1 -- 172.21.15.103:0/4135996099 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f897c043050 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 10 client.15147 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3bd34d700 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f896cff9640 1 -- 172.21.15.103:0/4135996099 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8980018d60 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f896cff9640 1 -- 172.21.15.103:0/4135996099 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f897c028790 con 0x55f3bd2d2bb0 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 10 client.15147 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 10 client.15147 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 20 client.15147 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd350f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 4 client.15147 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "status"}) -- 0x55f3bd235b10 con 0x55f3bd2e37f0 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.505+0000 7f89894e9640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd350f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f896cff9640 10 client.15147 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f89894e9640 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd350f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d573daf5fca6c0b server_cookie=e32ee962ad00993 in_seq=0 out_seq=0 2024-09-19T07:37:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f896cff9640 1 -- 172.21.15.103:0/4135996099 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55f3bd235b10 con 0x55f3bd2e37f0 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f896cff9640 10 client.15147 handle_command_reply: tid=0 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 client.15147 shutdown 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 msgr2=0x55f3bd34bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd34bef0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f897c01f980 tx=0x7f897c01fe20 comp rx=0 tx=0).stop 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 shutdown_connections 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f3bd2e37f0 0x55f3bd350f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3bd2d2bb0 0x55f3bd34bef0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3bd2c82f0 0x55f3bd2e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 --2- 172.21.15.103:0/4135996099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3bd2c79d0 0x55f3bd2e6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 >> 172.21.15.103:0/4135996099 conn(0x55f3bd2c3f60 msgr2=0x55f3bd2cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.506+0000 7f896cff9640 0 client.15147 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 shutdown_connections 2024-09-19T07:37:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898db26740 1 -- 172.21.15.103:0/4135996099 wait complete. 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898db26740 20 client.15147 trim_cache size 0 max 16384 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:50.520 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 94.762642149000001, 2024-09-19T07:37:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:50.521 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 120.55711001 2024-09-19T07:37:50.521 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:50.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89700a3240 msgr2=0x7f89700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89700a3240 0x7f89700a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f897c0092f0 tx=0x7f897c010740 comp rx=0 tx=0).stop 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984100b30 msgr2=0x7f89841a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984100b30 0x7f89841a46f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f898000c920 tx=0x7f898000cde0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 shutdown_connections 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f89700a3240 0x7f89700a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8984105f40 0x7f898419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8984101450 0x7f89841a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 --2- 172.21.15.103:0/170123439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8984100b30 0x7f89841a46f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.507+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 >> 172.21.15.103:0/170123439 conn(0x7f89840fc7a0 msgr2=0x7f89841038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:50.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.508+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 shutdown_connections 2024-09-19T07:37:50.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:50.508+0000 7f898b74d640 1 -- 172.21.15.103:0/170123439 wait complete. 2024-09-19T07:37:51.169 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T07:37:51.415 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:51.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:51 smithi103 ceph-mon[31201]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T07:37:51.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:51 smithi134 ceph-mon[35414]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T07:37:51.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:51 smithi174 ceph-mon[35370]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-19T07:37:51.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/3789463174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102ef0 msgr2=0x7fe4c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/3789463174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102ef0 0x7fe4c810da40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ca0 tx=0x7fe4c002a950 comp rx=0 tx=0).stop 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/3789463174 shutdown_connections 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/3789463174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102ef0 0x7fe4c810da40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/3789463174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c8102570 0x7fe4c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/3789463174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c8100e20 0x7fe4c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/3789463174 >> 172.21.15.103:0/3789463174 conn(0x7fe4c80fc7d0 msgr2=0x7fe4c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/3789463174 shutdown_connections 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/3789463174 wait complete. 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.895+0000 7fe4cf1ea640 1 Processor -- start 2024-09-19T07:37:51.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cf1ea640 1 -- start start 2024-09-19T07:37:51.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cf1ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c8100e20 0x7fe4c8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cf1ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102570 0x7fe4c8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cf1ea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c8102ef0 0x7fe4c81a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cf1ea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8110fc0 con 0x7fe4c8102570 2024-09-19T07:37:51.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cf1ea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8110e40 con 0x7fe4c8100e20 2024-09-19T07:37:51.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cf1ea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8111140 con 0x7fe4c8102ef0 2024-09-19T07:37:51.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102570 0x7fe4c8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102570 0x7fe4c8199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33364/0 (socket says 172.21.15.103:33364) 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2630291054 learned_addr learned my addr 172.21.15.103:0/2630291054 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c8102ef0 0x7fe4c81a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c8100e20 0x7fe4c8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2630291054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c8102ef0 msgr2=0x7fe4c81a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c8102ef0 0x7fe4c81a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2630291054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c8100e20 msgr2=0x7fe4c8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c8100e20 0x7fe4c8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2630291054 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c81a48f0 con 0x7fe4c8102570 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c8100e20 0x7fe4c8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102570 0x7fe4c8199870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400b4a0 tx=0x7fe4b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.896+0000 7fe4c5ffb640 1 -- 172.21.15.103:0/2630291054 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4014070 con 0x7fe4c8102570 2024-09-19T07:37:51.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.897+0000 7fe4c5ffb640 1 -- 172.21.15.103:0/2630291054 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b4010af0 con 0x7fe4c8102570 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.897+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c81a4ac0 con 0x7fe4c8102570 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.897+0000 7fe4c5ffb640 1 -- 172.21.15.103:0/2630291054 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4017d10 con 0x7fe4c8102570 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.897+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c81a5000 con 0x7fe4c8102570 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.899+0000 7fe4c5ffb640 1 -- 172.21.15.103:0/2630291054 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fe4b4017070 con 0x7fe4c8102570 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.899+0000 7fe4d15c3740 1 Processor -- start 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.899+0000 7fe4d15c3740 1 -- start start 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.899+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b318c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b327a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b32a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56418b27f090 con 0x56418b3191d0 2024-09-19T07:37:51.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56418b226a70 con 0x56418b3188b0 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56418b27b050 con 0x56418b327fc0 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4c5ffb640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4b80a3240 0x7fe4b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b327a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b327a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33366/0 (socket says 172.21.15.103:33366) 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 -- 172.21.15.103:0/2862975412 learned_addr learned my addr 172.21.15.103:0/2862975412 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4b80a3240 0x7fe4b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4c5ffb640 1 -- 172.21.15.103:0/2630291054 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe4b40cec80 con 0x7fe4c8102570 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b32a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b318c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 -- 172.21.15.103:0/2862975412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 msgr2=0x56418b32a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b32a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 -- 172.21.15.103:0/2862975412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 msgr2=0x56418b318c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b318c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 -- 172.21.15.103:0/2862975412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418b233130 con 0x56418b3191d0 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b327a80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009c50 tx=0x7fe4bc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f11d22879dd629 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4b80a3240 0x7fe4b80a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0030040 tx=0x7fe4c002fe00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4b2ffd640 1 -- 172.21.15.103:0/2862975412 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc016070 con 0x56418b3191d0 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4b2ffd640 1 -- 172.21.15.103:0/2862975412 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc011950 con 0x56418b3191d0 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.900+0000 7fe4b2ffd640 1 -- 172.21.15.103:0/2862975412 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0108e0 con 0x56418b3191d0 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2862975412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 msgr2=0x56418b327a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b327a80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009c50 tx=0x7fe4bc00e6c0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2862975412 shutdown_connections 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b32a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b327a80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2862975412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b318c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2862975412 >> 172.21.15.103:0/2862975412 conn(0x56418b314f60 msgr2=0x56418b315340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2862975412 shutdown_connections 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2862975412 wait complete. 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.901+0000 7fe4d15c3740 1 -- start start 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 1 Processor -- start 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b327870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3191d0 0x56418b324bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b325100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56418b32ab90 con 0x56418b327fc0 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56418b239f20 con 0x56418b3188b0 2024-09-19T07:37:51.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56418b32ad10 con 0x56418b3191d0 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4d15c3740 10 client.0 _ll_register_callbacks cb 0x56418b203030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b325100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b325100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33382/0 (socket says 172.21.15.103:33382) 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 -- 172.21.15.103:0/2859333124 learned_addr learned my addr 172.21.15.103:0/2859333124 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3191d0 0x56418b324bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b327870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 -- 172.21.15.103:0/2859333124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3191d0 msgr2=0x56418b324bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3191d0 0x56418b324bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 -- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 msgr2=0x56418b327870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b327870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 -- 172.21.15.103:0/2859333124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418aff02b0 con 0x56418b327fc0 2024-09-19T07:37:51.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b327870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b325100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0035000 tx=0x7fe4c002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d5a25661d3eb06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.902+0000 7fe4a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2859333124 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0034040 con 0x56418b327fc0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2859333124 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe4c0004040 con 0x56418b327fc0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4bc011950 con 0x56418b327fc0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2859333124 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4c0003e40 con 0x56418b327fc0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 10 client.15159 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56418b3364f0 con 0x56418b327fc0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2859333124 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4bc011950 con 0x56418b327fc0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2859333124 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe4c0058020 con 0x56418b327fc0 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:37:51.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 20 client.15159 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b338b50 0x56418b339fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 4 client.15159 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56418b28c600 con 0x56418b338b50 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.903+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b338b50 0x56418b339fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.904+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b338b50 0x56418b339fc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482fd125d5d9f5fb server_cookie=75d8642fcbd34700 in_seq=0 out_seq=0 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.904+0000 7fe4a7fff640 10 client.15159 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2859333124 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56418b28c600 con 0x56418b338b50 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4a7fff640 10 client.15159 handle_command_reply: tid=0 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 client.15159 shutdown 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 msgr2=0x56418b325100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b325100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0035000 tx=0x7fe4c002a950 comp rx=0 tx=0).stop 2024-09-19T07:37:51.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 shutdown_connections 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b338b50 0x56418b339fc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b325100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3191d0 0x56418b324bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2859333124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b327870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 >> 172.21.15.103:0/2859333124 conn(0x56418b314f60 msgr2=0x56418b31acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4a7fff640 0 client.15159 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 shutdown_connections 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2859333124 wait complete. 2024-09-19T07:37:51.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.905+0000 7fe4d15c3740 20 client.15159 trim_cache size 0 max 16384 2024-09-19T07:37:51.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.922+0000 7fe4d15c3740 1 Processor -- start 2024-09-19T07:37:51.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4d15c3740 1 -- start start 2024-09-19T07:37:51.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3188b0 0x56418b338480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3191d0 0x56418b3398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b3234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56418b338300 con 0x56418b327fc0 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56418b3389d0 con 0x56418b3191d0 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56418b3390a0 con 0x56418b3188b0 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b3234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b3234b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33394/0 (socket says 172.21.15.103:33394) 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 -- 172.21.15.103:0/397033022 learned_addr learned my addr 172.21.15.103:0/397033022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3191d0 0x56418b3398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3188b0 0x56418b338480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 -- 172.21.15.103:0/397033022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3188b0 msgr2=0x56418b338480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3188b0 0x56418b338480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 -- 172.21.15.103:0/397033022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3191d0 msgr2=0x56418b3398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3191d0 0x56418b3398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 -- 172.21.15.103:0/397033022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418b319000 con 0x56418b327fc0 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3191d0 0x56418b3398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.923+0000 7fe4cd787640 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b3234b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005e30 tx=0x7fe4c0003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7d83c52ab9acb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4b27fc640 1 -- 172.21.15.103:0/397033022 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002f930 con 0x56418b327fc0 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4b27fc640 1 -- 172.21.15.103:0/397033022 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4c004b070 con 0x56418b327fc0 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4b27fc640 1 -- 172.21.15.103:0/397033022 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0004700 con 0x56418b327fc0 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 -- 172.21.15.103:0/397033022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 msgr2=0x56418b3234b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b3234b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005e30 tx=0x7fe4c0003d70 comp rx=0 tx=0).stop 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 -- 172.21.15.103:0/397033022 shutdown_connections 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b327fc0 0x56418b3234b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3191d0 0x56418b3398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/397033022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b3188b0 0x56418b338480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 -- 172.21.15.103:0/397033022 >> 172.21.15.103:0/397033022 conn(0x56418b314f60 msgr2=0x56418b324b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:51.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 -- 172.21.15.103:0/397033022 shutdown_connections 2024-09-19T07:37:51.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.924+0000 7fe4d15c3740 1 -- 172.21.15.103:0/397033022 wait complete. 2024-09-19T07:37:51.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 -- start start 2024-09-19T07:37:51.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 Processor -- start 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b320e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b3213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b321910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56418b339e40 con 0x56418b3191d0 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56418b339770 con 0x56418b3188b0 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56418b33a510 con 0x56418b327fc0 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4d15c3740 10 client.0 _ll_register_callbacks cb 0x7fe4c0009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:37:51.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b3213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b3213d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33410/0 (socket says 172.21.15.103:33410) 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2670528535 learned_addr learned my addr 172.21.15.103:0/2670528535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4cd787640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b321910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b320e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2670528535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 msgr2=0x56418b321910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b321910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 msgr2=0x56418b320e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b320e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4c7fff640 1 -- 172.21.15.103:0/2670528535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418b3412c0 con 0x56418b3191d0 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.925+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b320e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4c7fff640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b3213d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4b409afa0 tx=0x7fe4b4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe7aa12c4ff44df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:51.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2670528535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b401e040 con 0x56418b3191d0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2670528535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fe4b4010ca0 con 0x56418b3191d0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c004b070 con 0x56418b3191d0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2670528535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4b40b4200 con 0x56418b3191d0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 10 client.15165 fetch_fsmap learned FSMap version 26 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56418b342620 con 0x56418b3191d0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2670528535 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4c004b070 con 0x56418b3191d0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2670528535 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe4b40bb0b0 con 0x56418b3191d0 2024-09-19T07:37:51.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 10 client.15165 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 10 client.15165 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 20 client.15165 populate_metadata read hostname 'smithi103' 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b3a73b0 0x56418b3a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 4 client.15165 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.926+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "status"}) -- 0x56418b286b10 con 0x56418b3a73b0 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b3a73b0 0x56418b3a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4a7fff640 10 client.15165 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4ccf86640 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b3a73b0 0x56418b3a60c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad6c72c2a03c2bc server_cookie=f0e3c26fae590bf5 in_seq=0 out_seq=0 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4a7fff640 1 -- 172.21.15.103:0/2670528535 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56418b286b10 con 0x56418b3a73b0 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4a7fff640 10 client.15165 handle_command_reply: tid=0 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 client.15165 shutdown 2024-09-19T07:37:51.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 msgr2=0x56418b3213d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b3213d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4b409afa0 tx=0x7fe4b4009390 comp rx=0 tx=0).stop 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 shutdown_connections 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56418b3a73b0 0x56418b3a60c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56418b327fc0 0x56418b321910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56418b3191d0 0x56418b3213d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 --2- 172.21.15.103:0/2670528535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56418b3188b0 0x56418b320e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 >> 172.21.15.103:0/2670528535 conn(0x56418b314f60 msgr2=0x56418b323470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.927+0000 7fe4a7fff640 0 client.15165 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 shutdown_connections 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4d15c3740 1 -- 172.21.15.103:0/2670528535 wait complete. 2024-09-19T07:37:51.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4d15c3740 20 client.15165 trim_cache size 0 max 16384 2024-09-19T07:37:51.937 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:37:51.937 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:37:51.937 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-19T07:37:51.937 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24494, 2024-09-19T07:37:51.937 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:37:51.937 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:37:51.937 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:37:51.938 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 103.08244899, 2024-09-19T07:37:51.938 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-19T07:37:51.938 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 84, 2024-09-19T07:37:51.938 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:37:51.938 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 113.41896617499999 2024-09-19T07:37:51.938 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:37:51.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4b80a3240 msgr2=0x7fe4b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4b80a3240 0x7fe4b80a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0030040 tx=0x7fe4c002fe00 comp rx=0 tx=0).stop 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102570 msgr2=0x7fe4c8199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102570 0x7fe4c8199870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400b4a0 tx=0x7fe4b400b960 comp rx=0 tx=0).stop 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 shutdown_connections 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fe4b80a3240 0x7fe4b80a5700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c8102ef0 0x7fe4c81a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c8102570 0x7fe4c8199870 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 --2- 172.21.15.103:0/2630291054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c8100e20 0x7fe4c8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.928+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 >> 172.21.15.103:0/2630291054 conn(0x7fe4c80fc7d0 msgr2=0x7fe4c8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.929+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 shutdown_connections 2024-09-19T07:37:51.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:51.929+0000 7fe4cf1ea640 1 -- 172.21.15.103:0/2630291054 wait complete. 2024-09-19T07:37:52.704 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T07:37:52.714 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T07:37:52.715 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T07:37:52.715 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T07:37:52.715 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T07:37:52.716 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T07:37:52.716 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:37:52.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T07:37:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 -- 172.21.15.103:0/2664310798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad681025f0 msgr2=0x7fad68108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 --2- 172.21.15.103:0/2664310798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad681025f0 0x7fad68108e80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fad58009ca0 tx=0x7fad5802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 -- 172.21.15.103:0/2664310798 shutdown_connections 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 --2- 172.21.15.103:0/2664310798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad681025f0 0x7fad68108e80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 --2- 172.21.15.103:0/2664310798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad68101c30 0x7fad681020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 --2- 172.21.15.103:0/2664310798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad68100a40 0x7fad68100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 -- 172.21.15.103:0/2664310798 >> 172.21.15.103:0/2664310798 conn(0x7fad680fc1e0 msgr2=0x7fad680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 -- 172.21.15.103:0/2664310798 shutdown_connections 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 -- 172.21.15.103:0/2664310798 wait complete. 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 Processor -- start 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.817+0000 7fad707e4640 1 -- start start 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad68100a40 0x7fad68196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad68101c30 0x7fad681971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad681025f0 0x7fad6819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810c6c0 con 0x7fad68101c30 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810c540 con 0x7fad68100a40 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad6810c840 con 0x7fad681025f0 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad68101c30 0x7fad681971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad68101c30 0x7fad681971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33424/0 (socket says 172.21.15.103:33424) 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 -- 172.21.15.103:0/3090841959 learned_addr learned my addr 172.21.15.103:0/3090841959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6e580640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad68100a40 0x7fad68196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6ed81640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad681025f0 0x7fad6819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 -- 172.21.15.103:0/3090841959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad681025f0 msgr2=0x7fad6819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad681025f0 0x7fad6819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 -- 172.21.15.103:0/3090841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad68100a40 msgr2=0x7fad68196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad68100a40 0x7fad68196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 -- 172.21.15.103:0/3090841959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6819e8c0 con 0x7fad68101c30 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6e580640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad68100a40 0x7fad68196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6ed81640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad681025f0 0x7fad6819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad6dd7f640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad68101c30 0x7fad681971c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00d770 tx=0x7fad5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad677fe640 1 -- 172.21.15.103:0/3090841959 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c017070 con 0x7fad68101c30 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad677fe640 1 -- 172.21.15.103:0/3090841959 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad5c005500 con 0x7fad68101c30 2024-09-19T07:37:52.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6819ebb0 con 0x7fad68101c30 2024-09-19T07:37:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad677fe640 1 -- 172.21.15.103:0/3090841959 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c004e00 con 0x7fad68101c30 2024-09-19T07:37:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.818+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6819f0b0 con 0x7fad68101c30 2024-09-19T07:37:52.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.819+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad68107de0 con 0x7fad68101c30 2024-09-19T07:37:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.821+0000 7fad677fe640 1 -- 172.21.15.103:0/3090841959 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fad5c004fa0 con 0x7fad68101c30 2024-09-19T07:37:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.821+0000 7fad677fe640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fad500a32f0 0x7fad500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.821+0000 7fad6e580640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fad500a32f0 0x7fad500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.821+0000 7fad677fe640 1 -- 172.21.15.103:0/3090841959 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7fad5c0cf7f0 con 0x7fad68101c30 2024-09-19T07:37:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.822+0000 7fad6e580640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fad500a32f0 0x7fad500a57b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fad60005fd0 tx=0x7fad60005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:52.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:52.824+0000 7fad677fe640 1 -- 172.21.15.103:0/3090841959 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fad5c09c510 con 0x7fad68101c30 2024-09-19T07:37:53.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.039+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fad6802da10 con 0x7fad68101c30 2024-09-19T07:37:53.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.214+0000 7fad677fe640 1 -- 172.21.15.103:0/3090841959 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fad5c09be10 con 0x7fad68101c30 2024-09-19T07:37:53.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.216+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fad500a32f0 msgr2=0x7fad500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:53.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.216+0000 7fad707e4640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fad500a32f0 0x7fad500a57b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fad60005fd0 tx=0x7fad60005ee0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.216+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad68101c30 msgr2=0x7fad681971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.216+0000 7fad707e4640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad68101c30 0x7fad681971c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00d770 tx=0x7fad5c00dc30 comp rx=0 tx=0).stop 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.216+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 shutdown_connections 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.216+0000 7fad707e4640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7fad500a32f0 0x7fad500a57b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.217+0000 7fad707e4640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad681025f0 0x7fad6819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.217+0000 7fad707e4640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad68101c30 0x7fad681971c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.217+0000 7fad707e4640 1 --2- 172.21.15.103:0/3090841959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad68100a40 0x7fad68196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.217+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 >> 172.21.15.103:0/3090841959 conn(0x7fad680fc1e0 msgr2=0x7fad68104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.217+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 shutdown_connections 2024-09-19T07:37:53.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:53.217+0000 7fad707e4640 1 -- 172.21.15.103:0/3090841959 wait complete. 2024-09-19T07:37:53.233 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T07:37:53.260 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T07:37:53.260 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T07:37:53.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:53.260 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-19T07:37:53.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:53.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-19T07:37:53.346 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2024-09-19T07:37:53.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:37:53.348 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-19T07:37:53.348 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T07:37:53.348 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-19T07:37:53.348 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-19T07:37:53.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T07:37:53.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-mon[35414]: pgmap v332: 97 pgs: 97 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T07:37:53.565 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3090841959' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T07:37:53.565 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.566 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.566 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-mgr[36786]: -- 172.21.15.134:0/3883021267 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc8d2010e0 con 0x55cc8d1df000 2024-09-19T07:37:53.566 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:53.362+0000 7efc7861d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:37:53.566 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:53.418+0000 7efc7861d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:37:53.566 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:53.564+0000 7efc7861d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:37:53.572 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-mon[35370]: pgmap v332: 97 pgs: 97 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T07:37:53.572 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3090841959' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T07:37:53.573 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.573 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.573 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-mgr[37182]: -- 172.21.15.174:0/3039035360 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfed7b70e0 con 0x55dfed794800 2024-09-19T07:37:53.573 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:53.367+0000 7f1e41aec040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:37:53.573 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:53.423+0000 7f1e41aec040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:37:53.573 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:53.570+0000 7f1e41aec040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:37:53.583 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-mon[31201]: pgmap v332: 97 pgs: 97 active+clean; 646 KiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-19T07:37:53.583 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3090841959' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T07:37:53.583 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ignoring --setuser ceph since I am not root 2024-09-19T07:37:53.583 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ignoring --setgroup ceph since I am not root 2024-09-19T07:37:53.583 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:53.368+0000 7f03f1849040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T07:37:53.583 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:53.428+0000 7f03f1849040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T07:37:53.583 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:53.583+0000 7f03f1849040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T07:37:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:53.829 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:53.629+0000 7efc7861d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:37:53.829 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:53.775+0000 7efc7861d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:37:53.884 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:53.657+0000 7f03f1849040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:37:53.884 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:53.809+0000 7f03f1849040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:37:53.915 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:53.637+0000 7f1e41aec040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T07:37:53.915 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:53.784+0000 7f1e41aec040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:37:54.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 -- 172.21.15.103:0/2853705743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20102d30 msgr2=0x7f5c20103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:54.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 --2- 172.21.15.103:0/2853705743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20102d30 0x7f5c20103110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009930 tx=0x7f5c1002a420 comp rx=0 tx=0).stop 2024-09-19T07:37:54.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 -- 172.21.15.103:0/2853705743 shutdown_connections 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 --2- 172.21.15.103:0/2853705743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c201082a0 0x7f5c2010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 --2- 172.21.15.103:0/2853705743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c201036e0 0x7f5c20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 --2- 172.21.15.103:0/2853705743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c20102d30 0x7f5c20103110 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 -- 172.21.15.103:0/2853705743 >> 172.21.15.103:0/2853705743 conn(0x7f5c200fc7d0 msgr2=0x7f5c200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 -- 172.21.15.103:0/2853705743 shutdown_connections 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.098+0000 7f5c25152640 1 -- 172.21.15.103:0/2853705743 wait complete. 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 Processor -- start 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 -- start start 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c20102d30 0x7f5c201985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c201036e0 0x7f5c20198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c201082a0 0x7f5c2019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c201120a0 con 0x7f5c201036e0 2024-09-19T07:37:54.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20111f20 con 0x7f5c20102d30 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c25152640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20112220 con 0x7f5c201082a0 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c1e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c201036e0 0x7f5c20198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c1e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c201036e0 0x7f5c20198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33468/0 (socket says 172.21.15.103:33468) 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c1e59c640 1 -- 172.21.15.103:0/263100506 learned_addr learned my addr 172.21.15.103:0/263100506 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c1ed9d640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c20102d30 0x7f5c201985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c1f59e640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c201082a0 0x7f5c2019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c1e59c640 1 -- 172.21.15.103:0/263100506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c201082a0 msgr2=0x7f5c2019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.099+0000 7f5c1e59c640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c201082a0 0x7f5c2019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c1e59c640 1 -- 172.21.15.103:0/263100506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c20102d30 msgr2=0x7f5c201985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c1e59c640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c20102d30 0x7f5c201985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c1e59c640 1 -- 172.21.15.103:0/263100506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2019d590 con 0x7f5c201036e0 2024-09-19T07:37:54.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c1ed9d640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c20102d30 0x7f5c201985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c1e59c640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c201036e0 0x7f5c20198b20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9c0 tx=0x7f5c0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c17fff640 1 -- 172.21.15.103:0/263100506 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800ccc0 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2019d880 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c17fff640 1 -- 172.21.15.103:0/263100506 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c08005500 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c17fff640 1 -- 172.21.15.103:0/263100506 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08010690 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.100+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201a5160 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.101+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2006a850 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.102+0000 7f5c17fff640 1 -- 172.21.15.103:0/263100506 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f5c08004e80 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.103+0000 7f5c17fff640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 0x7f5c0c0a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.103+0000 7f5c1ed9d640 1 -- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 msgr2=0x7f5c0c0a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.104+0000 7f5c17fff640 1 -- 172.21.15.103:0/263100506 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f5c080cff30 con 0x7f5c201036e0 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.104+0000 7f5c1ed9d640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 0x7f5c0c0a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:37:54.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.107+0000 7f5c17fff640 1 -- 172.21.15.103:0/263100506 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c08054210 con 0x7f5c201036e0 2024-09-19T07:37:54.220 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:53.966+0000 7f03f1849040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:37:54.227 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:53.925+0000 7f1e41aec040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:37:54.227 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:53.916+0000 7efc7861d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T07:37:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.305+0000 7f5c1ed9d640 1 -- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 msgr2=0x7f5c0c0a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:54.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.305+0000 7f5c1ed9d640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 0x7f5c0c0a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:37:54.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.315+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5c20103110 con 0x7f5c201036e0 2024-09-19T07:37:54.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.316+0000 7f5c17fff640 1 -- 172.21.15.103:0/263100506 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f5c20103110 con 0x7f5c201036e0 2024-09-19T07:37:54.315 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 msgr2=0x7f5c0c0a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 0x7f5c0c0a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c201036e0 msgr2=0x7f5c20198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c201036e0 0x7f5c20198b20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800e9c0 tx=0x7f5c0800ee80 comp rx=0 tx=0).stop 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 shutdown_connections 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f5c0c0a3290 0x7f5c0c0a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c201082a0 0x7f5c2019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c201036e0 0x7f5c20198b20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 --2- 172.21.15.103:0/263100506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c20102d30 0x7f5c201985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 >> 172.21.15.103:0/263100506 conn(0x7f5c200fc7d0 msgr2=0x7f5c20105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 shutdown_connections 2024-09-19T07:37:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:54.318+0000 7f5c25152640 1 -- 172.21.15.103:0/263100506 wait complete. 2024-09-19T07:37:54.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:54 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3090841959' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T07:37:54.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:54 smithi174 ceph-mon[35370]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T07:37:54.481 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:54.237+0000 7f1e41aec040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:37:54.481 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:54.297+0000 7f1e41aec040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:37:54.546 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:54 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3090841959' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T07:37:54.546 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:54 smithi103 ceph-mon[31201]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T07:37:54.547 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:54 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:54.293+0000 7f03f1849040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:37:54.547 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:54 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:54.362+0000 7f03f1849040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:37:54.547 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:54 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:54.546+0000 7f03f1849040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:37:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:54 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3090841959' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T07:37:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:54 smithi134 ceph-mon[35414]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T07:37:54.580 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:54.227+0000 7efc7861d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T07:37:54.580 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:54.287+0000 7efc7861d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T07:37:54.580 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:54.470+0000 7efc7861d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:37:54.915 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:54.480+0000 7f1e41aec040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T07:37:55.111 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-19T07:37:55.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:55.112 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-09-19T07:37:55.128 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T07:37:55.128 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-09-19T07:37:55.129 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T07:37:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-09-19T07:37:55.130 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T07:37:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-09-19T07:37:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-19T07:37:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42187sec preferred_lft 42187sec 2024-09-19T07:37:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-19T07:37:55.130 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-19T07:37:55.130 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T07:37:55.131 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:37:55.131 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T07:37:55.131 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-09-19T07:37:55.131 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-09-19T07:37:55.131 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-09-19T07:37:55.131 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T07:37:55.131 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T07:37:55.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:37:55.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:55.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:55.328 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T07:37:55.348 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:55 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/263100506' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:37:55.352 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-09-19T07:37:55.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:55.354 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-09-19T07:37:55.425 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-09-19T07:37:55.425 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T07:37:55.425 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T07:37:55.425 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T07:37:55.425 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T07:37:55.426 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:37:55.426 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T07:37:55.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T07:37:55.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T07:37:55.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T07:37:55.426 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T07:37:55.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:55 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/263100506' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:37:55.480 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:55.345+0000 7efc7861d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:37:55.480 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:55.413+0000 7efc7861d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:37:55.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:37:55.637 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:55 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:55.461+0000 7f03f1849040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:37:55.637 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:55 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:55.532+0000 7f03f1849040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:37:55.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:55 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/263100506' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T07:37:55.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:55.365+0000 7f1e41aec040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T07:37:55.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:55.434+0000 7f1e41aec040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T07:37:55.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:55.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:55.731 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-19T07:37:55.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:55.747 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-09-19T07:37:55.803 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:37:55.803 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T07:37:55.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-19T07:37:55.803 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-19T07:37:55.803 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T07:37:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:37:55.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:55.923 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-19T07:37:55.923 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:37:55.923 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T07:37:55.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-19T07:37:55.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T07:37:55.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-19T07:37:55.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-19T07:37:55.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-19T07:37:55.924 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T07:37:55.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:37:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:56.052 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T07:37:56.052 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-19T07:37:56.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-09-19T07:37:56.053 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T07:37:56.053 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-19T07:37:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:37:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:56.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:37:56.131 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /tmp/cephfs 2024-09-19T07:37:56.146 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-19T07:37:56.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T07:37:56.147 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /tmp/cephfs 2024-09-19T07:37:56.201 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T07:37:56.202 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T07:37:56.202 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/7cc5c1f1-01fb-4213-a2c6-8ec0edefbf83 2024-09-19T07:37:56.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:56.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/7cc5c1f1-01fb-4213-a2c6-8ec0edefbf83 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-09-19T07:37:56.296 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-19T07:37:56.296 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-19T07:37:56.296 INFO:teuthology.orchestra.run.smithi103.stdout:invalid new device string format 2024-09-19T07:37:56.377 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.119+0000 7efc7861d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:37:56.377 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.233+0000 7efc7861d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:37:56.401 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.142+0000 7f1e41aec040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:37:56.401 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.259+0000 7f1e41aec040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:37:56.423 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-19T07:37:56.423 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-19T07:37:56.423 INFO:teuthology.orchestra.run.smithi103.stdout:invalid new device string format 2024-09-19T07:37:56.424 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.103:3300,172.21.15.134:3300,172.21.15.174:3300" 2024-09-19T07:37:56.424 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.103:3300,172.21.15.134:3300,172.21.15.174:3300:/volumes/qa/sv_1/7cc5c1f1-01fb-4213-a2c6-8ec0edefbf83 2024-09-19T07:37:56.424 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=9fd714d2-7658-11ef-bceb-c7b262605968" will pass to kernel 2024-09-19T07:37:56.426 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T07:37:56.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:37:56.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /tmp/cephfs 2024-09-19T07:37:56.452 DEBUG:teuthology.orchestra.run.smithi103:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T07:37:56.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:37:56.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:56.551 INFO:teuthology.orchestra.run.smithi103.stdout:client15198 2024-09-19T07:37:56.551 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:37:56.552 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:37:56.560 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:56 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:56.288+0000 7f03f1849040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T07:37:56.560 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:56 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:56.409+0000 7f03f1849040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T07:37:56.568 DEBUG:teuthology.orchestra.run.smithi103:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T07:37:56.655 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.375+0000 7efc7861d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:37:56.655 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.437+0000 7efc7861d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:37:56.655 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.586+0000 7efc7861d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:37:56.655 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.653+0000 7efc7861d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:37:56.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:37:56.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:56.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.401+0000 7f1e41aec040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:37:56.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.463+0000 7f1e41aec040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:37:56.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.611+0000 7f1e41aec040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:37:56.666 INFO:teuthology.orchestra.run.smithi103.stdout:client15198 2024-09-19T07:37:56.666 DEBUG:teuthology.orchestra.run.smithi103:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T07:37:56.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T07:37:56.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:56.723 INFO:teuthology.orchestra.run.smithi103.stdout:client15198 2024-09-19T07:37:56.724 DEBUG:teuthology.orchestra.run.smithi103:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9fd714d2-7658-11ef-bceb-c7b262605968.client15198/status') 2024-09-19T07:37:56.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T07:37:56.783 INFO:teuthology.orchestra.run.smithi103.stdout:instance: client.15198 (3)192.168.0.1:0/1108023587 2024-09-19T07:37:56.784 INFO:teuthology.orchestra.run.smithi103.stdout:blocklisted: false 2024-09-19T07:37:56.784 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records in 2024-09-19T07:37:56.784 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records out 2024-09-19T07:37:56.784 INFO:teuthology.orchestra.run.smithi103.stderr:70 bytes copied, 2.3937e-05 s, 2.9 MB/s 2024-09-19T07:37:56.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T07:37:56.791 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T07:37:56.800 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T07:37:56.800 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:37:56.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-19T07:37:56.823 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:56 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:56.559+0000 7f03f1849040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T07:37:56.823 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:56 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:56.622+0000 7f03f1849040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T07:37:56.823 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:56 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:56.781+0000 7f03f1849040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T07:37:56.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/2798261064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c08100a40 msgr2=0x7f7c08100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/2798261064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c08100a40 0x7f7c08100e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009900 tx=0x7f7c0402a410 comp rx=0 tx=0).stop 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/2798261064 shutdown_connections 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/2798261064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c081025f0 0x7f7c08108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/2798261064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c08101c30 0x7f7c081020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/2798261064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c08100a40 0x7f7c08100e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/2798261064 >> 172.21.15.103:0/2798261064 conn(0x7f7c080fc1e0 msgr2=0x7f7c080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/2798261064 shutdown_connections 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.904+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/2798261064 wait complete. 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 Processor -- start 2024-09-19T07:37:56.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 -- start start 2024-09-19T07:37:56.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c08100a40 0x7f7c08196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:56.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c08101c30 0x7f7c08197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:56.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c081025f0 0x7f7c0819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:56.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810c6a0 con 0x7f7c081025f0 2024-09-19T07:37:56.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810c520 con 0x7f7c08101c30 2024-09-19T07:37:56.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0ff1f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0810c820 con 0x7f7c08100a40 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0e4bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c081025f0 0x7f7c0819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0e4bc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c081025f0 0x7f7c0819e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33500/0 (socket says 172.21.15.103:33500) 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0e4bc640 1 -- 172.21.15.103:0/3508577444 learned_addr learned my addr 172.21.15.103:0/3508577444 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0d4ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c08101c30 0x7f7c08197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.905+0000 7f7c0dcbb640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c08100a40 0x7f7c08196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0e4bc640 1 -- 172.21.15.103:0/3508577444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c08100a40 msgr2=0x7f7c08196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0e4bc640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c08100a40 0x7f7c08196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0e4bc640 1 -- 172.21.15.103:0/3508577444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c08101c30 msgr2=0x7f7c08197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0e4bc640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c08101c30 0x7f7c08197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0e4bc640 1 -- 172.21.15.103:0/3508577444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0819e900 con 0x7f7c081025f0 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0d4ba640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c08101c30 0x7f7c08197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0e4bc640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c081025f0 0x7f7c0819e260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000eb10 tx=0x7f7c0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:56.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7bfeffd640 1 -- 172.21.15.103:0/3508577444 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00004b20 con 0x7f7c081025f0 2024-09-19T07:37:56.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0819ebf0 con 0x7f7c081025f0 2024-09-19T07:37:56.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7bfeffd640 1 -- 172.21.15.103:0/3508577444 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c00004cc0 con 0x7f7c081025f0 2024-09-19T07:37:56.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7bfeffd640 1 -- 172.21.15.103:0/3508577444 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00010760 con 0x7f7c081025f0 2024-09-19T07:37:56.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.906+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0819f0c0 con 0x7f7c081025f0 2024-09-19T07:37:56.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.907+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c08107d60 con 0x7f7c081025f0 2024-09-19T07:37:56.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.908+0000 7f7bfeffd640 1 -- 172.21.15.103:0/3508577444 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f7c00010900 con 0x7f7c081025f0 2024-09-19T07:37:56.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.909+0000 7f7bfeffd640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 0x7f7bf40a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:56.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.909+0000 7f7c0dcbb640 1 -- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 msgr2=0x7f7bf40a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:56.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.910+0000 7f7bfeffd640 1 -- 172.21.15.103:0/3508577444 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f7c000d0130 con 0x7f7c081025f0 2024-09-19T07:37:56.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.910+0000 7f7c0dcbb640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 0x7f7bf40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:37:56.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:56.912+0000 7f7bfeffd640 1 -- 172.21.15.103:0/3508577444 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c00054210 con 0x7f7c081025f0 2024-09-19T07:37:56.938 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.713+0000 7efc7861d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:37:56.938 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.871+0000 7efc7861d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:37:56.938 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:56.937+0000 7efc7861d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:37:56.964 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.679+0000 7f1e41aec040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:37:56.964 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.739+0000 7f1e41aec040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:37:56.964 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.898+0000 7f1e41aec040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:37:57.083 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:56 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:56.854+0000 7f03f1849040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T07:37:57.083 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:56 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:56.919+0000 7f03f1849040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T07:37:57.083 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:57.082+0000 7f03f1849040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T07:37:57.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.111+0000 7f7c0dcbb640 1 -- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 msgr2=0x7f7bf40a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:57.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.111+0000 7f7c0dcbb640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 0x7f7bf40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T07:37:57.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.120+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f7c0802d7c0 con 0x7f7c081025f0 2024-09-19T07:37:57.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.172+0000 7f7bfeffd640 1 -- 172.21.15.103:0/3508577444 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f7c0009ce10 con 0x7f7c081025f0 2024-09-19T07:37:57.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 msgr2=0x7f7bf40a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:37:57.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 0x7f7bf40a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c081025f0 msgr2=0x7f7c0819e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c081025f0 0x7f7c0819e260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000eb10 tx=0x7f7c0000c2d0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 shutdown_connections 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f7bf40a3340 0x7f7bf40a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c081025f0 0x7f7c0819e260 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c08101c30 0x7f7c08197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 --2- 172.21.15.103:0/3508577444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c08100a40 0x7f7c08196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 >> 172.21.15.103:0/3508577444 conn(0x7f7c080fc1e0 msgr2=0x7f7c08104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 shutdown_connections 2024-09-19T07:37:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.175+0000 7f7c0ff1f640 1 -- 172.21.15.103:0/3508577444 wait complete. 2024-09-19T07:37:57.187 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T07:37:57.198 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T07:37:57.220 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:37:57.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3508577444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T07:37:57.221 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:57.003+0000 7efc7861d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:37:57.221 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:57.085+0000 7efc7861d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:37:57.221 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:57.143+0000 7efc7861d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:37:57.222 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:57.205+0000 7efc7861d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:37:57.234 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:57 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3508577444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T07:37:57.234 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:56.964+0000 7f1e41aec040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:37:57.234 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:57.030+0000 7f1e41aec040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:37:57.234 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:57.112+0000 7f1e41aec040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:37:57.234 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:57.173+0000 7f1e41aec040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:37:57.234 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:57.233+0000 7f1e41aec040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:37:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 -- 172.21.15.103:0/3461832374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 msgr2=0x7f2128100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 --2- 172.21.15.103:0/3461832374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128100e10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f211c009900 tx=0x7f211c02a410 comp rx=0 tx=0).stop 2024-09-19T07:37:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 -- 172.21.15.103:0/3461832374 shutdown_connections 2024-09-19T07:37:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 --2- 172.21.15.103:0/3461832374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21281025c0 0x7f2128108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 --2- 172.21.15.103:0/3461832374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2128101c00 0x7f2128102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 --2- 172.21.15.103:0/3461832374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128100e10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 -- 172.21.15.103:0/3461832374 >> 172.21.15.103:0/3461832374 conn(0x7f21280fc170 msgr2=0x7f21280fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 -- 172.21.15.103:0/3461832374 shutdown_connections 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.329+0000 7f212f1f6640 1 -- 172.21.15.103:0/3461832374 wait complete. 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 Processor -- start 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 -- start start 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2128101c00 0x7f212819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21281025c0 0x7f212819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f212810c690 con 0x7f2128100a10 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f212810c510 con 0x7f21281025c0 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212f1f6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f212810c810 con 0x7f2128101c00 2024-09-19T07:37:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212cf92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212cf92640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33522/0 (socket says 172.21.15.103:33522) 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212cf92640 1 -- 172.21.15.103:0/1622886876 learned_addr learned my addr 172.21.15.103:0/1622886876 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f212d793640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21281025c0 0x7f212819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.330+0000 7f2127fff640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2128101c00 0x7f212819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212cf92640 1 -- 172.21.15.103:0/1622886876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2128101c00 msgr2=0x7f212819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212cf92640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2128101c00 0x7f212819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212cf92640 1 -- 172.21.15.103:0/1622886876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21281025c0 msgr2=0x7f212819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212cf92640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21281025c0 0x7f212819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212cf92640 1 -- 172.21.15.103:0/1622886876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212819faf0 con 0x7f2128100a10 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212cf92640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128069ef0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f211c031610 tx=0x7f211c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f2125ffb640 1 -- 172.21.15.103:0/1622886876 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c03f030 con 0x7f2128100a10 2024-09-19T07:37:57.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212819fd20 con 0x7f2128100a10 2024-09-19T07:37:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f2125ffb640 1 -- 172.21.15.103:0/1622886876 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c003ce0 con 0x7f2128100a10 2024-09-19T07:37:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f2125ffb640 1 -- 172.21.15.103:0/1622886876 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c035070 con 0x7f2128100a10 2024-09-19T07:37:57.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.331+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21281a01b0 con 0x7f2128100a10 2024-09-19T07:37:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.332+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2128107ea0 con 0x7f2128100a10 2024-09-19T07:37:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.333+0000 7f2125ffb640 1 -- 172.21.15.103:0/1622886876 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f211c042030 con 0x7f2128100a10 2024-09-19T07:37:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.334+0000 7f2125ffb640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f21180a32a0 0x7f21180a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.334+0000 7f2127fff640 1 -- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f21180a32a0 msgr2=0x7f21180a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.334+0000 7f2125ffb640 1 -- 172.21.15.103:0/1622886876 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f211c0f00d0 con 0x7f2128100a10 2024-09-19T07:37:57.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.334+0000 7f2127fff640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f21180a32a0 0x7f21180a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:37:57.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.337+0000 7f2125ffb640 1 -- 172.21.15.103:0/1622886876 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f211c0bc7a0 con 0x7f2128100a10 2024-09-19T07:37:57.372 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:57 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3508577444' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-19T07:37:57.372 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:57.149+0000 7f03f1849040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T07:37:57.372 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:57.219+0000 7f03f1849040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T07:37:57.372 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:57.306+0000 7f03f1849040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T07:37:57.372 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:57.371+0000 7f03f1849040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T07:37:57.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.509+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f212802d970 con 0x7f2128100a10 2024-09-19T07:37:57.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.511+0000 7f2125ffb640 1 -- 172.21.15.103:0/1622886876 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22818 (secure 0 0 0) 0x7f211c0c0aa0 con 0x7f2128100a10 2024-09-19T07:37:57.511 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:57.512 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":84,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:36:17.882336+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f21180a32a0 msgr2=0x7f21180a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f21180a32a0 0x7f21180a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 msgr2=0x7f2128069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128069ef0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f211c031610 tx=0x7f211c0043b0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 shutdown_connections 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f21180a32a0 0x7f21180a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21281025c0 0x7f212819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2128101c00 0x7f212819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 --2- 172.21.15.103:0/1622886876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2128100a10 0x7f2128069ef0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 >> 172.21.15.103:0/1622886876 conn(0x7f21280fc170 msgr2=0x7f2128105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:57.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 shutdown_connections 2024-09-19T07:37:57.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.513+0000 7f212f1f6640 1 -- 172.21.15.103:0/1622886876 wait complete. 2024-09-19T07:37:57.525 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:31:30.307307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T07:36:04.766918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T07:36:06.565665+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:37:57.526 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:37:57.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 -- 172.21.15.103:0/1471900706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 msgr2=0x7f6c9c104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 --2- 172.21.15.103:0/1471900706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c104970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009900 tx=0x7f6c8c02a410 comp rx=0 tx=0).stop 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 -- 172.21.15.103:0/1471900706 shutdown_connections 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 --2- 172.21.15.103:0/1471900706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c1077e0 0x7f6c9c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 --2- 172.21.15.103:0/1471900706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c9c104eb0 0x7f6c9c1072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 --2- 172.21.15.103:0/1471900706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c104970 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 -- 172.21.15.103:0/1471900706 >> 172.21.15.103:0/1471900706 conn(0x7f6c9c0fc140 msgr2=0x7f6c9c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 -- 172.21.15.103:0/1471900706 shutdown_connections 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.630+0000 7f6ca0991640 1 -- 172.21.15.103:0/1471900706 wait complete. 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 Processor -- start 2024-09-19T07:37:57.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 -- start start 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c104eb0 0x7f6c9c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c9c1077e0 0x7f6c9c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c10d7c0 con 0x7f6c9c102580 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c10d640 con 0x7f6c9c104eb0 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6ca0991640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c10d940 con 0x7f6c9c1077e0 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33530/0 (socket says 172.21.15.103:33530) 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 -- 172.21.15.103:0/3045561885 learned_addr learned my addr 172.21.15.103:0/3045561885 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c99d9b640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c104eb0 0x7f6c9c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9ad9d640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c9c1077e0 0x7f6c9c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 -- 172.21.15.103:0/3045561885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c9c1077e0 msgr2=0x7f6c9c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c9c1077e0 0x7f6c9c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 -- 172.21.15.103:0/3045561885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c104eb0 msgr2=0x7f6c9c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c104eb0 0x7f6c9c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 -- 172.21.15.103:0/3045561885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c19faf0 con 0x7f6c9c102580 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.631+0000 7f6c9a59c640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c197ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009230 tx=0x7f6c8c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.632+0000 7f6c8b7fe640 1 -- 172.21.15.103:0/3045561885 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c03f030 con 0x7f6c9c102580 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.632+0000 7f6c8b7fe640 1 -- 172.21.15.103:0/3045561885 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8c003ce0 con 0x7f6c9c102580 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.632+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9c19fd80 con 0x7f6c9c102580 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.632+0000 7f6c8b7fe640 1 -- 172.21.15.103:0/3045561885 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c035070 con 0x7f6c9c102580 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.632+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9c1a01c0 con 0x7f6c9c102580 2024-09-19T07:37:57.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.633+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c9c0fdc30 con 0x7f6c9c102580 2024-09-19T07:37:57.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.635+0000 7f6c8b7fe640 1 -- 172.21.15.103:0/3045561885 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f6c8c042030 con 0x7f6c9c102580 2024-09-19T07:37:57.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.635+0000 7f6c8b7fe640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f6c840a32a0 0x7f6c840a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.635+0000 7f6c99d9b640 1 -- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f6c840a32a0 msgr2=0x7f6c840a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:57.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.635+0000 7f6c8b7fe640 1 -- 172.21.15.103:0/3045561885 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f6c8c0f02b0 con 0x7f6c9c102580 2024-09-19T07:37:57.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.635+0000 7f6c99d9b640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f6c840a32a0 0x7f6c840a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:37:57.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.638+0000 7f6c8b7fe640 1 -- 172.21.15.103:0/3045561885 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6c8c0bc980 con 0x7f6c9c102580 2024-09-19T07:37:57.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:57 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:57.432+0000 7f03f1849040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T07:37:57.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.806+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c9c062e90 con 0x7f6c9c102580 2024-09-19T07:37:57.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.806+0000 7f6c8b7fe640 1 -- 172.21.15.103:0/3045561885 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f6c8c0c0c80 con 0x7f6c9c102580 2024-09-19T07:37:57.807 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-19T07:37:57.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f6c840a32a0 msgr2=0x7f6c840a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:37:57.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f6c840a32a0 0x7f6c840a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 msgr2=0x7f6c9c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c197ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009230 tx=0x7f6c8c004360 comp rx=0 tx=0).stop 2024-09-19T07:37:57.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 shutdown_connections 2024-09-19T07:37:57.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f6c840a32a0 0x7f6c840a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c9c1077e0 0x7f6c9c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c104eb0 0x7f6c9c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 --2- 172.21.15.103:0/3045561885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c9c102580 0x7f6c9c197ed0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 >> 172.21.15.103:0/3045561885 conn(0x7f6c9c0fc140 msgr2=0x7f6c9c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:57.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 shutdown_connections 2024-09-19T07:37:57.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.808+0000 7f6ca0991640 1 -- 172.21.15.103:0/3045561885 wait complete. 2024-09-19T07:37:57.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 -- 172.21.15.103:0/3693676233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc1025b0 msgr2=0x7f99bc108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/3693676233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc1025b0 0x7f99bc108e40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009ca0 tx=0x7f99b402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 -- 172.21.15.103:0/3693676233 shutdown_connections 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/3693676233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc1025b0 0x7f99bc108e40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/3693676233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99bc101bf0 0x7f99bc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/3693676233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99bc100a00 0x7f99bc100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 -- 172.21.15.103:0/3693676233 >> 172.21.15.103:0/3693676233 conn(0x7f99bc0fc140 msgr2=0x7f99bc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 -- 172.21.15.103:0/3693676233 shutdown_connections 2024-09-19T07:37:57.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.924+0000 7f99c3c7b640 1 -- 172.21.15.103:0/3693676233 wait complete. 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 Processor -- start 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 -- start start 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc100a00 0x7f99bc197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99bc101bf0 0x7f99bc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99bc1025b0 0x7f99bc19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc10c660 con 0x7f99bc100a00 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc10c4e0 con 0x7f99bc101bf0 2024-09-19T07:37:57.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c3c7b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc10c7e0 con 0x7f99bc1025b0 2024-09-19T07:37:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc100a00 0x7f99bc197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc100a00 0x7f99bc197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33540/0 (socket says 172.21.15.103:33540) 2024-09-19T07:37:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 -- 172.21.15.103:0/2212802523 learned_addr learned my addr 172.21.15.103:0/2212802523 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c2218640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99bc1025b0 0x7f99bc19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1216640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99bc101bf0 0x7f99bc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 -- 172.21.15.103:0/2212802523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99bc1025b0 msgr2=0x7f99bc19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99bc1025b0 0x7f99bc19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 -- 172.21.15.103:0/2212802523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99bc101bf0 msgr2=0x7f99bc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99bc101bf0 0x7f99bc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.925+0000 7f99c1a17640 1 -- 172.21.15.103:0/2212802523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc19fa50 con 0x7f99bc100a00 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.926+0000 7f99c1a17640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc100a00 0x7f99bc197e30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99b800e9c0 tx=0x7f99b800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.926+0000 7f99b2ffd640 1 -- 172.21.15.103:0/2212802523 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b800cd80 con 0x7f99bc100a00 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.926+0000 7f99b2ffd640 1 -- 172.21.15.103:0/2212802523 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b80035f0 con 0x7f99bc100a00 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.926+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc19fd40 con 0x7f99bc100a00 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.926+0000 7f99b2ffd640 1 -- 172.21.15.103:0/2212802523 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8010690 con 0x7f99bc100a00 2024-09-19T07:37:57.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.926+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc1a0220 con 0x7f99bc100a00 2024-09-19T07:37:57.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.927+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99bc1a04b0 con 0x7f99bc100a00 2024-09-19T07:37:57.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.928+0000 7f99b2ffd640 1 -- 172.21.15.103:0/2212802523 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138373+0+0 (secure 0 0 0) 0x7f99b80037a0 con 0x7f99bc100a00 2024-09-19T07:37:57.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.929+0000 7f99b2ffd640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99a80a3320 0x7f99a80a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:57.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.929+0000 7f99c1216640 1 -- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99a80a3320 msgr2=0x7f99a80a57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:57.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.929+0000 7f99b2ffd640 1 -- 172.21.15.103:0/2212802523 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f99b80cfd90 con 0x7f99bc100a00 2024-09-19T07:37:57.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.929+0000 7f99c1216640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99a80a3320 0x7f99a80a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:37:57.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:57.932+0000 7f99b2ffd640 1 -- 172.21.15.103:0/2212802523 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f99b809cb00 con 0x7f99bc100a00 2024-09-19T07:37:58.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.107+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f99bc1a0840 con 0x7f99bc100a00 2024-09-19T07:37:58.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.107+0000 7f99b2ffd640 1 -- 172.21.15.103:0/2212802523 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f99b809c400 con 0x7f99bc100a00 2024-09-19T07:37:58.108 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-19T07:37:58.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99a80a3320 msgr2=0x7f99a80a57e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99a80a3320 0x7f99a80a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc100a00 msgr2=0x7f99bc197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc100a00 0x7f99bc197e30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99b800e9c0 tx=0x7f99b800ee80 comp rx=0 tx=0).stop 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 shutdown_connections 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f99a80a3320 0x7f99a80a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99bc1025b0 0x7f99bc19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99bc101bf0 0x7f99bc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 --2- 172.21.15.103:0/2212802523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99bc100a00 0x7f99bc197e30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 >> 172.21.15.103:0/2212802523 conn(0x7f99bc0fc140 msgr2=0x7f99bc069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.109+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 shutdown_connections 2024-09-19T07:37:58.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.110+0000 7f99c3c7b640 1 -- 172.21.15.103:0/2212802523 wait complete. 2024-09-19T07:37:58.121 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T07:37:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 -- 172.21.15.103:0/1806075018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be81025e0 msgr2=0x7f2be8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 --2- 172.21.15.103:0/1806075018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be81025e0 0x7f2be8108e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009ca0 tx=0x7f2be002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 -- 172.21.15.103:0/1806075018 shutdown_connections 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 --2- 172.21.15.103:0/1806075018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be81025e0 0x7f2be8108e70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 --2- 172.21.15.103:0/1806075018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be8101c20 0x7f2be81020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 --2- 172.21.15.103:0/1806075018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be8100a30 0x7f2be8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 -- 172.21.15.103:0/1806075018 >> 172.21.15.103:0/1806075018 conn(0x7f2be80fc1b0 msgr2=0x7f2be80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 -- 172.21.15.103:0/1806075018 shutdown_connections 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.229+0000 7f2bef759640 1 -- 172.21.15.103:0/1806075018 wait complete. 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 Processor -- start 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 -- start start 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be8100a30 0x7f2be8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be8101c20 0x7f2be81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be81025e0 0x7f2be819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810c6b0 con 0x7f2be8100a30 2024-09-19T07:37:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810c530 con 0x7f2be81025e0 2024-09-19T07:37:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bef759640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810c830 con 0x7f2be8101c20 2024-09-19T07:37:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be8100a30 0x7f2be8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be8100a30 0x7f2be8197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45556/0 (socket says 172.21.15.103:45556) 2024-09-19T07:37:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 -- 172.21.15.103:0/2804218351 learned_addr learned my addr 172.21.15.103:0/2804218351 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bedcf6640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be81025e0 0x7f2be819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2beccf4640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be8101c20 0x7f2be81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 -- 172.21.15.103:0/2804218351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be8101c20 msgr2=0x7f2be81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be8101c20 0x7f2be81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 -- 172.21.15.103:0/2804218351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be81025e0 msgr2=0x7f2be819f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be81025e0 0x7f2be819f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.230+0000 7f2bed4f5640 1 -- 172.21.15.103:0/2804218351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be819faa0 con 0x7f2be8100a30 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.231+0000 7f2bed4f5640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be8100a30 0x7f2be8197e80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e9c0 tx=0x7f2be400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.231+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be400cd80 con 0x7f2be8100a30 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.231+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be40035f0 con 0x7f2be8100a30 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.231+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4010690 con 0x7f2be8100a30 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.231+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be819fd90 con 0x7f2be8100a30 2024-09-19T07:37:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.231+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a0250 con 0x7f2be8100a30 2024-09-19T07:37:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.232+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be81a04e0 con 0x7f2be8100a30 2024-09-19T07:37:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.233+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138373+0+0 (secure 0 0 0) 0x7f2be40037a0 con 0x7f2be8100a30 2024-09-19T07:37:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.234+0000 7f2bde7fc640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2bd40a3320 0x7f2bd40a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.234+0000 7f2beccf4640 1 -- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2bd40a3320 msgr2=0x7f2bd40a57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.103:6800/2164654927 2024-09-19T07:37:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.234+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8240+0+0 (secure 0 0 0) 0x7f2be40cfd90 con 0x7f2be8100a30 2024-09-19T07:37:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.234+0000 7f2beccf4640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2bd40a3320 0x7f2bd40a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T07:37:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.238+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2be409cab0 con 0x7f2be8100a30 2024-09-19T07:37:58.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.283+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f2be40d6080 con 0x7f2be8100a30 2024-09-19T07:37:58.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.283+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2bd40a3320 msgr2=0x7f2bd40a57e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:37:58.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.283+0000 7f2bde7fc640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2bd40a3320 0x7f2bd40a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.329 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:37:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y[36782]: 2024-09-19T07:37:57.994+0000 7efc7861d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:37:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3508577444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T07:37:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:37:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1622886876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3045561885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: Standby manager daemon y restarted 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: Standby manager daemon y started 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: Standby manager daemon z restarted 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: Standby manager daemon z started 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:37:58.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:58 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2212802523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:37:58.415 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z[37178]: 2024-09-19T07:37:58.024+0000 7f1e41aec040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:37:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3508577444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T07:37:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:37:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1622886876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3045561885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:37:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: Standby manager daemon y restarted 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: Standby manager daemon y started 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: Standby manager daemon z restarted 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: Standby manager daemon z started 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:37:58.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:58 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2212802523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:37:58.470 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.268+0000 7f03f1849040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3508577444' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1622886876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3045561885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: Standby manager daemon y restarted 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: Standby manager daemon y started 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.134:0/2164812157' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: Standby manager daemon z restarted 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: Standby manager daemon z started 2024-09-19T07:37:58.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T07:37:58.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T07:37:58.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T07:37:58.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='mgr.? 172.21.15.174:0/1667358457' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T07:37:58.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2212802523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:37:58.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.486+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2be81a06b0 con 0x7f2be8100a30 2024-09-19T07:37:58.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.487+0000 7f2bde7fc640 1 -- 172.21.15.103:0/2804218351 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f2be406b050 con 0x7f2be8100a30 2024-09-19T07:37:58.488 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-19T07:37:58.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be8100a30 msgr2=0x7f2be8197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be8100a30 0x7f2be8197e80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e9c0 tx=0x7f2be400ee80 comp rx=0 tx=0).stop 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 shutdown_connections 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:6800/2164654927,v1:172.21.15.103:6801/2164654927] conn(0x7f2bd40a3320 0x7f2bd40a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2be81025e0 0x7f2be819f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2be8101c20 0x7f2be81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 --2- 172.21.15.103:0/2804218351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be8100a30 0x7f2be8197e80 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2be400e9c0 tx=0x7f2be400ee80 comp rx=0 tx=0).stop 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.490+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 >> 172.21.15.103:0/2804218351 conn(0x7f2be80fc1b0 msgr2=0x7f2be8069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.491+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 shutdown_connections 2024-09-19T07:37:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:58.491+0000 7f2bef759640 1 -- 172.21.15.103:0/2804218351 wait complete. 2024-09-19T07:37:58.507 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T07:37:58.508 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:37:58.787 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:37:58.916 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035919b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035919b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035919b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035919b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035919b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035c1a1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035c1a1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035c1a1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035c1a1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:58.917 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:58 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:58.559+0000 7f035c1a1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 -- 172.21.15.103:0/3893928527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 msgr2=0x7f19e810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 --2- 172.21.15.103:0/3893928527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e810ea80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009ca0 tx=0x7f19e002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 -- 172.21.15.103:0/3893928527 shutdown_connections 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 --2- 172.21.15.103:0/3893928527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e810ea80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 --2- 172.21.15.103:0/3893928527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e8102780 0x7f19e81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 --2- 172.21.15.103:0/3893928527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19e8101dd0 0x7f19e81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 -- 172.21.15.103:0/3893928527 >> 172.21.15.103:0/3893928527 conn(0x7f19e80fd8b0 msgr2=0x7f19e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:59.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 -- 172.21.15.103:0/3893928527 shutdown_connections 2024-09-19T07:37:59.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.334+0000 7f19f0673640 1 -- 172.21.15.103:0/3893928527 wait complete. 2024-09-19T07:37:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 Processor -- start 2024-09-19T07:37:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 -- start start 2024-09-19T07:37:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19e8101dd0 0x7f19e8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e8102780 0x7f19e81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:59.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8111f60 con 0x7f19e8106f00 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8111de0 con 0x7f19e8102780 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19f0673640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19e81120e0 con 0x7f19e8101dd0 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.335+0000 7f19eec10640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45690/0 (socket says 172.21.15.103:45690) 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 -- 172.21.15.103:0/2536183079 learned_addr learned my addr 172.21.15.103:0/2536183079 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19edc0e640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e8102780 0x7f19e81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19ee40f640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19e8101dd0 0x7f19e8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 -- 172.21.15.103:0/2536183079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19e8101dd0 msgr2=0x7f19e8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19e8101dd0 0x7f19e8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 -- 172.21.15.103:0/2536183079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e8102780 msgr2=0x7f19e81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:59.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e8102780 0x7f19e81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 -- 172.21.15.103:0/2536183079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e819d330 con 0x7f19e8106f00 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19edc0e640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e8102780 0x7f19e81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19eec10640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e819cc50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f19e0031a20 tx=0x7f19e0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19df7fe640 1 -- 172.21.15.103:0/2536183079 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0003c30 con 0x7f19e8106f00 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e819d5c0 con 0x7f19e8106f00 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19df7fe640 1 -- 172.21.15.103:0/2536183079 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e0003dd0 con 0x7f19e8106f00 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e81a4f00 con 0x7f19e8106f00 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.336+0000 7f19df7fe640 1 -- 172.21.15.103:0/2536183079 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e002fcf0 con 0x7f19e8106f00 2024-09-19T07:37:59.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.337+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e81021b0 con 0x7f19e8106f00 2024-09-19T07:37:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.339+0000 7f19df7fe640 1 -- 172.21.15.103:0/2536183079 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f19e0004050 con 0x7f19e8106f00 2024-09-19T07:37:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.339+0000 7f19df7fe640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19d40a3030 0x7f19d40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:37:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.339+0000 7f19ee40f640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19d40a3030 0x7f19d40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:37:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.339+0000 7f19df7fe640 1 -- 172.21.15.103:0/2536183079 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f19e00f0c80 con 0x7f19e8106f00 2024-09-19T07:37:59.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.340+0000 7f19ee40f640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19d40a3030 0x7f19d40a54f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f19e4005fd0 tx=0x7f19e400b590 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:37:59.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.347+0000 7f19df7fe640 1 -- 172.21.15.103:0/2536183079 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f19e00bdc10 con 0x7f19e8106f00 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: Active manager daemon x restarted 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: Activating manager daemon x 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: mgrmap e21: x(active, starting, since 0.0141949s), standbys: y, z 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:37:59.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:37:59.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:37:59.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: Manager daemon x is now available 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:37:59.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2804218351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:37:59.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-19T07:37:59.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: Active manager daemon x restarted 2024-09-19T07:37:59.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: Activating manager daemon x 2024-09-19T07:37:59.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T07:37:59.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: mgrmap e21: x(active, starting, since 0.0141949s), standbys: y, z 2024-09-19T07:37:59.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:37:59.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:37:59.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: Manager daemon x is now available 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:37:59.478 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:37:59.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:37:59.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:37:59.479 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:37:59 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2804218351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:37:59.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-19T07:37:59.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: Active manager daemon x restarted 2024-09-19T07:37:59.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: Activating manager daemon x 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: mgrmap e21: x(active, starting, since 0.0141949s), standbys: y, z 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:37:59.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T07:37:59.481 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: Manager daemon x is now available 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:37:59.482 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:37:59.483 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T07:37:59.483 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:37:59.483 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T07:37:59.483 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:37:59 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2804218351' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:37:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.561+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19e810a0e0 con 0x7f19e8106f00 2024-09-19T07:37:59.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.561+0000 7f19df7fe640 1 -- 172.21.15.103:0/2536183079 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f19e002fe90 con 0x7f19e8106f00 2024-09-19T07:37:59.563 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:37:59.563 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T07:37:57:172446+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:37:57.122252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:37:59.563 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:37:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19d40a3030 msgr2=0x7f19d40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19d40a3030 0x7f19d40a54f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f19e4005fd0 tx=0x7f19e400b590 comp rx=0 tx=0).stop 2024-09-19T07:37:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 msgr2=0x7f19e819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:37:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e819cc50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f19e0031a20 tx=0x7f19e0031a50 comp rx=0 tx=0).stop 2024-09-19T07:37:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 shutdown_connections 2024-09-19T07:37:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19d40a3030 0x7f19d40a54f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e8106f00 0x7f19e819cc50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e8102780 0x7f19e81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 --2- 172.21.15.103:0/2536183079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19e8101dd0 0x7f19e8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:37:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 >> 172.21.15.103:0/2536183079 conn(0x7f19e80fd8b0 msgr2=0x7f19e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:37:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 shutdown_connections 2024-09-19T07:37:59.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:37:59.564+0000 7f19f0673640 1 -- 172.21.15.103:0/2536183079 wait complete. 2024-09-19T07:38:00.204 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:37:59.873+0000 7f033f869640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:07:37:59] ENGINE Error in HTTPServer.serve 2024-09-19T07:38:00.204 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: Traceback (most recent call last): 2024-09-19T07:38:00.204 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:38:00.204 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._connections.run(self.expiration_interval) 2024-09-19T07:38:00.204 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._run(expiration_interval) 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: s = self.context.wrap_socket( 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: return self.sslsocket_class._create( 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self.do_handshake() 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:38:00.205 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: self._sslobj.do_handshake() 2024-09-19T07:38:00.206 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:38:00.206 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:37:59 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:00 smithi103 ceph-mon[31201]: mgrmap e22: x(active, since 1.0194s), standbys: y, z 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:00 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2536183079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:00 smithi103 ceph-mon[31201]: [19/Sep/2024:07:37:59] ENGINE Bus STARTING 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:00 smithi103 ceph-mon[31201]: [19/Sep/2024:07:37:59] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:00 smithi103 ceph-mon[31201]: [19/Sep/2024:07:37:59] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:00 smithi103 ceph-mon[31201]: [19/Sep/2024:07:37:59] ENGINE Bus STARTED 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:00 smithi103 ceph-mon[31201]: [19/Sep/2024:07:37:59] ENGINE Error in HTTPServer.serve 2024-09-19T07:38:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:38:00.471 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-19T07:38:00.472 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:38:00.472 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-19T07:38:00.472 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:38:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:00 smithi134 ceph-mon[35414]: mgrmap e22: x(active, since 1.0194s), standbys: y, z 2024-09-19T07:38:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:00 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2536183079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:00 smithi134 ceph-mon[35414]: [19/Sep/2024:07:37:59] ENGINE Bus STARTING 2024-09-19T07:38:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:00 smithi134 ceph-mon[35414]: [19/Sep/2024:07:37:59] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T07:38:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:00 smithi134 ceph-mon[35414]: [19/Sep/2024:07:37:59] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:00 smithi134 ceph-mon[35414]: [19/Sep/2024:07:37:59] ENGINE Bus STARTED 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:00 smithi134 ceph-mon[35414]: [19/Sep/2024:07:37:59] ENGINE Error in HTTPServer.serve 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: Traceback (most recent call last): 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._connections.run(self.expiration_interval) 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._run(expiration_interval) 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:38:00.580 INFO:journalctl@ceph.mon.b.smithi134.stdout: s = self.context.wrap_socket( 2024-09-19T07:38:00.581 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:38:00.581 INFO:journalctl@ceph.mon.b.smithi134.stdout: return self.sslsocket_class._create( 2024-09-19T07:38:00.581 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:38:00.581 INFO:journalctl@ceph.mon.b.smithi134.stdout: self.do_handshake() 2024-09-19T07:38:00.582 INFO:journalctl@ceph.mon.b.smithi134.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:38:00.582 INFO:journalctl@ceph.mon.b.smithi134.stdout: self._sslobj.do_handshake() 2024-09-19T07:38:00.582 INFO:journalctl@ceph.mon.b.smithi134.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:38:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:00 smithi174 ceph-mon[35370]: mgrmap e22: x(active, since 1.0194s), standbys: y, z 2024-09-19T07:38:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:00 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2536183079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:00 smithi174 ceph-mon[35370]: [19/Sep/2024:07:37:59] ENGINE Bus STARTING 2024-09-19T07:38:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:00 smithi174 ceph-mon[35370]: [19/Sep/2024:07:37:59] ENGINE Serving on http://172.21.15.103:8765 2024-09-19T07:38:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:00 smithi174 ceph-mon[35370]: [19/Sep/2024:07:37:59] ENGINE Serving on https://172.21.15.103:7150 2024-09-19T07:38:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:00 smithi174 ceph-mon[35370]: [19/Sep/2024:07:37:59] ENGINE Bus STARTED 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:00 smithi174 ceph-mon[35370]: [19/Sep/2024:07:37:59] ENGINE Error in HTTPServer.serve 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: Traceback (most recent call last): 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._connections.run(self.expiration_interval) 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._run(expiration_interval) 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: s = self.context.wrap_socket( 2024-09-19T07:38:00.666 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T07:38:00.667 INFO:journalctl@ceph.mon.c.smithi174.stdout: return self.sslsocket_class._create( 2024-09-19T07:38:00.667 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T07:38:00.667 INFO:journalctl@ceph.mon.c.smithi174.stdout: self.do_handshake() 2024-09-19T07:38:00.667 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T07:38:00.667 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._sslobj.do_handshake() 2024-09-19T07:38:00.667 INFO:journalctl@ceph.mon.c.smithi174.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T07:38:00.975 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T07:38:00.975 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T07:38:00.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:38:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 -- 172.21.15.103:0/1584616334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924102630 msgr2=0x7f2924104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 --2- 172.21.15.103:0/1584616334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924102630 0x7f2924104a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2914009900 tx=0x7f291402a410 comp rx=0 tx=0).stop 2024-09-19T07:38:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 -- 172.21.15.103:0/1584616334 shutdown_connections 2024-09-19T07:38:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 --2- 172.21.15.103:0/1584616334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2924107890 0x7f2924109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 --2- 172.21.15.103:0/1584616334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2924104f60 0x7f2924107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 --2- 172.21.15.103:0/1584616334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924102630 0x7f2924104a20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 -- 172.21.15.103:0/1584616334 >> 172.21.15.103:0/1584616334 conn(0x7f29240fc170 msgr2=0x7f29240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 -- 172.21.15.103:0/1584616334 shutdown_connections 2024-09-19T07:38:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 -- 172.21.15.103:0/1584616334 wait complete. 2024-09-19T07:38:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 Processor -- start 2024-09-19T07:38:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.094+0000 7f292a88c640 1 -- start start 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f292a88c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2924102630 0x7f2924197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f292a88c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2924104f60 0x7f2924198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f292a88c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924107890 0x7f292419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f292a88c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f292410d7b0 con 0x7f2924107890 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f292a88c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f292410d630 con 0x7f2924102630 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f292a88c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f292410d930 con 0x7f2924104f60 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924107890 0x7f292419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924107890 0x7f292419f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45740/0 (socket says 172.21.15.103:45740) 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 -- 172.21.15.103:0/3345446167 learned_addr learned my addr 172.21.15.103:0/3345446167 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2923fff640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2924102630 0x7f2924197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f29237fe640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2924104f60 0x7f2924198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 -- 172.21.15.103:0/3345446167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2924104f60 msgr2=0x7f2924198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2924104f60 0x7f2924198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 -- 172.21.15.103:0/3345446167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2924102630 msgr2=0x7f2924197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2924102630 0x7f2924197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.095+0000 7f2928e29640 1 -- 172.21.15.103:0/3345446167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292419fa50 con 0x7f2924107890 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.096+0000 7f2928e29640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924107890 0x7f292419f3e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f291c00ccb0 tx=0x7f291c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.096+0000 7f29217fa640 1 -- 172.21.15.103:0/3345446167 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c003970 con 0x7f2924107890 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.096+0000 7f29217fa640 1 -- 172.21.15.103:0/3345446167 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f291c004030 con 0x7f2924107890 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.096+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292419fd40 con 0x7f2924107890 2024-09-19T07:38:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.096+0000 7f29217fa640 1 -- 172.21.15.103:0/3345446167 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c00f720 con 0x7f2924107890 2024-09-19T07:38:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.096+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a01e0 con 0x7f2924107890 2024-09-19T07:38:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.097+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29240fdc60 con 0x7f2924107890 2024-09-19T07:38:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.098+0000 7f29217fa640 1 -- 172.21.15.103:0/3345446167 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7f291c0041e0 con 0x7f2924107890 2024-09-19T07:38:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.099+0000 7f29217fa640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29100a33c0 0x7f29100a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.099+0000 7f29217fa640 1 -- 172.21.15.103:0/3345446167 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f291c0cfce0 con 0x7f2924107890 2024-09-19T07:38:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.099+0000 7f2923fff640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29100a33c0 0x7f29100a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.100+0000 7f2923fff640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29100a33c0 0x7f29100a5880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29140314e0 tx=0x7f2914000bc0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:01.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.103+0000 7f29217fa640 1 -- 172.21.15.103:0/3345446167 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f291c014070 con 0x7f2924107890 2024-09-19T07:38:01.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.288+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2924062630 con 0x7f29100a33c0 2024-09-19T07:38:01.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.296+0000 7f29217fa640 1 -- 172.21.15.103:0/3345446167 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258076 (secure 0 0 0) 0x7f2924062630 con 0x7f29100a33c0 2024-09-19T07:38:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:01.301 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29100a33c0 msgr2=0x7f29100a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29100a33c0 0x7f29100a5880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29140314e0 tx=0x7f2914000bc0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924107890 msgr2=0x7f292419f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924107890 0x7f292419f3e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f291c00ccb0 tx=0x7f291c00a620 comp rx=0 tx=0).stop 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 shutdown_connections 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29100a33c0 0x7f29100a5880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2924107890 0x7f292419f3e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2924104f60 0x7f2924198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 --2- 172.21.15.103:0/3345446167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2924102630 0x7f2924197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 >> 172.21.15.103:0/3345446167 conn(0x7f29240fc170 msgr2=0x7f2924102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:01.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 shutdown_connections 2024-09-19T07:38:01.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.300+0000 7f292a88c640 1 -- 172.21.15.103:0/3345446167 wait complete. 2024-09-19T07:38:01.313 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-19T07:38:00.293143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193254,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1278,"num_read_kb":809,"num_write":559,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344136,"kb_used_data":11496,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597688,"statfs":{"total":1151940427776,"available":1151588032512,"internally_reserved":0,"allocated":11771904,"data_stored":8168242,"data_compressed":248501,"data_compressed_allocated":335872,"data_compressed_original":2834432,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.993569"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566027+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:37:58.566027+0000","last_peered":"2024-09-19T07:37:58.566027+0000","last_clean":"2024-09-19T07:37:58.566027+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:37:58.566027+0000","last_undegraded":"2024-09-19T07:37:58.566027+0000","last_fullsized":"2024-09-19T07:37:58.566027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566607+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:37:58.566607+0000","last_peered":"2024-09-19T07:37:58.566607+0000","last_clean":"2024-09-19T07:37:58.566607+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:37:58.566607+0000","last_undegraded":"2024-09-19T07:37:58.566607+0000","last_fullsized":"2024-09-19T07:37:58.566607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566643+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:37:58.566643+0000","last_peered":"2024-09-19T07:37:58.566643+0000","last_clean":"2024-09-19T07:37:58.566643+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:37:58.566643+0000","last_undegraded":"2024-09-19T07:37:58.566643+0000","last_fullsized":"2024-09-19T07:37:58.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395953+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:37:58.395953+0000","last_peered":"2024-09-19T07:37:58.395953+0000","last_clean":"2024-09-19T07:37:58.395953+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:37:58.395953+0000","last_undegraded":"2024-09-19T07:37:58.395953+0000","last_fullsized":"2024-09-19T07:37:58.395953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398077+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:37:58.398077+0000","last_peered":"2024-09-19T07:37:58.398077+0000","last_clean":"2024-09-19T07:37:58.398077+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:37:58.398077+0000","last_undegraded":"2024-09-19T07:37:58.398077+0000","last_fullsized":"2024-09-19T07:37:58.398077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408847+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:37:58.408847+0000","last_peered":"2024-09-19T07:37:58.408847+0000","last_clean":"2024-09-19T07:37:58.408847+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:37:58.408847+0000","last_undegraded":"2024-09-19T07:37:58.408847+0000","last_fullsized":"2024-09-19T07:37:58.408847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567869+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:37:58.567869+0000","last_peered":"2024-09-19T07:37:58.567869+0000","last_clean":"2024-09-19T07:37:58.567869+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:37:58.567869+0000","last_undegraded":"2024-09-19T07:37:58.567869+0000","last_fullsized":"2024-09-19T07:37:58.567869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398332+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:37:58.398332+0000","last_peered":"2024-09-19T07:37:58.398332+0000","last_clean":"2024-09-19T07:37:58.398332+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:37:58.398332+0000","last_undegraded":"2024-09-19T07:37:58.398332+0000","last_fullsized":"2024-09-19T07:37:58.398332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278587+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:37:58.278587+0000","last_peered":"2024-09-19T07:37:58.278587+0000","last_clean":"2024-09-19T07:37:58.278587+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:37:58.278587+0000","last_undegraded":"2024-09-19T07:37:58.278587+0000","last_fullsized":"2024-09-19T07:37:58.278587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409236+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:37:58.409236+0000","last_peered":"2024-09-19T07:37:58.409236+0000","last_clean":"2024-09-19T07:37:58.409236+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:37:58.409236+0000","last_undegraded":"2024-09-19T07:37:58.409236+0000","last_fullsized":"2024-09-19T07:37:58.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568581+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:37:58.568581+0000","last_peered":"2024-09-19T07:37:58.568581+0000","last_clean":"2024-09-19T07:37:58.568581+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:37:58.568581+0000","last_undegraded":"2024-09-19T07:37:58.568581+0000","last_fullsized":"2024-09-19T07:37:58.568581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566381+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:37:58.566381+0000","last_peered":"2024-09-19T07:37:58.566381+0000","last_clean":"2024-09-19T07:37:58.566381+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:37:58.566381+0000","last_undegraded":"2024-09-19T07:37:58.566381+0000","last_fullsized":"2024-09-19T07:37:58.566381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278860+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:37:58.278860+0000","last_peered":"2024-09-19T07:37:58.278860+0000","last_clean":"2024-09-19T07:37:58.278860+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:37:58.278860+0000","last_undegraded":"2024-09-19T07:37:58.278860+0000","last_fullsized":"2024-09-19T07:37:58.278860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566247+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:37:58.566247+0000","last_peered":"2024-09-19T07:37:58.566247+0000","last_clean":"2024-09-19T07:37:58.566247+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:37:58.566247+0000","last_undegraded":"2024-09-19T07:37:58.566247+0000","last_fullsized":"2024-09-19T07:37:58.566247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566034+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:37:58.566034+0000","last_peered":"2024-09-19T07:37:58.566034+0000","last_clean":"2024-09-19T07:37:58.566034+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:37:58.566034+0000","last_undegraded":"2024-09-19T07:37:58.566034+0000","last_fullsized":"2024-09-19T07:37:58.566034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277343+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:37:58.277343+0000","last_peered":"2024-09-19T07:37:58.277343+0000","last_clean":"2024-09-19T07:37:58.277343+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:37:58.277343+0000","last_undegraded":"2024-09-19T07:37:58.277343+0000","last_fullsized":"2024-09-19T07:37:58.277343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566048+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:37:58.566048+0000","last_peered":"2024-09-19T07:37:58.566048+0000","last_clean":"2024-09-19T07:37:58.566048+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:37:58.566048+0000","last_undegraded":"2024-09-19T07:37:58.566048+0000","last_fullsized":"2024-09-19T07:37:58.566048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395689+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:37:58.395689+0000","last_peered":"2024-09-19T07:37:58.395689+0000","last_clean":"2024-09-19T07:37:58.395689+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:37:58.395689+0000","last_undegraded":"2024-09-19T07:37:58.395689+0000","last_fullsized":"2024-09-19T07:37:58.395689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281089+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:37:58.281089+0000","last_peered":"2024-09-19T07:37:58.281089+0000","last_clean":"2024-09-19T07:37:58.281089+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:37:58.281089+0000","last_undegraded":"2024-09-19T07:37:58.281089+0000","last_fullsized":"2024-09-19T07:37:58.281089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565798+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:37:58.565798+0000","last_peered":"2024-09-19T07:37:58.565798+0000","last_clean":"2024-09-19T07:37:58.565798+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:37:58.565798+0000","last_undegraded":"2024-09-19T07:37:58.565798+0000","last_fullsized":"2024-09-19T07:37:58.565798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278866+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:37:58.278866+0000","last_peered":"2024-09-19T07:37:58.278866+0000","last_clean":"2024-09-19T07:37:58.278866+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:37:58.278866+0000","last_undegraded":"2024-09-19T07:37:58.278866+0000","last_fullsized":"2024-09-19T07:37:58.278866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277508+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:37:58.277508+0000","last_peered":"2024-09-19T07:37:58.277508+0000","last_clean":"2024-09-19T07:37:58.277508+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:37:58.277508+0000","last_undegraded":"2024-09-19T07:37:58.277508+0000","last_fullsized":"2024-09-19T07:37:58.277508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398644+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:37:58.398644+0000","last_peered":"2024-09-19T07:37:58.398644+0000","last_clean":"2024-09-19T07:37:58.398644+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:37:58.398644+0000","last_undegraded":"2024-09-19T07:37:58.398644+0000","last_fullsized":"2024-09-19T07:37:58.398644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405691+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:37:58.405691+0000","last_peered":"2024-09-19T07:37:58.405691+0000","last_clean":"2024-09-19T07:37:58.405691+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:37:58.405691+0000","last_undegraded":"2024-09-19T07:37:58.405691+0000","last_fullsized":"2024-09-19T07:37:58.405691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399976+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:37:58.399976+0000","last_peered":"2024-09-19T07:37:58.399976+0000","last_clean":"2024-09-19T07:37:58.399976+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:37:58.399976+0000","last_undegraded":"2024-09-19T07:37:58.399976+0000","last_fullsized":"2024-09-19T07:37:58.399976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281203+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:37:58.281203+0000","last_peered":"2024-09-19T07:37:58.281203+0000","last_clean":"2024-09-19T07:37:58.281203+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:37:58.281203+0000","last_undegraded":"2024-09-19T07:37:58.281203+0000","last_fullsized":"2024-09-19T07:37:58.281203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565955+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:37:58.565955+0000","last_peered":"2024-09-19T07:37:58.565955+0000","last_clean":"2024-09-19T07:37:58.565955+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:37:58.565955+0000","last_undegraded":"2024-09-19T07:37:58.565955+0000","last_fullsized":"2024-09-19T07:37:58.565955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278990+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:37:58.278990+0000","last_peered":"2024-09-19T07:37:58.278990+0000","last_clean":"2024-09-19T07:37:58.278990+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:37:58.278990+0000","last_undegraded":"2024-09-19T07:37:58.278990+0000","last_fullsized":"2024-09-19T07:37:58.278990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566392+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:37:58.566392+0000","last_peered":"2024-09-19T07:37:58.566392+0000","last_clean":"2024-09-19T07:37:58.566392+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:37:58.566392+0000","last_undegraded":"2024-09-19T07:37:58.566392+0000","last_fullsized":"2024-09-19T07:37:58.566392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280791+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:37:58.280791+0000","last_peered":"2024-09-19T07:37:58.280791+0000","last_clean":"2024-09-19T07:37:58.280791+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:37:58.280791+0000","last_undegraded":"2024-09-19T07:37:58.280791+0000","last_fullsized":"2024-09-19T07:37:58.280791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280919+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:37:58.280919+0000","last_peered":"2024-09-19T07:37:58.280919+0000","last_clean":"2024-09-19T07:37:58.280919+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:37:58.280919+0000","last_undegraded":"2024-09-19T07:37:58.280919+0000","last_fullsized":"2024-09-19T07:37:58.280919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565810+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:37:58.565810+0000","last_peered":"2024-09-19T07:37:58.565810+0000","last_clean":"2024-09-19T07:37:58.565810+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:37:58.565810+0000","last_undegraded":"2024-09-19T07:37:58.565810+0000","last_fullsized":"2024-09-19T07:37:58.565810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398122+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:37:58.398122+0000","last_peered":"2024-09-19T07:37:58.398122+0000","last_clean":"2024-09-19T07:37:58.398122+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:37:58.398122+0000","last_undegraded":"2024-09-19T07:37:58.398122+0000","last_fullsized":"2024-09-19T07:37:58.398122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406021+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:37:58.406021+0000","last_peered":"2024-09-19T07:37:58.406021+0000","last_clean":"2024-09-19T07:37:58.406021+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:37:58.406021+0000","last_undegraded":"2024-09-19T07:37:58.406021+0000","last_fullsized":"2024-09-19T07:37:58.406021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405841+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:37:58.405841+0000","last_peered":"2024-09-19T07:37:58.405841+0000","last_clean":"2024-09-19T07:37:58.405841+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:37:58.405841+0000","last_undegraded":"2024-09-19T07:37:58.405841+0000","last_fullsized":"2024-09-19T07:37:58.405841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408925+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:37:58.408925+0000","last_peered":"2024-09-19T07:37:58.408925+0000","last_clean":"2024-09-19T07:37:58.408925+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:37:58.408925+0000","last_undegraded":"2024-09-19T07:37:58.408925+0000","last_fullsized":"2024-09-19T07:37:58.408925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278608+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:37:58.278608+0000","last_peered":"2024-09-19T07:37:58.278608+0000","last_clean":"2024-09-19T07:37:58.278608+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:37:58.278608+0000","last_undegraded":"2024-09-19T07:37:58.278608+0000","last_fullsized":"2024-09-19T07:37:58.278608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398404+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:37:58.398404+0000","last_peered":"2024-09-19T07:37:58.398404+0000","last_clean":"2024-09-19T07:37:58.398404+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:37:58.398404+0000","last_undegraded":"2024-09-19T07:37:58.398404+0000","last_fullsized":"2024-09-19T07:37:58.398404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406017+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:37:58.406017+0000","last_peered":"2024-09-19T07:37:58.406017+0000","last_clean":"2024-09-19T07:37:58.406017+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:37:58.406017+0000","last_undegraded":"2024-09-19T07:37:58.406017+0000","last_fullsized":"2024-09-19T07:37:58.406017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395465+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:37:58.395465+0000","last_peered":"2024-09-19T07:37:58.395465+0000","last_clean":"2024-09-19T07:37:58.395465+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:37:58.395465+0000","last_undegraded":"2024-09-19T07:37:58.395465+0000","last_fullsized":"2024-09-19T07:37:58.395465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":229,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.042696+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:37:59.042696+0000","last_peered":"2024-09-19T07:37:59.042696+0000","last_clean":"2024-09-19T07:37:59.042696+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:37:59.042696+0000","last_undegraded":"2024-09-19T07:37:59.042696+0000","last_fullsized":"2024-09-19T07:37:59.042696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278242+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:37:58.278242+0000","last_peered":"2024-09-19T07:37:58.278242+0000","last_clean":"2024-09-19T07:37:58.278242+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:37:58.278242+0000","last_undegraded":"2024-09-19T07:37:58.278242+0000","last_fullsized":"2024-09-19T07:37:58.278242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568741+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:37:58.568741+0000","last_peered":"2024-09-19T07:37:58.568741+0000","last_clean":"2024-09-19T07:37:58.568741+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:37:58.568741+0000","last_undegraded":"2024-09-19T07:37:58.568741+0000","last_fullsized":"2024-09-19T07:37:58.568741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567021+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:37:58.567021+0000","last_peered":"2024-09-19T07:37:58.567021+0000","last_clean":"2024-09-19T07:37:58.567021+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:37:58.567021+0000","last_undegraded":"2024-09-19T07:37:58.567021+0000","last_fullsized":"2024-09-19T07:37:58.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405996+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:37:58.405996+0000","last_peered":"2024-09-19T07:37:58.405996+0000","last_clean":"2024-09-19T07:37:58.405996+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:37:58.405996+0000","last_undegraded":"2024-09-19T07:37:58.405996+0000","last_fullsized":"2024-09-19T07:37:58.405996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277409+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:37:58.277409+0000","last_peered":"2024-09-19T07:37:58.277409+0000","last_clean":"2024-09-19T07:37:58.277409+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:37:58.277409+0000","last_undegraded":"2024-09-19T07:37:58.277409+0000","last_fullsized":"2024-09-19T07:37:58.277409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":274,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.043101+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:37:59.043101+0000","last_peered":"2024-09-19T07:37:59.043101+0000","last_clean":"2024-09-19T07:37:59.043101+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:37:59.043101+0000","last_undegraded":"2024-09-19T07:37:59.043101+0000","last_fullsized":"2024-09-19T07:37:59.043101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":173,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398289+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:37:58.398289+0000","last_peered":"2024-09-19T07:37:58.398289+0000","last_clean":"2024-09-19T07:37:58.398289+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:37:58.398289+0000","last_undegraded":"2024-09-19T07:37:58.398289+0000","last_fullsized":"2024-09-19T07:37:58.398289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280980+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:37:58.280980+0000","last_peered":"2024-09-19T07:37:58.280980+0000","last_clean":"2024-09-19T07:37:58.280980+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:37:58.280980+0000","last_undegraded":"2024-09-19T07:37:58.280980+0000","last_fullsized":"2024-09-19T07:37:58.280980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278729+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:37:58.278729+0000","last_peered":"2024-09-19T07:37:58.278729+0000","last_clean":"2024-09-19T07:37:58.278729+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:37:58.278729+0000","last_undegraded":"2024-09-19T07:37:58.278729+0000","last_fullsized":"2024-09-19T07:37:58.278729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405861+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:37:58.405861+0000","last_peered":"2024-09-19T07:37:58.405861+0000","last_clean":"2024-09-19T07:37:58.405861+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:37:58.405861+0000","last_undegraded":"2024-09-19T07:37:58.405861+0000","last_fullsized":"2024-09-19T07:37:58.405861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568411+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:37:58.568411+0000","last_peered":"2024-09-19T07:37:58.568411+0000","last_clean":"2024-09-19T07:37:58.568411+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:37:58.568411+0000","last_undegraded":"2024-09-19T07:37:58.568411+0000","last_fullsized":"2024-09-19T07:37:58.568411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'9","reported_seq":138,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.042928+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:37:59.042928+0000","last_peered":"2024-09-19T07:37:59.042928+0000","last_clean":"2024-09-19T07:37:59.042928+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:37:59.042928+0000","last_undegraded":"2024-09-19T07:37:59.042928+0000","last_fullsized":"2024-09-19T07:37:59.042928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567829+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:37:58.567829+0000","last_peered":"2024-09-19T07:37:58.567829+0000","last_clean":"2024-09-19T07:37:58.567829+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:37:58.567829+0000","last_undegraded":"2024-09-19T07:37:58.567829+0000","last_fullsized":"2024-09-19T07:37:58.567829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565722+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:37:58.565722+0000","last_peered":"2024-09-19T07:37:58.565722+0000","last_clean":"2024-09-19T07:37:58.565722+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:37:58.565722+0000","last_undegraded":"2024-09-19T07:37:58.565722+0000","last_fullsized":"2024-09-19T07:37:58.565722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405806+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:37:58.405806+0000","last_peered":"2024-09-19T07:37:58.405806+0000","last_clean":"2024-09-19T07:37:58.405806+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:37:58.405806+0000","last_undegraded":"2024-09-19T07:37:58.405806+0000","last_fullsized":"2024-09-19T07:37:58.405806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398454+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:37:58.398454+0000","last_peered":"2024-09-19T07:37:58.398454+0000","last_clean":"2024-09-19T07:37:58.398454+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:37:58.398454+0000","last_undegraded":"2024-09-19T07:37:58.398454+0000","last_fullsized":"2024-09-19T07:37:58.398454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566085+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:37:58.566085+0000","last_peered":"2024-09-19T07:37:58.566085+0000","last_clean":"2024-09-19T07:37:58.566085+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:37:58.566085+0000","last_undegraded":"2024-09-19T07:37:58.566085+0000","last_fullsized":"2024-09-19T07:37:58.566085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278747+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:37:58.278747+0000","last_peered":"2024-09-19T07:37:58.278747+0000","last_clean":"2024-09-19T07:37:58.278747+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:37:58.278747+0000","last_undegraded":"2024-09-19T07:37:58.278747+0000","last_fullsized":"2024-09-19T07:37:58.278747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568790+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:37:58.568790+0000","last_peered":"2024-09-19T07:37:58.568790+0000","last_clean":"2024-09-19T07:37:58.568790+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:37:58.568790+0000","last_undegraded":"2024-09-19T07:37:58.568790+0000","last_fullsized":"2024-09-19T07:37:58.568790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'41","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.523245+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:37:58.523245+0000","last_peered":"2024-09-19T07:37:58.523245+0000","last_clean":"2024-09-19T07:37:58.523245+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:37:58.523245+0000","last_undegraded":"2024-09-19T07:37:58.523245+0000","last_fullsized":"2024-09-19T07:37:58.523245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568493+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:37:58.568493+0000","last_peered":"2024-09-19T07:37:58.568493+0000","last_clean":"2024-09-19T07:37:58.568493+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:37:58.568493+0000","last_undegraded":"2024-09-19T07:37:58.568493+0000","last_fullsized":"2024-09-19T07:37:58.568493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":137,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.043379+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:37:59.043379+0000","last_peered":"2024-09-19T07:37:59.043379+0000","last_clean":"2024-09-19T07:37:59.043379+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:37:59.043379+0000","last_undegraded":"2024-09-19T07:37:59.043379+0000","last_fullsized":"2024-09-19T07:37:59.043379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568589+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:37:58.568589+0000","last_peered":"2024-09-19T07:37:58.568589+0000","last_clean":"2024-09-19T07:37:58.568589+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:37:58.568589+0000","last_undegraded":"2024-09-19T07:37:58.568589+0000","last_fullsized":"2024-09-19T07:37:58.568589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567999+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:37:58.567999+0000","last_peered":"2024-09-19T07:37:58.567999+0000","last_clean":"2024-09-19T07:37:58.567999+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:37:58.567999+0000","last_undegraded":"2024-09-19T07:37:58.567999+0000","last_fullsized":"2024-09-19T07:37:58.567999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566436+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:37:58.566436+0000","last_peered":"2024-09-19T07:37:58.566436+0000","last_clean":"2024-09-19T07:37:58.566436+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:37:58.566436+0000","last_undegraded":"2024-09-19T07:37:58.566436+0000","last_fullsized":"2024-09-19T07:37:58.566436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399251+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:37:58.399251+0000","last_peered":"2024-09-19T07:37:58.399251+0000","last_clean":"2024-09-19T07:37:58.399251+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:37:58.399251+0000","last_undegraded":"2024-09-19T07:37:58.399251+0000","last_fullsized":"2024-09-19T07:37:58.399251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567920+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:37:58.567920+0000","last_peered":"2024-09-19T07:37:58.567920+0000","last_clean":"2024-09-19T07:37:58.567920+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:37:58.567920+0000","last_undegraded":"2024-09-19T07:37:58.567920+0000","last_fullsized":"2024-09-19T07:37:58.567920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278577+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:37:58.278577+0000","last_peered":"2024-09-19T07:37:58.278577+0000","last_clean":"2024-09-19T07:37:58.278577+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:37:58.278577+0000","last_undegraded":"2024-09-19T07:37:58.278577+0000","last_fullsized":"2024-09-19T07:37:58.278577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278764+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:37:58.278764+0000","last_peered":"2024-09-19T07:37:58.278764+0000","last_clean":"2024-09-19T07:37:58.278764+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:37:58.278764+0000","last_undegraded":"2024-09-19T07:37:58.278764+0000","last_fullsized":"2024-09-19T07:37:58.278764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405844+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:37:58.405844+0000","last_peered":"2024-09-19T07:37:58.405844+0000","last_clean":"2024-09-19T07:37:58.405844+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:37:58.405844+0000","last_undegraded":"2024-09-19T07:37:58.405844+0000","last_fullsized":"2024-09-19T07:37:58.405844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":134,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568315+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:37:58.568315+0000","last_peered":"2024-09-19T07:37:58.568315+0000","last_clean":"2024-09-19T07:37:58.568315+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:37:58.568315+0000","last_undegraded":"2024-09-19T07:37:58.568315+0000","last_fullsized":"2024-09-19T07:37:58.568315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408965+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:37:58.408965+0000","last_peered":"2024-09-19T07:37:58.408965+0000","last_clean":"2024-09-19T07:37:58.408965+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:37:58.408965+0000","last_undegraded":"2024-09-19T07:37:58.408965+0000","last_fullsized":"2024-09-19T07:37:58.408965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396146+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:37:58.396146+0000","last_peered":"2024-09-19T07:37:58.396146+0000","last_clean":"2024-09-19T07:37:58.396146+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:37:58.396146+0000","last_undegraded":"2024-09-19T07:37:58.396146+0000","last_fullsized":"2024-09-19T07:37:58.396146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409093+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:37:58.409093+0000","last_peered":"2024-09-19T07:37:58.409093+0000","last_clean":"2024-09-19T07:37:58.409093+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:37:58.409093+0000","last_undegraded":"2024-09-19T07:37:58.409093+0000","last_fullsized":"2024-09-19T07:37:58.409093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398419+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:37:58.398419+0000","last_peered":"2024-09-19T07:37:58.398419+0000","last_clean":"2024-09-19T07:37:58.398419+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:37:58.398419+0000","last_undegraded":"2024-09-19T07:37:58.398419+0000","last_fullsized":"2024-09-19T07:37:58.398419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278876+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:37:58.278876+0000","last_peered":"2024-09-19T07:37:58.278876+0000","last_clean":"2024-09-19T07:37:58.278876+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:37:58.278876+0000","last_undegraded":"2024-09-19T07:37:58.278876+0000","last_fullsized":"2024-09-19T07:37:58.278876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'51","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.530637+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:37:58.530637+0000","last_peered":"2024-09-19T07:37:58.530637+0000","last_clean":"2024-09-19T07:37:58.530637+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:37:58.530637+0000","last_undegraded":"2024-09-19T07:37:58.530637+0000","last_fullsized":"2024-09-19T07:37:58.530637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400100+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:37:58.400100+0000","last_peered":"2024-09-19T07:37:58.400100+0000","last_clean":"2024-09-19T07:37:58.400100+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:37:58.400100+0000","last_undegraded":"2024-09-19T07:37:58.400100+0000","last_fullsized":"2024-09-19T07:37:58.400100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566566+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:37:58.566566+0000","last_peered":"2024-09-19T07:37:58.566566+0000","last_clean":"2024-09-19T07:37:58.566566+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:37:58.566566+0000","last_undegraded":"2024-09-19T07:37:58.566566+0000","last_fullsized":"2024-09-19T07:37:58.566566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566745+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:37:58.566745+0000","last_peered":"2024-09-19T07:37:58.566745+0000","last_clean":"2024-09-19T07:37:58.566745+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:37:58.566745+0000","last_undegraded":"2024-09-19T07:37:58.566745+0000","last_fullsized":"2024-09-19T07:37:58.566745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277436+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:37:58.277436+0000","last_peered":"2024-09-19T07:37:58.277436+0000","last_clean":"2024-09-19T07:37:58.277436+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:37:58.277436+0000","last_undegraded":"2024-09-19T07:37:58.277436+0000","last_fullsized":"2024-09-19T07:37:58.277436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.397993+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:37:58.397993+0000","last_peered":"2024-09-19T07:37:58.397993+0000","last_clean":"2024-09-19T07:37:58.397993+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:37:58.397993+0000","last_undegraded":"2024-09-19T07:37:58.397993+0000","last_fullsized":"2024-09-19T07:37:58.397993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398888+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:37:58.398888+0000","last_peered":"2024-09-19T07:37:58.398888+0000","last_clean":"2024-09-19T07:37:58.398888+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:37:58.398888+0000","last_undegraded":"2024-09-19T07:37:58.398888+0000","last_fullsized":"2024-09-19T07:37:58.398888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565988+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:37:58.565988+0000","last_peered":"2024-09-19T07:37:58.565988+0000","last_clean":"2024-09-19T07:37:58.565988+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:37:58.565988+0000","last_undegraded":"2024-09-19T07:37:58.565988+0000","last_fullsized":"2024-09-19T07:37:58.565988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565778+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:37:58.565778+0000","last_peered":"2024-09-19T07:37:58.565778+0000","last_clean":"2024-09-19T07:37:58.565778+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:37:58.565778+0000","last_undegraded":"2024-09-19T07:37:58.565778+0000","last_fullsized":"2024-09-19T07:37:58.565778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399648+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:37:58.399648+0000","last_peered":"2024-09-19T07:37:58.399648+0000","last_clean":"2024-09-19T07:37:58.399648+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:37:58.399648+0000","last_undegraded":"2024-09-19T07:37:58.399648+0000","last_fullsized":"2024-09-19T07:37:58.399648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398085+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:37:58.398085+0000","last_peered":"2024-09-19T07:37:58.398085+0000","last_clean":"2024-09-19T07:37:58.398085+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:37:58.398085+0000","last_undegraded":"2024-09-19T07:37:58.398085+0000","last_fullsized":"2024-09-19T07:37:58.398085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568069+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:37:58.568069+0000","last_peered":"2024-09-19T07:37:58.568069+0000","last_clean":"2024-09-19T07:37:58.568069+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:37:58.568069+0000","last_undegraded":"2024-09-19T07:37:58.568069+0000","last_fullsized":"2024-09-19T07:37:58.568069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395362+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:37:58.395362+0000","last_peered":"2024-09-19T07:37:58.395362+0000","last_clean":"2024-09-19T07:37:58.395362+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:37:58.395362+0000","last_undegraded":"2024-09-19T07:37:58.395362+0000","last_fullsized":"2024-09-19T07:37:58.395362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398241+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:37:58.398241+0000","last_peered":"2024-09-19T07:37:58.398241+0000","last_clean":"2024-09-19T07:37:58.398241+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:37:58.398241+0000","last_undegraded":"2024-09-19T07:37:58.398241+0000","last_fullsized":"2024-09-19T07:37:58.398241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277959+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:37:58.277959+0000","last_peered":"2024-09-19T07:37:58.277959+0000","last_clean":"2024-09-19T07:37:58.277959+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:37:58.277959+0000","last_undegraded":"2024-09-19T07:37:58.277959+0000","last_fullsized":"2024-09-19T07:37:58.277959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398335+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:37:58.398335+0000","last_peered":"2024-09-19T07:37:58.398335+0000","last_clean":"2024-09-19T07:37:58.398335+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:37:58.398335+0000","last_undegraded":"2024-09-19T07:37:58.398335+0000","last_fullsized":"2024-09-19T07:37:58.398335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566969+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:37:58.566969+0000","last_peered":"2024-09-19T07:37:58.566969+0000","last_clean":"2024-09-19T07:37:58.566969+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:37:58.566969+0000","last_undegraded":"2024-09-19T07:37:58.566969+0000","last_fullsized":"2024-09-19T07:37:58.566969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400607+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:37:58.400607+0000","last_peered":"2024-09-19T07:37:58.400607+0000","last_clean":"2024-09-19T07:37:58.400607+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:37:58.400607+0000","last_undegraded":"2024-09-19T07:37:58.400607+0000","last_fullsized":"2024-09-19T07:37:58.400607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400322+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:37:58.400322+0000","last_peered":"2024-09-19T07:37:58.400322+0000","last_clean":"2024-09-19T07:37:58.400322+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:37:58.400322+0000","last_undegraded":"2024-09-19T07:37:58.400322+0000","last_fullsized":"2024-09-19T07:37:58.400322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'14","reported_seq":155,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396388+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:37:58.396388+0000","last_peered":"2024-09-19T07:37:58.396388+0000","last_clean":"2024-09-19T07:37:58.396388+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:37:58.396388+0000","last_undegraded":"2024-09-19T07:37:58.396388+0000","last_fullsized":"2024-09-19T07:37:58.396388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602652,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1142,"num_read_kb":690,"num_write":376,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1807956,"data_compressed":80685,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107370,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":451633,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":1243176,"data_compressed":52251,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441568,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":441162,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1044883,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528381,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":450775,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1174307,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":451422,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":974882,"data_compressed":34014,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":520934,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":520993,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":452507,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71591,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71532,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":203289,"data_compressed":7749,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:38:01.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:38:01.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.456+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3122806863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0101b30 msgr2=0x7fe2f0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.456+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3122806863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0101b30 0x7fe2f0101f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009900 tx=0x7fe2e402a410 comp rx=0 tx=0).stop 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.456+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3122806863 shutdown_connections 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.456+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3122806863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f01036e0 0x7fe2f0109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.456+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3122806863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0102d20 0x7fe2f01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3122806863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0101b30 0x7fe2f0101f30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3122806863 >> 172.21.15.103:0/3122806863 conn(0x7fe2f00fd2b0 msgr2=0x7fe2f00ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3122806863 shutdown_connections 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3122806863 wait complete. 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 Processor -- start 2024-09-19T07:38:01.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 -- start start 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0101b30 0x7fe2f0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0102d20 0x7fe2f01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f01036e0 0x7fe2f019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f010d7b0 con 0x7fe2f0102d20 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f010d630 con 0x7fe2f01036e0 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.457+0000 7fe2f5a1e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f010d930 con 0x7fe2f0101b30 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0102d20 0x7fe2f01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0102d20 0x7fe2f01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45764/0 (socket says 172.21.15.103:45764) 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2eeffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0101b30 0x7fe2f0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 -- 172.21.15.103:0/3919444165 learned_addr learned my addr 172.21.15.103:0/3919444165 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:01.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ef7fe640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f01036e0 0x7fe2f019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 -- 172.21.15.103:0/3919444165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0101b30 msgr2=0x7fe2f0197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0101b30 0x7fe2f0197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 -- 172.21.15.103:0/3919444165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f01036e0 msgr2=0x7fe2f019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f01036e0 0x7fe2f019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 -- 172.21.15.103:0/3919444165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f019fa80 con 0x7fe2f0102d20 2024-09-19T07:38:01.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ef7fe640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f01036e0 0x7fe2f019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:01.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2eeffd640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0101b30 0x7fe2f0197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:01.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2ee7fc640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0102d20 0x7fe2f01983a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000e9f0 tx=0x7fe2e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:01.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2f4a1c640 1 -- 172.21.15.103:0/3919444165 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000cd80 con 0x7fe2f0102d20 2024-09-19T07:38:01.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2f4a1c640 1 -- 172.21.15.103:0/3919444165 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e0005500 con 0x7fe2f0102d20 2024-09-19T07:38:01.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f019fd70 con 0x7fe2f0102d20 2024-09-19T07:38:01.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2f4a1c640 1 -- 172.21.15.103:0/3919444165 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0010690 con 0x7fe2f0102d20 2024-09-19T07:38:01.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.458+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f01a0210 con 0x7fe2f0102d20 2024-09-19T07:38:01.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.459+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f0108f70 con 0x7fe2f0102d20 2024-09-19T07:38:01.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.461+0000 7fe2f4a1c640 1 -- 172.21.15.103:0/3919444165 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7fe2e00046c0 con 0x7fe2f0102d20 2024-09-19T07:38:01.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.461+0000 7fe2f4a1c640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe2d80a33c0 0x7fe2d80a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.462+0000 7fe2eeffd640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe2d80a33c0 0x7fe2d80a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.462+0000 7fe2f4a1c640 1 -- 172.21.15.103:0/3919444165 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe2e00d0360 con 0x7fe2f0102d20 2024-09-19T07:38:01.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.462+0000 7fe2eeffd640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe2d80a33c0 0x7fe2d80a5880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4005ab0 tx=0x7fe2e4000bc0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:01.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.465+0000 7fe2f4a1c640 1 -- 172.21.15.103:0/3919444165 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe2e0054210 con 0x7fe2f0102d20 2024-09-19T07:38:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-19T07:38:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T07:38:01.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-19T07:38:01.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T07:38:01.585 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:01 smithi134 ceph-mon[35414]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-19T07:38:01.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.639+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2f0062630 con 0x7fe2d80a33c0 2024-09-19T07:38:01.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.645+0000 7fe2f4a1c640 1 -- 172.21.15.103:0/3919444165 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258076 (secure 0 0 0) 0x7fe2f0062630 con 0x7fe2d80a33c0 2024-09-19T07:38:01.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:01.650 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:38:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe2d80a33c0 msgr2=0x7fe2d80a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe2d80a33c0 0x7fe2d80a5880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4005ab0 tx=0x7fe2e4000bc0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0102d20 msgr2=0x7fe2f01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0102d20 0x7fe2f01983a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000e9f0 tx=0x7fe2e000edf0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 shutdown_connections 2024-09-19T07:38:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe2d80a33c0 0x7fe2d80a5880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f01036e0 0x7fe2f019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2f0102d20 0x7fe2f01983a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 --2- 172.21.15.103:0/3919444165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2f0101b30 0x7fe2f0197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.648+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 >> 172.21.15.103:0/3919444165 conn(0x7fe2f00fd2b0 msgr2=0x7fe2f0106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:01.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.649+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 shutdown_connections 2024-09-19T07:38:01.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.649+0000 7fe2f5a1e640 1 -- 172.21.15.103:0/3919444165 wait complete. 2024-09-19T07:38:01.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:01.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-19T07:38:01.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-19T07:38:01.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T07:38:01.663 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:01 smithi103 ceph-mon[31201]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-19T07:38:01.663 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-19T07:38:00.293143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193254,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1278,"num_read_kb":809,"num_write":559,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344136,"kb_used_data":11496,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597688,"statfs":{"total":1151940427776,"available":1151588032512,"internally_reserved":0,"allocated":11771904,"data_stored":8168242,"data_compressed":248501,"data_compressed_allocated":335872,"data_compressed_original":2834432,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.993569"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566027+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:37:58.566027+0000","last_peered":"2024-09-19T07:37:58.566027+0000","last_clean":"2024-09-19T07:37:58.566027+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:37:58.566027+0000","last_undegraded":"2024-09-19T07:37:58.566027+0000","last_fullsized":"2024-09-19T07:37:58.566027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566607+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:37:58.566607+0000","last_peered":"2024-09-19T07:37:58.566607+0000","last_clean":"2024-09-19T07:37:58.566607+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:37:58.566607+0000","last_undegraded":"2024-09-19T07:37:58.566607+0000","last_fullsized":"2024-09-19T07:37:58.566607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566643+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:37:58.566643+0000","last_peered":"2024-09-19T07:37:58.566643+0000","last_clean":"2024-09-19T07:37:58.566643+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:37:58.566643+0000","last_undegraded":"2024-09-19T07:37:58.566643+0000","last_fullsized":"2024-09-19T07:37:58.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395953+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:37:58.395953+0000","last_peered":"2024-09-19T07:37:58.395953+0000","last_clean":"2024-09-19T07:37:58.395953+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:37:58.395953+0000","last_undegraded":"2024-09-19T07:37:58.395953+0000","last_fullsized":"2024-09-19T07:37:58.395953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398077+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:37:58.398077+0000","last_peered":"2024-09-19T07:37:58.398077+0000","last_clean":"2024-09-19T07:37:58.398077+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:37:58.398077+0000","last_undegraded":"2024-09-19T07:37:58.398077+0000","last_fullsized":"2024-09-19T07:37:58.398077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408847+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:37:58.408847+0000","last_peered":"2024-09-19T07:37:58.408847+0000","last_clean":"2024-09-19T07:37:58.408847+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:37:58.408847+0000","last_undegraded":"2024-09-19T07:37:58.408847+0000","last_fullsized":"2024-09-19T07:37:58.408847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567869+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:37:58.567869+0000","last_peered":"2024-09-19T07:37:58.567869+0000","last_clean":"2024-09-19T07:37:58.567869+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:37:58.567869+0000","last_undegraded":"2024-09-19T07:37:58.567869+0000","last_fullsized":"2024-09-19T07:37:58.567869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398332+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:37:58.398332+0000","last_peered":"2024-09-19T07:37:58.398332+0000","last_clean":"2024-09-19T07:37:58.398332+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:37:58.398332+0000","last_undegraded":"2024-09-19T07:37:58.398332+0000","last_fullsized":"2024-09-19T07:37:58.398332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278587+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:37:58.278587+0000","last_peered":"2024-09-19T07:37:58.278587+0000","last_clean":"2024-09-19T07:37:58.278587+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:37:58.278587+0000","last_undegraded":"2024-09-19T07:37:58.278587+0000","last_fullsized":"2024-09-19T07:37:58.278587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409236+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:37:58.409236+0000","last_peered":"2024-09-19T07:37:58.409236+0000","last_clean":"2024-09-19T07:37:58.409236+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:37:58.409236+0000","last_undegraded":"2024-09-19T07:37:58.409236+0000","last_fullsized":"2024-09-19T07:37:58.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568581+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:37:58.568581+0000","last_peered":"2024-09-19T07:37:58.568581+0000","last_clean":"2024-09-19T07:37:58.568581+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:37:58.568581+0000","last_undegraded":"2024-09-19T07:37:58.568581+0000","last_fullsized":"2024-09-19T07:37:58.568581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566381+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:37:58.566381+0000","last_peered":"2024-09-19T07:37:58.566381+0000","last_clean":"2024-09-19T07:37:58.566381+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:37:58.566381+0000","last_undegraded":"2024-09-19T07:37:58.566381+0000","last_fullsized":"2024-09-19T07:37:58.566381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278860+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:37:58.278860+0000","last_peered":"2024-09-19T07:37:58.278860+0000","last_clean":"2024-09-19T07:37:58.278860+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:37:58.278860+0000","last_undegraded":"2024-09-19T07:37:58.278860+0000","last_fullsized":"2024-09-19T07:37:58.278860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566247+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:37:58.566247+0000","last_peered":"2024-09-19T07:37:58.566247+0000","last_clean":"2024-09-19T07:37:58.566247+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:37:58.566247+0000","last_undegraded":"2024-09-19T07:37:58.566247+0000","last_fullsized":"2024-09-19T07:37:58.566247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566034+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:37:58.566034+0000","last_peered":"2024-09-19T07:37:58.566034+0000","last_clean":"2024-09-19T07:37:58.566034+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:37:58.566034+0000","last_undegraded":"2024-09-19T07:37:58.566034+0000","last_fullsized":"2024-09-19T07:37:58.566034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277343+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:37:58.277343+0000","last_peered":"2024-09-19T07:37:58.277343+0000","last_clean":"2024-09-19T07:37:58.277343+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:37:58.277343+0000","last_undegraded":"2024-09-19T07:37:58.277343+0000","last_fullsized":"2024-09-19T07:37:58.277343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566048+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:37:58.566048+0000","last_peered":"2024-09-19T07:37:58.566048+0000","last_clean":"2024-09-19T07:37:58.566048+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:37:58.566048+0000","last_undegraded":"2024-09-19T07:37:58.566048+0000","last_fullsized":"2024-09-19T07:37:58.566048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395689+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:37:58.395689+0000","last_peered":"2024-09-19T07:37:58.395689+0000","last_clean":"2024-09-19T07:37:58.395689+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:37:58.395689+0000","last_undegraded":"2024-09-19T07:37:58.395689+0000","last_fullsized":"2024-09-19T07:37:58.395689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281089+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:37:58.281089+0000","last_peered":"2024-09-19T07:37:58.281089+0000","last_clean":"2024-09-19T07:37:58.281089+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:37:58.281089+0000","last_undegraded":"2024-09-19T07:37:58.281089+0000","last_fullsized":"2024-09-19T07:37:58.281089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565798+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:37:58.565798+0000","last_peered":"2024-09-19T07:37:58.565798+0000","last_clean":"2024-09-19T07:37:58.565798+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:37:58.565798+0000","last_undegraded":"2024-09-19T07:37:58.565798+0000","last_fullsized":"2024-09-19T07:37:58.565798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278866+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:37:58.278866+0000","last_peered":"2024-09-19T07:37:58.278866+0000","last_clean":"2024-09-19T07:37:58.278866+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:37:58.278866+0000","last_undegraded":"2024-09-19T07:37:58.278866+0000","last_fullsized":"2024-09-19T07:37:58.278866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277508+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:37:58.277508+0000","last_peered":"2024-09-19T07:37:58.277508+0000","last_clean":"2024-09-19T07:37:58.277508+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:37:58.277508+0000","last_undegraded":"2024-09-19T07:37:58.277508+0000","last_fullsized":"2024-09-19T07:37:58.277508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398644+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:37:58.398644+0000","last_peered":"2024-09-19T07:37:58.398644+0000","last_clean":"2024-09-19T07:37:58.398644+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:37:58.398644+0000","last_undegraded":"2024-09-19T07:37:58.398644+0000","last_fullsized":"2024-09-19T07:37:58.398644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405691+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:37:58.405691+0000","last_peered":"2024-09-19T07:37:58.405691+0000","last_clean":"2024-09-19T07:37:58.405691+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:37:58.405691+0000","last_undegraded":"2024-09-19T07:37:58.405691+0000","last_fullsized":"2024-09-19T07:37:58.405691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399976+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:37:58.399976+0000","last_peered":"2024-09-19T07:37:58.399976+0000","last_clean":"2024-09-19T07:37:58.399976+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:37:58.399976+0000","last_undegraded":"2024-09-19T07:37:58.399976+0000","last_fullsized":"2024-09-19T07:37:58.399976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281203+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:37:58.281203+0000","last_peered":"2024-09-19T07:37:58.281203+0000","last_clean":"2024-09-19T07:37:58.281203+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:37:58.281203+0000","last_undegraded":"2024-09-19T07:37:58.281203+0000","last_fullsized":"2024-09-19T07:37:58.281203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565955+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:37:58.565955+0000","last_peered":"2024-09-19T07:37:58.565955+0000","last_clean":"2024-09-19T07:37:58.565955+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:37:58.565955+0000","last_undegraded":"2024-09-19T07:37:58.565955+0000","last_fullsized":"2024-09-19T07:37:58.565955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278990+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:37:58.278990+0000","last_peered":"2024-09-19T07:37:58.278990+0000","last_clean":"2024-09-19T07:37:58.278990+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:37:58.278990+0000","last_undegraded":"2024-09-19T07:37:58.278990+0000","last_fullsized":"2024-09-19T07:37:58.278990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566392+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:37:58.566392+0000","last_peered":"2024-09-19T07:37:58.566392+0000","last_clean":"2024-09-19T07:37:58.566392+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:37:58.566392+0000","last_undegraded":"2024-09-19T07:37:58.566392+0000","last_fullsized":"2024-09-19T07:37:58.566392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280791+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:37:58.280791+0000","last_peered":"2024-09-19T07:37:58.280791+0000","last_clean":"2024-09-19T07:37:58.280791+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:37:58.280791+0000","last_undegraded":"2024-09-19T07:37:58.280791+0000","last_fullsized":"2024-09-19T07:37:58.280791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280919+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:37:58.280919+0000","last_peered":"2024-09-19T07:37:58.280919+0000","last_clean":"2024-09-19T07:37:58.280919+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:37:58.280919+0000","last_undegraded":"2024-09-19T07:37:58.280919+0000","last_fullsized":"2024-09-19T07:37:58.280919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565810+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:37:58.565810+0000","last_peered":"2024-09-19T07:37:58.565810+0000","last_clean":"2024-09-19T07:37:58.565810+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:37:58.565810+0000","last_undegraded":"2024-09-19T07:37:58.565810+0000","last_fullsized":"2024-09-19T07:37:58.565810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398122+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:37:58.398122+0000","last_peered":"2024-09-19T07:37:58.398122+0000","last_clean":"2024-09-19T07:37:58.398122+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:37:58.398122+0000","last_undegraded":"2024-09-19T07:37:58.398122+0000","last_fullsized":"2024-09-19T07:37:58.398122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406021+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:37:58.406021+0000","last_peered":"2024-09-19T07:37:58.406021+0000","last_clean":"2024-09-19T07:37:58.406021+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:37:58.406021+0000","last_undegraded":"2024-09-19T07:37:58.406021+0000","last_fullsized":"2024-09-19T07:37:58.406021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405841+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:37:58.405841+0000","last_peered":"2024-09-19T07:37:58.405841+0000","last_clean":"2024-09-19T07:37:58.405841+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:37:58.405841+0000","last_undegraded":"2024-09-19T07:37:58.405841+0000","last_fullsized":"2024-09-19T07:37:58.405841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408925+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:37:58.408925+0000","last_peered":"2024-09-19T07:37:58.408925+0000","last_clean":"2024-09-19T07:37:58.408925+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:37:58.408925+0000","last_undegraded":"2024-09-19T07:37:58.408925+0000","last_fullsized":"2024-09-19T07:37:58.408925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278608+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:37:58.278608+0000","last_peered":"2024-09-19T07:37:58.278608+0000","last_clean":"2024-09-19T07:37:58.278608+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:37:58.278608+0000","last_undegraded":"2024-09-19T07:37:58.278608+0000","last_fullsized":"2024-09-19T07:37:58.278608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398404+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:37:58.398404+0000","last_peered":"2024-09-19T07:37:58.398404+0000","last_clean":"2024-09-19T07:37:58.398404+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:37:58.398404+0000","last_undegraded":"2024-09-19T07:37:58.398404+0000","last_fullsized":"2024-09-19T07:37:58.398404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406017+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:37:58.406017+0000","last_peered":"2024-09-19T07:37:58.406017+0000","last_clean":"2024-09-19T07:37:58.406017+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:37:58.406017+0000","last_undegraded":"2024-09-19T07:37:58.406017+0000","last_fullsized":"2024-09-19T07:37:58.406017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395465+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:37:58.395465+0000","last_peered":"2024-09-19T07:37:58.395465+0000","last_clean":"2024-09-19T07:37:58.395465+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:37:58.395465+0000","last_undegraded":"2024-09-19T07:37:58.395465+0000","last_fullsized":"2024-09-19T07:37:58.395465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":229,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.042696+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:37:59.042696+0000","last_peered":"2024-09-19T07:37:59.042696+0000","last_clean":"2024-09-19T07:37:59.042696+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:37:59.042696+0000","last_undegraded":"2024-09-19T07:37:59.042696+0000","last_fullsized":"2024-09-19T07:37:59.042696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278242+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:37:58.278242+0000","last_peered":"2024-09-19T07:37:58.278242+0000","last_clean":"2024-09-19T07:37:58.278242+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:37:58.278242+0000","last_undegraded":"2024-09-19T07:37:58.278242+0000","last_fullsized":"2024-09-19T07:37:58.278242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568741+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:37:58.568741+0000","last_peered":"2024-09-19T07:37:58.568741+0000","last_clean":"2024-09-19T07:37:58.568741+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:37:58.568741+0000","last_undegraded":"2024-09-19T07:37:58.568741+0000","last_fullsized":"2024-09-19T07:37:58.568741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567021+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:37:58.567021+0000","last_peered":"2024-09-19T07:37:58.567021+0000","last_clean":"2024-09-19T07:37:58.567021+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:37:58.567021+0000","last_undegraded":"2024-09-19T07:37:58.567021+0000","last_fullsized":"2024-09-19T07:37:58.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405996+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:37:58.405996+0000","last_peered":"2024-09-19T07:37:58.405996+0000","last_clean":"2024-09-19T07:37:58.405996+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:37:58.405996+0000","last_undegraded":"2024-09-19T07:37:58.405996+0000","last_fullsized":"2024-09-19T07:37:58.405996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277409+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:37:58.277409+0000","last_peered":"2024-09-19T07:37:58.277409+0000","last_clean":"2024-09-19T07:37:58.277409+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:37:58.277409+0000","last_undegraded":"2024-09-19T07:37:58.277409+0000","last_fullsized":"2024-09-19T07:37:58.277409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":274,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.043101+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:37:59.043101+0000","last_peered":"2024-09-19T07:37:59.043101+0000","last_clean":"2024-09-19T07:37:59.043101+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:37:59.043101+0000","last_undegraded":"2024-09-19T07:37:59.043101+0000","last_fullsized":"2024-09-19T07:37:59.043101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":173,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398289+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:37:58.398289+0000","last_peered":"2024-09-19T07:37:58.398289+0000","last_clean":"2024-09-19T07:37:58.398289+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:37:58.398289+0000","last_undegraded":"2024-09-19T07:37:58.398289+0000","last_fullsized":"2024-09-19T07:37:58.398289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280980+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:37:58.280980+0000","last_peered":"2024-09-19T07:37:58.280980+0000","last_clean":"2024-09-19T07:37:58.280980+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:37:58.280980+0000","last_undegraded":"2024-09-19T07:37:58.280980+0000","last_fullsized":"2024-09-19T07:37:58.280980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278729+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:37:58.278729+0000","last_peered":"2024-09-19T07:37:58.278729+0000","last_clean":"2024-09-19T07:37:58.278729+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:37:58.278729+0000","last_undegraded":"2024-09-19T07:37:58.278729+0000","last_fullsized":"2024-09-19T07:37:58.278729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405861+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:37:58.405861+0000","last_peered":"2024-09-19T07:37:58.405861+0000","last_clean":"2024-09-19T07:37:58.405861+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:37:58.405861+0000","last_undegraded":"2024-09-19T07:37:58.405861+0000","last_fullsized":"2024-09-19T07:37:58.405861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568411+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:37:58.568411+0000","last_peered":"2024-09-19T07:37:58.568411+0000","last_clean":"2024-09-19T07:37:58.568411+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:37:58.568411+0000","last_undegraded":"2024-09-19T07:37:58.568411+0000","last_fullsized":"2024-09-19T07:37:58.568411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'9","reported_seq":138,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.042928+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:37:59.042928+0000","last_peered":"2024-09-19T07:37:59.042928+0000","last_clean":"2024-09-19T07:37:59.042928+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:37:59.042928+0000","last_undegraded":"2024-09-19T07:37:59.042928+0000","last_fullsized":"2024-09-19T07:37:59.042928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567829+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:37:58.567829+0000","last_peered":"2024-09-19T07:37:58.567829+0000","last_clean":"2024-09-19T07:37:58.567829+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:37:58.567829+0000","last_undegraded":"2024-09-19T07:37:58.567829+0000","last_fullsized":"2024-09-19T07:37:58.567829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565722+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:37:58.565722+0000","last_peered":"2024-09-19T07:37:58.565722+0000","last_clean":"2024-09-19T07:37:58.565722+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:37:58.565722+0000","last_undegraded":"2024-09-19T07:37:58.565722+0000","last_fullsized":"2024-09-19T07:37:58.565722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405806+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:37:58.405806+0000","last_peered":"2024-09-19T07:37:58.405806+0000","last_clean":"2024-09-19T07:37:58.405806+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:37:58.405806+0000","last_undegraded":"2024-09-19T07:37:58.405806+0000","last_fullsized":"2024-09-19T07:37:58.405806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398454+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:37:58.398454+0000","last_peered":"2024-09-19T07:37:58.398454+0000","last_clean":"2024-09-19T07:37:58.398454+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:37:58.398454+0000","last_undegraded":"2024-09-19T07:37:58.398454+0000","last_fullsized":"2024-09-19T07:37:58.398454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566085+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:37:58.566085+0000","last_peered":"2024-09-19T07:37:58.566085+0000","last_clean":"2024-09-19T07:37:58.566085+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:37:58.566085+0000","last_undegraded":"2024-09-19T07:37:58.566085+0000","last_fullsized":"2024-09-19T07:37:58.566085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278747+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:37:58.278747+0000","last_peered":"2024-09-19T07:37:58.278747+0000","last_clean":"2024-09-19T07:37:58.278747+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:37:58.278747+0000","last_undegraded":"2024-09-19T07:37:58.278747+0000","last_fullsized":"2024-09-19T07:37:58.278747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568790+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:37:58.568790+0000","last_peered":"2024-09-19T07:37:58.568790+0000","last_clean":"2024-09-19T07:37:58.568790+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:37:58.568790+0000","last_undegraded":"2024-09-19T07:37:58.568790+0000","last_fullsized":"2024-09-19T07:37:58.568790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'41","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.523245+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:37:58.523245+0000","last_peered":"2024-09-19T07:37:58.523245+0000","last_clean":"2024-09-19T07:37:58.523245+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:37:58.523245+0000","last_undegraded":"2024-09-19T07:37:58.523245+0000","last_fullsized":"2024-09-19T07:37:58.523245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568493+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:37:58.568493+0000","last_peered":"2024-09-19T07:37:58.568493+0000","last_clean":"2024-09-19T07:37:58.568493+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:37:58.568493+0000","last_undegraded":"2024-09-19T07:37:58.568493+0000","last_fullsized":"2024-09-19T07:37:58.568493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":137,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.043379+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:37:59.043379+0000","last_peered":"2024-09-19T07:37:59.043379+0000","last_clean":"2024-09-19T07:37:59.043379+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:37:59.043379+0000","last_undegraded":"2024-09-19T07:37:59.043379+0000","last_fullsized":"2024-09-19T07:37:59.043379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568589+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:37:58.568589+0000","last_peered":"2024-09-19T07:37:58.568589+0000","last_clean":"2024-09-19T07:37:58.568589+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:37:58.568589+0000","last_undegraded":"2024-09-19T07:37:58.568589+0000","last_fullsized":"2024-09-19T07:37:58.568589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567999+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:37:58.567999+0000","last_peered":"2024-09-19T07:37:58.567999+0000","last_clean":"2024-09-19T07:37:58.567999+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:37:58.567999+0000","last_undegraded":"2024-09-19T07:37:58.567999+0000","last_fullsized":"2024-09-19T07:37:58.567999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566436+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:37:58.566436+0000","last_peered":"2024-09-19T07:37:58.566436+0000","last_clean":"2024-09-19T07:37:58.566436+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:37:58.566436+0000","last_undegraded":"2024-09-19T07:37:58.566436+0000","last_fullsized":"2024-09-19T07:37:58.566436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399251+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:37:58.399251+0000","last_peered":"2024-09-19T07:37:58.399251+0000","last_clean":"2024-09-19T07:37:58.399251+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:37:58.399251+0000","last_undegraded":"2024-09-19T07:37:58.399251+0000","last_fullsized":"2024-09-19T07:37:58.399251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567920+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:37:58.567920+0000","last_peered":"2024-09-19T07:37:58.567920+0000","last_clean":"2024-09-19T07:37:58.567920+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:37:58.567920+0000","last_undegraded":"2024-09-19T07:37:58.567920+0000","last_fullsized":"2024-09-19T07:37:58.567920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278577+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:37:58.278577+0000","last_peered":"2024-09-19T07:37:58.278577+0000","last_clean":"2024-09-19T07:37:58.278577+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:37:58.278577+0000","last_undegraded":"2024-09-19T07:37:58.278577+0000","last_fullsized":"2024-09-19T07:37:58.278577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278764+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:37:58.278764+0000","last_peered":"2024-09-19T07:37:58.278764+0000","last_clean":"2024-09-19T07:37:58.278764+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:37:58.278764+0000","last_undegraded":"2024-09-19T07:37:58.278764+0000","last_fullsized":"2024-09-19T07:37:58.278764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405844+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:37:58.405844+0000","last_peered":"2024-09-19T07:37:58.405844+0000","last_clean":"2024-09-19T07:37:58.405844+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:37:58.405844+0000","last_undegraded":"2024-09-19T07:37:58.405844+0000","last_fullsized":"2024-09-19T07:37:58.405844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":134,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568315+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:37:58.568315+0000","last_peered":"2024-09-19T07:37:58.568315+0000","last_clean":"2024-09-19T07:37:58.568315+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:37:58.568315+0000","last_undegraded":"2024-09-19T07:37:58.568315+0000","last_fullsized":"2024-09-19T07:37:58.568315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408965+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:37:58.408965+0000","last_peered":"2024-09-19T07:37:58.408965+0000","last_clean":"2024-09-19T07:37:58.408965+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:37:58.408965+0000","last_undegraded":"2024-09-19T07:37:58.408965+0000","last_fullsized":"2024-09-19T07:37:58.408965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396146+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:37:58.396146+0000","last_peered":"2024-09-19T07:37:58.396146+0000","last_clean":"2024-09-19T07:37:58.396146+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:37:58.396146+0000","last_undegraded":"2024-09-19T07:37:58.396146+0000","last_fullsized":"2024-09-19T07:37:58.396146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409093+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:37:58.409093+0000","last_peered":"2024-09-19T07:37:58.409093+0000","last_clean":"2024-09-19T07:37:58.409093+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:37:58.409093+0000","last_undegraded":"2024-09-19T07:37:58.409093+0000","last_fullsized":"2024-09-19T07:37:58.409093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398419+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:37:58.398419+0000","last_peered":"2024-09-19T07:37:58.398419+0000","last_clean":"2024-09-19T07:37:58.398419+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:37:58.398419+0000","last_undegraded":"2024-09-19T07:37:58.398419+0000","last_fullsized":"2024-09-19T07:37:58.398419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278876+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:37:58.278876+0000","last_peered":"2024-09-19T07:37:58.278876+0000","last_clean":"2024-09-19T07:37:58.278876+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:37:58.278876+0000","last_undegraded":"2024-09-19T07:37:58.278876+0000","last_fullsized":"2024-09-19T07:37:58.278876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'51","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.530637+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:37:58.530637+0000","last_peered":"2024-09-19T07:37:58.530637+0000","last_clean":"2024-09-19T07:37:58.530637+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:37:58.530637+0000","last_undegraded":"2024-09-19T07:37:58.530637+0000","last_fullsized":"2024-09-19T07:37:58.530637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400100+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:37:58.400100+0000","last_peered":"2024-09-19T07:37:58.400100+0000","last_clean":"2024-09-19T07:37:58.400100+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:37:58.400100+0000","last_undegraded":"2024-09-19T07:37:58.400100+0000","last_fullsized":"2024-09-19T07:37:58.400100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566566+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:37:58.566566+0000","last_peered":"2024-09-19T07:37:58.566566+0000","last_clean":"2024-09-19T07:37:58.566566+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:37:58.566566+0000","last_undegraded":"2024-09-19T07:37:58.566566+0000","last_fullsized":"2024-09-19T07:37:58.566566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566745+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:37:58.566745+0000","last_peered":"2024-09-19T07:37:58.566745+0000","last_clean":"2024-09-19T07:37:58.566745+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:37:58.566745+0000","last_undegraded":"2024-09-19T07:37:58.566745+0000","last_fullsized":"2024-09-19T07:37:58.566745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277436+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:37:58.277436+0000","last_peered":"2024-09-19T07:37:58.277436+0000","last_clean":"2024-09-19T07:37:58.277436+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:37:58.277436+0000","last_undegraded":"2024-09-19T07:37:58.277436+0000","last_fullsized":"2024-09-19T07:37:58.277436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.397993+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:37:58.397993+0000","last_peered":"2024-09-19T07:37:58.397993+0000","last_clean":"2024-09-19T07:37:58.397993+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:37:58.397993+0000","last_undegraded":"2024-09-19T07:37:58.397993+0000","last_fullsized":"2024-09-19T07:37:58.397993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398888+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:37:58.398888+0000","last_peered":"2024-09-19T07:37:58.398888+0000","last_clean":"2024-09-19T07:37:58.398888+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:37:58.398888+0000","last_undegraded":"2024-09-19T07:37:58.398888+0000","last_fullsized":"2024-09-19T07:37:58.398888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565988+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:37:58.565988+0000","last_peered":"2024-09-19T07:37:58.565988+0000","last_clean":"2024-09-19T07:37:58.565988+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:37:58.565988+0000","last_undegraded":"2024-09-19T07:37:58.565988+0000","last_fullsized":"2024-09-19T07:37:58.565988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565778+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:37:58.565778+0000","last_peered":"2024-09-19T07:37:58.565778+0000","last_clean":"2024-09-19T07:37:58.565778+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:37:58.565778+0000","last_undegraded":"2024-09-19T07:37:58.565778+0000","last_fullsized":"2024-09-19T07:37:58.565778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399648+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:37:58.399648+0000","last_peered":"2024-09-19T07:37:58.399648+0000","last_clean":"2024-09-19T07:37:58.399648+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:37:58.399648+0000","last_undegraded":"2024-09-19T07:37:58.399648+0000","last_fullsized":"2024-09-19T07:37:58.399648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398085+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:37:58.398085+0000","last_peered":"2024-09-19T07:37:58.398085+0000","last_clean":"2024-09-19T07:37:58.398085+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:37:58.398085+0000","last_undegraded":"2024-09-19T07:37:58.398085+0000","last_fullsized":"2024-09-19T07:37:58.398085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568069+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:37:58.568069+0000","last_peered":"2024-09-19T07:37:58.568069+0000","last_clean":"2024-09-19T07:37:58.568069+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:37:58.568069+0000","last_undegraded":"2024-09-19T07:37:58.568069+0000","last_fullsized":"2024-09-19T07:37:58.568069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395362+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:37:58.395362+0000","last_peered":"2024-09-19T07:37:58.395362+0000","last_clean":"2024-09-19T07:37:58.395362+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:37:58.395362+0000","last_undegraded":"2024-09-19T07:37:58.395362+0000","last_fullsized":"2024-09-19T07:37:58.395362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398241+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:37:58.398241+0000","last_peered":"2024-09-19T07:37:58.398241+0000","last_clean":"2024-09-19T07:37:58.398241+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:37:58.398241+0000","last_undegraded":"2024-09-19T07:37:58.398241+0000","last_fullsized":"2024-09-19T07:37:58.398241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277959+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:37:58.277959+0000","last_peered":"2024-09-19T07:37:58.277959+0000","last_clean":"2024-09-19T07:37:58.277959+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:37:58.277959+0000","last_undegraded":"2024-09-19T07:37:58.277959+0000","last_fullsized":"2024-09-19T07:37:58.277959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398335+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:37:58.398335+0000","last_peered":"2024-09-19T07:37:58.398335+0000","last_clean":"2024-09-19T07:37:58.398335+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:37:58.398335+0000","last_undegraded":"2024-09-19T07:37:58.398335+0000","last_fullsized":"2024-09-19T07:37:58.398335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566969+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:37:58.566969+0000","last_peered":"2024-09-19T07:37:58.566969+0000","last_clean":"2024-09-19T07:37:58.566969+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:37:58.566969+0000","last_undegraded":"2024-09-19T07:37:58.566969+0000","last_fullsized":"2024-09-19T07:37:58.566969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400607+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:37:58.400607+0000","last_peered":"2024-09-19T07:37:58.400607+0000","last_clean":"2024-09-19T07:37:58.400607+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:37:58.400607+0000","last_undegraded":"2024-09-19T07:37:58.400607+0000","last_fullsized":"2024-09-19T07:37:58.400607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400322+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:37:58.400322+0000","last_peered":"2024-09-19T07:37:58.400322+0000","last_clean":"2024-09-19T07:37:58.400322+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:37:58.400322+0000","last_undegraded":"2024-09-19T07:37:58.400322+0000","last_fullsized":"2024-09-19T07:37:58.400322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'14","reported_seq":155,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396388+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:37:58.396388+0000","last_peered":"2024-09-19T07:37:58.396388+0000","last_clean":"2024-09-19T07:37:58.396388+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:37:58.396388+0000","last_undegraded":"2024-09-19T07:37:58.396388+0000","last_fullsized":"2024-09-19T07:37:58.396388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602652,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1142,"num_read_kb":690,"num_write":376,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1807956,"data_compressed":80685,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107370,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":451633,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":1243176,"data_compressed":52251,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441568,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":441162,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1044883,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528381,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":450775,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1174307,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":451422,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":974882,"data_compressed":34014,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":520934,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":520993,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":452507,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71591,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71532,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":203289,"data_compressed":7749,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:38:01.668 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T07:38:01.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T07:38:01.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:01.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-19T07:38:01.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:01.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.671 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.672 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T07:38:01.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:01.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-19T07:38:01.673 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T07:38:01.674 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:01 smithi174 ceph-mon[35370]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 -- 172.21.15.103:0/3377959729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 msgr2=0x7f0aec104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 --2- 172.21.15.103:0/3377959729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec104a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009900 tx=0x7f0adc02a410 comp rx=0 tx=0).stop 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 -- 172.21.15.103:0/3377959729 shutdown_connections 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 --2- 172.21.15.103:0/3377959729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec107890 0x7f0aec109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 --2- 172.21.15.103:0/3377959729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec104f60 0x7f0aec107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 --2- 172.21.15.103:0/3377959729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec104a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 -- 172.21.15.103:0/3377959729 >> 172.21.15.103:0/3377959729 conn(0x7f0aec0fc170 msgr2=0x7f0aec0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 -- 172.21.15.103:0/3377959729 shutdown_connections 2024-09-19T07:38:01.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 -- 172.21.15.103:0/3377959729 wait complete. 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 Processor -- start 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.808+0000 7f0af26d0640 1 -- start start 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af26d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af26d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec104f60 0x7f0aec1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af26d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec107890 0x7f0aec19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af26d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec10d800 con 0x7f0aec102630 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af26d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec10d680 con 0x7f0aec107890 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af26d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec10d980 con 0x7f0aec104f60 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45796/0 (socket says 172.21.15.103:45796) 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 -- 172.21.15.103:0/1388889144 learned_addr learned my addr 172.21.15.103:0/1388889144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af0c6d640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec107890 0x7f0aec19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0ae3fff640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec104f60 0x7f0aec1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 -- 172.21.15.103:0/1388889144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec104f60 msgr2=0x7f0aec1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec104f60 0x7f0aec1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 -- 172.21.15.103:0/1388889144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec107890 msgr2=0x7f0aec19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec107890 0x7f0aec19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 -- 172.21.15.103:0/1388889144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec19fad0 con 0x7f0aec102630 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0ae3fff640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec104f60 0x7f0aec1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0aebfff640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec197eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0adc031440 tx=0x7f0adc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0ae9ffb640 1 -- 172.21.15.103:0/1388889144 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc03f030 con 0x7f0aec102630 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0ae9ffb640 1 -- 172.21.15.103:0/1388889144 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0adc003ce0 con 0x7f0aec102630 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec19fd60 con 0x7f0aec102630 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.809+0000 7f0ae9ffb640 1 -- 172.21.15.103:0/1388889144 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc035070 con 0x7f0aec102630 2024-09-19T07:38:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.810+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec1a01a0 con 0x7f0aec102630 2024-09-19T07:38:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.811+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec0fdc60 con 0x7f0aec102630 2024-09-19T07:38:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.812+0000 7f0ae9ffb640 1 -- 172.21.15.103:0/1388889144 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7f0adc042030 con 0x7f0aec102630 2024-09-19T07:38:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.812+0000 7f0ae9ffb640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0ad80a33c0 0x7f0ad80a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:01.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.812+0000 7f0ae9ffb640 1 -- 172.21.15.103:0/1388889144 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0adc0f0280 con 0x7f0aec102630 2024-09-19T07:38:01.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.812+0000 7f0ae3fff640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0ad80a33c0 0x7f0ad80a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:01.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.813+0000 7f0ae3fff640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0ad80a33c0 0x7f0ad80a5880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0aec1993c0 tx=0x7f0ad4005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:01.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.816+0000 7f0ae9ffb640 1 -- 172.21.15.103:0/1388889144 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0adc0bcc70 con 0x7f0aec102630 2024-09-19T07:38:01.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.991+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0aec062630 con 0x7f0ad80a33c0 2024-09-19T07:38:01.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:01.998+0000 7f0ae9ffb640 1 -- 172.21.15.103:0/1388889144 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258076 (secure 0 0 0) 0x7f0aec062630 con 0x7f0ad80a33c0 2024-09-19T07:38:02.001 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:02.003 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:38:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0ad80a33c0 msgr2=0x7f0ad80a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0ad80a33c0 0x7f0ad80a5880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0aec1993c0 tx=0x7f0ad4005f90 comp rx=0 tx=0).stop 2024-09-19T07:38:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 msgr2=0x7f0aec197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec197eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0adc031440 tx=0x7f0adc032ce0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 shutdown_connections 2024-09-19T07:38:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0ad80a33c0 0x7f0ad80a5880 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0aec107890 0x7f0aec19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec104f60 0x7f0aec1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 --2- 172.21.15.103:0/1388889144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aec102630 0x7f0aec197eb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 >> 172.21.15.103:0/1388889144 conn(0x7f0aec0fc170 msgr2=0x7f0aec102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 shutdown_connections 2024-09-19T07:38:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.001+0000 7f0af26d0640 1 -- 172.21.15.103:0/1388889144 wait complete. 2024-09-19T07:38:02.016 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-19T07:38:00.293143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193254,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1278,"num_read_kb":809,"num_write":559,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344136,"kb_used_data":11496,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597688,"statfs":{"total":1151940427776,"available":1151588032512,"internally_reserved":0,"allocated":11771904,"data_stored":8168242,"data_compressed":248501,"data_compressed_allocated":335872,"data_compressed_original":2834432,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.993569"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566027+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:37:58.566027+0000","last_peered":"2024-09-19T07:37:58.566027+0000","last_clean":"2024-09-19T07:37:58.566027+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:37:58.566027+0000","last_undegraded":"2024-09-19T07:37:58.566027+0000","last_fullsized":"2024-09-19T07:37:58.566027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566607+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:37:58.566607+0000","last_peered":"2024-09-19T07:37:58.566607+0000","last_clean":"2024-09-19T07:37:58.566607+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:37:58.566607+0000","last_undegraded":"2024-09-19T07:37:58.566607+0000","last_fullsized":"2024-09-19T07:37:58.566607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566643+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:37:58.566643+0000","last_peered":"2024-09-19T07:37:58.566643+0000","last_clean":"2024-09-19T07:37:58.566643+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:37:58.566643+0000","last_undegraded":"2024-09-19T07:37:58.566643+0000","last_fullsized":"2024-09-19T07:37:58.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395953+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:37:58.395953+0000","last_peered":"2024-09-19T07:37:58.395953+0000","last_clean":"2024-09-19T07:37:58.395953+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:37:58.395953+0000","last_undegraded":"2024-09-19T07:37:58.395953+0000","last_fullsized":"2024-09-19T07:37:58.395953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398077+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:37:58.398077+0000","last_peered":"2024-09-19T07:37:58.398077+0000","last_clean":"2024-09-19T07:37:58.398077+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:37:58.398077+0000","last_undegraded":"2024-09-19T07:37:58.398077+0000","last_fullsized":"2024-09-19T07:37:58.398077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408847+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:37:58.408847+0000","last_peered":"2024-09-19T07:37:58.408847+0000","last_clean":"2024-09-19T07:37:58.408847+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:37:58.408847+0000","last_undegraded":"2024-09-19T07:37:58.408847+0000","last_fullsized":"2024-09-19T07:37:58.408847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567869+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:37:58.567869+0000","last_peered":"2024-09-19T07:37:58.567869+0000","last_clean":"2024-09-19T07:37:58.567869+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:37:58.567869+0000","last_undegraded":"2024-09-19T07:37:58.567869+0000","last_fullsized":"2024-09-19T07:37:58.567869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398332+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:37:58.398332+0000","last_peered":"2024-09-19T07:37:58.398332+0000","last_clean":"2024-09-19T07:37:58.398332+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:37:58.398332+0000","last_undegraded":"2024-09-19T07:37:58.398332+0000","last_fullsized":"2024-09-19T07:37:58.398332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278587+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:37:58.278587+0000","last_peered":"2024-09-19T07:37:58.278587+0000","last_clean":"2024-09-19T07:37:58.278587+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:37:58.278587+0000","last_undegraded":"2024-09-19T07:37:58.278587+0000","last_fullsized":"2024-09-19T07:37:58.278587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409236+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:37:58.409236+0000","last_peered":"2024-09-19T07:37:58.409236+0000","last_clean":"2024-09-19T07:37:58.409236+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:37:58.409236+0000","last_undegraded":"2024-09-19T07:37:58.409236+0000","last_fullsized":"2024-09-19T07:37:58.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568581+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:37:58.568581+0000","last_peered":"2024-09-19T07:37:58.568581+0000","last_clean":"2024-09-19T07:37:58.568581+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:37:58.568581+0000","last_undegraded":"2024-09-19T07:37:58.568581+0000","last_fullsized":"2024-09-19T07:37:58.568581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566381+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:37:58.566381+0000","last_peered":"2024-09-19T07:37:58.566381+0000","last_clean":"2024-09-19T07:37:58.566381+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:37:58.566381+0000","last_undegraded":"2024-09-19T07:37:58.566381+0000","last_fullsized":"2024-09-19T07:37:58.566381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278860+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:37:58.278860+0000","last_peered":"2024-09-19T07:37:58.278860+0000","last_clean":"2024-09-19T07:37:58.278860+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:37:58.278860+0000","last_undegraded":"2024-09-19T07:37:58.278860+0000","last_fullsized":"2024-09-19T07:37:58.278860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566247+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:37:58.566247+0000","last_peered":"2024-09-19T07:37:58.566247+0000","last_clean":"2024-09-19T07:37:58.566247+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:37:58.566247+0000","last_undegraded":"2024-09-19T07:37:58.566247+0000","last_fullsized":"2024-09-19T07:37:58.566247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566034+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:37:58.566034+0000","last_peered":"2024-09-19T07:37:58.566034+0000","last_clean":"2024-09-19T07:37:58.566034+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:37:58.566034+0000","last_undegraded":"2024-09-19T07:37:58.566034+0000","last_fullsized":"2024-09-19T07:37:58.566034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277343+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:37:58.277343+0000","last_peered":"2024-09-19T07:37:58.277343+0000","last_clean":"2024-09-19T07:37:58.277343+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:37:58.277343+0000","last_undegraded":"2024-09-19T07:37:58.277343+0000","last_fullsized":"2024-09-19T07:37:58.277343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566048+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:37:58.566048+0000","last_peered":"2024-09-19T07:37:58.566048+0000","last_clean":"2024-09-19T07:37:58.566048+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:37:58.566048+0000","last_undegraded":"2024-09-19T07:37:58.566048+0000","last_fullsized":"2024-09-19T07:37:58.566048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395689+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:37:58.395689+0000","last_peered":"2024-09-19T07:37:58.395689+0000","last_clean":"2024-09-19T07:37:58.395689+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:37:58.395689+0000","last_undegraded":"2024-09-19T07:37:58.395689+0000","last_fullsized":"2024-09-19T07:37:58.395689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281089+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:37:58.281089+0000","last_peered":"2024-09-19T07:37:58.281089+0000","last_clean":"2024-09-19T07:37:58.281089+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:37:58.281089+0000","last_undegraded":"2024-09-19T07:37:58.281089+0000","last_fullsized":"2024-09-19T07:37:58.281089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565798+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:37:58.565798+0000","last_peered":"2024-09-19T07:37:58.565798+0000","last_clean":"2024-09-19T07:37:58.565798+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:37:58.565798+0000","last_undegraded":"2024-09-19T07:37:58.565798+0000","last_fullsized":"2024-09-19T07:37:58.565798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278866+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:37:58.278866+0000","last_peered":"2024-09-19T07:37:58.278866+0000","last_clean":"2024-09-19T07:37:58.278866+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:37:58.278866+0000","last_undegraded":"2024-09-19T07:37:58.278866+0000","last_fullsized":"2024-09-19T07:37:58.278866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277508+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:37:58.277508+0000","last_peered":"2024-09-19T07:37:58.277508+0000","last_clean":"2024-09-19T07:37:58.277508+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:37:58.277508+0000","last_undegraded":"2024-09-19T07:37:58.277508+0000","last_fullsized":"2024-09-19T07:37:58.277508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398644+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:37:58.398644+0000","last_peered":"2024-09-19T07:37:58.398644+0000","last_clean":"2024-09-19T07:37:58.398644+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:37:58.398644+0000","last_undegraded":"2024-09-19T07:37:58.398644+0000","last_fullsized":"2024-09-19T07:37:58.398644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405691+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:37:58.405691+0000","last_peered":"2024-09-19T07:37:58.405691+0000","last_clean":"2024-09-19T07:37:58.405691+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:37:58.405691+0000","last_undegraded":"2024-09-19T07:37:58.405691+0000","last_fullsized":"2024-09-19T07:37:58.405691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399976+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:37:58.399976+0000","last_peered":"2024-09-19T07:37:58.399976+0000","last_clean":"2024-09-19T07:37:58.399976+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:37:58.399976+0000","last_undegraded":"2024-09-19T07:37:58.399976+0000","last_fullsized":"2024-09-19T07:37:58.399976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281203+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:37:58.281203+0000","last_peered":"2024-09-19T07:37:58.281203+0000","last_clean":"2024-09-19T07:37:58.281203+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:37:58.281203+0000","last_undegraded":"2024-09-19T07:37:58.281203+0000","last_fullsized":"2024-09-19T07:37:58.281203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565955+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:37:58.565955+0000","last_peered":"2024-09-19T07:37:58.565955+0000","last_clean":"2024-09-19T07:37:58.565955+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:37:58.565955+0000","last_undegraded":"2024-09-19T07:37:58.565955+0000","last_fullsized":"2024-09-19T07:37:58.565955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278990+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:37:58.278990+0000","last_peered":"2024-09-19T07:37:58.278990+0000","last_clean":"2024-09-19T07:37:58.278990+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:37:58.278990+0000","last_undegraded":"2024-09-19T07:37:58.278990+0000","last_fullsized":"2024-09-19T07:37:58.278990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566392+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:37:58.566392+0000","last_peered":"2024-09-19T07:37:58.566392+0000","last_clean":"2024-09-19T07:37:58.566392+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:37:58.566392+0000","last_undegraded":"2024-09-19T07:37:58.566392+0000","last_fullsized":"2024-09-19T07:37:58.566392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280791+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:37:58.280791+0000","last_peered":"2024-09-19T07:37:58.280791+0000","last_clean":"2024-09-19T07:37:58.280791+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:37:58.280791+0000","last_undegraded":"2024-09-19T07:37:58.280791+0000","last_fullsized":"2024-09-19T07:37:58.280791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280919+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:37:58.280919+0000","last_peered":"2024-09-19T07:37:58.280919+0000","last_clean":"2024-09-19T07:37:58.280919+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:37:58.280919+0000","last_undegraded":"2024-09-19T07:37:58.280919+0000","last_fullsized":"2024-09-19T07:37:58.280919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565810+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:37:58.565810+0000","last_peered":"2024-09-19T07:37:58.565810+0000","last_clean":"2024-09-19T07:37:58.565810+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:37:58.565810+0000","last_undegraded":"2024-09-19T07:37:58.565810+0000","last_fullsized":"2024-09-19T07:37:58.565810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398122+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:37:58.398122+0000","last_peered":"2024-09-19T07:37:58.398122+0000","last_clean":"2024-09-19T07:37:58.398122+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:37:58.398122+0000","last_undegraded":"2024-09-19T07:37:58.398122+0000","last_fullsized":"2024-09-19T07:37:58.398122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406021+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:37:58.406021+0000","last_peered":"2024-09-19T07:37:58.406021+0000","last_clean":"2024-09-19T07:37:58.406021+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:37:58.406021+0000","last_undegraded":"2024-09-19T07:37:58.406021+0000","last_fullsized":"2024-09-19T07:37:58.406021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405841+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:37:58.405841+0000","last_peered":"2024-09-19T07:37:58.405841+0000","last_clean":"2024-09-19T07:37:58.405841+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:37:58.405841+0000","last_undegraded":"2024-09-19T07:37:58.405841+0000","last_fullsized":"2024-09-19T07:37:58.405841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408925+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:37:58.408925+0000","last_peered":"2024-09-19T07:37:58.408925+0000","last_clean":"2024-09-19T07:37:58.408925+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:37:58.408925+0000","last_undegraded":"2024-09-19T07:37:58.408925+0000","last_fullsized":"2024-09-19T07:37:58.408925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278608+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:37:58.278608+0000","last_peered":"2024-09-19T07:37:58.278608+0000","last_clean":"2024-09-19T07:37:58.278608+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:37:58.278608+0000","last_undegraded":"2024-09-19T07:37:58.278608+0000","last_fullsized":"2024-09-19T07:37:58.278608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398404+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:37:58.398404+0000","last_peered":"2024-09-19T07:37:58.398404+0000","last_clean":"2024-09-19T07:37:58.398404+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:37:58.398404+0000","last_undegraded":"2024-09-19T07:37:58.398404+0000","last_fullsized":"2024-09-19T07:37:58.398404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406017+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:37:58.406017+0000","last_peered":"2024-09-19T07:37:58.406017+0000","last_clean":"2024-09-19T07:37:58.406017+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:37:58.406017+0000","last_undegraded":"2024-09-19T07:37:58.406017+0000","last_fullsized":"2024-09-19T07:37:58.406017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395465+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:37:58.395465+0000","last_peered":"2024-09-19T07:37:58.395465+0000","last_clean":"2024-09-19T07:37:58.395465+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:37:58.395465+0000","last_undegraded":"2024-09-19T07:37:58.395465+0000","last_fullsized":"2024-09-19T07:37:58.395465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":229,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.042696+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:37:59.042696+0000","last_peered":"2024-09-19T07:37:59.042696+0000","last_clean":"2024-09-19T07:37:59.042696+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:37:59.042696+0000","last_undegraded":"2024-09-19T07:37:59.042696+0000","last_fullsized":"2024-09-19T07:37:59.042696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278242+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:37:58.278242+0000","last_peered":"2024-09-19T07:37:58.278242+0000","last_clean":"2024-09-19T07:37:58.278242+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:37:58.278242+0000","last_undegraded":"2024-09-19T07:37:58.278242+0000","last_fullsized":"2024-09-19T07:37:58.278242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568741+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:37:58.568741+0000","last_peered":"2024-09-19T07:37:58.568741+0000","last_clean":"2024-09-19T07:37:58.568741+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:37:58.568741+0000","last_undegraded":"2024-09-19T07:37:58.568741+0000","last_fullsized":"2024-09-19T07:37:58.568741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567021+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:37:58.567021+0000","last_peered":"2024-09-19T07:37:58.567021+0000","last_clean":"2024-09-19T07:37:58.567021+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:37:58.567021+0000","last_undegraded":"2024-09-19T07:37:58.567021+0000","last_fullsized":"2024-09-19T07:37:58.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405996+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:37:58.405996+0000","last_peered":"2024-09-19T07:37:58.405996+0000","last_clean":"2024-09-19T07:37:58.405996+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:37:58.405996+0000","last_undegraded":"2024-09-19T07:37:58.405996+0000","last_fullsized":"2024-09-19T07:37:58.405996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277409+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:37:58.277409+0000","last_peered":"2024-09-19T07:37:58.277409+0000","last_clean":"2024-09-19T07:37:58.277409+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:37:58.277409+0000","last_undegraded":"2024-09-19T07:37:58.277409+0000","last_fullsized":"2024-09-19T07:37:58.277409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":274,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.043101+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:37:59.043101+0000","last_peered":"2024-09-19T07:37:59.043101+0000","last_clean":"2024-09-19T07:37:59.043101+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:37:59.043101+0000","last_undegraded":"2024-09-19T07:37:59.043101+0000","last_fullsized":"2024-09-19T07:37:59.043101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":173,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398289+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:37:58.398289+0000","last_peered":"2024-09-19T07:37:58.398289+0000","last_clean":"2024-09-19T07:37:58.398289+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:37:58.398289+0000","last_undegraded":"2024-09-19T07:37:58.398289+0000","last_fullsized":"2024-09-19T07:37:58.398289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280980+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:37:58.280980+0000","last_peered":"2024-09-19T07:37:58.280980+0000","last_clean":"2024-09-19T07:37:58.280980+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:37:58.280980+0000","last_undegraded":"2024-09-19T07:37:58.280980+0000","last_fullsized":"2024-09-19T07:37:58.280980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278729+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:37:58.278729+0000","last_peered":"2024-09-19T07:37:58.278729+0000","last_clean":"2024-09-19T07:37:58.278729+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:37:58.278729+0000","last_undegraded":"2024-09-19T07:37:58.278729+0000","last_fullsized":"2024-09-19T07:37:58.278729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405861+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:37:58.405861+0000","last_peered":"2024-09-19T07:37:58.405861+0000","last_clean":"2024-09-19T07:37:58.405861+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:37:58.405861+0000","last_undegraded":"2024-09-19T07:37:58.405861+0000","last_fullsized":"2024-09-19T07:37:58.405861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568411+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:37:58.568411+0000","last_peered":"2024-09-19T07:37:58.568411+0000","last_clean":"2024-09-19T07:37:58.568411+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:37:58.568411+0000","last_undegraded":"2024-09-19T07:37:58.568411+0000","last_fullsized":"2024-09-19T07:37:58.568411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'9","reported_seq":138,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.042928+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:37:59.042928+0000","last_peered":"2024-09-19T07:37:59.042928+0000","last_clean":"2024-09-19T07:37:59.042928+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:37:59.042928+0000","last_undegraded":"2024-09-19T07:37:59.042928+0000","last_fullsized":"2024-09-19T07:37:59.042928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567829+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:37:58.567829+0000","last_peered":"2024-09-19T07:37:58.567829+0000","last_clean":"2024-09-19T07:37:58.567829+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:37:58.567829+0000","last_undegraded":"2024-09-19T07:37:58.567829+0000","last_fullsized":"2024-09-19T07:37:58.567829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565722+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:37:58.565722+0000","last_peered":"2024-09-19T07:37:58.565722+0000","last_clean":"2024-09-19T07:37:58.565722+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:37:58.565722+0000","last_undegraded":"2024-09-19T07:37:58.565722+0000","last_fullsized":"2024-09-19T07:37:58.565722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405806+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:37:58.405806+0000","last_peered":"2024-09-19T07:37:58.405806+0000","last_clean":"2024-09-19T07:37:58.405806+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:37:58.405806+0000","last_undegraded":"2024-09-19T07:37:58.405806+0000","last_fullsized":"2024-09-19T07:37:58.405806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398454+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:37:58.398454+0000","last_peered":"2024-09-19T07:37:58.398454+0000","last_clean":"2024-09-19T07:37:58.398454+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:37:58.398454+0000","last_undegraded":"2024-09-19T07:37:58.398454+0000","last_fullsized":"2024-09-19T07:37:58.398454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566085+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:37:58.566085+0000","last_peered":"2024-09-19T07:37:58.566085+0000","last_clean":"2024-09-19T07:37:58.566085+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:37:58.566085+0000","last_undegraded":"2024-09-19T07:37:58.566085+0000","last_fullsized":"2024-09-19T07:37:58.566085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278747+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:37:58.278747+0000","last_peered":"2024-09-19T07:37:58.278747+0000","last_clean":"2024-09-19T07:37:58.278747+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:37:58.278747+0000","last_undegraded":"2024-09-19T07:37:58.278747+0000","last_fullsized":"2024-09-19T07:37:58.278747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568790+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:37:58.568790+0000","last_peered":"2024-09-19T07:37:58.568790+0000","last_clean":"2024-09-19T07:37:58.568790+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:37:58.568790+0000","last_undegraded":"2024-09-19T07:37:58.568790+0000","last_fullsized":"2024-09-19T07:37:58.568790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'41","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.523245+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:37:58.523245+0000","last_peered":"2024-09-19T07:37:58.523245+0000","last_clean":"2024-09-19T07:37:58.523245+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:37:58.523245+0000","last_undegraded":"2024-09-19T07:37:58.523245+0000","last_fullsized":"2024-09-19T07:37:58.523245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568493+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:37:58.568493+0000","last_peered":"2024-09-19T07:37:58.568493+0000","last_clean":"2024-09-19T07:37:58.568493+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:37:58.568493+0000","last_undegraded":"2024-09-19T07:37:58.568493+0000","last_fullsized":"2024-09-19T07:37:58.568493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":137,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:59.043379+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:37:59.043379+0000","last_peered":"2024-09-19T07:37:59.043379+0000","last_clean":"2024-09-19T07:37:59.043379+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:37:59.043379+0000","last_undegraded":"2024-09-19T07:37:59.043379+0000","last_fullsized":"2024-09-19T07:37:59.043379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568589+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:37:58.568589+0000","last_peered":"2024-09-19T07:37:58.568589+0000","last_clean":"2024-09-19T07:37:58.568589+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:37:58.568589+0000","last_undegraded":"2024-09-19T07:37:58.568589+0000","last_fullsized":"2024-09-19T07:37:58.568589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567999+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:37:58.567999+0000","last_peered":"2024-09-19T07:37:58.567999+0000","last_clean":"2024-09-19T07:37:58.567999+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:37:58.567999+0000","last_undegraded":"2024-09-19T07:37:58.567999+0000","last_fullsized":"2024-09-19T07:37:58.567999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566436+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:37:58.566436+0000","last_peered":"2024-09-19T07:37:58.566436+0000","last_clean":"2024-09-19T07:37:58.566436+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:37:58.566436+0000","last_undegraded":"2024-09-19T07:37:58.566436+0000","last_fullsized":"2024-09-19T07:37:58.566436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399251+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:37:58.399251+0000","last_peered":"2024-09-19T07:37:58.399251+0000","last_clean":"2024-09-19T07:37:58.399251+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:37:58.399251+0000","last_undegraded":"2024-09-19T07:37:58.399251+0000","last_fullsized":"2024-09-19T07:37:58.399251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567920+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:37:58.567920+0000","last_peered":"2024-09-19T07:37:58.567920+0000","last_clean":"2024-09-19T07:37:58.567920+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:37:58.567920+0000","last_undegraded":"2024-09-19T07:37:58.567920+0000","last_fullsized":"2024-09-19T07:37:58.567920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278577+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:37:58.278577+0000","last_peered":"2024-09-19T07:37:58.278577+0000","last_clean":"2024-09-19T07:37:58.278577+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:37:58.278577+0000","last_undegraded":"2024-09-19T07:37:58.278577+0000","last_fullsized":"2024-09-19T07:37:58.278577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278764+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:37:58.278764+0000","last_peered":"2024-09-19T07:37:58.278764+0000","last_clean":"2024-09-19T07:37:58.278764+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:37:58.278764+0000","last_undegraded":"2024-09-19T07:37:58.278764+0000","last_fullsized":"2024-09-19T07:37:58.278764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405844+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:37:58.405844+0000","last_peered":"2024-09-19T07:37:58.405844+0000","last_clean":"2024-09-19T07:37:58.405844+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:37:58.405844+0000","last_undegraded":"2024-09-19T07:37:58.405844+0000","last_fullsized":"2024-09-19T07:37:58.405844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":134,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568315+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:37:58.568315+0000","last_peered":"2024-09-19T07:37:58.568315+0000","last_clean":"2024-09-19T07:37:58.568315+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:37:58.568315+0000","last_undegraded":"2024-09-19T07:37:58.568315+0000","last_fullsized":"2024-09-19T07:37:58.568315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408965+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:37:58.408965+0000","last_peered":"2024-09-19T07:37:58.408965+0000","last_clean":"2024-09-19T07:37:58.408965+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:37:58.408965+0000","last_undegraded":"2024-09-19T07:37:58.408965+0000","last_fullsized":"2024-09-19T07:37:58.408965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396146+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:37:58.396146+0000","last_peered":"2024-09-19T07:37:58.396146+0000","last_clean":"2024-09-19T07:37:58.396146+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:37:58.396146+0000","last_undegraded":"2024-09-19T07:37:58.396146+0000","last_fullsized":"2024-09-19T07:37:58.396146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409093+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:37:58.409093+0000","last_peered":"2024-09-19T07:37:58.409093+0000","last_clean":"2024-09-19T07:37:58.409093+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:37:58.409093+0000","last_undegraded":"2024-09-19T07:37:58.409093+0000","last_fullsized":"2024-09-19T07:37:58.409093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398419+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:37:58.398419+0000","last_peered":"2024-09-19T07:37:58.398419+0000","last_clean":"2024-09-19T07:37:58.398419+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:37:58.398419+0000","last_undegraded":"2024-09-19T07:37:58.398419+0000","last_fullsized":"2024-09-19T07:37:58.398419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278876+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:37:58.278876+0000","last_peered":"2024-09-19T07:37:58.278876+0000","last_clean":"2024-09-19T07:37:58.278876+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:37:58.278876+0000","last_undegraded":"2024-09-19T07:37:58.278876+0000","last_fullsized":"2024-09-19T07:37:58.278876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'51","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.530637+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:37:58.530637+0000","last_peered":"2024-09-19T07:37:58.530637+0000","last_clean":"2024-09-19T07:37:58.530637+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:37:58.530637+0000","last_undegraded":"2024-09-19T07:37:58.530637+0000","last_fullsized":"2024-09-19T07:37:58.530637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400100+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:37:58.400100+0000","last_peered":"2024-09-19T07:37:58.400100+0000","last_clean":"2024-09-19T07:37:58.400100+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:37:58.400100+0000","last_undegraded":"2024-09-19T07:37:58.400100+0000","last_fullsized":"2024-09-19T07:37:58.400100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566566+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:37:58.566566+0000","last_peered":"2024-09-19T07:37:58.566566+0000","last_clean":"2024-09-19T07:37:58.566566+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:37:58.566566+0000","last_undegraded":"2024-09-19T07:37:58.566566+0000","last_fullsized":"2024-09-19T07:37:58.566566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566745+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:37:58.566745+0000","last_peered":"2024-09-19T07:37:58.566745+0000","last_clean":"2024-09-19T07:37:58.566745+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:37:58.566745+0000","last_undegraded":"2024-09-19T07:37:58.566745+0000","last_fullsized":"2024-09-19T07:37:58.566745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277436+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:37:58.277436+0000","last_peered":"2024-09-19T07:37:58.277436+0000","last_clean":"2024-09-19T07:37:58.277436+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:37:58.277436+0000","last_undegraded":"2024-09-19T07:37:58.277436+0000","last_fullsized":"2024-09-19T07:37:58.277436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.397993+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:37:58.397993+0000","last_peered":"2024-09-19T07:37:58.397993+0000","last_clean":"2024-09-19T07:37:58.397993+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:37:58.397993+0000","last_undegraded":"2024-09-19T07:37:58.397993+0000","last_fullsized":"2024-09-19T07:37:58.397993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398888+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:37:58.398888+0000","last_peered":"2024-09-19T07:37:58.398888+0000","last_clean":"2024-09-19T07:37:58.398888+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:37:58.398888+0000","last_undegraded":"2024-09-19T07:37:58.398888+0000","last_fullsized":"2024-09-19T07:37:58.398888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565988+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:37:58.565988+0000","last_peered":"2024-09-19T07:37:58.565988+0000","last_clean":"2024-09-19T07:37:58.565988+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:37:58.565988+0000","last_undegraded":"2024-09-19T07:37:58.565988+0000","last_fullsized":"2024-09-19T07:37:58.565988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565778+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:37:58.565778+0000","last_peered":"2024-09-19T07:37:58.565778+0000","last_clean":"2024-09-19T07:37:58.565778+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:37:58.565778+0000","last_undegraded":"2024-09-19T07:37:58.565778+0000","last_fullsized":"2024-09-19T07:37:58.565778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399648+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:37:58.399648+0000","last_peered":"2024-09-19T07:37:58.399648+0000","last_clean":"2024-09-19T07:37:58.399648+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:37:58.399648+0000","last_undegraded":"2024-09-19T07:37:58.399648+0000","last_fullsized":"2024-09-19T07:37:58.399648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398085+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:37:58.398085+0000","last_peered":"2024-09-19T07:37:58.398085+0000","last_clean":"2024-09-19T07:37:58.398085+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:37:58.398085+0000","last_undegraded":"2024-09-19T07:37:58.398085+0000","last_fullsized":"2024-09-19T07:37:58.398085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568069+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:37:58.568069+0000","last_peered":"2024-09-19T07:37:58.568069+0000","last_clean":"2024-09-19T07:37:58.568069+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:37:58.568069+0000","last_undegraded":"2024-09-19T07:37:58.568069+0000","last_fullsized":"2024-09-19T07:37:58.568069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395362+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:37:58.395362+0000","last_peered":"2024-09-19T07:37:58.395362+0000","last_clean":"2024-09-19T07:37:58.395362+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:37:58.395362+0000","last_undegraded":"2024-09-19T07:37:58.395362+0000","last_fullsized":"2024-09-19T07:37:58.395362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398241+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:37:58.398241+0000","last_peered":"2024-09-19T07:37:58.398241+0000","last_clean":"2024-09-19T07:37:58.398241+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:37:58.398241+0000","last_undegraded":"2024-09-19T07:37:58.398241+0000","last_fullsized":"2024-09-19T07:37:58.398241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277959+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:37:58.277959+0000","last_peered":"2024-09-19T07:37:58.277959+0000","last_clean":"2024-09-19T07:37:58.277959+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:37:58.277959+0000","last_undegraded":"2024-09-19T07:37:58.277959+0000","last_fullsized":"2024-09-19T07:37:58.277959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398335+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:37:58.398335+0000","last_peered":"2024-09-19T07:37:58.398335+0000","last_clean":"2024-09-19T07:37:58.398335+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:37:58.398335+0000","last_undegraded":"2024-09-19T07:37:58.398335+0000","last_fullsized":"2024-09-19T07:37:58.398335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566969+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:37:58.566969+0000","last_peered":"2024-09-19T07:37:58.566969+0000","last_clean":"2024-09-19T07:37:58.566969+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:37:58.566969+0000","last_undegraded":"2024-09-19T07:37:58.566969+0000","last_fullsized":"2024-09-19T07:37:58.566969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400607+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:37:58.400607+0000","last_peered":"2024-09-19T07:37:58.400607+0000","last_clean":"2024-09-19T07:37:58.400607+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:37:58.400607+0000","last_undegraded":"2024-09-19T07:37:58.400607+0000","last_fullsized":"2024-09-19T07:37:58.400607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400322+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:37:58.400322+0000","last_peered":"2024-09-19T07:37:58.400322+0000","last_clean":"2024-09-19T07:37:58.400322+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:37:58.400322+0000","last_undegraded":"2024-09-19T07:37:58.400322+0000","last_fullsized":"2024-09-19T07:37:58.400322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'14","reported_seq":155,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396388+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:37:58.396388+0000","last_peered":"2024-09-19T07:37:58.396388+0000","last_clean":"2024-09-19T07:37:58.396388+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:37:58.396388+0000","last_undegraded":"2024-09-19T07:37:58.396388+0000","last_fullsized":"2024-09-19T07:37:58.396388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602652,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1142,"num_read_kb":690,"num_write":376,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1807956,"data_compressed":80685,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107370,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514531,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":451633,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":1243176,"data_compressed":52251,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441568,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":441162,"data_compressed":5809,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1044883,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528381,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":450775,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1174307,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":451422,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":974882,"data_compressed":34014,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":520934,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":520993,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":452507,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71591,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71532,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":203289,"data_compressed":7749,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:38:02.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:38:02.333 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.333 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.335 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:02 smithi103 ceph-mon[31201]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.341 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T07:38:02.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:02 smithi174 ceph-mon[35370]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:38:02.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Updating smithi134:/etc/ceph/ceph.conf 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:02 smithi134 ceph-mon[35414]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T07:38:02.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 -- 172.21.15.103:0/2788809644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66601036e0 msgr2=0x7f6660107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 --2- 172.21.15.103:0/2788809644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66601036e0 0x7f6660107b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6654009930 tx=0x7f665402a420 comp rx=0 tx=0).stop 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 -- 172.21.15.103:0/2788809644 shutdown_connections 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 --2- 172.21.15.103:0/2788809644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66601082a0 0x7f666010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 --2- 172.21.15.103:0/2788809644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66601036e0 0x7f6660107b90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 --2- 172.21.15.103:0/2788809644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6660102d30 0x7f6660103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 -- 172.21.15.103:0/2788809644 >> 172.21.15.103:0/2788809644 conn(0x7f66600fc7d0 msgr2=0x7f66600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 -- 172.21.15.103:0/2788809644 shutdown_connections 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.843+0000 7f6665b55640 1 -- 172.21.15.103:0/2788809644 wait complete. 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 Processor -- start 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 -- start start 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6660102d30 0x7f66601985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66601036e0 0x7f6660198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66601082a0 0x7f666019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66601120a0 con 0x7f6660102d30 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111f20 con 0x7f66601082a0 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f6665b55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6660112220 con 0x7f66601036e0 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f665f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6660102d30 0x7f66601985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f665f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6660102d30 0x7f66601985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45820/0 (socket says 172.21.15.103:45820) 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.844+0000 7f665f7fe640 1 -- 172.21.15.103:0/2708312068 learned_addr learned my addr 172.21.15.103:0/2708312068 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665ffff640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66601082a0 0x7f666019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665effd640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66601036e0 0x7f6660198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665f7fe640 1 -- 172.21.15.103:0/2708312068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66601036e0 msgr2=0x7f6660198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665f7fe640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66601036e0 0x7f6660198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665f7fe640 1 -- 172.21.15.103:0/2708312068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66601082a0 msgr2=0x7f666019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:02.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665f7fe640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66601082a0 0x7f666019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665f7fe640 1 -- 172.21.15.103:0/2708312068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666019d550 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665effd640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66601036e0 0x7f6660198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665ffff640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66601082a0 0x7f666019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665f7fe640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6660102d30 0x7f66601985a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f665000e9f0 tx=0x7f665000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000ccc0 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650010910 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666019d840 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500188a0 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.845+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601a5120 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.847+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660103110 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.848+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 23) ==== 138476+0+0 (secure 0 0 0) 0x7f6650010ac0 con 0x7f6660102d30 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.848+0000 7f665cff9640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66480a33b0 0x7f66480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:02.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.848+0000 7f665effd640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66480a33b0 0x7f66480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:02.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.848+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f66500d0100 con 0x7f6660102d30 2024-09-19T07:38:02.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.849+0000 7f665effd640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66480a33b0 0x7f66480a5870 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6654009900 tx=0x7f6654005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:02.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:02.852+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f665009cb90 con 0x7f6660102d30 2024-09-19T07:38:03.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.009+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f66500cdb80 con 0x7f6660102d30 2024-09-19T07:38:03.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.107+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f666010a2b0 con 0x7f6660102d30 2024-09-19T07:38:03.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.108+0000 7f665cff9640 1 -- 172.21.15.103:0/2708312068 <== mon.0 v2:172.21.15.103:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f6650053c30 con 0x7f6660102d30 2024-09-19T07:38:03.109 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:03.109 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T07:37:57:172446+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:37:57.122252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:38:03.110 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:38:03.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66480a33b0 msgr2=0x7f66480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:03.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66480a33b0 0x7f66480a5870 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6654009900 tx=0x7f6654005a20 comp rx=0 tx=0).stop 2024-09-19T07:38:03.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6660102d30 msgr2=0x7f66601985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:03.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6660102d30 0x7f66601985a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f665000e9f0 tx=0x7f665000edf0 comp rx=0 tx=0).stop 2024-09-19T07:38:03.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 shutdown_connections 2024-09-19T07:38:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66480a33b0 0x7f66480a5870 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66601082a0 0x7f666019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66601036e0 0x7f6660198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 --2- 172.21.15.103:0/2708312068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6660102d30 0x7f66601985a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 >> 172.21.15.103:0/2708312068 conn(0x7f66600fc7d0 msgr2=0x7f6660105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 shutdown_connections 2024-09-19T07:38:03.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:03.110+0000 7f6665b55640 1 -- 172.21.15.103:0/2708312068 wait complete. 2024-09-19T07:38:03.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:38:04.159 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:04.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:04.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2708312068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:04.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:04.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:04.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:04.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:03 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-19T07:38:04.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2708312068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:04.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:04.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:04.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:04.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:03 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:38:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: Updating smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: Updating smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: Updating smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-19T07:38:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2708312068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:04.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:04.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:04.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:04.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:03 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:04.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 -- 172.21.15.103:0/4126254930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710106f10 msgr2=0x7fc71010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:04.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 --2- 172.21.15.103:0/4126254930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710106f10 0x7fc71010ea90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc708009ca0 tx=0x7fc70802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 -- 172.21.15.103:0/4126254930 shutdown_connections 2024-09-19T07:38:04.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 --2- 172.21.15.103:0/4126254930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710106f10 0x7fc71010ea90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 --2- 172.21.15.103:0/4126254930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc710102790 0x7fc7101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 --2- 172.21.15.103:0/4126254930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc710101de0 0x7fc7101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 -- 172.21.15.103:0/4126254930 >> 172.21.15.103:0/4126254930 conn(0x7fc7100fd8c0 msgr2=0x7fc7100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:04.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 -- 172.21.15.103:0/4126254930 shutdown_connections 2024-09-19T07:38:04.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 -- 172.21.15.103:0/4126254930 wait complete. 2024-09-19T07:38:04.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 Processor -- start 2024-09-19T07:38:04.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.628+0000 7fc716c59640 1 -- start start 2024-09-19T07:38:04.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc716c59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc710101de0 0x7fc7101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:04.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc716c59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710102790 0x7fc710198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:04.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc716c59640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc710106f10 0x7fc71019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:04.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc716c59640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc710111f70 con 0x7fc710102790 2024-09-19T07:38:04.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc716c59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc710111df0 con 0x7fc710101de0 2024-09-19T07:38:04.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc716c59640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101120f0 con 0x7fc710106f10 2024-09-19T07:38:04.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710102790 0x7fc710198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:04.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710102790 0x7fc710198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45836/0 (socket says 172.21.15.103:45836) 2024-09-19T07:38:04.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 -- 172.21.15.103:0/3810130792 learned_addr learned my addr 172.21.15.103:0/3810130792 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:04.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc7149f5640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc710101de0 0x7fc7101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:04.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc7151f6640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc710106f10 0x7fc71019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:04.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 -- 172.21.15.103:0/3810130792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc710106f10 msgr2=0x7fc71019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:04.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc710106f10 0x7fc71019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 -- 172.21.15.103:0/3810130792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc710101de0 msgr2=0x7fc7101983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:04.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc710101de0 0x7fc7101983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 -- 172.21.15.103:0/3810130792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71019d3a0 con 0x7fc710102790 2024-09-19T07:38:04.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70ffff640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710102790 0x7fc710198930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc70400c970 tx=0x7fc70400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:04.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70dffb640 1 -- 172.21.15.103:0/3810130792 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704007bb0 con 0x7fc710102790 2024-09-19T07:38:04.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc70dffb640 1 -- 172.21.15.103:0/3810130792 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc704007d50 con 0x7fc710102790 2024-09-19T07:38:04.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.629+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71019d690 con 0x7fc710102790 2024-09-19T07:38:04.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.630+0000 7fc70dffb640 1 -- 172.21.15.103:0/3810130792 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70400f450 con 0x7fc710102790 2024-09-19T07:38:04.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.630+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a4f70 con 0x7fc710102790 2024-09-19T07:38:04.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.631+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7101021c0 con 0x7fc710102790 2024-09-19T07:38:04.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.632+0000 7fc70dffb640 1 -- 172.21.15.103:0/3810130792 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fc704005020 con 0x7fc710102790 2024-09-19T07:38:04.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.632+0000 7fc70dffb640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc6f80a34c0 0x7fc6f80a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:04.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.632+0000 7fc7149f5640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc6f80a34c0 0x7fc6f80a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:04.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.632+0000 7fc70dffb640 1 -- 172.21.15.103:0/3810130792 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc7040cfaf0 con 0x7fc710102790 2024-09-19T07:38:04.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.633+0000 7fc7149f5640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc6f80a34c0 0x7fc6f80a5980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc700005d00 tx=0x7fc70000b410 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:04.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.637+0000 7fc70dffb640 1 -- 172.21.15.103:0/3810130792 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc704014030 con 0x7fc710102790 2024-09-19T07:38:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.805+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc71002d7c0 con 0x7fc710102790 2024-09-19T07:38:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.807+0000 7fc70dffb640 1 -- 172.21.15.103:0/3810130792 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23200 (secure 0 0 0) 0x7fc70409bd20 con 0x7fc710102790 2024-09-19T07:38:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:04.808 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":85,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:37:58.271170+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/3492331771":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/189968053":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2164654927":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/2266926610":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/532109813":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6800/2164654927":"2024-09-20T07:37:58.271121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:38:04.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc6f80a34c0 msgr2=0x7fc6f80a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:04.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc6f80a34c0 0x7fc6f80a5980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc700005d00 tx=0x7fc70000b410 comp rx=0 tx=0).stop 2024-09-19T07:38:04.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710102790 msgr2=0x7fc710198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:04.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710102790 0x7fc710198930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc70400c970 tx=0x7fc70400ce30 comp rx=0 tx=0).stop 2024-09-19T07:38:04.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 shutdown_connections 2024-09-19T07:38:04.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc6f80a34c0 0x7fc6f80a5980 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc710106f10 0x7fc71019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc710102790 0x7fc710198930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 --2- 172.21.15.103:0/3810130792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc710101de0 0x7fc7101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:04.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.809+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 >> 172.21.15.103:0/3810130792 conn(0x7fc7100fd8c0 msgr2=0x7fc7100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:04.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.810+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 shutdown_connections 2024-09-19T07:38:04.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:04.810+0000 7fc716c59640 1 -- 172.21.15.103:0/3810130792 wait complete. 2024-09-19T07:38:05.187 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:04 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3810130792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:38:05.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:04 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3810130792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:38:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:04 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3810130792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:38:05.635 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T07:38:05.635 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T07:38:05.645 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T07:38:05.645 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:38:05.645 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T07:38:05.646 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T07:38:05.646 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:38:05.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:38:05.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 -- 172.21.15.103:0/3586316597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38102d40 msgr2=0x7f0c38103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:05.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3586316597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38102d40 0x7f0c38103120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009900 tx=0x7f0c2802a480 comp rx=0 tx=0).stop 2024-09-19T07:38:05.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 -- 172.21.15.103:0/3586316597 shutdown_connections 2024-09-19T07:38:05.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3586316597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c3810c110 0x7f0c3810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3586316597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c38103660 0x7f0c3810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3586316597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38102d40 0x7f0c38103120 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 -- 172.21.15.103:0/3586316597 >> 172.21.15.103:0/3586316597 conn(0x7f0c38077990 msgr2=0x7f0c380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:05.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 -- 172.21.15.103:0/3586316597 shutdown_connections 2024-09-19T07:38:05.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.752+0000 7f0c3e103640 1 -- 172.21.15.103:0/3586316597 wait complete. 2024-09-19T07:38:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 Processor -- start 2024-09-19T07:38:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 -- start start 2024-09-19T07:38:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c38102d40 0x7f0c38198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c38198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:05.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c3810c110 0x7f0c3819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111ef0 con 0x7f0c38103660 2024-09-19T07:38:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111d70 con 0x7f0c38102d40 2024-09-19T07:38:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c3e103640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112070 con 0x7f0c3810c110 2024-09-19T07:38:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c36ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c38198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c377fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c38102d40 0x7f0c38198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:05.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c36ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c38198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45854/0 (socket says 172.21.15.103:45854) 2024-09-19T07:38:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.753+0000 7f0c36ffd640 1 -- 172.21.15.103:0/3227161485 learned_addr learned my addr 172.21.15.103:0/3227161485 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c37fff640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c3810c110 0x7f0c3819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c36ffd640 1 -- 172.21.15.103:0/3227161485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c3810c110 msgr2=0x7f0c3819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c36ffd640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c3810c110 0x7f0c3819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c36ffd640 1 -- 172.21.15.103:0/3227161485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c38102d40 msgr2=0x7f0c38198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c36ffd640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c38102d40 0x7f0c38198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c36ffd640 1 -- 172.21.15.103:0/3227161485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3819d420 con 0x7f0c38103660 2024-09-19T07:38:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c377fe640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c38102d40 0x7f0c38198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c36ffd640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c38198970 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b4a0 tx=0x7f0c2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c34ff9640 1 -- 172.21.15.103:0/3227161485 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c014070 con 0x7f0c38103660 2024-09-19T07:38:05.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c34ff9640 1 -- 172.21.15.103:0/3227161485 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c010af0 con 0x7f0c38103660 2024-09-19T07:38:05.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c34ff9640 1 -- 172.21.15.103:0/3227161485 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c017d10 con 0x7f0c38103660 2024-09-19T07:38:05.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3819d710 con 0x7f0c38103660 2024-09-19T07:38:05.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.754+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c381a4fa0 con 0x7f0c38103660 2024-09-19T07:38:05.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.755+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c38103120 con 0x7f0c38103660 2024-09-19T07:38:05.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.757+0000 7f0c34ff9640 1 -- 172.21.15.103:0/3227161485 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f0c2c017070 con 0x7f0c38103660 2024-09-19T07:38:05.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.758+0000 7f0c34ff9640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0c200a3500 0x7f0c200a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:05.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.758+0000 7f0c377fe640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0c200a3500 0x7f0c200a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:05.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.758+0000 7f0c34ff9640 1 -- 172.21.15.103:0/3227161485 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0c2c0cf110 con 0x7f0c38103660 2024-09-19T07:38:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.759+0000 7f0c377fe640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0c200a3500 0x7f0c200a59c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009230 tx=0x7f0c280058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:05.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.763+0000 7f0c34ff9640 1 -- 172.21.15.103:0/3227161485 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0c2c09baa0 con 0x7f0c38103660 2024-09-19T07:38:05.908 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:05.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.948+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f0c38106c10 con 0x7f0c38103660 2024-09-19T07:38:05.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.952+0000 7f0c34ff9640 1 -- 172.21.15.103:0/3227161485 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v25)=0 v25) ==== 129+0+0 (secure 0 0 0) 0x7f0c2c09b3a0 con 0x7f0c38103660 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0c200a3500 msgr2=0x7f0c200a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0c200a3500 0x7f0c200a59c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009230 tx=0x7f0c280058d0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 msgr2=0x7f0c38198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c38198970 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b4a0 tx=0x7f0c2c00b960 comp rx=0 tx=0).stop 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 shutdown_connections 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0c200a3500 0x7f0c200a59c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c3810c110 0x7f0c3819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c38103660 0x7f0c38198970 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 --2- 172.21.15.103:0/3227161485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c38102d40 0x7f0c38198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 >> 172.21.15.103:0/3227161485 conn(0x7f0c38077990 msgr2=0x7f0c380fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:05.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 shutdown_connections 2024-09-19T07:38:05.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:05.954+0000 7f0c3e103640 1 -- 172.21.15.103:0/3227161485 wait complete. 2024-09-19T07:38:05.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T07:38:06.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 -- 172.21.15.103:0/2650775442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 msgr2=0x7f391810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 --2- 172.21.15.103:0/2650775442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f391810ba80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3908009900 tx=0x7f390802a480 comp rx=0 tx=0).stop 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 -- 172.21.15.103:0/2650775442 shutdown_connections 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 --2- 172.21.15.103:0/2650775442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391810c0d0 0x7f391810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 --2- 172.21.15.103:0/2650775442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f391810ba80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 --2- 172.21.15.103:0/2650775442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3918102c80 0x7f3918103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 -- 172.21.15.103:0/2650775442 >> 172.21.15.103:0/2650775442 conn(0x7f3918077990 msgr2=0x7f39180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 -- 172.21.15.103:0/2650775442 shutdown_connections 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.072+0000 7f391e3e4640 1 -- 172.21.15.103:0/2650775442 wait complete. 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 Processor -- start 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 -- start start 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3918102c80 0x7f3918198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f39181988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391810c0d0 0x7f391819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3918111eb0 con 0x7f39181035a0 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3918111d30 con 0x7f3918102c80 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391e3e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3918112030 con 0x7f391810c0d0 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f39177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f39181988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f3917fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3918102c80 0x7f3918198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f39177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f39181988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45872/0 (socket says 172.21.15.103:45872) 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f39177fe640 1 -- 172.21.15.103:0/1781136807 learned_addr learned my addr 172.21.15.103:0/1781136807 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.073+0000 7f391c981640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391810c0d0 0x7f391819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39177fe640 1 -- 172.21.15.103:0/1781136807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391810c0d0 msgr2=0x7f391819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39177fe640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391810c0d0 0x7f391819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39177fe640 1 -- 172.21.15.103:0/1781136807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3918102c80 msgr2=0x7f3918198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39177fe640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3918102c80 0x7f3918198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39177fe640 1 -- 172.21.15.103:0/1781136807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391819d350 con 0x7f39181035a0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f391c981640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391810c0d0 0x7f391819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39177fe640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f39181988a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3908006010 tx=0x7f39080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39157fa640 1 -- 172.21.15.103:0/1781136807 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390803f030 con 0x7f39181035a0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39157fa640 1 -- 172.21.15.103:0/1781136807 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3908003ce0 con 0x7f39181035a0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391819d5e0 con 0x7f39181035a0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f39157fa640 1 -- 172.21.15.103:0/1781136807 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908035070 con 0x7f39181035a0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.074+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39181a4ed0 con 0x7f39181035a0 2024-09-19T07:38:06.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.075+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3918103060 con 0x7f39181035a0 2024-09-19T07:38:06.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.076+0000 7f39157fa640 1 -- 172.21.15.103:0/1781136807 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f3908042030 con 0x7f39181035a0 2024-09-19T07:38:06.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.077+0000 7f39157fa640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f39040a3580 0x7f39040a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.077+0000 7f3917fff640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f39040a3580 0x7f39040a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.078+0000 7f39157fa640 1 -- 172.21.15.103:0/1781136807 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f39080f0390 con 0x7f39181035a0 2024-09-19T07:38:06.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.078+0000 7f3917fff640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f39040a3580 0x7f39040a5a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f390000bbe0 tx=0x7f390000b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.082+0000 7f39157fa640 1 -- 172.21.15.103:0/1781136807 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f39080bccd0 con 0x7f39181035a0 2024-09-19T07:38:06.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:05 smithi103 ceph-mon[31201]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-19T07:38:06.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:05 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3227161485' entity='client.admin' 2024-09-19T07:38:06.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:05 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:06.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:05 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:06.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:05 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:06.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:05 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:06.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.256+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f39181a5130 con 0x7f39181035a0 2024-09-19T07:38:06.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.260+0000 7f39157fa640 1 -- 172.21.15.103:0/1781136807 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) ==== 138+0+0 (secure 0 0 0) 0x7f39080bc5d0 con 0x7f39181035a0 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f39040a3580 msgr2=0x7f39040a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f39040a3580 0x7f39040a5a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f390000bbe0 tx=0x7f390000b390 comp rx=0 tx=0).stop 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 msgr2=0x7f39181988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f39181988a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3908006010 tx=0x7f39080042f0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 shutdown_connections 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f39040a3580 0x7f39040a5a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f391810c0d0 0x7f391819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39181035a0 0x7f39181988a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 --2- 172.21.15.103:0/1781136807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3918102c80 0x7f3918198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 >> 172.21.15.103:0/1781136807 conn(0x7f3918077990 msgr2=0x7f39180fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 shutdown_connections 2024-09-19T07:38:06.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.263+0000 7f391e3e4640 1 -- 172.21.15.103:0/1781136807 wait complete. 2024-09-19T07:38:06.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T07:38:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:05 smithi134 ceph-mon[35414]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-19T07:38:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:05 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3227161485' entity='client.admin' 2024-09-19T07:38:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:05 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:05 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:05 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:05 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.380+0000 7f1c594d0640 1 -- 172.21.15.103:0/1560526725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 msgr2=0x7f1c5410d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.380+0000 7f1c594d0640 1 --2- 172.21.15.103:0/1560526725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c5410d7d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009ca0 tx=0x7f1c4402a950 comp rx=0 tx=0).stop 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.380+0000 7f1c594d0640 1 -- 172.21.15.103:0/1560526725 shutdown_connections 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.380+0000 7f1c594d0640 1 --2- 172.21.15.103:0/1560526725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c5410d7d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.380+0000 7f1c594d0640 1 --2- 172.21.15.103:0/1560526725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c541016d0 0x7f1c54101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.380+0000 7f1c594d0640 1 --2- 172.21.15.103:0/1560526725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c54105af0 0x7f1c54105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.380+0000 7f1c594d0640 1 -- 172.21.15.103:0/1560526725 >> 172.21.15.103:0/1560526725 conn(0x7f1c54075ec0 msgr2=0x7f1c540762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.381+0000 7f1c594d0640 1 -- 172.21.15.103:0/1560526725 shutdown_connections 2024-09-19T07:38:06.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.381+0000 7f1c594d0640 1 -- 172.21.15.103:0/1560526725 wait complete. 2024-09-19T07:38:06.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.381+0000 7f1c594d0640 1 Processor -- start 2024-09-19T07:38:06.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.381+0000 7f1c594d0640 1 -- start start 2024-09-19T07:38:06.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.381+0000 7f1c594d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c541016d0 0x7f1c54199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.381+0000 7f1c594d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c54199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c594d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c54105af0 0x7f1c541a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c594d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54110de0 con 0x7f1c54102050 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c594d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54110c60 con 0x7f1c54105af0 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c594d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54110f60 con 0x7f1c541016d0 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c54199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c54199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45890/0 (socket says 172.21.15.103:45890) 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 -- 172.21.15.103:0/3276902154 learned_addr learned my addr 172.21.15.103:0/3276902154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c52ffd640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c541016d0 0x7f1c54199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c537fe640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c54105af0 0x7f1c541a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 -- 172.21.15.103:0/3276902154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c541016d0 msgr2=0x7f1c54199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c541016d0 0x7f1c54199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 -- 172.21.15.103:0/3276902154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c54105af0 msgr2=0x7f1c541a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c54105af0 0x7f1c541a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 -- 172.21.15.103:0/3276902154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c541a46d0 con 0x7f1c54102050 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c527fc640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c54199650 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00d950 tx=0x7f1c3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c4bfff640 1 -- 172.21.15.103:0/3276902154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c017070 con 0x7f1c54102050 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c4bfff640 1 -- 172.21.15.103:0/3276902154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c3c009c30 con 0x7f1c54102050 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c4bfff640 1 -- 172.21.15.103:0/3276902154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00f6b0 con 0x7f1c54102050 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c541a48a0 con 0x7f1c54102050 2024-09-19T07:38:06.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.382+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c541a4dc0 con 0x7f1c54102050 2024-09-19T07:38:06.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.383+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c54101b10 con 0x7f1c54102050 2024-09-19T07:38:06.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.384+0000 7f1c4bfff640 1 -- 172.21.15.103:0/3276902154 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f1c3c0109d0 con 0x7f1c54102050 2024-09-19T07:38:06.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.385+0000 7f1c4bfff640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1c400a3500 0x7f1c400a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.385+0000 7f1c52ffd640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1c400a3500 0x7f1c400a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.385+0000 7f1c4bfff640 1 -- 172.21.15.103:0/3276902154 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f1c3c0cfb00 con 0x7f1c54102050 2024-09-19T07:38:06.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.386+0000 7f1c52ffd640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1c400a3500 0x7f1c400a59c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005fd0 tx=0x7f1c4c00b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.389+0000 7f1c4bfff640 1 -- 172.21.15.103:0/3276902154 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1c3c009de0 con 0x7f1c54102050 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.390+0000 7f7d85256640 1 -- 172.21.15.103:0/4236526786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 msgr2=0x7f7d80101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.390+0000 7f7d85256640 1 --2- 172.21.15.103:0/4236526786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80101850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009900 tx=0x7f7d7002a410 comp rx=0 tx=0).stop 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.390+0000 7f7d85256640 1 -- 172.21.15.103:0/4236526786 shutdown_connections 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.390+0000 7f7d85256640 1 --2- 172.21.15.103:0/4236526786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d80105de0 0x7f7d8010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.390+0000 7f7d85256640 1 --2- 172.21.15.103:0/4236526786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80101e20 0x7f7d801056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.390+0000 7f7d85256640 1 --2- 172.21.15.103:0/4236526786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80101850 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.390+0000 7f7d85256640 1 -- 172.21.15.103:0/4236526786 >> 172.21.15.103:0/4236526786 conn(0x7f7d80077230 msgr2=0x7f7d800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.391+0000 7f7d85256640 1 -- 172.21.15.103:0/4236526786 shutdown_connections 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.391+0000 7f7d85256640 1 -- 172.21.15.103:0/4236526786 wait complete. 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.391+0000 7f7d85256640 1 Processor -- start 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d85256640 1 -- start start 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d85256640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d85256640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d80101e20 0x7f7d80198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d85256640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80105de0 0x7f7d8019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d85256640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80110e40 con 0x7f7d80101470 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d85256640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80110cc0 con 0x7f7d80101e20 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d85256640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80110fc0 con 0x7f7d80105de0 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45912/0 (socket says 172.21.15.103:45912) 2024-09-19T07:38:06.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 -- 172.21.15.103:0/3987023770 learned_addr learned my addr 172.21.15.103:0/3987023770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80105de0 0x7f7d8019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d80101e20 0x7f7d80198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 -- 172.21.15.103:0/3987023770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80105de0 msgr2=0x7f7d8019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80105de0 0x7f7d8019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 -- 172.21.15.103:0/3987023770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d80101e20 msgr2=0x7f7d80198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d80101e20 0x7f7d80198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7ed9d640 1 -- 172.21.15.103:0/3987023770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8019d3d0 con 0x7f7d80101470 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.392+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d80101e20 0x7f7d80198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.393+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80198420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7d70006010 tx=0x7f7d70004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.393+0000 7f7d67fff640 1 -- 172.21.15.103:0/3987023770 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7003f030 con 0x7f7d80101470 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.393+0000 7f7d67fff640 1 -- 172.21.15.103:0/3987023770 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d70004440 con 0x7f7d80101470 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.393+0000 7f7d67fff640 1 -- 172.21.15.103:0/3987023770 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70035070 con 0x7f7d80101470 2024-09-19T07:38:06.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.393+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8019d660 con 0x7f7d80101470 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.393+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d800fc630 con 0x7f7d80101470 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d67fff640 1 -- 172.21.15.103:0/3987023770 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f7d70042030 con 0x7f7d80101470 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 Processor -- start 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 -- start start 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4741f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b47424a0 0x5566b474c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b474cd60 0x5566b474f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566b46a7280 con 0x5566b474cd60 2024-09-19T07:38:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5566b464eb10 con 0x5566b47424a0 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d8762f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566b46a3240 con 0x5566b4741b80 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d67fff640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7d680a34c0 0x7f7d680a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b474cd60 0x5566b474f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b474cd60 0x5566b474f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45916/0 (socket says 172.21.15.103:45916) 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d7e59c640 1 -- 172.21.15.103:0/1751261656 learned_addr learned my addr 172.21.15.103:0/1751261656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d67fff640 1 -- 172.21.15.103:0/3987023770 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7d700f0260 con 0x7f7d80101470 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.396+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b47424a0 0x5566b474c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7e59c640 1 -- 172.21.15.103:0/1751261656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 msgr2=0x5566b4741f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4741f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4741f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7e59c640 1 -- 172.21.15.103:0/1751261656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b47424a0 msgr2=0x5566b474c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b47424a0 0x5566b474c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7e59c640 1 -- 172.21.15.103:0/1751261656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b465b1f0 con 0x5566b474cd60 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b47424a0 0x5566b474c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7d680a34c0 0x7f7d680a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b474cd60 0x5566b474f150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009ce0 tx=0x7f7d740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beab4cdbc5db9c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d657fa640 1 -- 172.21.15.103:0/1751261656 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74012450 con 0x5566b474cd60 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d657fa640 1 -- 172.21.15.103:0/1751261656 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d74012ad0 con 0x5566b474cd60 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.397+0000 7f7d657fa640 1 -- 172.21.15.103:0/1751261656 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400fc90 con 0x5566b474cd60 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 -- 172.21.15.103:0/1751261656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b474cd60 msgr2=0x5566b474f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b474cd60 0x5566b474f150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009ce0 tx=0x7f7d740117b0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7d680a34c0 0x7f7d680a5980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d78005c40 tx=0x7f7d78015040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 -- 172.21.15.103:0/1751261656 shutdown_connections 2024-09-19T07:38:06.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b474cd60 0x5566b474f150 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b47424a0 0x5566b474c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1751261656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4741f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 -- 172.21.15.103:0/1751261656 >> 172.21.15.103:0/1751261656 conn(0x5566b473e110 msgr2=0x5566b473e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 -- 172.21.15.103:0/1751261656 shutdown_connections 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.398+0000 7f7d8762f740 1 -- 172.21.15.103:0/1751261656 wait complete. 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 -- start start 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 Processor -- start 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4749be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b474a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566b474f930 con 0x5566b47424a0 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5566b4661f10 con 0x5566b474cd60 2024-09-19T07:38:06.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566b474fab0 con 0x5566b4741b80 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b474a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d8762f740 10 client.0 _ll_register_callbacks cb 0x5566b462aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b474a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45932/0 (socket says 172.21.15.103:45932) 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7e59c640 1 -- 172.21.15.103:0/1835408988 learned_addr learned my addr 172.21.15.103:0/1835408988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4749be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7ed9d640 1 -- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 msgr2=0x5566b474a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7ed9d640 1 -- 172.21.15.103:0/1835408988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 msgr2=0x5566b474a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b474a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7ed9d640 1 -- 172.21.15.103:0/1835408988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b44186a0 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.399+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b474a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4749be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ed900 tx=0x7f7d700bc270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59dfbd31787e6798 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d62ff5640 1 -- 172.21.15.103:0/1835408988 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70049070 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d62ff5640 1 -- 172.21.15.103:0/1835408988 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7d700edbb0 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d74012ad0 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d62ff5640 1 -- 172.21.15.103:0/1835408988 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d700eddb0 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d8762f740 10 client.24955 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566b474f930 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d62ff5640 1 -- 172.21.15.103:0/1835408988 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d74012ad0 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d62ff5640 1 -- 172.21.15.103:0/1835408988 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d700bc6d0 con 0x5566b4741b80 2024-09-19T07:38:06.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d8762f740 10 client.24955 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d8762f740 10 client.24955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d8762f740 20 client.24955 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.400+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b475c860 0x5566b475ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.401+0000 7f7d8762f740 4 client.24955 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.401+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566b46b47f0 con 0x5566b475c860 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.401+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b475c860 0x5566b475ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.401+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b475c860 0x5566b475ecd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4a7dba2be0e136 server_cookie=218ccb99bee30c95 in_seq=0 out_seq=0 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.401+0000 7f7d62ff5640 10 client.24955 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d62ff5640 1 -- 172.21.15.103:0/1835408988 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566b46b47f0 con 0x5566b475c860 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d62ff5640 10 client.24955 handle_command_reply: tid=0 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 client.24955 shutdown 2024-09-19T07:38:06.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 msgr2=0x5566b4749be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4749be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ed900 tx=0x7f7d700bc270 comp rx=0 tx=0).stop 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 shutdown_connections 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b475c860 0x5566b475ecd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b474a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d62ff5640 0 client.24955 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 --2- 172.21.15.103:0/1835408988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4749be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 >> 172.21.15.103:0/1835408988 conn(0x5566b473e110 msgr2=0x5566b4743df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 shutdown_connections 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.402+0000 7f7d8762f740 1 -- 172.21.15.103:0/1835408988 wait complete. 2024-09-19T07:38:06.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.403+0000 7f7d8762f740 20 client.24955 trim_cache size 0 max 16384 2024-09-19T07:38:06.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:05 smithi174 ceph-mon[35370]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-19T07:38:06.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:05 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3227161485' entity='client.admin' 2024-09-19T07:38:06.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:05 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:06.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:05 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:06.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:05 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:06.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:05 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:06.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.419+0000 7f7d8762f740 1 Processor -- start 2024-09-19T07:38:06.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.419+0000 7f7d8762f740 1 -- start start 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b473d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b47645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b4764af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d8762f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801afab0 con 0x5566b47424a0 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d8762f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b15f0 con 0x5566b474cd60 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d8762f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b0180 con 0x5566b4741b80 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b47645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b47645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45944/0 (socket says 172.21.15.103:45944) 2024-09-19T07:38:06.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 -- 172.21.15.103:0/485737678 learned_addr learned my addr 172.21.15.103:0/485737678 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b4764af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b473d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 -- 172.21.15.103:0/485737678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 msgr2=0x5566b473d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b473d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 -- 172.21.15.103:0/485737678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 msgr2=0x5566b4764af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b4764af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 -- 172.21.15.103:0/485737678 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b474c4b0 con 0x5566b47424a0 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b4764af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b47645b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400f810 tx=0x7f7d74009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93150034c32f90bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.420+0000 7f7d64ff9640 1 -- 172.21.15.103:0/485737678 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74009800 con 0x5566b47424a0 2024-09-19T07:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d64ff9640 1 -- 172.21.15.103:0/485737678 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7400fe80 con 0x5566b47424a0 2024-09-19T07:38:06.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d64ff9640 1 -- 172.21.15.103:0/485737678 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740126b0 con 0x5566b47424a0 2024-09-19T07:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 -- 172.21.15.103:0/485737678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 msgr2=0x5566b47645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b47645b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400f810 tx=0x7f7d74009fb0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 -- 172.21.15.103:0/485737678 shutdown_connections 2024-09-19T07:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b4764af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b47645b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 --2- 172.21.15.103:0/485737678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b473d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 -- 172.21.15.103:0/485737678 >> 172.21.15.103:0/485737678 conn(0x5566b473e110 msgr2=0x5566b473ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 -- 172.21.15.103:0/485737678 shutdown_connections 2024-09-19T07:38:06.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.421+0000 7f7d8762f740 1 -- 172.21.15.103:0/485737678 wait complete. 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 -- start start 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 Processor -- start 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4754d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b4755250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b0f20 con 0x5566b47424a0 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b0850 con 0x5566b474cd60 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801ada90 con 0x5566b4741b80 2024-09-19T07:38:06.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d8762f740 10 client.0 _ll_register_callbacks cb 0x7f7d700b6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b4755250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b4755250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45956/0 (socket says 172.21.15.103:45956) 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 -- 172.21.15.103:0/2477821124 learned_addr learned my addr 172.21.15.103:0/2477821124 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4754d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 -- 172.21.15.103:0/2477821124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 msgr2=0x5566b4754d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4754d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 -- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 msgr2=0x5566b474fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7e59c640 1 -- 172.21.15.103:0/2477821124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b4750200 con 0x5566b47424a0 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.422+0000 7f7d7f59e640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d7e59c640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b4755250 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005450 tx=0x7f7d7400f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1fc18f4d3bc506 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d62ff5640 1 -- 172.21.15.103:0/2477821124 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400fa40 con 0x5566b47424a0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566b46aed00 con 0x5566b47424a0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d62ff5640 1 -- 172.21.15.103:0/2477821124 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7d7400fbe0 con 0x5566b47424a0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d62ff5640 1 -- 172.21.15.103:0/2477821124 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d74012450 con 0x5566b47424a0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 10 client.15342 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566b47c7d70 con 0x5566b47424a0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d62ff5640 1 -- 172.21.15.103:0/2477821124 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5566b46aed00 con 0x5566b47424a0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d62ff5640 1 -- 172.21.15.103:0/2477821124 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d7402da50 con 0x5566b47424a0 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 10 client.15342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 20 client.15342 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:06.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b47c82e0 0x5566b47ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 4 client.15342 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.423+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5566b462b110 con 0x5566b47c82e0 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.424+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b47c82e0 0x5566b47ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.424+0000 7f7d62ff5640 10 client.15342 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.424+0000 7f7d7ed9d640 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b47c82e0 0x5566b47ca700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b92d002b90640b server_cookie=9defa13ec6ca46c2 in_seq=0 out_seq=0 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d62ff5640 1 -- 172.21.15.103:0/2477821124 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5566b462b110 con 0x5566b47c82e0 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d62ff5640 10 client.15342 handle_command_reply: tid=0 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 client.15342 shutdown 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 msgr2=0x5566b4755250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b4755250 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005450 tx=0x7f7d7400f5e0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 shutdown_connections 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566b47c82e0 0x5566b47ca700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566b474cd60 0x5566b474fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566b47424a0 0x5566b4755250 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 --2- 172.21.15.103:0/2477821124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566b4741b80 0x5566b4754d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d62ff5640 0 client.15342 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.425+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 >> 172.21.15.103:0/2477821124 conn(0x5566b473e110 msgr2=0x5566b473d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 shutdown_connections 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d8762f740 1 -- 172.21.15.103:0/2477821124 wait complete. 2024-09-19T07:38:06.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d8762f740 20 client.15342 trim_cache size 0 max 16384 2024-09-19T07:38:06.435 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:06.435 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:38:06.435 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "17324f75-dbbd-4936-9d4b-c86c7f6195f1", 2024-09-19T07:38:06.435 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:38:06.435 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7d680a34c0 msgr2=0x7f7d680a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7d680a34c0 0x7f7d680a5980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7d78005c40 tx=0x7f7d78015040 comp rx=0 tx=0).stop 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 msgr2=0x7f7d80198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80198420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7d70006010 tx=0x7f7d70004040 comp rx=0 tx=0).stop 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 shutdown_connections 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7d680a34c0 0x7f7d680a5980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80105de0 0x7f7d8019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d80101e20 0x7f7d80198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 --2- 172.21.15.103:0/3987023770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80101470 0x7f7d80198420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.426+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 >> 172.21.15.103:0/3987023770 conn(0x7f7d80077230 msgr2=0x7f7d80100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.427+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 shutdown_connections 2024-09-19T07:38:06.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.427+0000 7f7d85256640 1 -- 172.21.15.103:0/3987023770 wait complete. 2024-09-19T07:38:06.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.574+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f1c541a5090 con 0x7f1c54102050 2024-09-19T07:38:06.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.578+0000 7f1c4bfff640 1 -- 172.21.15.103:0/3276902154 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) ==== 133+0+0 (secure 0 0 0) 0x7f1c3c09c380 con 0x7f1c54102050 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1c400a3500 msgr2=0x7f1c400a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1c400a3500 0x7f1c400a59c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005fd0 tx=0x7f1c4c00b390 comp rx=0 tx=0).stop 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 msgr2=0x7f1c54199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c54199650 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00d950 tx=0x7f1c3c00de10 comp rx=0 tx=0).stop 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 shutdown_connections 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1c400a3500 0x7f1c400a59c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c54105af0 0x7f1c541a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c54102050 0x7f1c54199650 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 --2- 172.21.15.103:0/3276902154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c541016d0 0x7f1c54199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 >> 172.21.15.103:0/3276902154 conn(0x7f1c54075ec0 msgr2=0x7f1c540ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 shutdown_connections 2024-09-19T07:38:06.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.581+0000 7f1c594d0640 1 -- 172.21.15.103:0/3276902154 wait complete. 2024-09-19T07:38:06.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T07:38:06.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 -- 172.21.15.103:0/4254766920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7878102d10 msgr2=0x7f78781030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 --2- 172.21.15.103:0/4254766920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7878102d10 0x7f78781030f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7868009900 tx=0x7f786802a480 comp rx=0 tx=0).stop 2024-09-19T07:38:06.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 -- 172.21.15.103:0/4254766920 shutdown_connections 2024-09-19T07:38:06.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 --2- 172.21.15.103:0/4254766920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f787810c0e0 0x7f787810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 --2- 172.21.15.103:0/4254766920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7878103630 0x7f787810bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 --2- 172.21.15.103:0/4254766920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7878102d10 0x7f78781030f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 -- 172.21.15.103:0/4254766920 >> 172.21.15.103:0/4254766920 conn(0x7f78780fc6c0 msgr2=0x7f78780feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 -- 172.21.15.103:0/4254766920 shutdown_connections 2024-09-19T07:38:06.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.704+0000 7f787dcba640 1 -- 172.21.15.103:0/4254766920 wait complete. 2024-09-19T07:38:06.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 Processor -- start 2024-09-19T07:38:06.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 -- start start 2024-09-19T07:38:06.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7878102d10 0x7f78781986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7878103630 0x7f7878198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f787810c0e0 0x7f787819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7878111f70 con 0x7f787810c0e0 2024-09-19T07:38:06.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7878111df0 con 0x7f7878103630 2024-09-19T07:38:06.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f787dcba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78781120f0 con 0x7f7878102d10 2024-09-19T07:38:06.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f7877fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f787810c0e0 0x7f787819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f7877fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f787810c0e0 0x7f787819cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45968/0 (socket says 172.21.15.103:45968) 2024-09-19T07:38:06.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f7877fff640 1 -- 172.21.15.103:0/2399360577 learned_addr learned my addr 172.21.15.103:0/2399360577 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:06.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f7876ffd640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7878103630 0x7f7878198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.705+0000 7f78777fe640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7878102d10 0x7f78781986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7877fff640 1 -- 172.21.15.103:0/2399360577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7878102d10 msgr2=0x7f78781986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7877fff640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7878102d10 0x7f78781986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7877fff640 1 -- 172.21.15.103:0/2399360577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7878103630 msgr2=0x7f7878198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7877fff640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7878103630 0x7f7878198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7877fff640 1 -- 172.21.15.103:0/2399360577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787819d6b0 con 0x7f787810c0e0 2024-09-19T07:38:06.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7877fff640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f787810c0e0 0x7f787819cf80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f787000b970 tx=0x7f787000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7874ff9640 1 -- 172.21.15.103:0/2399360577 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870019070 con 0x7f787810c0e0 2024-09-19T07:38:06.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7874ff9640 1 -- 172.21.15.103:0/2399360577 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787000cb40 con 0x7f787810c0e0 2024-09-19T07:38:06.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787819d9a0 con 0x7f787810c0e0 2024-09-19T07:38:06.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f7874ff9640 1 -- 172.21.15.103:0/2399360577 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870007650 con 0x7f787810c0e0 2024-09-19T07:38:06.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.706+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78781a5230 con 0x7f787810c0e0 2024-09-19T07:38:06.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.707+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78781030f0 con 0x7f787810c0e0 2024-09-19T07:38:06.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.708+0000 7f7874ff9640 1 -- 172.21.15.103:0/2399360577 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f787001c070 con 0x7f787810c0e0 2024-09-19T07:38:06.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.709+0000 7f7874ff9640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78600a3500 0x7f78600a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:06.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.709+0000 7f78777fe640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78600a3500 0x7f78600a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:06.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.709+0000 7f7874ff9640 1 -- 172.21.15.103:0/2399360577 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f78700cf740 con 0x7f787810c0e0 2024-09-19T07:38:06.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.709+0000 7f78777fe640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78600a3500 0x7f78600a59c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7868000c00 tx=0x7f7868005b50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:06.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.712+0000 7f7874ff9640 1 -- 172.21.15.103:0/2399360577 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f787009bb30 con 0x7f787810c0e0 2024-09-19T07:38:06.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.903+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f787806b8f0 con 0x7f78600a3500 2024-09-19T07:38:06.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.969+0000 7f7874ff9640 1 -- 172.21.15.103:0/2399360577 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f787806b8f0 con 0x7f78600a3500 2024-09-19T07:38:06.971 INFO:teuthology.orchestra.run.smithi103.stdout:Schedule set for path / 2024-09-19T07:38:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.971+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78600a3500 msgr2=0x7f78600a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.971+0000 7f787dcba640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78600a3500 0x7f78600a59c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7868000c00 tx=0x7f7868005b50 comp rx=0 tx=0).stop 2024-09-19T07:38:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.971+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f787810c0e0 msgr2=0x7f787819cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.971+0000 7f787dcba640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f787810c0e0 0x7f787819cf80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f787000b970 tx=0x7f787000be30 comp rx=0 tx=0).stop 2024-09-19T07:38:06.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.971+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 shutdown_connections 2024-09-19T07:38:06.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.971+0000 7f787dcba640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78600a3500 0x7f78600a59c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.972+0000 7f787dcba640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f787810c0e0 0x7f787819cf80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.972+0000 7f787dcba640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7878103630 0x7f7878198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.972+0000 7f787dcba640 1 --2- 172.21.15.103:0/2399360577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7878102d10 0x7f78781986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:06.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.972+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 >> 172.21.15.103:0/2399360577 conn(0x7f78780fc6c0 msgr2=0x7f78780fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:06.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.972+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 shutdown_connections 2024-09-19T07:38:06.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:06.972+0000 7f787dcba640 1 -- 172.21.15.103:0/2399360577 wait complete. 2024-09-19T07:38:06.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T07:38:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- 172.21.15.103:0/936304436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 msgr2=0x7f5e54100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/936304436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54100e30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009930 tx=0x7f5e3c02a420 comp rx=0 tx=0).stop 2024-09-19T07:38:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- 172.21.15.103:0/936304436 shutdown_connections 2024-09-19T07:38:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/936304436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e54101370 0x7f5e5410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/936304436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54100e30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/936304436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e541069e0 0x7f5e54106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- 172.21.15.103:0/936304436 >> 172.21.15.103:0/936304436 conn(0x7f5e540fc6f0 msgr2=0x7f5e540feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- 172.21.15.103:0/936304436 shutdown_connections 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- 172.21.15.103:0/936304436 wait complete. 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 Processor -- start 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- start start 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e54101370 0x7f5e54199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e541069e0 0x7f5e541a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54110e20 con 0x7f5e541009d0 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54110ca0 con 0x7f5e541069e0 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.086+0000 7f5e59ca4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54110fa0 con 0x7f5e54101370 2024-09-19T07:38:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45978/0 (socket says 172.21.15.103:45978) 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 -- 172.21.15.103:0/458594457 learned_addr learned my addr 172.21.15.103:0/458594457 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e4bfff640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e54101370 0x7f5e54199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e53fff640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e541069e0 0x7f5e541a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 -- 172.21.15.103:0/458594457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e54101370 msgr2=0x7f5e54199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e54101370 0x7f5e54199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 -- 172.21.15.103:0/458594457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e541069e0 msgr2=0x7f5e541a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e541069e0 0x7f5e541a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 -- 172.21.15.103:0/458594457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e541a4710 con 0x7f5e541009d0 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e4bfff640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e54101370 0x7f5e54199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e53fff640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e541069e0 0x7f5e541a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e537fe640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54199150 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400d950 tx=0x7f5e4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:07.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e517fa640 1 -- 172.21.15.103:0/458594457 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44017070 con 0x7f5e541009d0 2024-09-19T07:38:07.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e517fa640 1 -- 172.21.15.103:0/458594457 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e44009c30 con 0x7f5e541009d0 2024-09-19T07:38:07.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e541a48e0 con 0x7f5e541009d0 2024-09-19T07:38:07.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e517fa640 1 -- 172.21.15.103:0/458594457 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400f6b0 con 0x7f5e541009d0 2024-09-19T07:38:07.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.087+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e541a4e00 con 0x7f5e541009d0 2024-09-19T07:38:07.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.088+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e54102110 con 0x7f5e541009d0 2024-09-19T07:38:07.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.089+0000 7f5e517fa640 1 -- 172.21.15.103:0/458594457 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f5e440109d0 con 0x7f5e541009d0 2024-09-19T07:38:07.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.090+0000 7f5e517fa640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5e400a3500 0x7f5e400a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.090+0000 7f5e517fa640 1 -- 172.21.15.103:0/458594457 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5e440cfb00 con 0x7f5e541009d0 2024-09-19T07:38:07.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.090+0000 7f5e4bfff640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5e400a3500 0x7f5e400a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.091+0000 7f5e4bfff640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5e400a3500 0x7f5e400a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c000c00 tx=0x7f5e3c005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:07.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.094+0000 7f5e517fa640 1 -- 172.21.15.103:0/458594457 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5e44009de0 con 0x7f5e541009d0 2024-09-19T07:38:07.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:38:07.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.292+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f5e540fd500 con 0x7f5e400a3500 2024-09-19T07:38:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.353+0000 7f5e517fa640 1 -- 172.21.15.103:0/458594457 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f5e540fd500 con 0x7f5e400a3500 2024-09-19T07:38:07.354 INFO:teuthology.orchestra.run.smithi103.stdout:Retention added to path / 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.355+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5e400a3500 msgr2=0x7f5e400a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.355+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5e400a3500 0x7f5e400a59c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c000c00 tx=0x7f5e3c005a20 comp rx=0 tx=0).stop 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.355+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 msgr2=0x7f5e54199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.355+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54199150 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400d950 tx=0x7f5e4400de10 comp rx=0 tx=0).stop 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 shutdown_connections 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5e400a3500 0x7f5e400a59c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e541069e0 0x7f5e541a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e54101370 0x7f5e54199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 --2- 172.21.15.103:0/458594457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e541009d0 0x7f5e54199150 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 >> 172.21.15.103:0/458594457 conn(0x7f5e540fc6f0 msgr2=0x7f5e540fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:07.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 shutdown_connections 2024-09-19T07:38:07.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.356+0000 7f5e59ca4640 1 -- 172.21.15.103:0/458594457 wait complete. 2024-09-19T07:38:07.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 -- 172.21.15.103:0/1410505387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc1026f0 msgr2=0x7f57bc10d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 --2- 172.21.15.103:0/1410505387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc1026f0 0x7f57bc10d110 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009cd0 tx=0x7f57b402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 -- 172.21.15.103:0/1410505387 shutdown_connections 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 --2- 172.21.15.103:0/1410505387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc1026f0 0x7f57bc10d110 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 --2- 172.21.15.103:0/1410505387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57bc101d70 0x7f57bc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 --2- 172.21.15.103:0/1410505387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc100490 0x7f57bc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 -- 172.21.15.103:0/1410505387 >> 172.21.15.103:0/1410505387 conn(0x7f57bc0770f0 msgr2=0x7f57bc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 -- 172.21.15.103:0/1410505387 shutdown_connections 2024-09-19T07:38:07.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.474+0000 7f57c2bff640 1 -- 172.21.15.103:0/1410505387 wait complete. 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 Processor -- start 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 -- start start 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc100490 0x7f57bc197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101d70 0x7f57bc1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57bc1026f0 0x7f57bc19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc110ed0 con 0x7f57bc101d70 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc110d50 con 0x7f57bc100490 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c2bff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc111050 con 0x7f57bc1026f0 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101d70 0x7f57bc1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101d70 0x7f57bc1977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46000/0 (socket says 172.21.15.103:46000) 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57bbfff640 1 -- 172.21.15.103:0/4140845199 learned_addr learned my addr 172.21.15.103:0/4140845199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:07.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c099b640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc100490 0x7f57bc197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57c119c640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57bc1026f0 0x7f57bc19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57bbfff640 1 -- 172.21.15.103:0/4140845199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57bc1026f0 msgr2=0x7f57bc19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57bbfff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57bc1026f0 0x7f57bc19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57bbfff640 1 -- 172.21.15.103:0/4140845199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc100490 msgr2=0x7f57bc197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.475+0000 7f57bbfff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc100490 0x7f57bc197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.476+0000 7f57bbfff640 1 -- 172.21.15.103:0/4140845199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc19c250 con 0x7f57bc101d70 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.476+0000 7f57bbfff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101d70 0x7f57bc1977a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f57a800e9f0 tx=0x7f57a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.476+0000 7f57b9ffb640 1 -- 172.21.15.103:0/4140845199 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a800cd80 con 0x7f57bc101d70 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.476+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57bc19c540 con 0x7f57bc101d70 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.476+0000 7f57b9ffb640 1 -- 172.21.15.103:0/4140845199 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a8005500 con 0x7f57bc101d70 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.476+0000 7f57b9ffb640 1 -- 172.21.15.103:0/4140845199 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8010690 con 0x7f57bc101d70 2024-09-19T07:38:07.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.476+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57bc1a3dd0 con 0x7f57bc101d70 2024-09-19T07:38:07.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.477+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57bc100870 con 0x7f57bc101d70 2024-09-19T07:38:07.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.478+0000 7f57b9ffb640 1 -- 172.21.15.103:0/4140845199 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f57a80046c0 con 0x7f57bc101d70 2024-09-19T07:38:07.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.479+0000 7f57b9ffb640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f57ac0a3500 0x7f57ac0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.479+0000 7f57b9ffb640 1 -- 172.21.15.103:0/4140845199 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f57a80d02f0 con 0x7f57bc101d70 2024-09-19T07:38:07.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.479+0000 7f57c099b640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f57ac0a3500 0x7f57ac0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.480+0000 7f57c099b640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f57ac0a3500 0x7f57ac0a59c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f57b000bbe0 tx=0x7f57b000b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:07.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.484+0000 7f57b9ffb640 1 -- 172.21.15.103:0/4140845199 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f57a809cb50 con 0x7f57bc101d70 2024-09-19T07:38:07.496 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1781136807' entity='client.admin' 2024-09-19T07:38:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:07.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3276902154' entity='client.admin' 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:07.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:07 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:07.621 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1781136807' entity='client.admin' 2024-09-19T07:38:07.621 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:07.621 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:07.621 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:07.621 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3276902154' entity='client.admin' 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:07.622 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:07 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1781136807' entity='client.admin' 2024-09-19T07:38:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3276902154' entity='client.admin' 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:38:07.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:07 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:38:07.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.686+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f57bc1047d0 con 0x7f57ac0a3500 2024-09-19T07:38:07.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.692+0000 7f57b9ffb640 1 -- 172.21.15.103:0/4140845199 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f57bc1047d0 con 0x7f57ac0a3500 2024-09-19T07:38:07.693 INFO:teuthology.orchestra.run.smithi103.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T07:38:06", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T07:38:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f57ac0a3500 msgr2=0x7f57ac0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f57ac0a3500 0x7f57ac0a59c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f57b000bbe0 tx=0x7f57b000b390 comp rx=0 tx=0).stop 2024-09-19T07:38:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101d70 msgr2=0x7f57bc1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101d70 0x7f57bc1977a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f57a800e9f0 tx=0x7f57a800edf0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 shutdown_connections 2024-09-19T07:38:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f57ac0a3500 0x7f57ac0a59c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57bc1026f0 0x7f57bc19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101d70 0x7f57bc1977a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 --2- 172.21.15.103:0/4140845199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc100490 0x7f57bc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 >> 172.21.15.103:0/4140845199 conn(0x7f57bc0770f0 msgr2=0x7f57bc1040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 shutdown_connections 2024-09-19T07:38:07.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.694+0000 7f57c2bff640 1 -- 172.21.15.103:0/4140845199 wait complete. 2024-09-19T07:38:07.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 -- 172.21.15.103:0/3707927799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c01035a0 msgr2=0x7ff4c010ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3707927799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c01035a0 0x7ff4c010ba80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009900 tx=0x7ff4b002a480 comp rx=0 tx=0).stop 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 -- 172.21.15.103:0/3707927799 shutdown_connections 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3707927799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4c010c0d0 0x7ff4c010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3707927799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c01035a0 0x7ff4c010ba80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3707927799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4c0102c80 0x7ff4c0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 -- 172.21.15.103:0/3707927799 >> 172.21.15.103:0/3707927799 conn(0x7ff4c0077990 msgr2=0x7ff4c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:07.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 -- 172.21.15.103:0/3707927799 shutdown_connections 2024-09-19T07:38:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.823+0000 7ff4c7711640 1 -- 172.21.15.103:0/3707927799 wait complete. 2024-09-19T07:38:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 Processor -- start 2024-09-19T07:38:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 -- start start 2024-09-19T07:38:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4c0102c80 0x7ff4c0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4c01035a0 0x7ff4c01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c010c0d0 0x7ff4c019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0111eb0 con 0x7ff4c010c0d0 2024-09-19T07:38:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0111d30 con 0x7ff4c0102c80 2024-09-19T07:38:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c7711640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0112030 con 0x7ff4c01035a0 2024-09-19T07:38:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c010c0d0 0x7ff4c019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c010c0d0 0x7ff4c019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46020/0 (socket says 172.21.15.103:46020) 2024-09-19T07:38:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 -- 172.21.15.103:0/3662927364 learned_addr learned my addr 172.21.15.103:0/3662927364 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:07.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c54ad640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4c0102c80 0x7ff4c0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c4cac640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4c01035a0 0x7ff4c01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 -- 172.21.15.103:0/3662927364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4c01035a0 msgr2=0x7ff4c01988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4c01035a0 0x7ff4c01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 -- 172.21.15.103:0/3662927364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4c0102c80 msgr2=0x7ff4c0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4c0102c80 0x7ff4c0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 -- 172.21.15.103:0/3662927364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c019d370 con 0x7ff4c010c0d0 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.824+0000 7ff4c5cae640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c010c0d0 0x7ff4c019cc40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009fd0 tx=0x7ff4b800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:07.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.825+0000 7ff4b67fc640 1 -- 172.21.15.103:0/3662927364 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8017070 con 0x7ff4c010c0d0 2024-09-19T07:38:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.825+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c019d660 con 0x7ff4c010c0d0 2024-09-19T07:38:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.825+0000 7ff4b67fc640 1 -- 172.21.15.103:0/3662927364 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b8003e50 con 0x7ff4c010c0d0 2024-09-19T07:38:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.825+0000 7ff4b67fc640 1 -- 172.21.15.103:0/3662927364 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80106c0 con 0x7ff4c010c0d0 2024-09-19T07:38:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.825+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c01a4ef0 con 0x7ff4c010c0d0 2024-09-19T07:38:07.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.826+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c0103060 con 0x7ff4c010c0d0 2024-09-19T07:38:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.827+0000 7ff4b67fc640 1 -- 172.21.15.103:0/3662927364 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7ff4b8003970 con 0x7ff4c010c0d0 2024-09-19T07:38:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.827+0000 7ff4b67fc640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff4ac0a3500 0x7ff4ac0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.827+0000 7ff4b67fc640 1 -- 172.21.15.103:0/3662927364 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff4b80cfbe0 con 0x7ff4c010c0d0 2024-09-19T07:38:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.827+0000 7ff4c54ad640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff4ac0a3500 0x7ff4ac0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:07.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.828+0000 7ff4c54ad640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff4ac0a3500 0x7ff4ac0a59c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009bd0 tx=0x7ff4bc005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:07.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.832+0000 7ff4b67fc640 1 -- 172.21.15.103:0/3662927364 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4b809c460 con 0x7ff4c010c0d0 2024-09-19T07:38:08.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 -- 172.21.15.103:0/2058571581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880106f00 msgr2=0x7fe88010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 --2- 172.21.15.103:0/2058571581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880106f00 0x7fe88010ea80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe878009ca0 tx=0x7fe87802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 -- 172.21.15.103:0/2058571581 shutdown_connections 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 --2- 172.21.15.103:0/2058571581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880106f00 0x7fe88010ea80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 --2- 172.21.15.103:0/2058571581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe880102780 0x7fe8801067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 --2- 172.21.15.103:0/2058571581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe880101dd0 0x7fe8801021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 -- 172.21.15.103:0/2058571581 >> 172.21.15.103:0/2058571581 conn(0x7fe8800fd8b0 msgr2=0x7fe8800ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 -- 172.21.15.103:0/2058571581 shutdown_connections 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 -- 172.21.15.103:0/2058571581 wait complete. 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.989+0000 7fe885acd640 1 Processor -- start 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe885acd640 1 -- start start 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe885acd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880101dd0 0x7fe880198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe885acd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe880102780 0x7fe880198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe885acd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe880106f00 0x7fe88019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe885acd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe880111f60 con 0x7fe880101dd0 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe885acd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe880111de0 con 0x7fe880102780 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe885acd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801120e0 con 0x7fe880106f00 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880101dd0 0x7fe880198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880101dd0 0x7fe880198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46044/0 (socket says 172.21.15.103:46044) 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 -- 172.21.15.103:0/3079076413 learned_addr learned my addr 172.21.15.103:0/3079076413 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87ffff640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe880106f00 0x7fe88019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87effd640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe880102780 0x7fe880198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 -- 172.21.15.103:0/3079076413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe880106f00 msgr2=0x7fe88019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe880106f00 0x7fe88019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 -- 172.21.15.103:0/3079076413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe880102780 msgr2=0x7fe880198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe880102780 0x7fe880198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.990+0000 7fe87f7fe640 1 -- 172.21.15.103:0/3079076413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88019d300 con 0x7fe880101dd0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.991+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880101dd0 0x7fe880198350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9f0 tx=0x7fe86800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.991+0000 7fe87cff9640 1 -- 172.21.15.103:0/3079076413 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86800cd80 con 0x7fe880101dd0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.991+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88019d5f0 con 0x7fe880101dd0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.991+0000 7fe87cff9640 1 -- 172.21.15.103:0/3079076413 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe868005500 con 0x7fe880101dd0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.991+0000 7fe87cff9640 1 -- 172.21.15.103:0/3079076413 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868010690 con 0x7fe880101dd0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.991+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8801a4ed0 con 0x7fe880101dd0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe87cff9640 1 -- 172.21.15.103:0/3079076413 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fe8680046c0 con 0x7fe880101dd0 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 Processor -- start 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 -- start start 2024-09-19T07:38:08.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59ed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561af59532e0 con 0x561af59eddc0 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561af58face0 con 0x561af59ed4a0 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.994+0000 7fe887ea6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561af594f2a0 con 0x561af59f8680 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46050/0 (socket says 172.21.15.103:46050) 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 -- 172.21.15.103:0/1235261206 learned_addr learned my addr 172.21.15.103:0/1235261206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87effd640 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87ffff640 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87cff9640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe86c0a3440 0x7fe86c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 -- 172.21.15.103:0/1235261206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 msgr2=0x561af59faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59faa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 -- 172.21.15.103:0/1235261206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 msgr2=0x561af59ed880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59ed880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 -- 172.21.15.103:0/1235261206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af58df440 con 0x561af59eddc0 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87ffff640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe86c0a3440 0x7fe86c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87cff9640 1 -- 172.21.15.103:0/3079076413 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe8680d0340 con 0x7fe880101dd0 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.995+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f8140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe8680cd980 tx=0x7fe868066ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3685388bee72b7b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe875ffb640 1 -- 172.21.15.103:0/1235261206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86809c000 con 0x561af59eddc0 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe875ffb640 1 -- 172.21.15.103:0/1235261206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8680673d0 con 0x561af59eddc0 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 -- 172.21.15.103:0/1235261206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 msgr2=0x561af59f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe87ffff640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe86c0a3440 0x7fe86c0a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe878004af0 tx=0x7fe8780095c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f8140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe8680cd980 tx=0x7fe868066ec0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 -- 172.21.15.103:0/1235261206 shutdown_connections 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f8140 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1235261206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59ed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 -- 172.21.15.103:0/1235261206 >> 172.21.15.103:0/1235261206 conn(0x561af59e9a30 msgr2=0x561af59e9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 -- 172.21.15.103:0/1235261206 shutdown_connections 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.996+0000 7fe887ea6740 1 -- 172.21.15.103:0/1235261206 wait complete. 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 -- start start 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 Processor -- start 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af59f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59f8680 0x561af59f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561af59fb250 con 0x561af59eddc0 2024-09-19T07:38:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561af590e050 con 0x561af59f8680 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561af59fb3d0 con 0x561af59ed4a0 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe887ea6740 10 client.0 _ll_register_callbacks cb 0x561af594e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe87effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe87effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46052/0 (socket says 172.21.15.103:46052) 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe87effd640 1 -- 172.21.15.103:0/1017206764 learned_addr learned my addr 172.21.15.103:0/1017206764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.997+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af59f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87ffff640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59f8680 0x561af59f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87effd640 1 -- 172.21.15.103:0/1017206764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 msgr2=0x561af59f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87effd640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af59f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87effd640 1 -- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59f8680 msgr2=0x561af59f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87effd640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59f8680 0x561af59f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87effd640 1 -- 172.21.15.103:0/1017206764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af56c43c0 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af59f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe87effd640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f5280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cd60 tx=0x7fe8700039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce45934b3ab5b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe85effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe85effd640 1 -- 172.21.15.103:0/1017206764 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870011910 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe85effd640 1 -- 172.21.15.103:0/1017206764 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe870011ab0 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8680673d0 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe85effd640 1 -- 172.21.15.103:0/1017206764 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe870011cb0 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe887ea6740 10 client.15378 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561af5a06c80 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.998+0000 7fe85effd640 1 -- 172.21.15.103:0/1017206764 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8680673d0 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe85effd640 1 -- 172.21.15.103:0/1017206764 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe870018070 con 0x561af59eddc0 2024-09-19T07:38:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe887ea6740 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe887ea6740 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe887ea6740 20 client.15378 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a08190 0x561af5a0a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe887ea6740 4 client.15378 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af5960850 con 0x561af5a08190 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a08190 0x561af5a0a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a08190 0x561af5a0a5f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee8d20032c5749a server_cookie=e583a2ecb472ef4 in_seq=0 out_seq=0 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:07.999+0000 7fe85effd640 10 client.15378 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe85effd640 1 -- 172.21.15.103:0/1017206764 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561af5960850 con 0x561af5a08190 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe85effd640 10 client.15378 handle_command_reply: tid=0 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 client.15378 shutdown 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 msgr2=0x561af59f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f5280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cd60 tx=0x7fe8700039c0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 shutdown_connections 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a08190 0x561af5a0a5f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59f8680 0x561af59f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59f5280 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 --2- 172.21.15.103:0/1017206764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af59f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe85effd640 0 client.15378 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 >> 172.21.15.103:0/1017206764 conn(0x561af59e9a30 msgr2=0x561af59e8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 shutdown_connections 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 1 -- 172.21.15.103:0/1017206764 wait complete. 2024-09-19T07:38:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.000+0000 7fe887ea6740 20 client.15378 trim_cache size 0 max 16384 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 Processor -- start 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 -- start start 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af5a044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59eddc0 0x561af5a0fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59f8680 0x561af5a10400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801ad890 con 0x561af59f8680 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801af3d0 con 0x561af59eddc0 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe887ea6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801adf60 con 0x561af59ed4a0 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe87ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59f8680 0x561af5a10400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe87ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59f8680 0x561af5a10400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46068/0 (socket says 172.21.15.103:46068) 2024-09-19T07:38:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.018+0000 7fe87ffff640 1 -- 172.21.15.103:0/2399931508 learned_addr learned my addr 172.21.15.103:0/2399931508 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af5a044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87effd640 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59eddc0 0x561af5a0fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87ffff640 1 -- 172.21.15.103:0/2399931508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 msgr2=0x561af5a044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87ffff640 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af5a044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87ffff640 1 -- 172.21.15.103:0/2399931508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59eddc0 msgr2=0x561af5a0fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87ffff640 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59eddc0 0x561af5a0fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87ffff640 1 -- 172.21.15.103:0/2399931508 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af59f4230 con 0x561af59f8680 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87effd640 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59eddc0 0x561af5a0fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe87ffff640 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59f8680 0x561af5a10400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe878035000 tx=0x7fe878043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291fa001120570c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe8757fa640 1 -- 172.21.15.103:0/2399931508 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8780315d0 con 0x561af59f8680 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe8757fa640 1 -- 172.21.15.103:0/2399931508 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe878036040 con 0x561af59f8680 2024-09-19T07:38:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.019+0000 7fe8757fa640 1 -- 172.21.15.103:0/2399931508 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87802f870 con 0x561af59f8680 2024-09-19T07:38:08.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 -- 172.21.15.103:0/2399931508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59f8680 msgr2=0x561af5a10400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59f8680 0x561af5a10400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe878035000 tx=0x7fe878043000 comp rx=0 tx=0).stop 2024-09-19T07:38:08.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 -- 172.21.15.103:0/2399931508 shutdown_connections 2024-09-19T07:38:08.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59f8680 0x561af5a10400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59eddc0 0x561af5a0fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2399931508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59ed4a0 0x561af5a044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 -- 172.21.15.103:0/2399931508 >> 172.21.15.103:0/2399931508 conn(0x561af59e9a30 msgr2=0x561af59f08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 -- 172.21.15.103:0/2399931508 shutdown_connections 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 -- 172.21.15.103:0/2399931508 wait complete. 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 -- start start 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.020+0000 7fe887ea6740 1 Processor -- start 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe887ea6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe887ea6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801aed00 con 0x561af59eddc0 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe887ea6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801ae630 con 0x561af59ed4a0 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe887ea6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8801ab870 con 0x561af59f8680 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe887ea6740 10 client.0 _ll_register_callbacks cb 0x7fe870004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:08.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59fcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46076/0 (socket says 172.21.15.103:46076) 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 -- 172.21.15.103:0/2274431530 learned_addr learned my addr 172.21.15.103:0/2274431530 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87ffff640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 -- 172.21.15.103:0/2274431530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 msgr2=0x561af59fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 -- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 msgr2=0x561af59fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59fc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 -- 172.21.15.103:0/2274431530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af5a720e0 con 0x561af59eddc0 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87ffff640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe87effd640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59fcb60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe8700227e0 tx=0x7fe8700119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661ae11444f3fcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.021+0000 7fe85effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:08.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe85effd640 1 -- 172.21.15.103:0/2274431530 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870005450 con 0x561af59eddc0 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe85effd640 1 -- 172.21.15.103:0/2274431530 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe8700055f0 con 0x561af59eddc0 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe878036040 con 0x561af59eddc0 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe85effd640 1 -- 172.21.15.103:0/2274431530 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe8700223f0 con 0x561af59eddc0 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 10 client.15384 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561af5a73340 con 0x561af59eddc0 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe85effd640 1 -- 172.21.15.103:0/2274431530 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe878036040 con 0x561af59eddc0 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe85effd640 1 -- 172.21.15.103:0/2274431530 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe870018040 con 0x561af59eddc0 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 20 client.15384 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a73a10 0x561af5a75e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 4 client.15384 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:38:08.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561af595ad60 con 0x561af5a73a10 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.022+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a73a10 0x561af5a75e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe87f7fe640 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a73a10 0x561af5a75e30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557a4ff2cc45d385 server_cookie=a2ac7d644a091ef5 in_seq=0 out_seq=0 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe85effd640 10 client.15384 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe85effd640 1 -- 172.21.15.103:0/2274431530 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561af595ad60 con 0x561af5a73a10 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe85effd640 10 client.15384 handle_command_reply: tid=0 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 client.15384 shutdown 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 msgr2=0x561af59fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59fcb60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe8700227e0 tx=0x7fe8700119c0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 shutdown_connections 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561af5a73a10 0x561af5a75e30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561af59f8680 0x561af59fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af59eddc0 0x561af59fcb60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe85effd640 0 client.15384 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 --2- 172.21.15.103:0/2274431530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561af59ed4a0 0x561af59fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 >> 172.21.15.103:0/2274431530 conn(0x561af59e9a30 msgr2=0x561af59f6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 shutdown_connections 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.023+0000 7fe887ea6740 1 -- 172.21.15.103:0/2274431530 wait complete. 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe887ea6740 20 client.15384 trim_cache size 0 max 16384 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:38:08.030 INFO:teuthology.orchestra.run.smithi103.stdout: "17324f75-dbbd-4936-9d4b-c86c7f6195f1": { 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "17324f75-dbbd-4936-9d4b-c86c7f6195f1", 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe86c0a3440 msgr2=0x7fe86c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe86c0a3440 0x7fe86c0a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe878004af0 tx=0x7fe8780095c0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880101dd0 msgr2=0x7fe880198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880101dd0 0x7fe880198350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9f0 tx=0x7fe86800edf0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 shutdown_connections 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe86c0a3440 0x7fe86c0a5900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe880106f00 0x7fe88019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe880102780 0x7fe880198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 --2- 172.21.15.103:0/3079076413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe880101dd0 0x7fe880198350 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 >> 172.21.15.103:0/3079076413 conn(0x7fe8800fd8b0 msgr2=0x7fe8800feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 shutdown_connections 2024-09-19T07:38:08.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.024+0000 7fe885acd640 1 -- 172.21.15.103:0/3079076413 wait complete. 2024-09-19T07:38:08.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.037+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff4c00fd730 con 0x7ff4ac0a3500 2024-09-19T07:38:08.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.044+0000 7ff4b67fc640 1 -- 172.21.15.103:0/3662927364 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff4c00fd730 con 0x7ff4ac0a3500 2024-09-19T07:38:08.045 INFO:teuthology.orchestra.run.smithi103.stdout:/ 1m 6m3h 2024-09-19T07:38:08.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff4ac0a3500 msgr2=0x7ff4ac0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff4ac0a3500 0x7ff4ac0a59c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009bd0 tx=0x7ff4bc005f90 comp rx=0 tx=0).stop 2024-09-19T07:38:08.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c010c0d0 msgr2=0x7ff4c019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:08.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c010c0d0 0x7ff4c019cc40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009fd0 tx=0x7ff4b800b260 comp rx=0 tx=0).stop 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 shutdown_connections 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff4ac0a3500 0x7ff4ac0a59c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4c010c0d0 0x7ff4c019cc40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4c01035a0 0x7ff4c01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 --2- 172.21.15.103:0/3662927364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4c0102c80 0x7ff4c0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 >> 172.21.15.103:0/3662927364 conn(0x7ff4c0077990 msgr2=0x7ff4c00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 shutdown_connections 2024-09-19T07:38:08.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:08.046+0000 7ff4c7711640 1 -- 172.21.15.103:0/3662927364 wait complete. 2024-09-19T07:38:08.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T07:38:08.088 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T07:38:08.099 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T07:38:08.139 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T07:38:08.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:38:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:08 smithi103 ceph-mon[31201]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T07:38:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:08 smithi103 ceph-mon[31201]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:08 smithi134 ceph-mon[35414]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T07:38:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:08 smithi134 ceph-mon[35414]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:08 smithi174 ceph-mon[35370]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-19T07:38:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:08 smithi174 ceph-mon[35370]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:08.732 INFO:tasks.cephfs.filesystem:scrub status for tag:17324f75-dbbd-4936-9d4b-c86c7f6195f1 - {'path': '/', 'tag': '17324f75-dbbd-4936-9d4b-c86c7f6195f1', 'options': 'recursive,force'} 2024-09-19T07:38:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 -- 172.21.15.103:0/3119169732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 msgr2=0x7f6718103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 --2- 172.21.15.103:0/3119169732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f6718103110 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6714009930 tx=0x7f671402a420 comp rx=0 tx=0).stop 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 -- 172.21.15.103:0/3119169732 shutdown_connections 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 --2- 172.21.15.103:0/3119169732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67181082a0 0x7f671810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 --2- 172.21.15.103:0/3119169732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67181036e0 0x7f6718107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 --2- 172.21.15.103:0/3119169732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f6718103110 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 -- 172.21.15.103:0/3119169732 >> 172.21.15.103:0/3119169732 conn(0x7f67180fc7d0 msgr2=0x7f67180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 -- 172.21.15.103:0/3119169732 shutdown_connections 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.355+0000 7f671f39c640 1 -- 172.21.15.103:0/3119169732 wait complete. 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 Processor -- start 2024-09-19T07:38:09.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 -- start start 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f67181985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67181036e0 0x7f6718198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67181082a0 0x7f671819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67181120a0 con 0x7f6718102d30 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6718111f20 con 0x7f67181036e0 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671f39c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6718112220 con 0x7f67181082a0 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671d138640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f67181985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671d138640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f67181985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34930/0 (socket says 172.21.15.103:34930) 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671d138640 1 -- 172.21.15.103:0/3807063790 learned_addr learned my addr 172.21.15.103:0/3807063790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671c937640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67181036e0 0x7f6718198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.356+0000 7f671d939640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67181082a0 0x7f671819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:09.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671d138640 1 -- 172.21.15.103:0/3807063790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67181082a0 msgr2=0x7f671819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671d138640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67181082a0 0x7f671819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671d138640 1 -- 172.21.15.103:0/3807063790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67181036e0 msgr2=0x7f6718198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671d138640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67181036e0 0x7f6718198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671d138640 1 -- 172.21.15.103:0/3807063790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671819d570 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671d138640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f67181985c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6714009230 tx=0x7f6714004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f670e7fc640 1 -- 172.21.15.103:0/3807063790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671403f030 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f670e7fc640 1 -- 172.21.15.103:0/3807063790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6714003ad0 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671819d800 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f670e7fc640 1 -- 172.21.15.103:0/3807063790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714035070 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.357+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67181a5140 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.358+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671806a850 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.360+0000 7f670e7fc640 1 -- 172.21.15.103:0/3807063790 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6714042030 con 0x7f6718102d30 2024-09-19T07:38:09.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.360+0000 7f670e7fc640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f67040a3440 0x7f67040a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:09.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.360+0000 7f671c937640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f67040a3440 0x7f67040a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:09.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.361+0000 7f670e7fc640 1 -- 172.21.15.103:0/3807063790 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f67140f0730 con 0x7f6718102d30 2024-09-19T07:38:09.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.361+0000 7f671c937640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f67040a3440 0x7f67040a5900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f67180fee00 tx=0x7f670800a2d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:09.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.364+0000 7f670e7fc640 1 -- 172.21.15.103:0/3807063790 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f67140bc960 con 0x7f6718102d30 2024-09-19T07:38:09.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.527+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6718062630 con 0x7f67040a3440 2024-09-19T07:38:09.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:09 smithi103 ceph-mon[31201]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:09 smithi103 ceph-mon[31201]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:09 smithi103 ceph-mon[31201]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.534+0000 7f670e7fc640 1 -- 172.21.15.103:0/3807063790 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258088 (secure 0 0 0) 0x7f6718062630 con 0x7f67040a3440 2024-09-19T07:38:09.537 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:09.538 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:38:09.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f67040a3440 msgr2=0x7f67040a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:09.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f67040a3440 0x7f67040a5900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f67180fee00 tx=0x7f670800a2d0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 msgr2=0x7f67181985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:09.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f67181985c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6714009230 tx=0x7f6714004270 comp rx=0 tx=0).stop 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 shutdown_connections 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f67040a3440 0x7f67040a5900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67181082a0 0x7f671819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67181036e0 0x7f6718198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 --2- 172.21.15.103:0/3807063790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6718102d30 0x7f67181985c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 >> 172.21.15.103:0/3807063790 conn(0x7f67180fc7d0 msgr2=0x7f6718105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 shutdown_connections 2024-09-19T07:38:09.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:09.537+0000 7f671f39c640 1 -- 172.21.15.103:0/3807063790 wait complete. 2024-09-19T07:38:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:09 smithi134 ceph-mon[35414]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:09 smithi134 ceph-mon[35414]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:09 smithi134 ceph-mon[35414]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:09 smithi174 ceph-mon[35370]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:09 smithi174 ceph-mon[35370]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:09 smithi174 ceph-mon[35370]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:38:10.242 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-19T07:38:08.294739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193254,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1429,"num_read_kb":1045,"num_write":579,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":952,"ondisk_log_size":952,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344216,"kb_used_data":11568,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597608,"statfs":{"total":1151940427776,"available":1151587950592,"internally_reserved":0,"allocated":11845632,"data_stored":8972586,"data_compressed":292599,"data_compressed_allocated":393216,"data_compressed_original":3637248,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":236,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.995165"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566027+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:37:58.566027+0000","last_peered":"2024-09-19T07:37:58.566027+0000","last_clean":"2024-09-19T07:37:58.566027+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:37:58.566027+0000","last_undegraded":"2024-09-19T07:37:58.566027+0000","last_fullsized":"2024-09-19T07:37:58.566027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566607+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:37:58.566607+0000","last_peered":"2024-09-19T07:37:58.566607+0000","last_clean":"2024-09-19T07:37:58.566607+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:37:58.566607+0000","last_undegraded":"2024-09-19T07:37:58.566607+0000","last_fullsized":"2024-09-19T07:37:58.566607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566643+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:37:58.566643+0000","last_peered":"2024-09-19T07:37:58.566643+0000","last_clean":"2024-09-19T07:37:58.566643+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:37:58.566643+0000","last_undegraded":"2024-09-19T07:37:58.566643+0000","last_fullsized":"2024-09-19T07:37:58.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395953+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:37:58.395953+0000","last_peered":"2024-09-19T07:37:58.395953+0000","last_clean":"2024-09-19T07:37:58.395953+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:37:58.395953+0000","last_undegraded":"2024-09-19T07:37:58.395953+0000","last_fullsized":"2024-09-19T07:37:58.395953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398077+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:37:58.398077+0000","last_peered":"2024-09-19T07:37:58.398077+0000","last_clean":"2024-09-19T07:37:58.398077+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:37:58.398077+0000","last_undegraded":"2024-09-19T07:37:58.398077+0000","last_fullsized":"2024-09-19T07:37:58.398077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408847+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:37:58.408847+0000","last_peered":"2024-09-19T07:37:58.408847+0000","last_clean":"2024-09-19T07:37:58.408847+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:37:58.408847+0000","last_undegraded":"2024-09-19T07:37:58.408847+0000","last_fullsized":"2024-09-19T07:37:58.408847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567869+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:37:58.567869+0000","last_peered":"2024-09-19T07:37:58.567869+0000","last_clean":"2024-09-19T07:37:58.567869+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:37:58.567869+0000","last_undegraded":"2024-09-19T07:37:58.567869+0000","last_fullsized":"2024-09-19T07:37:58.567869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398332+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:37:58.398332+0000","last_peered":"2024-09-19T07:37:58.398332+0000","last_clean":"2024-09-19T07:37:58.398332+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:37:58.398332+0000","last_undegraded":"2024-09-19T07:37:58.398332+0000","last_fullsized":"2024-09-19T07:37:58.398332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278587+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:37:58.278587+0000","last_peered":"2024-09-19T07:37:58.278587+0000","last_clean":"2024-09-19T07:37:58.278587+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:37:58.278587+0000","last_undegraded":"2024-09-19T07:37:58.278587+0000","last_fullsized":"2024-09-19T07:37:58.278587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409236+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:37:58.409236+0000","last_peered":"2024-09-19T07:37:58.409236+0000","last_clean":"2024-09-19T07:37:58.409236+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:37:58.409236+0000","last_undegraded":"2024-09-19T07:37:58.409236+0000","last_fullsized":"2024-09-19T07:37:58.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568581+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:37:58.568581+0000","last_peered":"2024-09-19T07:37:58.568581+0000","last_clean":"2024-09-19T07:37:58.568581+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:37:58.568581+0000","last_undegraded":"2024-09-19T07:37:58.568581+0000","last_fullsized":"2024-09-19T07:37:58.568581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566381+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:37:58.566381+0000","last_peered":"2024-09-19T07:37:58.566381+0000","last_clean":"2024-09-19T07:37:58.566381+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:37:58.566381+0000","last_undegraded":"2024-09-19T07:37:58.566381+0000","last_fullsized":"2024-09-19T07:37:58.566381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278860+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:37:58.278860+0000","last_peered":"2024-09-19T07:37:58.278860+0000","last_clean":"2024-09-19T07:37:58.278860+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:37:58.278860+0000","last_undegraded":"2024-09-19T07:37:58.278860+0000","last_fullsized":"2024-09-19T07:37:58.278860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566247+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:37:58.566247+0000","last_peered":"2024-09-19T07:37:58.566247+0000","last_clean":"2024-09-19T07:37:58.566247+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:37:58.566247+0000","last_undegraded":"2024-09-19T07:37:58.566247+0000","last_fullsized":"2024-09-19T07:37:58.566247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566034+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:37:58.566034+0000","last_peered":"2024-09-19T07:37:58.566034+0000","last_clean":"2024-09-19T07:37:58.566034+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:37:58.566034+0000","last_undegraded":"2024-09-19T07:37:58.566034+0000","last_fullsized":"2024-09-19T07:37:58.566034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277343+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:37:58.277343+0000","last_peered":"2024-09-19T07:37:58.277343+0000","last_clean":"2024-09-19T07:37:58.277343+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:37:58.277343+0000","last_undegraded":"2024-09-19T07:37:58.277343+0000","last_fullsized":"2024-09-19T07:37:58.277343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566048+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:37:58.566048+0000","last_peered":"2024-09-19T07:37:58.566048+0000","last_clean":"2024-09-19T07:37:58.566048+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:37:58.566048+0000","last_undegraded":"2024-09-19T07:37:58.566048+0000","last_fullsized":"2024-09-19T07:37:58.566048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395689+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:37:58.395689+0000","last_peered":"2024-09-19T07:37:58.395689+0000","last_clean":"2024-09-19T07:37:58.395689+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:37:58.395689+0000","last_undegraded":"2024-09-19T07:37:58.395689+0000","last_fullsized":"2024-09-19T07:37:58.395689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281089+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:37:58.281089+0000","last_peered":"2024-09-19T07:37:58.281089+0000","last_clean":"2024-09-19T07:37:58.281089+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:37:58.281089+0000","last_undegraded":"2024-09-19T07:37:58.281089+0000","last_fullsized":"2024-09-19T07:37:58.281089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565798+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:37:58.565798+0000","last_peered":"2024-09-19T07:37:58.565798+0000","last_clean":"2024-09-19T07:37:58.565798+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:37:58.565798+0000","last_undegraded":"2024-09-19T07:37:58.565798+0000","last_fullsized":"2024-09-19T07:37:58.565798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278866+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:37:58.278866+0000","last_peered":"2024-09-19T07:37:58.278866+0000","last_clean":"2024-09-19T07:37:58.278866+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:37:58.278866+0000","last_undegraded":"2024-09-19T07:37:58.278866+0000","last_fullsized":"2024-09-19T07:37:58.278866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277508+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:37:58.277508+0000","last_peered":"2024-09-19T07:37:58.277508+0000","last_clean":"2024-09-19T07:37:58.277508+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:37:58.277508+0000","last_undegraded":"2024-09-19T07:37:58.277508+0000","last_fullsized":"2024-09-19T07:37:58.277508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398644+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:37:58.398644+0000","last_peered":"2024-09-19T07:37:58.398644+0000","last_clean":"2024-09-19T07:37:58.398644+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:37:58.398644+0000","last_undegraded":"2024-09-19T07:37:58.398644+0000","last_fullsized":"2024-09-19T07:37:58.398644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405691+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:37:58.405691+0000","last_peered":"2024-09-19T07:37:58.405691+0000","last_clean":"2024-09-19T07:37:58.405691+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:37:58.405691+0000","last_undegraded":"2024-09-19T07:37:58.405691+0000","last_fullsized":"2024-09-19T07:37:58.405691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399976+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:37:58.399976+0000","last_peered":"2024-09-19T07:37:58.399976+0000","last_clean":"2024-09-19T07:37:58.399976+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:37:58.399976+0000","last_undegraded":"2024-09-19T07:37:58.399976+0000","last_fullsized":"2024-09-19T07:37:58.399976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281203+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:37:58.281203+0000","last_peered":"2024-09-19T07:37:58.281203+0000","last_clean":"2024-09-19T07:37:58.281203+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:37:58.281203+0000","last_undegraded":"2024-09-19T07:37:58.281203+0000","last_fullsized":"2024-09-19T07:37:58.281203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565955+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:37:58.565955+0000","last_peered":"2024-09-19T07:37:58.565955+0000","last_clean":"2024-09-19T07:37:58.565955+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:37:58.565955+0000","last_undegraded":"2024-09-19T07:37:58.565955+0000","last_fullsized":"2024-09-19T07:37:58.565955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278990+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:37:58.278990+0000","last_peered":"2024-09-19T07:37:58.278990+0000","last_clean":"2024-09-19T07:37:58.278990+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:37:58.278990+0000","last_undegraded":"2024-09-19T07:37:58.278990+0000","last_fullsized":"2024-09-19T07:37:58.278990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566392+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:37:58.566392+0000","last_peered":"2024-09-19T07:37:58.566392+0000","last_clean":"2024-09-19T07:37:58.566392+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:37:58.566392+0000","last_undegraded":"2024-09-19T07:37:58.566392+0000","last_fullsized":"2024-09-19T07:37:58.566392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280791+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:37:58.280791+0000","last_peered":"2024-09-19T07:37:58.280791+0000","last_clean":"2024-09-19T07:37:58.280791+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:37:58.280791+0000","last_undegraded":"2024-09-19T07:37:58.280791+0000","last_fullsized":"2024-09-19T07:37:58.280791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280919+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:37:58.280919+0000","last_peered":"2024-09-19T07:37:58.280919+0000","last_clean":"2024-09-19T07:37:58.280919+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:37:58.280919+0000","last_undegraded":"2024-09-19T07:37:58.280919+0000","last_fullsized":"2024-09-19T07:37:58.280919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565810+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:37:58.565810+0000","last_peered":"2024-09-19T07:37:58.565810+0000","last_clean":"2024-09-19T07:37:58.565810+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:37:58.565810+0000","last_undegraded":"2024-09-19T07:37:58.565810+0000","last_fullsized":"2024-09-19T07:37:58.565810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398122+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:37:58.398122+0000","last_peered":"2024-09-19T07:37:58.398122+0000","last_clean":"2024-09-19T07:37:58.398122+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:37:58.398122+0000","last_undegraded":"2024-09-19T07:37:58.398122+0000","last_fullsized":"2024-09-19T07:37:58.398122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":136,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:03.047798+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:38:03.047798+0000","last_peered":"2024-09-19T07:38:03.047798+0000","last_clean":"2024-09-19T07:38:03.047798+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:38:03.047798+0000","last_undegraded":"2024-09-19T07:38:03.047798+0000","last_fullsized":"2024-09-19T07:38:03.047798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405841+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:37:58.405841+0000","last_peered":"2024-09-19T07:37:58.405841+0000","last_clean":"2024-09-19T07:37:58.405841+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:37:58.405841+0000","last_undegraded":"2024-09-19T07:37:58.405841+0000","last_fullsized":"2024-09-19T07:37:58.405841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408925+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:37:58.408925+0000","last_peered":"2024-09-19T07:37:58.408925+0000","last_clean":"2024-09-19T07:37:58.408925+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:37:58.408925+0000","last_undegraded":"2024-09-19T07:37:58.408925+0000","last_fullsized":"2024-09-19T07:37:58.408925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278608+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:37:58.278608+0000","last_peered":"2024-09-19T07:37:58.278608+0000","last_clean":"2024-09-19T07:37:58.278608+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:37:58.278608+0000","last_undegraded":"2024-09-19T07:37:58.278608+0000","last_fullsized":"2024-09-19T07:37:58.278608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398404+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:37:58.398404+0000","last_peered":"2024-09-19T07:37:58.398404+0000","last_clean":"2024-09-19T07:37:58.398404+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:37:58.398404+0000","last_undegraded":"2024-09-19T07:37:58.398404+0000","last_fullsized":"2024-09-19T07:37:58.398404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406017+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:37:58.406017+0000","last_peered":"2024-09-19T07:37:58.406017+0000","last_clean":"2024-09-19T07:37:58.406017+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:37:58.406017+0000","last_undegraded":"2024-09-19T07:37:58.406017+0000","last_fullsized":"2024-09-19T07:37:58.406017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395465+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:37:58.395465+0000","last_peered":"2024-09-19T07:37:58.395465+0000","last_clean":"2024-09-19T07:37:58.395465+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:37:58.395465+0000","last_undegraded":"2024-09-19T07:37:58.395465+0000","last_fullsized":"2024-09-19T07:37:58.395465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":239,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:04.048492+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:38:04.048492+0000","last_peered":"2024-09-19T07:38:04.048492+0000","last_clean":"2024-09-19T07:38:04.048492+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:38:04.048492+0000","last_undegraded":"2024-09-19T07:38:04.048492+0000","last_fullsized":"2024-09-19T07:38:04.048492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278242+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:37:58.278242+0000","last_peered":"2024-09-19T07:37:58.278242+0000","last_clean":"2024-09-19T07:37:58.278242+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:37:58.278242+0000","last_undegraded":"2024-09-19T07:37:58.278242+0000","last_fullsized":"2024-09-19T07:37:58.278242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568741+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:37:58.568741+0000","last_peered":"2024-09-19T07:37:58.568741+0000","last_clean":"2024-09-19T07:37:58.568741+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:37:58.568741+0000","last_undegraded":"2024-09-19T07:37:58.568741+0000","last_fullsized":"2024-09-19T07:37:58.568741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567021+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:37:58.567021+0000","last_peered":"2024-09-19T07:37:58.567021+0000","last_clean":"2024-09-19T07:37:58.567021+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:37:58.567021+0000","last_undegraded":"2024-09-19T07:37:58.567021+0000","last_fullsized":"2024-09-19T07:37:58.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405996+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:37:58.405996+0000","last_peered":"2024-09-19T07:37:58.405996+0000","last_clean":"2024-09-19T07:37:58.405996+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:37:58.405996+0000","last_undegraded":"2024-09-19T07:37:58.405996+0000","last_fullsized":"2024-09-19T07:37:58.405996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277409+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:37:58.277409+0000","last_peered":"2024-09-19T07:37:58.277409+0000","last_clean":"2024-09-19T07:37:58.277409+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:37:58.277409+0000","last_undegraded":"2024-09-19T07:37:58.277409+0000","last_fullsized":"2024-09-19T07:37:58.277409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":284,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:04.048524+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:38:04.048524+0000","last_peered":"2024-09-19T07:38:04.048524+0000","last_clean":"2024-09-19T07:38:04.048524+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:38:04.048524+0000","last_undegraded":"2024-09-19T07:38:04.048524+0000","last_fullsized":"2024-09-19T07:38:04.048524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":222,"num_read_kb":178,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398289+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:37:58.398289+0000","last_peered":"2024-09-19T07:37:58.398289+0000","last_clean":"2024-09-19T07:37:58.398289+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:37:58.398289+0000","last_undegraded":"2024-09-19T07:37:58.398289+0000","last_fullsized":"2024-09-19T07:37:58.398289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280980+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:37:58.280980+0000","last_peered":"2024-09-19T07:37:58.280980+0000","last_clean":"2024-09-19T07:37:58.280980+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:37:58.280980+0000","last_undegraded":"2024-09-19T07:37:58.280980+0000","last_fullsized":"2024-09-19T07:37:58.280980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278729+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:37:58.278729+0000","last_peered":"2024-09-19T07:37:58.278729+0000","last_clean":"2024-09-19T07:37:58.278729+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:37:58.278729+0000","last_undegraded":"2024-09-19T07:37:58.278729+0000","last_fullsized":"2024-09-19T07:37:58.278729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405861+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:37:58.405861+0000","last_peered":"2024-09-19T07:37:58.405861+0000","last_clean":"2024-09-19T07:37:58.405861+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:37:58.405861+0000","last_undegraded":"2024-09-19T07:37:58.405861+0000","last_fullsized":"2024-09-19T07:37:58.405861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568411+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:37:58.568411+0000","last_peered":"2024-09-19T07:37:58.568411+0000","last_clean":"2024-09-19T07:37:58.568411+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:37:58.568411+0000","last_undegraded":"2024-09-19T07:37:58.568411+0000","last_fullsized":"2024-09-19T07:37:58.568411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'9","reported_seq":143,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:03.047936+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:38:03.047936+0000","last_peered":"2024-09-19T07:38:03.047936+0000","last_clean":"2024-09-19T07:38:03.047936+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:38:03.047936+0000","last_undegraded":"2024-09-19T07:38:03.047936+0000","last_fullsized":"2024-09-19T07:38:03.047936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567829+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:37:58.567829+0000","last_peered":"2024-09-19T07:37:58.567829+0000","last_clean":"2024-09-19T07:37:58.567829+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:37:58.567829+0000","last_undegraded":"2024-09-19T07:37:58.567829+0000","last_fullsized":"2024-09-19T07:37:58.567829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565722+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:37:58.565722+0000","last_peered":"2024-09-19T07:37:58.565722+0000","last_clean":"2024-09-19T07:37:58.565722+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:37:58.565722+0000","last_undegraded":"2024-09-19T07:37:58.565722+0000","last_fullsized":"2024-09-19T07:37:58.565722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405806+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:37:58.405806+0000","last_peered":"2024-09-19T07:37:58.405806+0000","last_clean":"2024-09-19T07:37:58.405806+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:37:58.405806+0000","last_undegraded":"2024-09-19T07:37:58.405806+0000","last_fullsized":"2024-09-19T07:37:58.405806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398454+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:37:58.398454+0000","last_peered":"2024-09-19T07:37:58.398454+0000","last_clean":"2024-09-19T07:37:58.398454+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:37:58.398454+0000","last_undegraded":"2024-09-19T07:37:58.398454+0000","last_fullsized":"2024-09-19T07:37:58.398454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566085+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:37:58.566085+0000","last_peered":"2024-09-19T07:37:58.566085+0000","last_clean":"2024-09-19T07:37:58.566085+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:37:58.566085+0000","last_undegraded":"2024-09-19T07:37:58.566085+0000","last_fullsized":"2024-09-19T07:37:58.566085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278747+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:37:58.278747+0000","last_peered":"2024-09-19T07:37:58.278747+0000","last_clean":"2024-09-19T07:37:58.278747+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:37:58.278747+0000","last_undegraded":"2024-09-19T07:37:58.278747+0000","last_fullsized":"2024-09-19T07:37:58.278747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568790+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:37:58.568790+0000","last_peered":"2024-09-19T07:37:58.568790+0000","last_clean":"2024-09-19T07:37:58.568790+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:37:58.568790+0000","last_undegraded":"2024-09-19T07:37:58.568790+0000","last_fullsized":"2024-09-19T07:37:58.568790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'41","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.523245+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:37:58.523245+0000","last_peered":"2024-09-19T07:37:58.523245+0000","last_clean":"2024-09-19T07:37:58.523245+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:37:58.523245+0000","last_undegraded":"2024-09-19T07:37:58.523245+0000","last_fullsized":"2024-09-19T07:37:58.523245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568493+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:37:58.568493+0000","last_peered":"2024-09-19T07:37:58.568493+0000","last_clean":"2024-09-19T07:37:58.568493+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:37:58.568493+0000","last_undegraded":"2024-09-19T07:37:58.568493+0000","last_fullsized":"2024-09-19T07:37:58.568493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:03.048059+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:38:03.048059+0000","last_peered":"2024-09-19T07:38:03.048059+0000","last_clean":"2024-09-19T07:38:03.048059+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:38:03.048059+0000","last_undegraded":"2024-09-19T07:38:03.048059+0000","last_fullsized":"2024-09-19T07:38:03.048059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568589+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:37:58.568589+0000","last_peered":"2024-09-19T07:37:58.568589+0000","last_clean":"2024-09-19T07:37:58.568589+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:37:58.568589+0000","last_undegraded":"2024-09-19T07:37:58.568589+0000","last_fullsized":"2024-09-19T07:37:58.568589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":394,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:00.401620+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:38:00.401620+0000","last_peered":"2024-09-19T07:38:00.401620+0000","last_clean":"2024-09-19T07:38:00.401620+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:38:00.401620+0000","last_undegraded":"2024-09-19T07:38:00.401620+0000","last_fullsized":"2024-09-19T07:38:00.401620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566436+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:37:58.566436+0000","last_peered":"2024-09-19T07:37:58.566436+0000","last_clean":"2024-09-19T07:37:58.566436+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:37:58.566436+0000","last_undegraded":"2024-09-19T07:37:58.566436+0000","last_fullsized":"2024-09-19T07:37:58.566436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399251+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:37:58.399251+0000","last_peered":"2024-09-19T07:37:58.399251+0000","last_clean":"2024-09-19T07:37:58.399251+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:37:58.399251+0000","last_undegraded":"2024-09-19T07:37:58.399251+0000","last_fullsized":"2024-09-19T07:37:58.399251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567920+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:37:58.567920+0000","last_peered":"2024-09-19T07:37:58.567920+0000","last_clean":"2024-09-19T07:37:58.567920+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:37:58.567920+0000","last_undegraded":"2024-09-19T07:37:58.567920+0000","last_fullsized":"2024-09-19T07:37:58.567920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278577+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:37:58.278577+0000","last_peered":"2024-09-19T07:37:58.278577+0000","last_clean":"2024-09-19T07:37:58.278577+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:37:58.278577+0000","last_undegraded":"2024-09-19T07:37:58.278577+0000","last_fullsized":"2024-09-19T07:37:58.278577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":136,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:03.047483+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:38:03.047483+0000","last_peered":"2024-09-19T07:38:03.047483+0000","last_clean":"2024-09-19T07:38:03.047483+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:38:03.047483+0000","last_undegraded":"2024-09-19T07:38:03.047483+0000","last_fullsized":"2024-09-19T07:38:03.047483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405844+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:37:58.405844+0000","last_peered":"2024-09-19T07:37:58.405844+0000","last_clean":"2024-09-19T07:37:58.405844+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:37:58.405844+0000","last_undegraded":"2024-09-19T07:37:58.405844+0000","last_fullsized":"2024-09-19T07:37:58.405844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:03.047544+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:38:03.047544+0000","last_peered":"2024-09-19T07:38:03.047544+0000","last_clean":"2024-09-19T07:38:03.047544+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:38:03.047544+0000","last_undegraded":"2024-09-19T07:38:03.047544+0000","last_fullsized":"2024-09-19T07:38:03.047544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408965+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:37:58.408965+0000","last_peered":"2024-09-19T07:37:58.408965+0000","last_clean":"2024-09-19T07:37:58.408965+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:37:58.408965+0000","last_undegraded":"2024-09-19T07:37:58.408965+0000","last_fullsized":"2024-09-19T07:37:58.408965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396146+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:37:58.396146+0000","last_peered":"2024-09-19T07:37:58.396146+0000","last_clean":"2024-09-19T07:37:58.396146+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:37:58.396146+0000","last_undegraded":"2024-09-19T07:37:58.396146+0000","last_fullsized":"2024-09-19T07:37:58.396146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409093+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:37:58.409093+0000","last_peered":"2024-09-19T07:37:58.409093+0000","last_clean":"2024-09-19T07:37:58.409093+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:37:58.409093+0000","last_undegraded":"2024-09-19T07:37:58.409093+0000","last_fullsized":"2024-09-19T07:37:58.409093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398419+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:37:58.398419+0000","last_peered":"2024-09-19T07:37:58.398419+0000","last_clean":"2024-09-19T07:37:58.398419+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:37:58.398419+0000","last_undegraded":"2024-09-19T07:37:58.398419+0000","last_fullsized":"2024-09-19T07:37:58.398419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278876+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:37:58.278876+0000","last_peered":"2024-09-19T07:37:58.278876+0000","last_clean":"2024-09-19T07:37:58.278876+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:37:58.278876+0000","last_undegraded":"2024-09-19T07:37:58.278876+0000","last_fullsized":"2024-09-19T07:37:58.278876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'51","reported_seq":98,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.530637+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:37:58.530637+0000","last_peered":"2024-09-19T07:37:58.530637+0000","last_clean":"2024-09-19T07:37:58.530637+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:37:58.530637+0000","last_undegraded":"2024-09-19T07:37:58.530637+0000","last_fullsized":"2024-09-19T07:37:58.530637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400100+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:37:58.400100+0000","last_peered":"2024-09-19T07:37:58.400100+0000","last_clean":"2024-09-19T07:37:58.400100+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:37:58.400100+0000","last_undegraded":"2024-09-19T07:37:58.400100+0000","last_fullsized":"2024-09-19T07:37:58.400100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566566+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:37:58.566566+0000","last_peered":"2024-09-19T07:37:58.566566+0000","last_clean":"2024-09-19T07:37:58.566566+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:37:58.566566+0000","last_undegraded":"2024-09-19T07:37:58.566566+0000","last_fullsized":"2024-09-19T07:37:58.566566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566745+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:37:58.566745+0000","last_peered":"2024-09-19T07:37:58.566745+0000","last_clean":"2024-09-19T07:37:58.566745+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:37:58.566745+0000","last_undegraded":"2024-09-19T07:37:58.566745+0000","last_fullsized":"2024-09-19T07:37:58.566745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277436+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:37:58.277436+0000","last_peered":"2024-09-19T07:37:58.277436+0000","last_clean":"2024-09-19T07:37:58.277436+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:37:58.277436+0000","last_undegraded":"2024-09-19T07:37:58.277436+0000","last_fullsized":"2024-09-19T07:37:58.277436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.397993+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:37:58.397993+0000","last_peered":"2024-09-19T07:37:58.397993+0000","last_clean":"2024-09-19T07:37:58.397993+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:37:58.397993+0000","last_undegraded":"2024-09-19T07:37:58.397993+0000","last_fullsized":"2024-09-19T07:37:58.397993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398888+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:37:58.398888+0000","last_peered":"2024-09-19T07:37:58.398888+0000","last_clean":"2024-09-19T07:37:58.398888+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:37:58.398888+0000","last_undegraded":"2024-09-19T07:37:58.398888+0000","last_fullsized":"2024-09-19T07:37:58.398888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565988+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:37:58.565988+0000","last_peered":"2024-09-19T07:37:58.565988+0000","last_clean":"2024-09-19T07:37:58.565988+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:37:58.565988+0000","last_undegraded":"2024-09-19T07:37:58.565988+0000","last_fullsized":"2024-09-19T07:37:58.565988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565778+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:37:58.565778+0000","last_peered":"2024-09-19T07:37:58.565778+0000","last_clean":"2024-09-19T07:37:58.565778+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:37:58.565778+0000","last_undegraded":"2024-09-19T07:37:58.565778+0000","last_fullsized":"2024-09-19T07:37:58.565778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399648+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:37:58.399648+0000","last_peered":"2024-09-19T07:37:58.399648+0000","last_clean":"2024-09-19T07:37:58.399648+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:37:58.399648+0000","last_undegraded":"2024-09-19T07:37:58.399648+0000","last_fullsized":"2024-09-19T07:37:58.399648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398085+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:37:58.398085+0000","last_peered":"2024-09-19T07:37:58.398085+0000","last_clean":"2024-09-19T07:37:58.398085+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:37:58.398085+0000","last_undegraded":"2024-09-19T07:37:58.398085+0000","last_fullsized":"2024-09-19T07:37:58.398085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568069+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:37:58.568069+0000","last_peered":"2024-09-19T07:37:58.568069+0000","last_clean":"2024-09-19T07:37:58.568069+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:37:58.568069+0000","last_undegraded":"2024-09-19T07:37:58.568069+0000","last_fullsized":"2024-09-19T07:37:58.568069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395362+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:37:58.395362+0000","last_peered":"2024-09-19T07:37:58.395362+0000","last_clean":"2024-09-19T07:37:58.395362+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:37:58.395362+0000","last_undegraded":"2024-09-19T07:37:58.395362+0000","last_fullsized":"2024-09-19T07:37:58.395362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398241+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:37:58.398241+0000","last_peered":"2024-09-19T07:37:58.398241+0000","last_clean":"2024-09-19T07:37:58.398241+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:37:58.398241+0000","last_undegraded":"2024-09-19T07:37:58.398241+0000","last_fullsized":"2024-09-19T07:37:58.398241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277959+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:37:58.277959+0000","last_peered":"2024-09-19T07:37:58.277959+0000","last_clean":"2024-09-19T07:37:58.277959+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:37:58.277959+0000","last_undegraded":"2024-09-19T07:37:58.277959+0000","last_fullsized":"2024-09-19T07:37:58.277959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398335+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:37:58.398335+0000","last_peered":"2024-09-19T07:37:58.398335+0000","last_clean":"2024-09-19T07:37:58.398335+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:37:58.398335+0000","last_undegraded":"2024-09-19T07:37:58.398335+0000","last_fullsized":"2024-09-19T07:37:58.398335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566969+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:37:58.566969+0000","last_peered":"2024-09-19T07:37:58.566969+0000","last_clean":"2024-09-19T07:37:58.566969+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:37:58.566969+0000","last_undegraded":"2024-09-19T07:37:58.566969+0000","last_fullsized":"2024-09-19T07:37:58.566969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400607+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:37:58.400607+0000","last_peered":"2024-09-19T07:37:58.400607+0000","last_clean":"2024-09-19T07:37:58.400607+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:37:58.400607+0000","last_undegraded":"2024-09-19T07:37:58.400607+0000","last_fullsized":"2024-09-19T07:37:58.400607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400322+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:37:58.400322+0000","last_peered":"2024-09-19T07:37:58.400322+0000","last_clean":"2024-09-19T07:37:58.400322+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:37:58.400322+0000","last_undegraded":"2024-09-19T07:37:58.400322+0000","last_fullsized":"2024-09-19T07:37:58.400322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'14","reported_seq":160,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:03.047105+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:38:03.047105+0000","last_peered":"2024-09-19T07:38:03.047105+0000","last_clean":"2024-09-19T07:38:03.047105+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:38:03.047105+0000","last_undegraded":"2024-09-19T07:38:03.047105+0000","last_fullsized":"2024-09-19T07:38:03.047105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602652,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1192,"num_read_kb":715,"num_write":376,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1807956,"data_compressed":80685,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107370,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514532,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":451633,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":1243176,"data_compressed":52251,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906991,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":843740,"data_compressed":27858,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":954368,"data_stored":843334,"data_compressed":27858,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364856,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1044883,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528382,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":450775,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1174307,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":451422,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":974882,"data_compressed":34014,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":520934,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":520993,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":452507,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71591,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71532,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":203289,"data_compressed":7749,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:38:10.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:38:10.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:10 smithi103 ceph-mon[31201]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T07:38:10.510 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:10 smithi134 ceph-mon[35414]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T07:38:10.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:10 smithi174 ceph-mon[35370]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-19T07:38:10.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 -- 172.21.15.103:0/16760520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f622410bfc0 msgr2=0x7f622410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:11.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 --2- 172.21.15.103:0/16760520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f622410bfc0 0x7f622410e460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f621c009ca0 tx=0x7f621c02a950 comp rx=0 tx=0).stop 2024-09-19T07:38:11.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 -- 172.21.15.103:0/16760520 shutdown_connections 2024-09-19T07:38:11.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 --2- 172.21.15.103:0/16760520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f622410bfc0 0x7f622410e460 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 --2- 172.21.15.103:0/16760520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62241034b0 0x7f622410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 --2- 172.21.15.103:0/16760520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6224102b90 0x7f6224102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 -- 172.21.15.103:0/16760520 >> 172.21.15.103:0/16760520 conn(0x7f62240fc610 msgr2=0x7f62240fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:11.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 -- 172.21.15.103:0/16760520 shutdown_connections 2024-09-19T07:38:11.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.986+0000 7f622b1a1640 1 -- 172.21.15.103:0/16760520 wait complete. 2024-09-19T07:38:11.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 Processor -- start 2024-09-19T07:38:11.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 -- start start 2024-09-19T07:38:11.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6224102b90 0x7f6224198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:11.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62241034b0 0x7f62241987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:11.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f622410bfc0 0x7f622419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:11.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111d90 con 0x7f62241034b0 2024-09-19T07:38:11.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111c10 con 0x7f6224102b90 2024-09-19T07:38:11.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622b1a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111f10 con 0x7f622410bfc0 2024-09-19T07:38:11.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62241034b0 0x7f62241987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:11.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62241034b0 0x7f62241987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34944/0 (socket says 172.21.15.103:34944) 2024-09-19T07:38:11.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 -- 172.21.15.103:0/2822455799 learned_addr learned my addr 172.21.15.103:0/2822455799 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:11.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622a19f640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6224102b90 0x7f6224198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:11.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622a9a0640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f622410bfc0 0x7f622419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:11.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 -- 172.21.15.103:0/2822455799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f622410bfc0 msgr2=0x7f622419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:11.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f622410bfc0 0x7f622419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 -- 172.21.15.103:0/2822455799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6224102b90 msgr2=0x7f6224198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:11.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6224102b90 0x7f6224198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.987+0000 7f622999e640 1 -- 172.21.15.103:0/2822455799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622419d240 con 0x7f62241034b0 2024-09-19T07:38:11.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.988+0000 7f622999e640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62241034b0 0x7f62241987d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f621400d740 tx=0x7f621400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:11.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.988+0000 7f621b7fe640 1 -- 172.21.15.103:0/2822455799 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214017070 con 0x7f62241034b0 2024-09-19T07:38:11.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.988+0000 7f621b7fe640 1 -- 172.21.15.103:0/2822455799 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621400de50 con 0x7f62241034b0 2024-09-19T07:38:11.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.988+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622419d530 con 0x7f62241034b0 2024-09-19T07:38:11.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.988+0000 7f621b7fe640 1 -- 172.21.15.103:0/2822455799 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400f620 con 0x7f62241034b0 2024-09-19T07:38:11.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.988+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241a4e10 con 0x7f62241034b0 2024-09-19T07:38:11.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.989+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6224102f70 con 0x7f62241034b0 2024-09-19T07:38:11.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.990+0000 7f621b7fe640 1 -- 172.21.15.103:0/2822455799 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6214009c30 con 0x7f62241034b0 2024-09-19T07:38:11.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.991+0000 7f621b7fe640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f62100a34c0 0x7f62100a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:11.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.991+0000 7f621b7fe640 1 -- 172.21.15.103:0/2822455799 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f62140cfa70 con 0x7f62241034b0 2024-09-19T07:38:11.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.991+0000 7f622a19f640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f62100a34c0 0x7f62100a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:11.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.992+0000 7f622a19f640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f62100a34c0 0x7f62100a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62200058c0 tx=0x7f622000a4c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:11.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:10.994+0000 7f621b7fe640 1 -- 172.21.15.103:0/2822455799 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f621409c370 con 0x7f62241034b0 2024-09-19T07:38:11.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.165+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62241070c0 con 0x7f62100a34c0 2024-09-19T07:38:11.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.177+0000 7f621b7fe640 1 -- 172.21.15.103:0/2822455799 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258114 (secure 0 0 0) 0x7f62241070c0 con 0x7f62100a34c0 2024-09-19T07:38:11.180 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:11.181 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:38:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f62100a34c0 msgr2=0x7f62100a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f62100a34c0 0x7f62100a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62200058c0 tx=0x7f622000a4c0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62241034b0 msgr2=0x7f62241987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62241034b0 0x7f62241987d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f621400d740 tx=0x7f621400dc00 comp rx=0 tx=0).stop 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 shutdown_connections 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f62100a34c0 0x7f62100a5980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f622410bfc0 0x7f622419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62241034b0 0x7f62241987d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 --2- 172.21.15.103:0/2822455799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6224102b90 0x7f6224198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 >> 172.21.15.103:0/2822455799 conn(0x7f62240fc610 msgr2=0x7f62240fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 shutdown_connections 2024-09-19T07:38:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:11.180+0000 7f622b1a1640 1 -- 172.21.15.103:0/2822455799 wait complete. 2024-09-19T07:38:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:11 smithi103 ceph-mon[31201]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:11 smithi134 ceph-mon[35414]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:11 smithi174 ceph-mon[35370]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:11.836 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-19T07:38:10.295048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389886,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1734,"num_read_kb":1302,"num_write":705,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1031,"ondisk_log_size":1031,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344300,"kb_used_data":11640,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597524,"statfs":{"total":1151940427776,"available":1151587864576,"internally_reserved":0,"allocated":11919360,"data_stored":9562482,"data_compressed":320166,"data_compressed_allocated":430080,"data_compressed_original":4190208,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":456,"num_read_kb":493,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.995474"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566027+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:37:58.566027+0000","last_peered":"2024-09-19T07:37:58.566027+0000","last_clean":"2024-09-19T07:37:58.566027+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:37:58.566027+0000","last_undegraded":"2024-09-19T07:37:58.566027+0000","last_fullsized":"2024-09-19T07:37:58.566027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566607+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:37:58.566607+0000","last_peered":"2024-09-19T07:37:58.566607+0000","last_clean":"2024-09-19T07:37:58.566607+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:37:58.566607+0000","last_undegraded":"2024-09-19T07:37:58.566607+0000","last_fullsized":"2024-09-19T07:37:58.566607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566643+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:37:58.566643+0000","last_peered":"2024-09-19T07:37:58.566643+0000","last_clean":"2024-09-19T07:37:58.566643+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:37:58.566643+0000","last_undegraded":"2024-09-19T07:37:58.566643+0000","last_fullsized":"2024-09-19T07:37:58.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395953+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:37:58.395953+0000","last_peered":"2024-09-19T07:37:58.395953+0000","last_clean":"2024-09-19T07:37:58.395953+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:37:58.395953+0000","last_undegraded":"2024-09-19T07:37:58.395953+0000","last_fullsized":"2024-09-19T07:37:58.395953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398077+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:37:58.398077+0000","last_peered":"2024-09-19T07:37:58.398077+0000","last_clean":"2024-09-19T07:37:58.398077+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:37:58.398077+0000","last_undegraded":"2024-09-19T07:37:58.398077+0000","last_fullsized":"2024-09-19T07:37:58.398077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408847+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:37:58.408847+0000","last_peered":"2024-09-19T07:37:58.408847+0000","last_clean":"2024-09-19T07:37:58.408847+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:37:58.408847+0000","last_undegraded":"2024-09-19T07:37:58.408847+0000","last_fullsized":"2024-09-19T07:37:58.408847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567869+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:37:58.567869+0000","last_peered":"2024-09-19T07:37:58.567869+0000","last_clean":"2024-09-19T07:37:58.567869+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:37:58.567869+0000","last_undegraded":"2024-09-19T07:37:58.567869+0000","last_fullsized":"2024-09-19T07:37:58.567869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398332+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:37:58.398332+0000","last_peered":"2024-09-19T07:37:58.398332+0000","last_clean":"2024-09-19T07:37:58.398332+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:37:58.398332+0000","last_undegraded":"2024-09-19T07:37:58.398332+0000","last_fullsized":"2024-09-19T07:37:58.398332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278587+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:37:58.278587+0000","last_peered":"2024-09-19T07:37:58.278587+0000","last_clean":"2024-09-19T07:37:58.278587+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:37:58.278587+0000","last_undegraded":"2024-09-19T07:37:58.278587+0000","last_fullsized":"2024-09-19T07:37:58.278587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409236+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:37:58.409236+0000","last_peered":"2024-09-19T07:37:58.409236+0000","last_clean":"2024-09-19T07:37:58.409236+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:37:58.409236+0000","last_undegraded":"2024-09-19T07:37:58.409236+0000","last_fullsized":"2024-09-19T07:37:58.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568581+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:37:58.568581+0000","last_peered":"2024-09-19T07:37:58.568581+0000","last_clean":"2024-09-19T07:37:58.568581+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:37:58.568581+0000","last_undegraded":"2024-09-19T07:37:58.568581+0000","last_fullsized":"2024-09-19T07:37:58.568581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566381+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:37:58.566381+0000","last_peered":"2024-09-19T07:37:58.566381+0000","last_clean":"2024-09-19T07:37:58.566381+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:37:58.566381+0000","last_undegraded":"2024-09-19T07:37:58.566381+0000","last_fullsized":"2024-09-19T07:37:58.566381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278860+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:37:58.278860+0000","last_peered":"2024-09-19T07:37:58.278860+0000","last_clean":"2024-09-19T07:37:58.278860+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:37:58.278860+0000","last_undegraded":"2024-09-19T07:37:58.278860+0000","last_fullsized":"2024-09-19T07:37:58.278860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566247+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:37:58.566247+0000","last_peered":"2024-09-19T07:37:58.566247+0000","last_clean":"2024-09-19T07:37:58.566247+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:37:58.566247+0000","last_undegraded":"2024-09-19T07:37:58.566247+0000","last_fullsized":"2024-09-19T07:37:58.566247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566034+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:37:58.566034+0000","last_peered":"2024-09-19T07:37:58.566034+0000","last_clean":"2024-09-19T07:37:58.566034+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:37:58.566034+0000","last_undegraded":"2024-09-19T07:37:58.566034+0000","last_fullsized":"2024-09-19T07:37:58.566034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277343+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:37:58.277343+0000","last_peered":"2024-09-19T07:37:58.277343+0000","last_clean":"2024-09-19T07:37:58.277343+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:37:58.277343+0000","last_undegraded":"2024-09-19T07:37:58.277343+0000","last_fullsized":"2024-09-19T07:37:58.277343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566048+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:37:58.566048+0000","last_peered":"2024-09-19T07:37:58.566048+0000","last_clean":"2024-09-19T07:37:58.566048+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:37:58.566048+0000","last_undegraded":"2024-09-19T07:37:58.566048+0000","last_fullsized":"2024-09-19T07:37:58.566048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395689+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:37:58.395689+0000","last_peered":"2024-09-19T07:37:58.395689+0000","last_clean":"2024-09-19T07:37:58.395689+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:37:58.395689+0000","last_undegraded":"2024-09-19T07:37:58.395689+0000","last_fullsized":"2024-09-19T07:37:58.395689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281089+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:37:58.281089+0000","last_peered":"2024-09-19T07:37:58.281089+0000","last_clean":"2024-09-19T07:37:58.281089+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:37:58.281089+0000","last_undegraded":"2024-09-19T07:37:58.281089+0000","last_fullsized":"2024-09-19T07:37:58.281089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565798+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:37:58.565798+0000","last_peered":"2024-09-19T07:37:58.565798+0000","last_clean":"2024-09-19T07:37:58.565798+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:37:58.565798+0000","last_undegraded":"2024-09-19T07:37:58.565798+0000","last_fullsized":"2024-09-19T07:37:58.565798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278866+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:37:58.278866+0000","last_peered":"2024-09-19T07:37:58.278866+0000","last_clean":"2024-09-19T07:37:58.278866+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:37:58.278866+0000","last_undegraded":"2024-09-19T07:37:58.278866+0000","last_fullsized":"2024-09-19T07:37:58.278866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277508+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:37:58.277508+0000","last_peered":"2024-09-19T07:37:58.277508+0000","last_clean":"2024-09-19T07:37:58.277508+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:37:58.277508+0000","last_undegraded":"2024-09-19T07:37:58.277508+0000","last_fullsized":"2024-09-19T07:37:58.277508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398644+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:37:58.398644+0000","last_peered":"2024-09-19T07:37:58.398644+0000","last_clean":"2024-09-19T07:37:58.398644+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:37:58.398644+0000","last_undegraded":"2024-09-19T07:37:58.398644+0000","last_fullsized":"2024-09-19T07:37:58.398644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405691+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:37:58.405691+0000","last_peered":"2024-09-19T07:37:58.405691+0000","last_clean":"2024-09-19T07:37:58.405691+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:37:58.405691+0000","last_undegraded":"2024-09-19T07:37:58.405691+0000","last_fullsized":"2024-09-19T07:37:58.405691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399976+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:37:58.399976+0000","last_peered":"2024-09-19T07:37:58.399976+0000","last_clean":"2024-09-19T07:37:58.399976+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:37:58.399976+0000","last_undegraded":"2024-09-19T07:37:58.399976+0000","last_fullsized":"2024-09-19T07:37:58.399976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281203+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:37:58.281203+0000","last_peered":"2024-09-19T07:37:58.281203+0000","last_clean":"2024-09-19T07:37:58.281203+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:37:58.281203+0000","last_undegraded":"2024-09-19T07:37:58.281203+0000","last_fullsized":"2024-09-19T07:37:58.281203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565955+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:37:58.565955+0000","last_peered":"2024-09-19T07:37:58.565955+0000","last_clean":"2024-09-19T07:37:58.565955+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:37:58.565955+0000","last_undegraded":"2024-09-19T07:37:58.565955+0000","last_fullsized":"2024-09-19T07:37:58.565955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278990+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:37:58.278990+0000","last_peered":"2024-09-19T07:37:58.278990+0000","last_clean":"2024-09-19T07:37:58.278990+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:37:58.278990+0000","last_undegraded":"2024-09-19T07:37:58.278990+0000","last_fullsized":"2024-09-19T07:37:58.278990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566392+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:37:58.566392+0000","last_peered":"2024-09-19T07:37:58.566392+0000","last_clean":"2024-09-19T07:37:58.566392+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:37:58.566392+0000","last_undegraded":"2024-09-19T07:37:58.566392+0000","last_fullsized":"2024-09-19T07:37:58.566392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280791+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:37:58.280791+0000","last_peered":"2024-09-19T07:37:58.280791+0000","last_clean":"2024-09-19T07:37:58.280791+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:37:58.280791+0000","last_undegraded":"2024-09-19T07:37:58.280791+0000","last_fullsized":"2024-09-19T07:37:58.280791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280919+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:37:58.280919+0000","last_peered":"2024-09-19T07:37:58.280919+0000","last_clean":"2024-09-19T07:37:58.280919+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:37:58.280919+0000","last_undegraded":"2024-09-19T07:37:58.280919+0000","last_fullsized":"2024-09-19T07:37:58.280919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565810+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:37:58.565810+0000","last_peered":"2024-09-19T07:37:58.565810+0000","last_clean":"2024-09-19T07:37:58.565810+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:37:58.565810+0000","last_undegraded":"2024-09-19T07:37:58.565810+0000","last_fullsized":"2024-09-19T07:37:58.565810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398122+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:37:58.398122+0000","last_peered":"2024-09-19T07:37:58.398122+0000","last_clean":"2024-09-19T07:37:58.398122+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:37:58.398122+0000","last_undegraded":"2024-09-19T07:37:58.398122+0000","last_fullsized":"2024-09-19T07:37:58.398122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":141,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.054316+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:38:08.054316+0000","last_peered":"2024-09-19T07:38:08.054316+0000","last_clean":"2024-09-19T07:38:08.054316+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:38:08.054316+0000","last_undegraded":"2024-09-19T07:38:08.054316+0000","last_fullsized":"2024-09-19T07:38:08.054316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405841+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:37:58.405841+0000","last_peered":"2024-09-19T07:37:58.405841+0000","last_clean":"2024-09-19T07:37:58.405841+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:37:58.405841+0000","last_undegraded":"2024-09-19T07:37:58.405841+0000","last_fullsized":"2024-09-19T07:37:58.405841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408925+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:37:58.408925+0000","last_peered":"2024-09-19T07:37:58.408925+0000","last_clean":"2024-09-19T07:37:58.408925+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:37:58.408925+0000","last_undegraded":"2024-09-19T07:37:58.408925+0000","last_fullsized":"2024-09-19T07:37:58.408925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278608+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:37:58.278608+0000","last_peered":"2024-09-19T07:37:58.278608+0000","last_clean":"2024-09-19T07:37:58.278608+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:37:58.278608+0000","last_undegraded":"2024-09-19T07:37:58.278608+0000","last_fullsized":"2024-09-19T07:37:58.278608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398404+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:37:58.398404+0000","last_peered":"2024-09-19T07:37:58.398404+0000","last_clean":"2024-09-19T07:37:58.398404+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:37:58.398404+0000","last_undegraded":"2024-09-19T07:37:58.398404+0000","last_fullsized":"2024-09-19T07:37:58.398404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406017+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:37:58.406017+0000","last_peered":"2024-09-19T07:37:58.406017+0000","last_clean":"2024-09-19T07:37:58.406017+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:37:58.406017+0000","last_undegraded":"2024-09-19T07:37:58.406017+0000","last_fullsized":"2024-09-19T07:37:58.406017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395465+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:37:58.395465+0000","last_peered":"2024-09-19T07:37:58.395465+0000","last_clean":"2024-09-19T07:37:58.395465+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:37:58.395465+0000","last_undegraded":"2024-09-19T07:37:58.395465+0000","last_fullsized":"2024-09-19T07:37:58.395465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":249,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054953+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:38:09.054953+0000","last_peered":"2024-09-19T07:38:09.054953+0000","last_clean":"2024-09-19T07:38:09.054953+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:38:09.054953+0000","last_undegraded":"2024-09-19T07:38:09.054953+0000","last_fullsized":"2024-09-19T07:38:09.054953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278242+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:37:58.278242+0000","last_peered":"2024-09-19T07:37:58.278242+0000","last_clean":"2024-09-19T07:37:58.278242+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:37:58.278242+0000","last_undegraded":"2024-09-19T07:37:58.278242+0000","last_fullsized":"2024-09-19T07:37:58.278242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568741+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:37:58.568741+0000","last_peered":"2024-09-19T07:37:58.568741+0000","last_clean":"2024-09-19T07:37:58.568741+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:37:58.568741+0000","last_undegraded":"2024-09-19T07:37:58.568741+0000","last_fullsized":"2024-09-19T07:37:58.568741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567021+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:37:58.567021+0000","last_peered":"2024-09-19T07:37:58.567021+0000","last_clean":"2024-09-19T07:37:58.567021+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:37:58.567021+0000","last_undegraded":"2024-09-19T07:37:58.567021+0000","last_fullsized":"2024-09-19T07:37:58.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405996+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:37:58.405996+0000","last_peered":"2024-09-19T07:37:58.405996+0000","last_clean":"2024-09-19T07:37:58.405996+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:37:58.405996+0000","last_undegraded":"2024-09-19T07:37:58.405996+0000","last_fullsized":"2024-09-19T07:37:58.405996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277409+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:37:58.277409+0000","last_peered":"2024-09-19T07:37:58.277409+0000","last_clean":"2024-09-19T07:37:58.277409+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:37:58.277409+0000","last_undegraded":"2024-09-19T07:37:58.277409+0000","last_fullsized":"2024-09-19T07:37:58.277409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":295,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054953+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:38:09.054953+0000","last_peered":"2024-09-19T07:38:09.054953+0000","last_clean":"2024-09-19T07:38:09.054953+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:38:09.054953+0000","last_undegraded":"2024-09-19T07:38:09.054953+0000","last_fullsized":"2024-09-19T07:38:09.054953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":233,"num_read_kb":184,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398289+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:37:58.398289+0000","last_peered":"2024-09-19T07:37:58.398289+0000","last_clean":"2024-09-19T07:37:58.398289+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:37:58.398289+0000","last_undegraded":"2024-09-19T07:37:58.398289+0000","last_fullsized":"2024-09-19T07:37:58.398289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280980+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:37:58.280980+0000","last_peered":"2024-09-19T07:37:58.280980+0000","last_clean":"2024-09-19T07:37:58.280980+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:37:58.280980+0000","last_undegraded":"2024-09-19T07:37:58.280980+0000","last_fullsized":"2024-09-19T07:37:58.280980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278729+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:37:58.278729+0000","last_peered":"2024-09-19T07:37:58.278729+0000","last_clean":"2024-09-19T07:37:58.278729+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:37:58.278729+0000","last_undegraded":"2024-09-19T07:37:58.278729+0000","last_fullsized":"2024-09-19T07:37:58.278729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405861+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:37:58.405861+0000","last_peered":"2024-09-19T07:37:58.405861+0000","last_clean":"2024-09-19T07:37:58.405861+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:37:58.405861+0000","last_undegraded":"2024-09-19T07:37:58.405861+0000","last_fullsized":"2024-09-19T07:37:58.405861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568411+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:37:58.568411+0000","last_peered":"2024-09-19T07:37:58.568411+0000","last_clean":"2024-09-19T07:37:58.568411+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:37:58.568411+0000","last_undegraded":"2024-09-19T07:37:58.568411+0000","last_fullsized":"2024-09-19T07:37:58.568411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'9","reported_seq":148,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.054007+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:38:08.054007+0000","last_peered":"2024-09-19T07:38:08.054007+0000","last_clean":"2024-09-19T07:38:08.054007+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:38:08.054007+0000","last_undegraded":"2024-09-19T07:38:08.054007+0000","last_fullsized":"2024-09-19T07:38:08.054007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567829+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:37:58.567829+0000","last_peered":"2024-09-19T07:37:58.567829+0000","last_clean":"2024-09-19T07:37:58.567829+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:37:58.567829+0000","last_undegraded":"2024-09-19T07:37:58.567829+0000","last_fullsized":"2024-09-19T07:37:58.567829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565722+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:37:58.565722+0000","last_peered":"2024-09-19T07:37:58.565722+0000","last_clean":"2024-09-19T07:37:58.565722+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:37:58.565722+0000","last_undegraded":"2024-09-19T07:37:58.565722+0000","last_fullsized":"2024-09-19T07:37:58.565722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405806+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:37:58.405806+0000","last_peered":"2024-09-19T07:37:58.405806+0000","last_clean":"2024-09-19T07:37:58.405806+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:37:58.405806+0000","last_undegraded":"2024-09-19T07:37:58.405806+0000","last_fullsized":"2024-09-19T07:37:58.405806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398454+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:37:58.398454+0000","last_peered":"2024-09-19T07:37:58.398454+0000","last_clean":"2024-09-19T07:37:58.398454+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:37:58.398454+0000","last_undegraded":"2024-09-19T07:37:58.398454+0000","last_fullsized":"2024-09-19T07:37:58.398454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566085+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:37:58.566085+0000","last_peered":"2024-09-19T07:37:58.566085+0000","last_clean":"2024-09-19T07:37:58.566085+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:37:58.566085+0000","last_undegraded":"2024-09-19T07:37:58.566085+0000","last_fullsized":"2024-09-19T07:37:58.566085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278747+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:37:58.278747+0000","last_peered":"2024-09-19T07:37:58.278747+0000","last_clean":"2024-09-19T07:37:58.278747+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:37:58.278747+0000","last_undegraded":"2024-09-19T07:37:58.278747+0000","last_fullsized":"2024-09-19T07:37:58.278747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568790+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:37:58.568790+0000","last_peered":"2024-09-19T07:37:58.568790+0000","last_clean":"2024-09-19T07:37:58.568790+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:37:58.568790+0000","last_undegraded":"2024-09-19T07:37:58.568790+0000","last_fullsized":"2024-09-19T07:37:58.568790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'67","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:07.691483+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:38:07.691483+0000","last_peered":"2024-09-19T07:38:07.691483+0000","last_clean":"2024-09-19T07:38:07.691483+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:38:07.691483+0000","last_undegraded":"2024-09-19T07:38:07.691483+0000","last_fullsized":"2024-09-19T07:38:07.691483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568493+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:37:58.568493+0000","last_peered":"2024-09-19T07:37:58.568493+0000","last_clean":"2024-09-19T07:37:58.568493+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:37:58.568493+0000","last_undegraded":"2024-09-19T07:37:58.568493+0000","last_fullsized":"2024-09-19T07:37:58.568493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":147,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.054072+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:38:08.054072+0000","last_peered":"2024-09-19T07:38:08.054072+0000","last_clean":"2024-09-19T07:38:08.054072+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:38:08.054072+0000","last_undegraded":"2024-09-19T07:38:08.054072+0000","last_fullsized":"2024-09-19T07:38:08.054072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568589+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:37:58.568589+0000","last_peered":"2024-09-19T07:37:58.568589+0000","last_clean":"2024-09-19T07:37:58.568589+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:37:58.568589+0000","last_undegraded":"2024-09-19T07:37:58.568589+0000","last_fullsized":"2024-09-19T07:37:58.568589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":394,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:00.401620+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:38:00.401620+0000","last_peered":"2024-09-19T07:38:00.401620+0000","last_clean":"2024-09-19T07:38:00.401620+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:38:00.401620+0000","last_undegraded":"2024-09-19T07:38:00.401620+0000","last_fullsized":"2024-09-19T07:38:00.401620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566487+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:37:58.566487+0000","last_peered":"2024-09-19T07:37:58.566487+0000","last_clean":"2024-09-19T07:37:58.566487+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:37:58.566487+0000","last_undegraded":"2024-09-19T07:37:58.566487+0000","last_fullsized":"2024-09-19T07:37:58.566487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399251+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:37:58.399251+0000","last_peered":"2024-09-19T07:37:58.399251+0000","last_clean":"2024-09-19T07:37:58.399251+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:37:58.399251+0000","last_undegraded":"2024-09-19T07:37:58.399251+0000","last_fullsized":"2024-09-19T07:37:58.399251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567968+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:37:58.567968+0000","last_peered":"2024-09-19T07:37:58.567968+0000","last_clean":"2024-09-19T07:37:58.567968+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:37:58.567968+0000","last_undegraded":"2024-09-19T07:37:58.567968+0000","last_fullsized":"2024-09-19T07:37:58.567968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278630+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:37:58.278630+0000","last_peered":"2024-09-19T07:37:58.278630+0000","last_clean":"2024-09-19T07:37:58.278630+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:37:58.278630+0000","last_undegraded":"2024-09-19T07:37:58.278630+0000","last_fullsized":"2024-09-19T07:37:58.278630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054712+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:38:09.054712+0000","last_peered":"2024-09-19T07:38:09.054712+0000","last_clean":"2024-09-19T07:38:09.054712+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:38:09.054712+0000","last_undegraded":"2024-09-19T07:38:09.054712+0000","last_fullsized":"2024-09-19T07:38:09.054712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405844+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:37:58.405844+0000","last_peered":"2024-09-19T07:37:58.405844+0000","last_clean":"2024-09-19T07:37:58.405844+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:37:58.405844+0000","last_undegraded":"2024-09-19T07:37:58.405844+0000","last_fullsized":"2024-09-19T07:37:58.405844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054910+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:38:09.054910+0000","last_peered":"2024-09-19T07:38:09.054910+0000","last_clean":"2024-09-19T07:38:09.054910+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:38:09.054910+0000","last_undegraded":"2024-09-19T07:38:09.054910+0000","last_fullsized":"2024-09-19T07:38:09.054910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408965+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:37:58.408965+0000","last_peered":"2024-09-19T07:37:58.408965+0000","last_clean":"2024-09-19T07:37:58.408965+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:37:58.408965+0000","last_undegraded":"2024-09-19T07:37:58.408965+0000","last_fullsized":"2024-09-19T07:37:58.408965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396146+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:37:58.396146+0000","last_peered":"2024-09-19T07:37:58.396146+0000","last_clean":"2024-09-19T07:37:58.396146+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:37:58.396146+0000","last_undegraded":"2024-09-19T07:37:58.396146+0000","last_fullsized":"2024-09-19T07:37:58.396146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409093+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:37:58.409093+0000","last_peered":"2024-09-19T07:37:58.409093+0000","last_clean":"2024-09-19T07:37:58.409093+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:37:58.409093+0000","last_undegraded":"2024-09-19T07:37:58.409093+0000","last_fullsized":"2024-09-19T07:37:58.409093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398419+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:37:58.398419+0000","last_peered":"2024-09-19T07:37:58.398419+0000","last_clean":"2024-09-19T07:37:58.398419+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:37:58.398419+0000","last_undegraded":"2024-09-19T07:37:58.398419+0000","last_fullsized":"2024-09-19T07:37:58.398419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278876+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:37:58.278876+0000","last_peered":"2024-09-19T07:37:58.278876+0000","last_clean":"2024-09-19T07:37:58.278876+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:37:58.278876+0000","last_undegraded":"2024-09-19T07:37:58.278876+0000","last_fullsized":"2024-09-19T07:37:58.278876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'104","reported_seq":220,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.043011+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:38:08.043011+0000","last_peered":"2024-09-19T07:38:08.043011+0000","last_clean":"2024-09-19T07:38:08.043011+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:38:08.043011+0000","last_undegraded":"2024-09-19T07:38:08.043011+0000","last_fullsized":"2024-09-19T07:38:08.043011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400100+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:37:58.400100+0000","last_peered":"2024-09-19T07:37:58.400100+0000","last_clean":"2024-09-19T07:37:58.400100+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:37:58.400100+0000","last_undegraded":"2024-09-19T07:37:58.400100+0000","last_fullsized":"2024-09-19T07:37:58.400100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566566+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:37:58.566566+0000","last_peered":"2024-09-19T07:37:58.566566+0000","last_clean":"2024-09-19T07:37:58.566566+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:37:58.566566+0000","last_undegraded":"2024-09-19T07:37:58.566566+0000","last_fullsized":"2024-09-19T07:37:58.566566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566745+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:37:58.566745+0000","last_peered":"2024-09-19T07:37:58.566745+0000","last_clean":"2024-09-19T07:37:58.566745+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:37:58.566745+0000","last_undegraded":"2024-09-19T07:37:58.566745+0000","last_fullsized":"2024-09-19T07:37:58.566745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277436+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:37:58.277436+0000","last_peered":"2024-09-19T07:37:58.277436+0000","last_clean":"2024-09-19T07:37:58.277436+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:37:58.277436+0000","last_undegraded":"2024-09-19T07:37:58.277436+0000","last_fullsized":"2024-09-19T07:37:58.277436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.397993+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:37:58.397993+0000","last_peered":"2024-09-19T07:37:58.397993+0000","last_clean":"2024-09-19T07:37:58.397993+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:37:58.397993+0000","last_undegraded":"2024-09-19T07:37:58.397993+0000","last_fullsized":"2024-09-19T07:37:58.397993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398888+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:37:58.398888+0000","last_peered":"2024-09-19T07:37:58.398888+0000","last_clean":"2024-09-19T07:37:58.398888+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:37:58.398888+0000","last_undegraded":"2024-09-19T07:37:58.398888+0000","last_fullsized":"2024-09-19T07:37:58.398888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565988+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:37:58.565988+0000","last_peered":"2024-09-19T07:37:58.565988+0000","last_clean":"2024-09-19T07:37:58.565988+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:37:58.565988+0000","last_undegraded":"2024-09-19T07:37:58.565988+0000","last_fullsized":"2024-09-19T07:37:58.565988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565778+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:37:58.565778+0000","last_peered":"2024-09-19T07:37:58.565778+0000","last_clean":"2024-09-19T07:37:58.565778+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:37:58.565778+0000","last_undegraded":"2024-09-19T07:37:58.565778+0000","last_fullsized":"2024-09-19T07:37:58.565778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399648+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:37:58.399648+0000","last_peered":"2024-09-19T07:37:58.399648+0000","last_clean":"2024-09-19T07:37:58.399648+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:37:58.399648+0000","last_undegraded":"2024-09-19T07:37:58.399648+0000","last_fullsized":"2024-09-19T07:37:58.399648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398085+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:37:58.398085+0000","last_peered":"2024-09-19T07:37:58.398085+0000","last_clean":"2024-09-19T07:37:58.398085+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:37:58.398085+0000","last_undegraded":"2024-09-19T07:37:58.398085+0000","last_fullsized":"2024-09-19T07:37:58.398085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568069+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:37:58.568069+0000","last_peered":"2024-09-19T07:37:58.568069+0000","last_clean":"2024-09-19T07:37:58.568069+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:37:58.568069+0000","last_undegraded":"2024-09-19T07:37:58.568069+0000","last_fullsized":"2024-09-19T07:37:58.568069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395362+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:37:58.395362+0000","last_peered":"2024-09-19T07:37:58.395362+0000","last_clean":"2024-09-19T07:37:58.395362+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:37:58.395362+0000","last_undegraded":"2024-09-19T07:37:58.395362+0000","last_fullsized":"2024-09-19T07:37:58.395362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398241+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:37:58.398241+0000","last_peered":"2024-09-19T07:37:58.398241+0000","last_clean":"2024-09-19T07:37:58.398241+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:37:58.398241+0000","last_undegraded":"2024-09-19T07:37:58.398241+0000","last_fullsized":"2024-09-19T07:37:58.398241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277959+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:37:58.277959+0000","last_peered":"2024-09-19T07:37:58.277959+0000","last_clean":"2024-09-19T07:37:58.277959+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:37:58.277959+0000","last_undegraded":"2024-09-19T07:37:58.277959+0000","last_fullsized":"2024-09-19T07:37:58.277959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398335+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:37:58.398335+0000","last_peered":"2024-09-19T07:37:58.398335+0000","last_clean":"2024-09-19T07:37:58.398335+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:37:58.398335+0000","last_undegraded":"2024-09-19T07:37:58.398335+0000","last_fullsized":"2024-09-19T07:37:58.398335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566969+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:37:58.566969+0000","last_peered":"2024-09-19T07:37:58.566969+0000","last_clean":"2024-09-19T07:37:58.566969+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:37:58.566969+0000","last_undegraded":"2024-09-19T07:37:58.566969+0000","last_fullsized":"2024-09-19T07:37:58.566969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400655+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:37:58.400655+0000","last_peered":"2024-09-19T07:37:58.400655+0000","last_clean":"2024-09-19T07:37:58.400655+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:37:58.400655+0000","last_undegraded":"2024-09-19T07:37:58.400655+0000","last_fullsized":"2024-09-19T07:37:58.400655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400381+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:37:58.400381+0000","last_peered":"2024-09-19T07:37:58.400381+0000","last_clean":"2024-09-19T07:37:58.400381+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:37:58.400381+0000","last_undegraded":"2024-09-19T07:37:58.400381+0000","last_fullsized":"2024-09-19T07:37:58.400381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'14","reported_seq":167,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054382+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:38:09.054382+0000","last_peered":"2024-09-19T07:38:09.054382+0000","last_clean":"2024-09-19T07:38:09.054382+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:38:09.054382+0000","last_undegraded":"2024-09-19T07:38:09.054382+0000","last_fullsized":"2024-09-19T07:38:09.054382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799284,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1497,"num_read_kb":972,"num_write":502,"num_write_kb":2381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":786432,"data_stored":2397852,"data_compressed":108252,"data_compressed_allocated":147456,"data_compressed_original":2125824,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107370,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514533,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":451633,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1130496,"data_stored":1439808,"data_compressed":61297,"data_compressed_allocated":81920,"data_compressed_original":929792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":843740,"data_compressed":28001,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":843334,"data_compressed":28001,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1044883,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":450775,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":1370939,"data_compressed":59704,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":451422,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":1171514,"data_compressed":43203,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":520934,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":520993,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":452507,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71591,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71532,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331052,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":399921,"data_compressed":16795,"data_compressed_allocated":24576,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:38:11.840 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T07:38:11.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T07:38:12.090 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:12 smithi103 ceph-mon[31201]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-19T07:38:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:12 smithi103 ceph-mon[31201]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:12.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 -- 172.21.15.103:0/4268276621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 msgr2=0x7f9a7c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:12.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 --2- 172.21.15.103:0/4268276621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c100ee0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009930 tx=0x7f9a6402a420 comp rx=0 tx=0).stop 2024-09-19T07:38:12.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 -- 172.21.15.103:0/4268276621 shutdown_connections 2024-09-19T07:38:12.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 --2- 172.21.15.103:0/4268276621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a7c101420 0x7f9a7c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 --2- 172.21.15.103:0/4268276621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c100ee0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 --2- 172.21.15.103:0/4268276621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7c106a80 0x7f9a7c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 -- 172.21.15.103:0/4268276621 >> 172.21.15.103:0/4268276621 conn(0x7f9a7c0fc7a0 msgr2=0x7f9a7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 -- 172.21.15.103:0/4268276621 shutdown_connections 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.560+0000 7f9a80b78640 1 -- 172.21.15.103:0/4268276621 wait complete. 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 Processor -- start 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 -- start start 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c0fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a7c101420 0x7f9a7c0fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7c106a80 0x7f9a7c0fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c110ec0 con 0x7f9a7c100a80 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c110d40 con 0x7f9a7c106a80 2024-09-19T07:38:12.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a80b78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c111040 con 0x7f9a7c101420 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c0fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a7a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c0fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34964/0 (socket says 172.21.15.103:34964) 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.561+0000 7f9a7a59c640 1 -- 172.21.15.103:0/775174733 learned_addr learned my addr 172.21.15.103:0/775174733 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a79d9b640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a7c101420 0x7f9a7c0fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7ad9d640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7c106a80 0x7f9a7c0fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7a59c640 1 -- 172.21.15.103:0/775174733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a7c101420 msgr2=0x7f9a7c0fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7a59c640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a7c101420 0x7f9a7c0fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7a59c640 1 -- 172.21.15.103:0/775174733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7c106a80 msgr2=0x7f9a7c0fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7a59c640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7c106a80 0x7f9a7c0fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7a59c640 1 -- 172.21.15.103:0/775174733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c1a6930 con 0x7f9a7c100a80 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7ad9d640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7c106a80 0x7f9a7c0fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:12.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a7a59c640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c0fcd90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00ba50 tx=0x7f9a6c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a737fe640 1 -- 172.21.15.103:0/775174733 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c019070 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a737fe640 1 -- 172.21.15.103:0/775174733 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a6c00cbe0 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c1a6c20 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a737fe640 1 -- 172.21.15.103:0/775174733 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0076e0 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.562+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c1a7180 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.563+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c100ee0 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.564+0000 7f9a737fe640 1 -- 172.21.15.103:0/775174733 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f9a6c01c070 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.565+0000 7f9a737fe640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9a680a34c0 0x7f9a680a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.565+0000 7f9a79d9b640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9a680a34c0 0x7f9a680a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.565+0000 7f9a737fe640 1 -- 172.21.15.103:0/775174733 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9a6c0cf7b0 con 0x7f9a7c100a80 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.566+0000 7f9a79d9b640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9a680a34c0 0x7f9a680a5980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009900 tx=0x7f9a640058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:12.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.568+0000 7f9a737fe640 1 -- 172.21.15.103:0/775174733 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9a6c09baf0 con 0x7f9a7c100a80 2024-09-19T07:38:12.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:12 smithi134 ceph-mon[35414]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-19T07:38:12.583 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:12 smithi134 ceph-mon[35414]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:12 smithi174 ceph-mon[35370]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-19T07:38:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:12 smithi174 ceph-mon[35370]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:12.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.732+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a7c062630 con 0x7f9a680a34c0 2024-09-19T07:38:12.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.738+0000 7f9a737fe640 1 -- 172.21.15.103:0/775174733 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258115 (secure 0 0 0) 0x7f9a7c062630 con 0x7f9a680a34c0 2024-09-19T07:38:12.741 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:12.742 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9a680a34c0 msgr2=0x7f9a680a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9a680a34c0 0x7f9a680a5980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009900 tx=0x7f9a640058d0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 msgr2=0x7f9a7c0fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c0fcd90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00ba50 tx=0x7f9a6c00bf10 comp rx=0 tx=0).stop 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 shutdown_connections 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9a680a34c0 0x7f9a680a5980 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7c106a80 0x7f9a7c0fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a7c101420 0x7f9a7c0fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 --2- 172.21.15.103:0/775174733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a7c100a80 0x7f9a7c0fcd90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:12.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 >> 172.21.15.103:0/775174733 conn(0x7f9a7c0fc7a0 msgr2=0x7f9a7c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:12.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 shutdown_connections 2024-09-19T07:38:12.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:12.741+0000 7f9a80b78640 1 -- 172.21.15.103:0/775174733 wait complete. 2024-09-19T07:38:13.376 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:13 smithi103 ceph-mon[31201]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-19T07:38:13.376 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:13 smithi103 ceph-mon[31201]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:13.405 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-19T07:38:12.295852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389886,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1734,"num_read_kb":1302,"num_write":705,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1031,"ondisk_log_size":1031,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":344300,"kb_used_data":11640,"kb_used_omap":18,"kb_used_meta":332397,"kb_avail":1124597524,"statfs":{"total":1151940427776,"available":1151587864576,"internally_reserved":0,"allocated":11919360,"data_stored":9562482,"data_compressed":320166,"data_compressed_allocated":430080,"data_compressed_original":4190208,"omap_allocated":19142,"internal_metadata":340374842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":456,"num_read_kb":493,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002709"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566027+0000","last_change":"2024-09-19T07:36:07.850219+0000","last_active":"2024-09-19T07:37:58.566027+0000","last_peered":"2024-09-19T07:37:58.566027+0000","last_clean":"2024-09-19T07:37:58.566027+0000","last_became_active":"2024-09-19T07:36:07.849469+0000","last_became_peered":"2024-09-19T07:36:07.849469+0000","last_unstale":"2024-09-19T07:37:58.566027+0000","last_undegraded":"2024-09-19T07:37:58.566027+0000","last_fullsized":"2024-09-19T07:37:58.566027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:19.688713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566607+0000","last_change":"2024-09-19T07:36:07.845873+0000","last_active":"2024-09-19T07:37:58.566607+0000","last_peered":"2024-09-19T07:37:58.566607+0000","last_clean":"2024-09-19T07:37:58.566607+0000","last_became_active":"2024-09-19T07:36:07.845545+0000","last_became_peered":"2024-09-19T07:36:07.845545+0000","last_unstale":"2024-09-19T07:37:58.566607+0000","last_undegraded":"2024-09-19T07:37:58.566607+0000","last_fullsized":"2024-09-19T07:37:58.566607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:20.102752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566643+0000","last_change":"2024-09-19T07:36:07.855776+0000","last_active":"2024-09-19T07:37:58.566643+0000","last_peered":"2024-09-19T07:37:58.566643+0000","last_clean":"2024-09-19T07:37:58.566643+0000","last_became_active":"2024-09-19T07:36:07.855475+0000","last_became_peered":"2024-09-19T07:36:07.855475+0000","last_unstale":"2024-09-19T07:37:58.566643+0000","last_undegraded":"2024-09-19T07:37:58.566643+0000","last_fullsized":"2024-09-19T07:37:58.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:10.918613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395953+0000","last_change":"2024-09-19T07:36:17.896911+0000","last_active":"2024-09-19T07:37:58.395953+0000","last_peered":"2024-09-19T07:37:58.395953+0000","last_clean":"2024-09-19T07:37:58.395953+0000","last_became_active":"2024-09-19T07:36:17.896537+0000","last_became_peered":"2024-09-19T07:36:17.896537+0000","last_unstale":"2024-09-19T07:37:58.395953+0000","last_undegraded":"2024-09-19T07:37:58.395953+0000","last_fullsized":"2024-09-19T07:37:58.395953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:19.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398077+0000","last_change":"2024-09-19T07:36:07.862977+0000","last_active":"2024-09-19T07:37:58.398077+0000","last_peered":"2024-09-19T07:37:58.398077+0000","last_clean":"2024-09-19T07:37:58.398077+0000","last_became_active":"2024-09-19T07:36:07.858984+0000","last_became_peered":"2024-09-19T07:36:07.858984+0000","last_unstale":"2024-09-19T07:37:58.398077+0000","last_undegraded":"2024-09-19T07:37:58.398077+0000","last_fullsized":"2024-09-19T07:37:58.398077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.056550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408847+0000","last_change":"2024-09-19T07:36:17.897391+0000","last_active":"2024-09-19T07:37:58.408847+0000","last_peered":"2024-09-19T07:37:58.408847+0000","last_clean":"2024-09-19T07:37:58.408847+0000","last_became_active":"2024-09-19T07:36:17.896909+0000","last_became_peered":"2024-09-19T07:36:17.896909+0000","last_unstale":"2024-09-19T07:37:58.408847+0000","last_undegraded":"2024-09-19T07:37:58.408847+0000","last_fullsized":"2024-09-19T07:37:58.408847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:45.903359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567869+0000","last_change":"2024-09-19T07:36:07.854028+0000","last_active":"2024-09-19T07:37:58.567869+0000","last_peered":"2024-09-19T07:37:58.567869+0000","last_clean":"2024-09-19T07:37:58.567869+0000","last_became_active":"2024-09-19T07:36:07.851422+0000","last_became_peered":"2024-09-19T07:36:07.851422+0000","last_unstale":"2024-09-19T07:37:58.567869+0000","last_undegraded":"2024-09-19T07:37:58.567869+0000","last_fullsized":"2024-09-19T07:37:58.567869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:04.838085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398332+0000","last_change":"2024-09-19T07:36:07.850322+0000","last_active":"2024-09-19T07:37:58.398332+0000","last_peered":"2024-09-19T07:37:58.398332+0000","last_clean":"2024-09-19T07:37:58.398332+0000","last_became_active":"2024-09-19T07:36:07.850028+0000","last_became_peered":"2024-09-19T07:36:07.850028+0000","last_unstale":"2024-09-19T07:37:58.398332+0000","last_undegraded":"2024-09-19T07:37:58.398332+0000","last_fullsized":"2024-09-19T07:37:58.398332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:12.444138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278587+0000","last_change":"2024-09-19T07:36:07.859543+0000","last_active":"2024-09-19T07:37:58.278587+0000","last_peered":"2024-09-19T07:37:58.278587+0000","last_clean":"2024-09-19T07:37:58.278587+0000","last_became_active":"2024-09-19T07:36:07.859011+0000","last_became_peered":"2024-09-19T07:36:07.859011+0000","last_unstale":"2024-09-19T07:37:58.278587+0000","last_undegraded":"2024-09-19T07:37:58.278587+0000","last_fullsized":"2024-09-19T07:37:58.278587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:45.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409236+0000","last_change":"2024-09-19T07:36:07.853910+0000","last_active":"2024-09-19T07:37:58.409236+0000","last_peered":"2024-09-19T07:37:58.409236+0000","last_clean":"2024-09-19T07:37:58.409236+0000","last_became_active":"2024-09-19T07:36:07.853374+0000","last_became_peered":"2024-09-19T07:36:07.853374+0000","last_unstale":"2024-09-19T07:37:58.409236+0000","last_undegraded":"2024-09-19T07:37:58.409236+0000","last_fullsized":"2024-09-19T07:37:58.409236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:38.935114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568581+0000","last_change":"2024-09-19T07:36:07.864236+0000","last_active":"2024-09-19T07:37:58.568581+0000","last_peered":"2024-09-19T07:37:58.568581+0000","last_clean":"2024-09-19T07:37:58.568581+0000","last_became_active":"2024-09-19T07:36:07.863777+0000","last_became_peered":"2024-09-19T07:36:07.863777+0000","last_unstale":"2024-09-19T07:37:58.568581+0000","last_undegraded":"2024-09-19T07:37:58.568581+0000","last_fullsized":"2024-09-19T07:37:58.568581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:29.080299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566381+0000","last_change":"2024-09-19T07:36:07.860810+0000","last_active":"2024-09-19T07:37:58.566381+0000","last_peered":"2024-09-19T07:37:58.566381+0000","last_clean":"2024-09-19T07:37:58.566381+0000","last_became_active":"2024-09-19T07:36:07.859681+0000","last_became_peered":"2024-09-19T07:36:07.859681+0000","last_unstale":"2024-09-19T07:37:58.566381+0000","last_undegraded":"2024-09-19T07:37:58.566381+0000","last_fullsized":"2024-09-19T07:37:58.566381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:16.696485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278860+0000","last_change":"2024-09-19T07:36:07.851401+0000","last_active":"2024-09-19T07:37:58.278860+0000","last_peered":"2024-09-19T07:37:58.278860+0000","last_clean":"2024-09-19T07:37:58.278860+0000","last_became_active":"2024-09-19T07:36:07.851060+0000","last_became_peered":"2024-09-19T07:36:07.851060+0000","last_unstale":"2024-09-19T07:37:58.278860+0000","last_undegraded":"2024-09-19T07:37:58.278860+0000","last_fullsized":"2024-09-19T07:37:58.278860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:52.862314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566247+0000","last_change":"2024-09-19T07:36:07.858957+0000","last_active":"2024-09-19T07:37:58.566247+0000","last_peered":"2024-09-19T07:37:58.566247+0000","last_clean":"2024-09-19T07:37:58.566247+0000","last_became_active":"2024-09-19T07:36:07.858723+0000","last_became_peered":"2024-09-19T07:36:07.858723+0000","last_unstale":"2024-09-19T07:37:58.566247+0000","last_undegraded":"2024-09-19T07:37:58.566247+0000","last_fullsized":"2024-09-19T07:37:58.566247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:57.135173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566034+0000","last_change":"2024-09-19T07:36:07.855619+0000","last_active":"2024-09-19T07:37:58.566034+0000","last_peered":"2024-09-19T07:37:58.566034+0000","last_clean":"2024-09-19T07:37:58.566034+0000","last_became_active":"2024-09-19T07:36:07.855295+0000","last_became_peered":"2024-09-19T07:36:07.855295+0000","last_unstale":"2024-09-19T07:37:58.566034+0000","last_undegraded":"2024-09-19T07:37:58.566034+0000","last_fullsized":"2024-09-19T07:37:58.566034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:48.511058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277343+0000","last_change":"2024-09-19T07:36:07.859984+0000","last_active":"2024-09-19T07:37:58.277343+0000","last_peered":"2024-09-19T07:37:58.277343+0000","last_clean":"2024-09-19T07:37:58.277343+0000","last_became_active":"2024-09-19T07:36:07.858909+0000","last_became_peered":"2024-09-19T07:36:07.858909+0000","last_unstale":"2024-09-19T07:37:58.277343+0000","last_undegraded":"2024-09-19T07:37:58.277343+0000","last_fullsized":"2024-09-19T07:37:58.277343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:32.264436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566048+0000","last_change":"2024-09-19T07:36:07.850551+0000","last_active":"2024-09-19T07:37:58.566048+0000","last_peered":"2024-09-19T07:37:58.566048+0000","last_clean":"2024-09-19T07:37:58.566048+0000","last_became_active":"2024-09-19T07:36:07.849931+0000","last_became_peered":"2024-09-19T07:36:07.849931+0000","last_unstale":"2024-09-19T07:37:58.566048+0000","last_undegraded":"2024-09-19T07:37:58.566048+0000","last_fullsized":"2024-09-19T07:37:58.566048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:57:37.995500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395689+0000","last_change":"2024-09-19T07:36:07.848160+0000","last_active":"2024-09-19T07:37:58.395689+0000","last_peered":"2024-09-19T07:37:58.395689+0000","last_clean":"2024-09-19T07:37:58.395689+0000","last_became_active":"2024-09-19T07:36:07.847753+0000","last_became_peered":"2024-09-19T07:36:07.847753+0000","last_unstale":"2024-09-19T07:37:58.395689+0000","last_undegraded":"2024-09-19T07:37:58.395689+0000","last_fullsized":"2024-09-19T07:37:58.395689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:40:17.825722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281089+0000","last_change":"2024-09-19T07:36:07.858542+0000","last_active":"2024-09-19T07:37:58.281089+0000","last_peered":"2024-09-19T07:37:58.281089+0000","last_clean":"2024-09-19T07:37:58.281089+0000","last_became_active":"2024-09-19T07:36:07.858057+0000","last_became_peered":"2024-09-19T07:36:07.858057+0000","last_unstale":"2024-09-19T07:37:58.281089+0000","last_undegraded":"2024-09-19T07:37:58.281089+0000","last_fullsized":"2024-09-19T07:37:58.281089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.542608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565798+0000","last_change":"2024-09-19T07:36:17.897489+0000","last_active":"2024-09-19T07:37:58.565798+0000","last_peered":"2024-09-19T07:37:58.565798+0000","last_clean":"2024-09-19T07:37:58.565798+0000","last_became_active":"2024-09-19T07:36:17.897098+0000","last_became_peered":"2024-09-19T07:36:17.897098+0000","last_unstale":"2024-09-19T07:37:58.565798+0000","last_undegraded":"2024-09-19T07:37:58.565798+0000","last_fullsized":"2024-09-19T07:37:58.565798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:20.107990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278866+0000","last_change":"2024-09-19T07:36:07.853942+0000","last_active":"2024-09-19T07:37:58.278866+0000","last_peered":"2024-09-19T07:37:58.278866+0000","last_clean":"2024-09-19T07:37:58.278866+0000","last_became_active":"2024-09-19T07:36:07.853626+0000","last_became_peered":"2024-09-19T07:36:07.853626+0000","last_unstale":"2024-09-19T07:37:58.278866+0000","last_undegraded":"2024-09-19T07:37:58.278866+0000","last_fullsized":"2024-09-19T07:37:58.278866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:40.634298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277508+0000","last_change":"2024-09-19T07:36:07.860601+0000","last_active":"2024-09-19T07:37:58.277508+0000","last_peered":"2024-09-19T07:37:58.277508+0000","last_clean":"2024-09-19T07:37:58.277508+0000","last_became_active":"2024-09-19T07:36:07.859093+0000","last_became_peered":"2024-09-19T07:36:07.859093+0000","last_unstale":"2024-09-19T07:37:58.277508+0000","last_undegraded":"2024-09-19T07:37:58.277508+0000","last_fullsized":"2024-09-19T07:37:58.277508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:50.105002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398644+0000","last_change":"2024-09-19T07:36:07.854584+0000","last_active":"2024-09-19T07:37:58.398644+0000","last_peered":"2024-09-19T07:37:58.398644+0000","last_clean":"2024-09-19T07:37:58.398644+0000","last_became_active":"2024-09-19T07:36:07.854255+0000","last_became_peered":"2024-09-19T07:36:07.854255+0000","last_unstale":"2024-09-19T07:37:58.398644+0000","last_undegraded":"2024-09-19T07:37:58.398644+0000","last_fullsized":"2024-09-19T07:37:58.398644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:38.864030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405691+0000","last_change":"2024-09-19T07:36:07.850888+0000","last_active":"2024-09-19T07:37:58.405691+0000","last_peered":"2024-09-19T07:37:58.405691+0000","last_clean":"2024-09-19T07:37:58.405691+0000","last_became_active":"2024-09-19T07:36:07.850576+0000","last_became_peered":"2024-09-19T07:36:07.850576+0000","last_unstale":"2024-09-19T07:37:58.405691+0000","last_undegraded":"2024-09-19T07:37:58.405691+0000","last_fullsized":"2024-09-19T07:37:58.405691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:45:54.150062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399976+0000","last_change":"2024-09-19T07:36:07.852059+0000","last_active":"2024-09-19T07:37:58.399976+0000","last_peered":"2024-09-19T07:37:58.399976+0000","last_clean":"2024-09-19T07:37:58.399976+0000","last_became_active":"2024-09-19T07:36:07.851559+0000","last_became_peered":"2024-09-19T07:36:07.851559+0000","last_unstale":"2024-09-19T07:37:58.399976+0000","last_undegraded":"2024-09-19T07:37:58.399976+0000","last_fullsized":"2024-09-19T07:37:58.399976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:14.232016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.281203+0000","last_change":"2024-09-19T07:36:07.864474+0000","last_active":"2024-09-19T07:37:58.281203+0000","last_peered":"2024-09-19T07:37:58.281203+0000","last_clean":"2024-09-19T07:37:58.281203+0000","last_became_active":"2024-09-19T07:36:07.864129+0000","last_became_peered":"2024-09-19T07:36:07.864129+0000","last_unstale":"2024-09-19T07:37:58.281203+0000","last_undegraded":"2024-09-19T07:37:58.281203+0000","last_fullsized":"2024-09-19T07:37:58.281203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:50:59.984621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565955+0000","last_change":"2024-09-19T07:36:07.860174+0000","last_active":"2024-09-19T07:37:58.565955+0000","last_peered":"2024-09-19T07:37:58.565955+0000","last_clean":"2024-09-19T07:37:58.565955+0000","last_became_active":"2024-09-19T07:36:07.859807+0000","last_became_peered":"2024-09-19T07:36:07.859807+0000","last_unstale":"2024-09-19T07:37:58.565955+0000","last_undegraded":"2024-09-19T07:37:58.565955+0000","last_fullsized":"2024-09-19T07:37:58.565955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:26.507167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278990+0000","last_change":"2024-09-19T07:36:07.847007+0000","last_active":"2024-09-19T07:37:58.278990+0000","last_peered":"2024-09-19T07:37:58.278990+0000","last_clean":"2024-09-19T07:37:58.278990+0000","last_became_active":"2024-09-19T07:36:07.846718+0000","last_became_peered":"2024-09-19T07:36:07.846718+0000","last_unstale":"2024-09-19T07:37:58.278990+0000","last_undegraded":"2024-09-19T07:37:58.278990+0000","last_fullsized":"2024-09-19T07:37:58.278990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:46.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566392+0000","last_change":"2024-09-19T07:36:07.851536+0000","last_active":"2024-09-19T07:37:58.566392+0000","last_peered":"2024-09-19T07:37:58.566392+0000","last_clean":"2024-09-19T07:37:58.566392+0000","last_became_active":"2024-09-19T07:36:07.851085+0000","last_became_peered":"2024-09-19T07:36:07.851085+0000","last_unstale":"2024-09-19T07:37:58.566392+0000","last_undegraded":"2024-09-19T07:37:58.566392+0000","last_fullsized":"2024-09-19T07:37:58.566392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:10.117602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280791+0000","last_change":"2024-09-19T07:36:07.858922+0000","last_active":"2024-09-19T07:37:58.280791+0000","last_peered":"2024-09-19T07:37:58.280791+0000","last_clean":"2024-09-19T07:37:58.280791+0000","last_became_active":"2024-09-19T07:36:07.857824+0000","last_became_peered":"2024-09-19T07:36:07.857824+0000","last_unstale":"2024-09-19T07:37:58.280791+0000","last_undegraded":"2024-09-19T07:37:58.280791+0000","last_fullsized":"2024-09-19T07:37:58.280791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:15.720903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280919+0000","last_change":"2024-09-19T07:36:07.857780+0000","last_active":"2024-09-19T07:37:58.280919+0000","last_peered":"2024-09-19T07:37:58.280919+0000","last_clean":"2024-09-19T07:37:58.280919+0000","last_became_active":"2024-09-19T07:36:07.857408+0000","last_became_peered":"2024-09-19T07:36:07.857408+0000","last_unstale":"2024-09-19T07:37:58.280919+0000","last_undegraded":"2024-09-19T07:37:58.280919+0000","last_fullsized":"2024-09-19T07:37:58.280919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:22.833459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565810+0000","last_change":"2024-09-19T07:36:07.854785+0000","last_active":"2024-09-19T07:37:58.565810+0000","last_peered":"2024-09-19T07:37:58.565810+0000","last_clean":"2024-09-19T07:37:58.565810+0000","last_became_active":"2024-09-19T07:36:07.854432+0000","last_became_peered":"2024-09-19T07:36:07.854432+0000","last_unstale":"2024-09-19T07:37:58.565810+0000","last_undegraded":"2024-09-19T07:37:58.565810+0000","last_fullsized":"2024-09-19T07:37:58.565810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:56.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398122+0000","last_change":"2024-09-19T07:36:07.863097+0000","last_active":"2024-09-19T07:37:58.398122+0000","last_peered":"2024-09-19T07:37:58.398122+0000","last_clean":"2024-09-19T07:37:58.398122+0000","last_became_active":"2024-09-19T07:36:07.859152+0000","last_became_peered":"2024-09-19T07:36:07.859152+0000","last_unstale":"2024-09-19T07:37:58.398122+0000","last_undegraded":"2024-09-19T07:37:58.398122+0000","last_fullsized":"2024-09-19T07:37:58.398122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:29.557171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":141,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.054316+0000","last_change":"2024-09-19T07:36:05.840522+0000","last_active":"2024-09-19T07:38:08.054316+0000","last_peered":"2024-09-19T07:38:08.054316+0000","last_clean":"2024-09-19T07:38:08.054316+0000","last_became_active":"2024-09-19T07:36:05.840141+0000","last_became_peered":"2024-09-19T07:36:05.840141+0000","last_unstale":"2024-09-19T07:38:08.054316+0000","last_undegraded":"2024-09-19T07:38:08.054316+0000","last_fullsized":"2024-09-19T07:38:08.054316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:32.833956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405841+0000","last_change":"2024-09-19T07:36:07.850122+0000","last_active":"2024-09-19T07:37:58.405841+0000","last_peered":"2024-09-19T07:37:58.405841+0000","last_clean":"2024-09-19T07:37:58.405841+0000","last_became_active":"2024-09-19T07:36:07.849813+0000","last_became_peered":"2024-09-19T07:36:07.849813+0000","last_unstale":"2024-09-19T07:37:58.405841+0000","last_undegraded":"2024-09-19T07:37:58.405841+0000","last_fullsized":"2024-09-19T07:37:58.405841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:21.118320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408925+0000","last_change":"2024-09-19T07:36:05.834456+0000","last_active":"2024-09-19T07:37:58.408925+0000","last_peered":"2024-09-19T07:37:58.408925+0000","last_clean":"2024-09-19T07:37:58.408925+0000","last_became_active":"2024-09-19T07:36:05.834092+0000","last_became_peered":"2024-09-19T07:36:05.834092+0000","last_unstale":"2024-09-19T07:37:58.408925+0000","last_undegraded":"2024-09-19T07:37:58.408925+0000","last_fullsized":"2024-09-19T07:37:58.408925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:11.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278608+0000","last_change":"2024-09-19T07:36:05.832648+0000","last_active":"2024-09-19T07:37:58.278608+0000","last_peered":"2024-09-19T07:37:58.278608+0000","last_clean":"2024-09-19T07:37:58.278608+0000","last_became_active":"2024-09-19T07:36:05.832321+0000","last_became_peered":"2024-09-19T07:36:05.832321+0000","last_unstale":"2024-09-19T07:37:58.278608+0000","last_undegraded":"2024-09-19T07:37:58.278608+0000","last_fullsized":"2024-09-19T07:37:58.278608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:15.767206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398404+0000","last_change":"2024-09-19T07:36:07.843688+0000","last_active":"2024-09-19T07:37:58.398404+0000","last_peered":"2024-09-19T07:37:58.398404+0000","last_clean":"2024-09-19T07:37:58.398404+0000","last_became_active":"2024-09-19T07:36:07.843373+0000","last_became_peered":"2024-09-19T07:36:07.843373+0000","last_unstale":"2024-09-19T07:37:58.398404+0000","last_undegraded":"2024-09-19T07:37:58.398404+0000","last_fullsized":"2024-09-19T07:37:58.398404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:49.049744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.406017+0000","last_change":"2024-09-19T07:36:05.835276+0000","last_active":"2024-09-19T07:37:58.406017+0000","last_peered":"2024-09-19T07:37:58.406017+0000","last_clean":"2024-09-19T07:37:58.406017+0000","last_became_active":"2024-09-19T07:36:05.834916+0000","last_became_peered":"2024-09-19T07:36:05.834916+0000","last_unstale":"2024-09-19T07:37:58.406017+0000","last_undegraded":"2024-09-19T07:37:58.406017+0000","last_fullsized":"2024-09-19T07:37:58.406017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:18.535996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395465+0000","last_change":"2024-09-19T07:36:07.842981+0000","last_active":"2024-09-19T07:37:58.395465+0000","last_peered":"2024-09-19T07:37:58.395465+0000","last_clean":"2024-09-19T07:37:58.395465+0000","last_became_active":"2024-09-19T07:36:07.842688+0000","last_became_peered":"2024-09-19T07:36:07.842688+0000","last_unstale":"2024-09-19T07:37:58.395465+0000","last_undegraded":"2024-09-19T07:37:58.395465+0000","last_fullsized":"2024-09-19T07:37:58.395465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:36.570177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":249,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054953+0000","last_change":"2024-09-19T07:36:05.834543+0000","last_active":"2024-09-19T07:38:09.054953+0000","last_peered":"2024-09-19T07:38:09.054953+0000","last_clean":"2024-09-19T07:38:09.054953+0000","last_became_active":"2024-09-19T07:36:05.834134+0000","last_became_peered":"2024-09-19T07:36:05.834134+0000","last_unstale":"2024-09-19T07:38:09.054953+0000","last_undegraded":"2024-09-19T07:38:09.054953+0000","last_fullsized":"2024-09-19T07:38:09.054953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:10.499797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278242+0000","last_change":"2024-09-19T07:36:07.846930+0000","last_active":"2024-09-19T07:37:58.278242+0000","last_peered":"2024-09-19T07:37:58.278242+0000","last_clean":"2024-09-19T07:37:58.278242+0000","last_became_active":"2024-09-19T07:36:07.846118+0000","last_became_peered":"2024-09-19T07:36:07.846118+0000","last_unstale":"2024-09-19T07:37:58.278242+0000","last_undegraded":"2024-09-19T07:37:58.278242+0000","last_fullsized":"2024-09-19T07:37:58.278242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:49.217854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568741+0000","last_change":"2024-09-19T07:36:17.940616+0000","last_active":"2024-09-19T07:37:58.568741+0000","last_peered":"2024-09-19T07:37:58.568741+0000","last_clean":"2024-09-19T07:37:58.568741+0000","last_became_active":"2024-09-19T07:36:17.937752+0000","last_became_peered":"2024-09-19T07:36:17.937752+0000","last_unstale":"2024-09-19T07:37:58.568741+0000","last_undegraded":"2024-09-19T07:37:58.568741+0000","last_fullsized":"2024-09-19T07:37:58.568741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:16.152646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567021+0000","last_change":"2024-09-19T07:36:07.851100+0000","last_active":"2024-09-19T07:37:58.567021+0000","last_peered":"2024-09-19T07:37:58.567021+0000","last_clean":"2024-09-19T07:37:58.567021+0000","last_became_active":"2024-09-19T07:36:07.850665+0000","last_became_peered":"2024-09-19T07:36:07.850665+0000","last_unstale":"2024-09-19T07:37:58.567021+0000","last_undegraded":"2024-09-19T07:37:58.567021+0000","last_fullsized":"2024-09-19T07:37:58.567021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:18.398265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405996+0000","last_change":"2024-09-19T07:36:05.982502+0000","last_active":"2024-09-19T07:37:58.405996+0000","last_peered":"2024-09-19T07:37:58.405996+0000","last_clean":"2024-09-19T07:37:58.405996+0000","last_became_active":"2024-09-19T07:36:05.982122+0000","last_became_peered":"2024-09-19T07:36:05.982122+0000","last_unstale":"2024-09-19T07:37:58.405996+0000","last_undegraded":"2024-09-19T07:37:58.405996+0000","last_fullsized":"2024-09-19T07:37:58.405996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:14.307759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277409+0000","last_change":"2024-09-19T07:36:07.863779+0000","last_active":"2024-09-19T07:37:58.277409+0000","last_peered":"2024-09-19T07:37:58.277409+0000","last_clean":"2024-09-19T07:37:58.277409+0000","last_became_active":"2024-09-19T07:36:07.863385+0000","last_became_peered":"2024-09-19T07:36:07.863385+0000","last_unstale":"2024-09-19T07:37:58.277409+0000","last_undegraded":"2024-09-19T07:37:58.277409+0000","last_fullsized":"2024-09-19T07:37:58.277409+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:28.667313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":295,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054953+0000","last_change":"2024-09-19T07:36:05.837579+0000","last_active":"2024-09-19T07:38:09.054953+0000","last_peered":"2024-09-19T07:38:09.054953+0000","last_clean":"2024-09-19T07:38:09.054953+0000","last_became_active":"2024-09-19T07:36:05.837192+0000","last_became_peered":"2024-09-19T07:36:05.837192+0000","last_unstale":"2024-09-19T07:38:09.054953+0000","last_undegraded":"2024-09-19T07:38:09.054953+0000","last_fullsized":"2024-09-19T07:38:09.054953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:32.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68633,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":233,"num_read_kb":184,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398289+0000","last_change":"2024-09-19T07:36:07.865175+0000","last_active":"2024-09-19T07:37:58.398289+0000","last_peered":"2024-09-19T07:37:58.398289+0000","last_clean":"2024-09-19T07:37:58.398289+0000","last_became_active":"2024-09-19T07:36:07.863797+0000","last_became_peered":"2024-09-19T07:36:07.863797+0000","last_unstale":"2024-09-19T07:37:58.398289+0000","last_undegraded":"2024-09-19T07:37:58.398289+0000","last_fullsized":"2024-09-19T07:37:58.398289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:13.953658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.280980+0000","last_change":"2024-09-19T07:36:05.834548+0000","last_active":"2024-09-19T07:37:58.280980+0000","last_peered":"2024-09-19T07:37:58.280980+0000","last_clean":"2024-09-19T07:37:58.280980+0000","last_became_active":"2024-09-19T07:36:05.834169+0000","last_became_peered":"2024-09-19T07:36:05.834169+0000","last_unstale":"2024-09-19T07:37:58.280980+0000","last_undegraded":"2024-09-19T07:37:58.280980+0000","last_fullsized":"2024-09-19T07:37:58.280980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:59.261661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278729+0000","last_change":"2024-09-19T07:36:07.854640+0000","last_active":"2024-09-19T07:37:58.278729+0000","last_peered":"2024-09-19T07:37:58.278729+0000","last_clean":"2024-09-19T07:37:58.278729+0000","last_became_active":"2024-09-19T07:36:07.854340+0000","last_became_peered":"2024-09-19T07:36:07.854340+0000","last_unstale":"2024-09-19T07:37:58.278729+0000","last_undegraded":"2024-09-19T07:37:58.278729+0000","last_fullsized":"2024-09-19T07:37:58.278729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:35.731338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405861+0000","last_change":"2024-09-19T07:36:05.837861+0000","last_active":"2024-09-19T07:37:58.405861+0000","last_peered":"2024-09-19T07:37:58.405861+0000","last_clean":"2024-09-19T07:37:58.405861+0000","last_became_active":"2024-09-19T07:36:05.837437+0000","last_became_peered":"2024-09-19T07:36:05.837437+0000","last_unstale":"2024-09-19T07:37:58.405861+0000","last_undegraded":"2024-09-19T07:37:58.405861+0000","last_fullsized":"2024-09-19T07:37:58.405861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:06:44.479731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568411+0000","last_change":"2024-09-19T07:36:07.857343+0000","last_active":"2024-09-19T07:37:58.568411+0000","last_peered":"2024-09-19T07:37:58.568411+0000","last_clean":"2024-09-19T07:37:58.568411+0000","last_became_active":"2024-09-19T07:36:07.857040+0000","last_became_peered":"2024-09-19T07:36:07.857040+0000","last_unstale":"2024-09-19T07:37:58.568411+0000","last_undegraded":"2024-09-19T07:37:58.568411+0000","last_fullsized":"2024-09-19T07:37:58.568411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:06:48.687502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'9","reported_seq":148,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.054007+0000","last_change":"2024-09-19T07:36:05.840078+0000","last_active":"2024-09-19T07:38:08.054007+0000","last_peered":"2024-09-19T07:38:08.054007+0000","last_clean":"2024-09-19T07:38:08.054007+0000","last_became_active":"2024-09-19T07:36:05.839713+0000","last_became_peered":"2024-09-19T07:36:05.839713+0000","last_unstale":"2024-09-19T07:38:08.054007+0000","last_undegraded":"2024-09-19T07:38:08.054007+0000","last_fullsized":"2024-09-19T07:38:08.054007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:31.887082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567829+0000","last_change":"2024-09-19T07:36:07.856261+0000","last_active":"2024-09-19T07:37:58.567829+0000","last_peered":"2024-09-19T07:37:58.567829+0000","last_clean":"2024-09-19T07:37:58.567829+0000","last_became_active":"2024-09-19T07:36:07.855918+0000","last_became_peered":"2024-09-19T07:36:07.855918+0000","last_unstale":"2024-09-19T07:37:58.567829+0000","last_undegraded":"2024-09-19T07:37:58.567829+0000","last_fullsized":"2024-09-19T07:37:58.567829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:45.015700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565722+0000","last_change":"2024-09-19T07:36:17.910272+0000","last_active":"2024-09-19T07:37:58.565722+0000","last_peered":"2024-09-19T07:37:58.565722+0000","last_clean":"2024-09-19T07:37:58.565722+0000","last_became_active":"2024-09-19T07:36:17.896856+0000","last_became_peered":"2024-09-19T07:36:17.896856+0000","last_unstale":"2024-09-19T07:37:58.565722+0000","last_undegraded":"2024-09-19T07:37:58.565722+0000","last_fullsized":"2024-09-19T07:37:58.565722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:30.893770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405806+0000","last_change":"2024-09-19T07:36:07.856885+0000","last_active":"2024-09-19T07:37:58.405806+0000","last_peered":"2024-09-19T07:37:58.405806+0000","last_clean":"2024-09-19T07:37:58.405806+0000","last_became_active":"2024-09-19T07:36:07.855298+0000","last_became_peered":"2024-09-19T07:36:07.855298+0000","last_unstale":"2024-09-19T07:37:58.405806+0000","last_undegraded":"2024-09-19T07:37:58.405806+0000","last_fullsized":"2024-09-19T07:37:58.405806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:18.111098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398454+0000","last_change":"2024-09-19T07:36:05.835550+0000","last_active":"2024-09-19T07:37:58.398454+0000","last_peered":"2024-09-19T07:37:58.398454+0000","last_clean":"2024-09-19T07:37:58.398454+0000","last_became_active":"2024-09-19T07:36:05.835170+0000","last_became_peered":"2024-09-19T07:36:05.835170+0000","last_unstale":"2024-09-19T07:37:58.398454+0000","last_undegraded":"2024-09-19T07:37:58.398454+0000","last_fullsized":"2024-09-19T07:37:58.398454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:27.397785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566085+0000","last_change":"2024-09-19T07:36:07.850975+0000","last_active":"2024-09-19T07:37:58.566085+0000","last_peered":"2024-09-19T07:37:58.566085+0000","last_clean":"2024-09-19T07:37:58.566085+0000","last_became_active":"2024-09-19T07:36:07.850496+0000","last_became_peered":"2024-09-19T07:36:07.850496+0000","last_unstale":"2024-09-19T07:37:58.566085+0000","last_undegraded":"2024-09-19T07:37:58.566085+0000","last_fullsized":"2024-09-19T07:37:58.566085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.578383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278747+0000","last_change":"2024-09-19T07:36:17.942042+0000","last_active":"2024-09-19T07:37:58.278747+0000","last_peered":"2024-09-19T07:37:58.278747+0000","last_clean":"2024-09-19T07:37:58.278747+0000","last_became_active":"2024-09-19T07:36:17.938060+0000","last_became_peered":"2024-09-19T07:36:17.938060+0000","last_unstale":"2024-09-19T07:37:58.278747+0000","last_undegraded":"2024-09-19T07:37:58.278747+0000","last_fullsized":"2024-09-19T07:37:58.278747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:25.072357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568790+0000","last_change":"2024-09-19T07:36:07.864756+0000","last_active":"2024-09-19T07:37:58.568790+0000","last_peered":"2024-09-19T07:37:58.568790+0000","last_clean":"2024-09-19T07:37:58.568790+0000","last_became_active":"2024-09-19T07:36:07.864355+0000","last_became_peered":"2024-09-19T07:36:07.864355+0000","last_unstale":"2024-09-19T07:37:58.568790+0000","last_undegraded":"2024-09-19T07:37:58.568790+0000","last_fullsized":"2024-09-19T07:37:58.568790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:34.383433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'67","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:07.691483+0000","last_change":"2024-09-19T07:36:05.834066+0000","last_active":"2024-09-19T07:38:07.691483+0000","last_peered":"2024-09-19T07:38:07.691483+0000","last_clean":"2024-09-19T07:38:07.691483+0000","last_became_active":"2024-09-19T07:36:05.833700+0000","last_became_peered":"2024-09-19T07:36:05.833700+0000","last_unstale":"2024-09-19T07:38:07.691483+0000","last_undegraded":"2024-09-19T07:38:07.691483+0000","last_fullsized":"2024-09-19T07:38:07.691483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:03.639374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568493+0000","last_change":"2024-09-19T07:36:07.855361+0000","last_active":"2024-09-19T07:37:58.568493+0000","last_peered":"2024-09-19T07:37:58.568493+0000","last_clean":"2024-09-19T07:37:58.568493+0000","last_became_active":"2024-09-19T07:36:07.854972+0000","last_became_peered":"2024-09-19T07:36:07.854972+0000","last_unstale":"2024-09-19T07:37:58.568493+0000","last_undegraded":"2024-09-19T07:37:58.568493+0000","last_fullsized":"2024-09-19T07:37:58.568493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:05.895986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":147,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.054072+0000","last_change":"2024-09-19T07:36:05.983194+0000","last_active":"2024-09-19T07:38:08.054072+0000","last_peered":"2024-09-19T07:38:08.054072+0000","last_clean":"2024-09-19T07:38:08.054072+0000","last_became_active":"2024-09-19T07:36:05.982554+0000","last_became_peered":"2024-09-19T07:36:05.982554+0000","last_unstale":"2024-09-19T07:38:08.054072+0000","last_undegraded":"2024-09-19T07:38:08.054072+0000","last_fullsized":"2024-09-19T07:38:08.054072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:45.681596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568589+0000","last_change":"2024-09-19T07:36:07.861494+0000","last_active":"2024-09-19T07:37:58.568589+0000","last_peered":"2024-09-19T07:37:58.568589+0000","last_clean":"2024-09-19T07:37:58.568589+0000","last_became_active":"2024-09-19T07:36:07.861140+0000","last_became_peered":"2024-09-19T07:36:07.861140+0000","last_unstale":"2024-09-19T07:37:58.568589+0000","last_undegraded":"2024-09-19T07:37:58.568589+0000","last_fullsized":"2024-09-19T07:37:58.568589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:23:18.327728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":394,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:00.401620+0000","last_change":"2024-09-19T07:34:56.017419+0000","last_active":"2024-09-19T07:38:00.401620+0000","last_peered":"2024-09-19T07:38:00.401620+0000","last_clean":"2024-09-19T07:38:00.401620+0000","last_became_active":"2024-09-19T07:34:56.016961+0000","last_became_peered":"2024-09-19T07:34:56.016961+0000","last_unstale":"2024-09-19T07:38:00.401620+0000","last_undegraded":"2024-09-19T07:38:00.401620+0000","last_fullsized":"2024-09-19T07:38:00.401620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:31:30.357447+0000","last_clean_scrub_stamp":"2024-09-19T07:31:30.357447+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:48.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566487+0000","last_change":"2024-09-19T07:36:05.835958+0000","last_active":"2024-09-19T07:37:58.566487+0000","last_peered":"2024-09-19T07:37:58.566487+0000","last_clean":"2024-09-19T07:37:58.566487+0000","last_became_active":"2024-09-19T07:36:05.835603+0000","last_became_peered":"2024-09-19T07:36:05.835603+0000","last_unstale":"2024-09-19T07:37:58.566487+0000","last_undegraded":"2024-09-19T07:37:58.566487+0000","last_fullsized":"2024-09-19T07:37:58.566487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:51.143359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399251+0000","last_change":"2024-09-19T07:36:07.850329+0000","last_active":"2024-09-19T07:37:58.399251+0000","last_peered":"2024-09-19T07:37:58.399251+0000","last_clean":"2024-09-19T07:37:58.399251+0000","last_became_active":"2024-09-19T07:36:07.850012+0000","last_became_peered":"2024-09-19T07:36:07.850012+0000","last_unstale":"2024-09-19T07:37:58.399251+0000","last_undegraded":"2024-09-19T07:37:58.399251+0000","last_fullsized":"2024-09-19T07:37:58.399251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:11.652180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.567968+0000","last_change":"2024-09-19T07:36:05.838252+0000","last_active":"2024-09-19T07:37:58.567968+0000","last_peered":"2024-09-19T07:37:58.567968+0000","last_clean":"2024-09-19T07:37:58.567968+0000","last_became_active":"2024-09-19T07:36:05.837695+0000","last_became_peered":"2024-09-19T07:36:05.837695+0000","last_unstale":"2024-09-19T07:37:58.567968+0000","last_undegraded":"2024-09-19T07:37:58.567968+0000","last_fullsized":"2024-09-19T07:37:58.567968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:41.922937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278630+0000","last_change":"2024-09-19T07:36:07.854542+0000","last_active":"2024-09-19T07:37:58.278630+0000","last_peered":"2024-09-19T07:37:58.278630+0000","last_clean":"2024-09-19T07:37:58.278630+0000","last_became_active":"2024-09-19T07:36:07.853483+0000","last_became_peered":"2024-09-19T07:36:07.853483+0000","last_unstale":"2024-09-19T07:37:58.278630+0000","last_undegraded":"2024-09-19T07:37:58.278630+0000","last_fullsized":"2024-09-19T07:37:58.278630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:45.618570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054712+0000","last_change":"2024-09-19T07:36:05.982565+0000","last_active":"2024-09-19T07:38:09.054712+0000","last_peered":"2024-09-19T07:38:09.054712+0000","last_clean":"2024-09-19T07:38:09.054712+0000","last_became_active":"2024-09-19T07:36:05.982298+0000","last_became_peered":"2024-09-19T07:36:05.982298+0000","last_unstale":"2024-09-19T07:38:09.054712+0000","last_undegraded":"2024-09-19T07:38:09.054712+0000","last_fullsized":"2024-09-19T07:38:09.054712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:41.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.405844+0000","last_change":"2024-09-19T07:36:07.856425+0000","last_active":"2024-09-19T07:37:58.405844+0000","last_peered":"2024-09-19T07:37:58.405844+0000","last_clean":"2024-09-19T07:37:58.405844+0000","last_became_active":"2024-09-19T07:36:07.854873+0000","last_became_peered":"2024-09-19T07:36:07.854873+0000","last_unstale":"2024-09-19T07:37:58.405844+0000","last_undegraded":"2024-09-19T07:37:58.405844+0000","last_fullsized":"2024-09-19T07:37:58.405844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:56.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'7","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054910+0000","last_change":"2024-09-19T07:36:05.835013+0000","last_active":"2024-09-19T07:38:09.054910+0000","last_peered":"2024-09-19T07:38:09.054910+0000","last_clean":"2024-09-19T07:38:09.054910+0000","last_became_active":"2024-09-19T07:36:05.834611+0000","last_became_peered":"2024-09-19T07:36:05.834611+0000","last_unstale":"2024-09-19T07:38:09.054910+0000","last_undegraded":"2024-09-19T07:38:09.054910+0000","last_fullsized":"2024-09-19T07:38:09.054910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:18.829988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.408965+0000","last_change":"2024-09-19T07:36:07.849780+0000","last_active":"2024-09-19T07:37:58.408965+0000","last_peered":"2024-09-19T07:37:58.408965+0000","last_clean":"2024-09-19T07:37:58.408965+0000","last_became_active":"2024-09-19T07:36:07.848763+0000","last_became_peered":"2024-09-19T07:36:07.848763+0000","last_unstale":"2024-09-19T07:37:58.408965+0000","last_undegraded":"2024-09-19T07:37:58.408965+0000","last_fullsized":"2024-09-19T07:37:58.408965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:57.817323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.396146+0000","last_change":"2024-09-19T07:36:05.982567+0000","last_active":"2024-09-19T07:37:58.396146+0000","last_peered":"2024-09-19T07:37:58.396146+0000","last_clean":"2024-09-19T07:37:58.396146+0000","last_became_active":"2024-09-19T07:36:05.982235+0000","last_became_peered":"2024-09-19T07:36:05.982235+0000","last_unstale":"2024-09-19T07:37:58.396146+0000","last_undegraded":"2024-09-19T07:37:58.396146+0000","last_fullsized":"2024-09-19T07:37:58.396146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:01.215609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.409093+0000","last_change":"2024-09-19T07:36:07.843923+0000","last_active":"2024-09-19T07:37:58.409093+0000","last_peered":"2024-09-19T07:37:58.409093+0000","last_clean":"2024-09-19T07:37:58.409093+0000","last_became_active":"2024-09-19T07:36:07.843593+0000","last_became_peered":"2024-09-19T07:36:07.843593+0000","last_unstale":"2024-09-19T07:37:58.409093+0000","last_undegraded":"2024-09-19T07:37:58.409093+0000","last_fullsized":"2024-09-19T07:37:58.409093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:42.157198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398419+0000","last_change":"2024-09-19T07:36:05.983030+0000","last_active":"2024-09-19T07:37:58.398419+0000","last_peered":"2024-09-19T07:37:58.398419+0000","last_clean":"2024-09-19T07:37:58.398419+0000","last_became_active":"2024-09-19T07:36:05.982591+0000","last_became_peered":"2024-09-19T07:36:05.982591+0000","last_unstale":"2024-09-19T07:37:58.398419+0000","last_undegraded":"2024-09-19T07:37:58.398419+0000","last_fullsized":"2024-09-19T07:37:58.398419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.181900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.278876+0000","last_change":"2024-09-19T07:36:07.853592+0000","last_active":"2024-09-19T07:37:58.278876+0000","last_peered":"2024-09-19T07:37:58.278876+0000","last_clean":"2024-09-19T07:37:58.278876+0000","last_became_active":"2024-09-19T07:36:07.853063+0000","last_became_peered":"2024-09-19T07:36:07.853063+0000","last_unstale":"2024-09-19T07:37:58.278876+0000","last_undegraded":"2024-09-19T07:37:58.278876+0000","last_fullsized":"2024-09-19T07:37:58.278876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:25.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'104","reported_seq":220,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:08.043011+0000","last_change":"2024-09-19T07:36:05.828937+0000","last_active":"2024-09-19T07:38:08.043011+0000","last_peered":"2024-09-19T07:38:08.043011+0000","last_clean":"2024-09-19T07:38:08.043011+0000","last_became_active":"2024-09-19T07:36:05.828659+0000","last_became_peered":"2024-09-19T07:36:05.828659+0000","last_unstale":"2024-09-19T07:38:08.043011+0000","last_undegraded":"2024-09-19T07:38:08.043011+0000","last_fullsized":"2024-09-19T07:38:08.043011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:53.004973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400100+0000","last_change":"2024-09-19T07:36:07.857537+0000","last_active":"2024-09-19T07:37:58.400100+0000","last_peered":"2024-09-19T07:37:58.400100+0000","last_clean":"2024-09-19T07:37:58.400100+0000","last_became_active":"2024-09-19T07:36:07.857227+0000","last_became_peered":"2024-09-19T07:36:07.857227+0000","last_unstale":"2024-09-19T07:37:58.400100+0000","last_undegraded":"2024-09-19T07:37:58.400100+0000","last_fullsized":"2024-09-19T07:37:58.400100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:27.544220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566566+0000","last_change":"2024-09-19T07:36:05.983057+0000","last_active":"2024-09-19T07:37:58.566566+0000","last_peered":"2024-09-19T07:37:58.566566+0000","last_clean":"2024-09-19T07:37:58.566566+0000","last_became_active":"2024-09-19T07:36:05.982686+0000","last_became_peered":"2024-09-19T07:36:05.982686+0000","last_unstale":"2024-09-19T07:37:58.566566+0000","last_undegraded":"2024-09-19T07:37:58.566566+0000","last_fullsized":"2024-09-19T07:37:58.566566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.690920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566745+0000","last_change":"2024-09-19T07:36:07.851885+0000","last_active":"2024-09-19T07:37:58.566745+0000","last_peered":"2024-09-19T07:37:58.566745+0000","last_clean":"2024-09-19T07:37:58.566745+0000","last_became_active":"2024-09-19T07:36:07.851143+0000","last_became_peered":"2024-09-19T07:36:07.851143+0000","last_unstale":"2024-09-19T07:37:58.566745+0000","last_undegraded":"2024-09-19T07:37:58.566745+0000","last_fullsized":"2024-09-19T07:37:58.566745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:53.307622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277436+0000","last_change":"2024-09-19T07:36:05.982839+0000","last_active":"2024-09-19T07:37:58.277436+0000","last_peered":"2024-09-19T07:37:58.277436+0000","last_clean":"2024-09-19T07:37:58.277436+0000","last_became_active":"2024-09-19T07:36:05.982416+0000","last_became_peered":"2024-09-19T07:36:05.982416+0000","last_unstale":"2024-09-19T07:37:58.277436+0000","last_undegraded":"2024-09-19T07:37:58.277436+0000","last_fullsized":"2024-09-19T07:37:58.277436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:15.646319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.397993+0000","last_change":"2024-09-19T07:36:07.865694+0000","last_active":"2024-09-19T07:37:58.397993+0000","last_peered":"2024-09-19T07:37:58.397993+0000","last_clean":"2024-09-19T07:37:58.397993+0000","last_became_active":"2024-09-19T07:36:07.865064+0000","last_became_peered":"2024-09-19T07:36:07.865064+0000","last_unstale":"2024-09-19T07:37:58.397993+0000","last_undegraded":"2024-09-19T07:37:58.397993+0000","last_fullsized":"2024-09-19T07:37:58.397993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:09.394342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398888+0000","last_change":"2024-09-19T07:36:05.836811+0000","last_active":"2024-09-19T07:37:58.398888+0000","last_peered":"2024-09-19T07:37:58.398888+0000","last_clean":"2024-09-19T07:37:58.398888+0000","last_became_active":"2024-09-19T07:36:05.836519+0000","last_became_peered":"2024-09-19T07:36:05.836519+0000","last_unstale":"2024-09-19T07:37:58.398888+0000","last_undegraded":"2024-09-19T07:37:58.398888+0000","last_fullsized":"2024-09-19T07:37:58.398888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:05.137366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565988+0000","last_change":"2024-09-19T07:36:07.853537+0000","last_active":"2024-09-19T07:37:58.565988+0000","last_peered":"2024-09-19T07:37:58.565988+0000","last_clean":"2024-09-19T07:37:58.565988+0000","last_became_active":"2024-09-19T07:36:07.853107+0000","last_became_peered":"2024-09-19T07:36:07.853107+0000","last_unstale":"2024-09-19T07:37:58.565988+0000","last_undegraded":"2024-09-19T07:37:58.565988+0000","last_fullsized":"2024-09-19T07:37:58.565988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.920624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.565778+0000","last_change":"2024-09-19T07:36:17.912633+0000","last_active":"2024-09-19T07:37:58.565778+0000","last_peered":"2024-09-19T07:37:58.565778+0000","last_clean":"2024-09-19T07:37:58.565778+0000","last_became_active":"2024-09-19T07:36:17.897874+0000","last_became_peered":"2024-09-19T07:36:17.897874+0000","last_unstale":"2024-09-19T07:37:58.565778+0000","last_undegraded":"2024-09-19T07:37:58.565778+0000","last_fullsized":"2024-09-19T07:37:58.565778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:47.937919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.399648+0000","last_change":"2024-09-19T07:36:07.846937+0000","last_active":"2024-09-19T07:37:58.399648+0000","last_peered":"2024-09-19T07:37:58.399648+0000","last_clean":"2024-09-19T07:37:58.399648+0000","last_became_active":"2024-09-19T07:36:07.846648+0000","last_became_peered":"2024-09-19T07:36:07.846648+0000","last_unstale":"2024-09-19T07:37:58.399648+0000","last_undegraded":"2024-09-19T07:37:58.399648+0000","last_fullsized":"2024-09-19T07:37:58.399648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:37.975811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398085+0000","last_change":"2024-09-19T07:36:05.841304+0000","last_active":"2024-09-19T07:37:58.398085+0000","last_peered":"2024-09-19T07:37:58.398085+0000","last_clean":"2024-09-19T07:37:58.398085+0000","last_became_active":"2024-09-19T07:36:05.840966+0000","last_became_peered":"2024-09-19T07:36:05.840966+0000","last_unstale":"2024-09-19T07:37:58.398085+0000","last_undegraded":"2024-09-19T07:37:58.398085+0000","last_fullsized":"2024-09-19T07:37:58.398085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:24.587509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.568069+0000","last_change":"2024-09-19T07:36:07.862988+0000","last_active":"2024-09-19T07:37:58.568069+0000","last_peered":"2024-09-19T07:37:58.568069+0000","last_clean":"2024-09-19T07:37:58.568069+0000","last_became_active":"2024-09-19T07:36:07.862635+0000","last_became_peered":"2024-09-19T07:36:07.862635+0000","last_unstale":"2024-09-19T07:37:58.568069+0000","last_undegraded":"2024-09-19T07:37:58.568069+0000","last_fullsized":"2024-09-19T07:37:58.568069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:29.398343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.395362+0000","last_change":"2024-09-19T07:36:17.936693+0000","last_active":"2024-09-19T07:37:58.395362+0000","last_peered":"2024-09-19T07:37:58.395362+0000","last_clean":"2024-09-19T07:37:58.395362+0000","last_became_active":"2024-09-19T07:36:17.936456+0000","last_became_peered":"2024-09-19T07:36:17.936456+0000","last_unstale":"2024-09-19T07:37:58.395362+0000","last_undegraded":"2024-09-19T07:37:58.395362+0000","last_fullsized":"2024-09-19T07:37:58.395362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:16.156825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398241+0000","last_change":"2024-09-19T07:36:05.829400+0000","last_active":"2024-09-19T07:37:58.398241+0000","last_peered":"2024-09-19T07:37:58.398241+0000","last_clean":"2024-09-19T07:37:58.398241+0000","last_became_active":"2024-09-19T07:36:05.829025+0000","last_became_peered":"2024-09-19T07:36:05.829025+0000","last_unstale":"2024-09-19T07:37:58.398241+0000","last_undegraded":"2024-09-19T07:37:58.398241+0000","last_fullsized":"2024-09-19T07:37:58.398241+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:42.701999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.277959+0000","last_change":"2024-09-19T07:36:07.847504+0000","last_active":"2024-09-19T07:37:58.277959+0000","last_peered":"2024-09-19T07:37:58.277959+0000","last_clean":"2024-09-19T07:37:58.277959+0000","last_became_active":"2024-09-19T07:36:07.847174+0000","last_became_peered":"2024-09-19T07:36:07.847174+0000","last_unstale":"2024-09-19T07:37:58.277959+0000","last_undegraded":"2024-09-19T07:37:58.277959+0000","last_fullsized":"2024-09-19T07:37:58.277959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:10.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.398335+0000","last_change":"2024-09-19T07:36:05.838789+0000","last_active":"2024-09-19T07:37:58.398335+0000","last_peered":"2024-09-19T07:37:58.398335+0000","last_clean":"2024-09-19T07:37:58.398335+0000","last_became_active":"2024-09-19T07:36:05.838384+0000","last_became_peered":"2024-09-19T07:36:05.838384+0000","last_unstale":"2024-09-19T07:37:58.398335+0000","last_undegraded":"2024-09-19T07:37:58.398335+0000","last_fullsized":"2024-09-19T07:37:58.398335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:18.339447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.566969+0000","last_change":"2024-09-19T07:36:07.849900+0000","last_active":"2024-09-19T07:37:58.566969+0000","last_peered":"2024-09-19T07:37:58.566969+0000","last_clean":"2024-09-19T07:37:58.566969+0000","last_became_active":"2024-09-19T07:36:07.849489+0000","last_became_peered":"2024-09-19T07:36:07.849489+0000","last_unstale":"2024-09-19T07:37:58.566969+0000","last_undegraded":"2024-09-19T07:37:58.566969+0000","last_fullsized":"2024-09-19T07:37:58.566969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:47.045953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400655+0000","last_change":"2024-09-19T07:36:05.982638+0000","last_active":"2024-09-19T07:37:58.400655+0000","last_peered":"2024-09-19T07:37:58.400655+0000","last_clean":"2024-09-19T07:37:58.400655+0000","last_became_active":"2024-09-19T07:36:05.982340+0000","last_became_peered":"2024-09-19T07:36:05.982340+0000","last_unstale":"2024-09-19T07:37:58.400655+0000","last_undegraded":"2024-09-19T07:37:58.400655+0000","last_fullsized":"2024-09-19T07:37:58.400655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:54.726621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:37:58.400381+0000","last_change":"2024-09-19T07:36:07.857154+0000","last_active":"2024-09-19T07:37:58.400381+0000","last_peered":"2024-09-19T07:37:58.400381+0000","last_clean":"2024-09-19T07:37:58.400381+0000","last_became_active":"2024-09-19T07:36:07.856715+0000","last_became_peered":"2024-09-19T07:36:07.856715+0000","last_unstale":"2024-09-19T07:37:58.400381+0000","last_undegraded":"2024-09-19T07:37:58.400381+0000","last_fullsized":"2024-09-19T07:37:58.400381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:06.820807+0000","last_clean_scrub_stamp":"2024-09-19T07:36:06.820807+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:46.504131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'14","reported_seq":167,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-19T07:38:09.054382+0000","last_change":"2024-09-19T07:36:17.904823+0000","last_active":"2024-09-19T07:38:09.054382+0000","last_peered":"2024-09-19T07:38:09.054382+0000","last_clean":"2024-09-19T07:38:09.054382+0000","last_became_active":"2024-09-19T07:36:17.897036+0000","last_became_peered":"2024-09-19T07:36:17.897036+0000","last_unstale":"2024-09-19T07:38:09.054382+0000","last_undegraded":"2024-09-19T07:38:09.054382+0000","last_fullsized":"2024-09-19T07:38:09.054382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T07:36:04.817064+0000","last_clean_scrub_stamp":"2024-09-19T07:36:04.817064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:32.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799284,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1497,"num_read_kb":972,"num_write":502,"num_write_kb":2381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":786432,"data_stored":2397852,"data_compressed":108252,"data_compressed_allocated":147456,"data_compressed_original":2125824,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107370,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514533,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":451633,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1130496,"data_stored":1439808,"data_compressed":61297,"data_compressed_allocated":81920,"data_compressed_original":929792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":843740,"data_compressed":28001,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":843334,"data_compressed":28001,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":1044883,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":450775,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":1370939,"data_compressed":59704,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":451422,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":1171514,"data_compressed":43203,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":520934,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":520993,"data_compressed":10305,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":452507,"data_compressed":8712,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35790,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71591,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71532,"data_compressed":1593,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331052,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":399921,"data_compressed":16795,"data_compressed_allocated":24576,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T07:38:13.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:38:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:13 smithi134 ceph-mon[35414]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-19T07:38:13.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:13 smithi134 ceph-mon[35414]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:13.658 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:13 smithi174 ceph-mon[35370]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-19T07:38:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:13 smithi174 ceph-mon[35370]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T07:38:14.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 -- 172.21.15.103:0/542341969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644106f10 msgr2=0x7f164410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:14.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 --2- 172.21.15.103:0/542341969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644106f10 0x7f164410ea90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f163c009ca0 tx=0x7f163c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 -- 172.21.15.103:0/542341969 shutdown_connections 2024-09-19T07:38:14.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 --2- 172.21.15.103:0/542341969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644106f10 0x7f164410ea90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 --2- 172.21.15.103:0/542341969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1644102790 0x7f16441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 --2- 172.21.15.103:0/542341969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1644101de0 0x7f16441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 -- 172.21.15.103:0/542341969 >> 172.21.15.103:0/542341969 conn(0x7f16440fd8c0 msgr2=0x7f16440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 -- 172.21.15.103:0/542341969 shutdown_connections 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.142+0000 7f1648dd1640 1 -- 172.21.15.103:0/542341969 wait complete. 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 Processor -- start 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 -- start start 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644101de0 0x7f1644198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1644102790 0x7f1644198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1644106f10 0x7f164419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1644111f70 con 0x7f1644101de0 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1644111df0 con 0x7f1644106f10 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.143+0000 7f1648dd1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16441120f0 con 0x7f1644102790 2024-09-19T07:38:14.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644101de0 0x7f1644198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644101de0 0x7f1644198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34976/0 (socket says 172.21.15.103:34976) 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 -- 172.21.15.103:0/2058372972 learned_addr learned my addr 172.21.15.103:0/2058372972 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f1639d9b640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1644102790 0x7f1644198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f1642d9d640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1644106f10 0x7f164419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 -- 172.21.15.103:0/2058372972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1644102790 msgr2=0x7f1644198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1644102790 0x7f1644198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 -- 172.21.15.103:0/2058372972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1644106f10 msgr2=0x7f164419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1644106f10 0x7f164419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 -- 172.21.15.103:0/2058372972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164419d2f0 con 0x7f1644101de0 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f1642d9d640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1644106f10 0x7f164419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f164259c640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644101de0 0x7f1644198340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f163400d770 tx=0x7f163400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f163bfff640 1 -- 172.21.15.103:0/2058372972 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634017070 con 0x7f1644101de0 2024-09-19T07:38:14.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164419d5e0 con 0x7f1644101de0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f163bfff640 1 -- 172.21.15.103:0/2058372972 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1634009820 con 0x7f1644101de0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f163bfff640 1 -- 172.21.15.103:0/2058372972 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634004cd0 con 0x7f1644101de0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.144+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16441a4ec0 con 0x7f1644101de0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.145+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16441021c0 con 0x7f1644101de0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.146+0000 7f163bfff640 1 -- 172.21.15.103:0/2058372972 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f16340099d0 con 0x7f1644101de0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.147+0000 7f163bfff640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16300a3440 0x7f16300a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.147+0000 7f1639d9b640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16300a3440 0x7f16300a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.147+0000 7f163bfff640 1 -- 172.21.15.103:0/2058372972 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f16340cf950 con 0x7f1644101de0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.147+0000 7f1639d9b640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16300a3440 0x7f16300a5900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1644108870 tx=0x7f162c00a2d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:14.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.151+0000 7f163bfff640 1 -- 172.21.15.103:0/2058372972 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f163409c1d0 con 0x7f1644101de0 2024-09-19T07:38:14.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.354+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f164410a1b0 con 0x7f1644101de0 2024-09-19T07:38:14.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.355+0000 7f163bfff640 1 -- 172.21.15.103:0/2058372972 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f163409bad0 con 0x7f1644101de0 2024-09-19T07:38:14.356 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:14.356 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T07:37:57:172446+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:37:57.122252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:38:14.356 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:38:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16300a3440 msgr2=0x7f16300a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16300a3440 0x7f16300a5900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1644108870 tx=0x7f162c00a2d0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644101de0 msgr2=0x7f1644198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644101de0 0x7f1644198340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f163400d770 tx=0x7f163400db70 comp rx=0 tx=0).stop 2024-09-19T07:38:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 shutdown_connections 2024-09-19T07:38:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16300a3440 0x7f16300a5900 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1644106f10 0x7f164419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1644102790 0x7f1644198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 --2- 172.21.15.103:0/2058372972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1644101de0 0x7f1644198340 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 >> 172.21.15.103:0/2058372972 conn(0x7f16440fd8c0 msgr2=0x7f16440febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.357+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 shutdown_connections 2024-09-19T07:38:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:14.358+0000 7f1648dd1640 1 -- 172.21.15.103:0/2058372972 wait complete. 2024-09-19T07:38:15.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:38:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:15.606 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:15 smithi103 ceph-mon[31201]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-19T07:38:15.607 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2058372972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:15 smithi174 ceph-mon[35370]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-19T07:38:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2058372972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.795+0000 7fbc5eade640 1 -- 172.21.15.103:0/410929933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58100e20 msgr2=0x7fbc58101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.795+0000 7fbc5eade640 1 --2- 172.21.15.103:0/410929933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58100e20 0x7fbc58101200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009900 tx=0x7fbc4c02a480 comp rx=0 tx=0).stop 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.795+0000 7fbc5eade640 1 -- 172.21.15.103:0/410929933 shutdown_connections 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.795+0000 7fbc5eade640 1 --2- 172.21.15.103:0/410929933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc58102ef0 0x7fbc5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.795+0000 7fbc5eade640 1 --2- 172.21.15.103:0/410929933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc58102570 0x7fbc581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.795+0000 7fbc5eade640 1 --2- 172.21.15.103:0/410929933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58100e20 0x7fbc58101200 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.795+0000 7fbc5eade640 1 -- 172.21.15.103:0/410929933 >> 172.21.15.103:0/410929933 conn(0x7fbc580fc7d0 msgr2=0x7fbc580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 -- 172.21.15.103:0/410929933 shutdown_connections 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 -- 172.21.15.103:0/410929933 wait complete. 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 Processor -- start 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 -- start start 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc58100e20 0x7fbc58199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58102570 0x7fbc58199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc58102ef0 0x7fbc581a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58110fa0 con 0x7fbc58102570 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58110e20 con 0x7fbc58102ef0 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.796+0000 7fbc5eade640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58111120 con 0x7fbc58100e20 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58102570 0x7fbc58199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58102570 0x7fbc58199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34990/0 (socket says 172.21.15.103:34990) 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 -- 172.21.15.103:0/1818286047 learned_addr learned my addr 172.21.15.103:0/1818286047 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 -- 172.21.15.103:0/1818286047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc58100e20 msgr2=0x7fbc58199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc58100e20 0x7fbc58199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 -- 172.21.15.103:0/1818286047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc58102ef0 msgr2=0x7fbc581a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc58102ef0 0x7fbc581a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 -- 172.21.15.103:0/1818286047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc581a48f0 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc57fff640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58102570 0x7fbc58199870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c920 tx=0x7fbc4400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc55ffb640 1 -- 172.21.15.103:0/1818286047 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44007c70 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc55ffb640 1 -- 172.21.15.103:0/1818286047 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44007e10 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc55ffb640 1 -- 172.21.15.103:0/1818286047 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44018890 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc581a4ac0 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.797+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc581a5000 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.798+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc58101200 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.799+0000 7fbc55ffb640 1 -- 172.21.15.103:0/1818286047 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fbc44018a70 con 0x7fbc58102570 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.800+0000 7fbc55ffb640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbc480a34c0 0x7fbc480a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.800+0000 7fbc5c87a640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbc480a34c0 0x7fbc480a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.800+0000 7fbc55ffb640 1 -- 172.21.15.103:0/1818286047 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fbc440d0080 con 0x7fbc58102570 2024-09-19T07:38:15.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.801+0000 7fbc5c87a640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbc480a34c0 0x7fbc480a5980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0096c0 tx=0x7fbc4c000bc0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:15.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:15.804+0000 7fbc55ffb640 1 -- 172.21.15.103:0/1818286047 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbc44014030 con 0x7fbc58102570 2024-09-19T07:38:15.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:15 smithi134 ceph-mon[35414]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-19T07:38:15.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2058372972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.012+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc5806a560 con 0x7fbc58102570 2024-09-19T07:38:16.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.013+0000 7fbc55ffb640 1 -- 172.21.15.103:0/1818286047 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7fbc4409c900 con 0x7fbc58102570 2024-09-19T07:38:16.014 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:16.014 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T07:37:57:172446+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:37:57.122252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:38:16.014 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbc480a34c0 msgr2=0x7fbc480a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbc480a34c0 0x7fbc480a5980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0096c0 tx=0x7fbc4c000bc0 comp rx=0 tx=0).stop 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58102570 msgr2=0x7fbc58199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58102570 0x7fbc58199870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c920 tx=0x7fbc4400cde0 comp rx=0 tx=0).stop 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 shutdown_connections 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbc480a34c0 0x7fbc480a5980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc58102ef0 0x7fbc581a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc58102570 0x7fbc58199870 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:16.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 --2- 172.21.15.103:0/1818286047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc58100e20 0x7fbc58199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 >> 172.21.15.103:0/1818286047 conn(0x7fbc580fc7d0 msgr2=0x7fbc58106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 shutdown_connections 2024-09-19T07:38:16.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:16.015+0000 7fbc5eade640 1 -- 172.21.15.103:0/1818286047 wait complete. 2024-09-19T07:38:16.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:16 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1818286047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:16 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1818286047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:16.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:38:16.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:16 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1818286047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:16.922 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:17.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 -- 172.21.15.103:0/893392123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c106a80 msgr2=0x7f327c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 --2- 172.21.15.103:0/893392123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c106a80 0x7f327c106e60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3270009930 tx=0x7f327002a420 comp rx=0 tx=0).stop 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 -- 172.21.15.103:0/893392123 shutdown_connections 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 --2- 172.21.15.103:0/893392123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f327c101420 0x7f327c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 --2- 172.21.15.103:0/893392123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f327c100a80 0x7f327c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 --2- 172.21.15.103:0/893392123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c106a80 0x7f327c106e60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 -- 172.21.15.103:0/893392123 >> 172.21.15.103:0/893392123 conn(0x7f327c0fc7a0 msgr2=0x7f327c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 -- 172.21.15.103:0/893392123 shutdown_connections 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.399+0000 7f3282b3c640 1 -- 172.21.15.103:0/893392123 wait complete. 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 Processor -- start 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 -- start start 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c100a80 0x7f327c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:17.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f327c101420 0x7f327c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f327c106a80 0x7f327c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f327c110e40 con 0x7f327c100a80 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f327c110cc0 con 0x7f327c101420 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f3282b3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f327c110fc0 con 0x7f327c106a80 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c100a80 0x7f327c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c100a80 0x7f327c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35008/0 (socket says 172.21.15.103:35008) 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 -- 172.21.15.103:0/1410214940 learned_addr learned my addr 172.21.15.103:0/1410214940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f327bfff640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f327c101420 0x7f327c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:17.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32810d9640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f327c106a80 0x7f327c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 -- 172.21.15.103:0/1410214940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f327c106a80 msgr2=0x7f327c1a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f327c106a80 0x7f327c1a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 -- 172.21.15.103:0/1410214940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f327c101420 msgr2=0x7f327c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f327c101420 0x7f327c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 -- 172.21.15.103:0/1410214940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c1a4850 con 0x7f327c100a80 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32810d9640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f327c106a80 0x7f327c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f327bfff640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f327c101420 0x7f327c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:17.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.400+0000 7f32808d8640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c100a80 0x7f327c199290 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3270009a50 tx=0x7f3270004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.401+0000 7f3279ffb640 1 -- 172.21.15.103:0/1410214940 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327003f030 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.401+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c1a4a20 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.401+0000 7f3279ffb640 1 -- 172.21.15.103:0/1410214940 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3270003ce0 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.401+0000 7f3279ffb640 1 -- 172.21.15.103:0/1410214940 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270035070 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.401+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c1a4ed0 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.402+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c100ee0 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.403+0000 7f3279ffb640 1 -- 172.21.15.103:0/1410214940 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f3270042030 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.403+0000 7f3279ffb640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f326c0a3440 0x7f326c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.404+0000 7f327bfff640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f326c0a3440 0x7f326c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.404+0000 7f3279ffb640 1 -- 172.21.15.103:0/1410214940 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f32700f0520 con 0x7f327c100a80 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.404+0000 7f327bfff640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f326c0a3440 0x7f326c0a5900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f327c06b2e0 tx=0x7f326800a5c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:17.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.408+0000 7f3279ffb640 1 -- 172.21.15.103:0/1410214940 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32700bcda0 con 0x7f327c100a80 2024-09-19T07:38:17.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.576+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f327c02d7c0 con 0x7f327c100a80 2024-09-19T07:38:17.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:17 smithi103 ceph-mon[31201]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-09-19T07:38:17.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.577+0000 7f3279ffb640 1 -- 172.21.15.103:0/1410214940 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23200 (secure 0 0 0) 0x7f32700bc6a0 con 0x7f327c100a80 2024-09-19T07:38:17.578 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:17.579 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":85,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:37:58.271170+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/3492331771":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/189968053":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2164654927":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/2266926610":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/532109813":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6800/2164654927":"2024-09-20T07:37:58.271121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:38:17.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f326c0a3440 msgr2=0x7f326c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:17.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f326c0a3440 0x7f326c0a5900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f327c06b2e0 tx=0x7f326800a5c0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c100a80 msgr2=0x7f327c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c100a80 0x7f327c199290 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3270009a50 tx=0x7f3270004270 comp rx=0 tx=0).stop 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 shutdown_connections 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f326c0a3440 0x7f326c0a5900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f327c106a80 0x7f327c1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f327c101420 0x7f327c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 --2- 172.21.15.103:0/1410214940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f327c100a80 0x7f327c199290 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 >> 172.21.15.103:0/1410214940 conn(0x7f327c0fc7a0 msgr2=0x7f327c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 shutdown_connections 2024-09-19T07:38:17.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:17.580+0000 7f3282b3c640 1 -- 172.21.15.103:0/1410214940 wait complete. 2024-09-19T07:38:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:17 smithi174 ceph-mon[35370]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-09-19T07:38:17.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:17 smithi134 ceph-mon[35414]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-09-19T07:38:18.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T07:38:18.544 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:18.665 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:18 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1410214940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:38:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:18 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1410214940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:38:18.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:18 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1410214940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:38:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:19 smithi174 ceph-mon[35370]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-19T07:38:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:38:19.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:19 smithi103 ceph-mon[31201]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-19T07:38:19.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:38:19.704 INFO:teuthology.orchestra.run.smithi103.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T07:38:19.704 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T07:38:19.704 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T07:38:19.705 DEBUG:tasks.quiescer:Yielding 2024-09-19T07:38:19.705 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T07:38:19.715 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T07:38:19.716 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:38:19.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-19T07:38:19.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:38:19.753 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:19.767 INFO:teuthology.orchestra.run.smithi103.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-19T07:38:19.768 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:3300,172.21.15.134:3300,172.21.15.174:3300:/volumes/qa/sv_1/7cc5c1f1-01fb-4213-a2c6-8ec0edefbf83 340G 0 340G 0% /tmp/cephfs 2024-09-19T07:38:19.770 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-19T07:38:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:19 smithi134 ceph-mon[35414]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-19T07:38:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:38:19.861 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-19T07:38:19.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-19T07:38:19.975 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 -- 172.21.15.103:0/574919184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac102d90 msgr2=0x7f63ac103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/574919184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac102d90 0x7f63ac103170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f639c009930 tx=0x7f639c02a420 comp rx=0 tx=0).stop 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 -- 172.21.15.103:0/574919184 shutdown_connections 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/574919184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63ac108290 0x7f63ac10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/574919184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63ac103740 0x7f63ac107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/574919184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac102d90 0x7f63ac103170 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 -- 172.21.15.103:0/574919184 >> 172.21.15.103:0/574919184 conn(0x7f63ac0fc7d0 msgr2=0x7f63ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 -- 172.21.15.103:0/574919184 shutdown_connections 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.453+0000 7f63b1f8d640 1 -- 172.21.15.103:0/574919184 wait complete. 2024-09-19T07:38:20.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.454+0000 7f63b1f8d640 1 Processor -- start 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.454+0000 7f63b1f8d640 1 -- start start 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.454+0000 7f63b1f8d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63ac102d90 0x7f63ac198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63b1f8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63ac103740 0x7f63ac198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63b1f8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac108290 0x7f63ac19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63b1f8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac112090 con 0x7f63ac108290 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63b1f8d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac111f10 con 0x7f63ac102d90 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63b1f8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac112210 con 0x7f63ac103740 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac108290 0x7f63ac19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:20.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac108290 0x7f63ac19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54990/0 (socket says 172.21.15.103:54990) 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 -- 172.21.15.103:0/3616257421 learned_addr learned my addr 172.21.15.103:0/3616257421 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63a3fff640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63ac103740 0x7f63ac198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63ab7fe640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63ac102d90 0x7f63ac198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 -- 172.21.15.103:0/3616257421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63ac103740 msgr2=0x7f63ac198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63ac103740 0x7f63ac198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 -- 172.21.15.103:0/3616257421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63ac102d90 msgr2=0x7f63ac198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63ac102d90 0x7f63ac198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 -- 172.21.15.103:0/3616257421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac19d4c0 con 0x7f63ac108290 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63abfff640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac108290 0x7f63ac19cde0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f63a400cce0 tx=0x7f63a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63a97fa640 1 -- 172.21.15.103:0/3616257421 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a400f450 con 0x7f63ac108290 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63a97fa640 1 -- 172.21.15.103:0/3616257421 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a400fad0 con 0x7f63ac108290 2024-09-19T07:38:20.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac19d7b0 con 0x7f63ac108290 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63a97fa640 1 -- 172.21.15.103:0/3616257421 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4018c10 con 0x7f63ac108290 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.455+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac1a5090 con 0x7f63ac108290 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.457+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac06a7f0 con 0x7f63ac108290 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.458+0000 7f63a97fa640 1 -- 172.21.15.103:0/3616257421 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f63a400f5f0 con 0x7f63ac108290 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.458+0000 7f63a97fa640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63980a3440 0x7f63980a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.458+0000 7f63ab7fe640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63980a3440 0x7f63980a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.459+0000 7f63a97fa640 1 -- 172.21.15.103:0/3616257421 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f63a40d04e0 con 0x7f63ac108290 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.459+0000 7f63ab7fe640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63980a3440 0x7f63980a5900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:20.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.463+0000 7f63a97fa640 1 -- 172.21.15.103:0/3616257421 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f63a4014030 con 0x7f63ac108290 2024-09-19T07:38:20.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.668+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63ac10a2a0 con 0x7f63ac108290 2024-09-19T07:38:20.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.668+0000 7f63a97fa640 1 -- 172.21.15.103:0/3616257421 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f63a409cd60 con 0x7f63ac108290 2024-09-19T07:38:20.669 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:20.669 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T07:37:57:172446+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:37:57.122252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:38:20.670 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:38:20.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.670+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63980a3440 msgr2=0x7f63980a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:20.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.670+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63980a3440 0x7f63980a5900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c005a20 comp rx=0 tx=0).stop 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.670+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac108290 msgr2=0x7f63ac19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.670+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac108290 0x7f63ac19cde0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f63a400cce0 tx=0x7f63a400a620 comp rx=0 tx=0).stop 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 shutdown_connections 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63980a3440 0x7f63980a5900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63ac108290 0x7f63ac19cde0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63ac103740 0x7f63ac198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 --2- 172.21.15.103:0/3616257421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63ac102d90 0x7f63ac198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 >> 172.21.15.103:0/3616257421 conn(0x7f63ac0fc7d0 msgr2=0x7f63ac105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 shutdown_connections 2024-09-19T07:38:20.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:20.671+0000 7f63b1f8d640 1 -- 172.21.15.103:0/3616257421 wait complete. 2024-09-19T07:38:21.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:21 smithi174 ceph-mon[35370]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-19T07:38:21.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:21 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3616257421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:21 smithi103 ceph-mon[31201]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-19T07:38:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:21 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3616257421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:21.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:21 smithi134 ceph-mon[35414]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-19T07:38:21.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:21 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3616257421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:22.118 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-19T07:38:22.196 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-19T07:38:22.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-19T07:38:22.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T07:36:08.395229+0000', 'modified': '2024-09-19T07:37:57.122252+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24494, 'mds_1': 24466, 'mds_2': 24455, 'mds_3': 24454, 'mds_4': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14568': {'gid': 14568, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6839/1660338093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 1660338093}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 1660338093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14580': {'gid': 14580, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6841/1521905484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6840', 'nonce': 1521905484}, {'type': 'v1', 'addr': '172.21.15.103:6841', 'nonce': 1521905484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2440407987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2440407987}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2440407987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/1885230172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 1885230172}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 1885230172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/19326296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 19326296}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 19326296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6835/214062550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 214062550}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 214062550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/2527370860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2527370860}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2527370860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.134:6837/3959604171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3959604171}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3959604171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/2514182394', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2514182394}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2514182394}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/3683561216', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3683561216}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3683561216}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24448, 24454, 24455, 24494, 24466]} 2024-09-19T07:38:22.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-19T07:38:22.217 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-19T07:38:22.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-19T07:38:22.370 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/: 2024-09-19T07:38:22.370 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:22.370 INFO:teuthology.orchestra.run.smithi103.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:38:22.370 INFO:teuthology.orchestra.run.smithi103.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 .. 2024-09-19T07:38:22.371 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 data 2024-09-19T07:38:22.371 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:22.371 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data: 2024-09-19T07:38:22.371 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:22.371 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:22.371 INFO:teuthology.orchestra.run.smithi103.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 .. 2024-09-19T07:38:22.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-19T07:38:22.460 INFO:teuthology.orchestra.run.smithi103.stdout:[Service] 2024-09-19T07:38:22.461 INFO:teuthology.orchestra.run.smithi103.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-19T07:38:22.461 INFO:teuthology.orchestra.run.smithi103.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-19T07:38:22.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-19T07:38:22.521 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:22.554 INFO:teuthology.orchestra.run.smithi103.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-19T07:38:23.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 -- 172.21.15.103:0/480672975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 msgr2=0x7f25e8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 --2- 172.21.15.103:0/480672975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8107b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009930 tx=0x7f25dc02a420 comp rx=0 tx=0).stop 2024-09-19T07:38:23.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 -- 172.21.15.103:0/480672975 shutdown_connections 2024-09-19T07:38:23.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 --2- 172.21.15.103:0/480672975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e8108290 0x7f25e810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 --2- 172.21.15.103:0/480672975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8107b80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 --2- 172.21.15.103:0/480672975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e8102d90 0x7f25e8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 -- 172.21.15.103:0/480672975 >> 172.21.15.103:0/480672975 conn(0x7f25e80fc7d0 msgr2=0x7f25e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 -- 172.21.15.103:0/480672975 shutdown_connections 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.024+0000 7f25ecf95640 1 -- 172.21.15.103:0/480672975 wait complete. 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 Processor -- start 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 -- start start 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e8102d90 0x7f25e8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e8108290 0x7f25e819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8112090 con 0x7f25e8103740 2024-09-19T07:38:23.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8111f10 con 0x7f25e8108290 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.025+0000 7f25ecf95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8112210 con 0x7f25e8102d90 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55006/0 (socket says 172.21.15.103:55006) 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 -- 172.21.15.103:0/699226037 learned_addr learned my addr 172.21.15.103:0/699226037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e8108290 0x7f25e819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e659c640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e8102d90 0x7f25e8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 -- 172.21.15.103:0/699226037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e8102d90 msgr2=0x7f25e8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e8102d90 0x7f25e8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 -- 172.21.15.103:0/699226037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e8108290 msgr2=0x7f25e819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e8108290 0x7f25e819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 -- 172.21.15.103:0/699226037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e819d600 con 0x7f25e8103740 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e8108290 0x7f25e819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8198b90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f25dc005940 tx=0x7f25dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25d77fe640 1 -- 172.21.15.103:0/699226037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc03f030 con 0x7f25e8103740 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25d77fe640 1 -- 172.21.15.103:0/699226037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25dc003ad0 con 0x7f25e8103740 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e819d890 con 0x7f25e8103740 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25d77fe640 1 -- 172.21.15.103:0/699226037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc035070 con 0x7f25e8103740 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.026+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81a51d0 con 0x7f25e8103740 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.028+0000 7f25d77fe640 1 -- 172.21.15.103:0/699226037 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f25dc042030 con 0x7f25e8103740 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 Processor -- start 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 -- start start 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfbf9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc04670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfc06fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb60090 con 0x5576bfc04bb0 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb07a70 con 0x5576bfbf99d0 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25ef36e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb5c050 con 0x5576bfbfa2f0 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25e6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfbf9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25e6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfbf9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45764/0 (socket says 172.21.15.103:45764) 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfc06fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3861387105 learned_addr learned my addr 172.21.15.103:0/3861387105 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.029+0000 7f25e659c640 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc04670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25d77fe640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f25d00a34c0 0x7f25d00a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3861387105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 msgr2=0x5576bfc04670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc04670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3861387105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 msgr2=0x5576bfc06fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfc06fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3861387105 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bfb14130 con 0x5576bfbf99d0 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f25d00a34c0 0x7f25d00a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25d77fe640 1 -- 172.21.15.103:0/699226037 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f25dc0f0590 con 0x7f25e8103740 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfbf9db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25e0010420 tx=0x7f25e00108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307c67f7ad2a6913 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25d4ff9640 1 -- 172.21.15.103:0/3861387105 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0023070 con 0x5576bfbf99d0 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25d4ff9640 1 -- 172.21.15.103:0/3861387105 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25e001c470 con 0x5576bfbf99d0 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.030+0000 7f25d4ff9640 1 -- 172.21.15.103:0/3861387105 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e001d640 con 0x5576bfbf99d0 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 -- 172.21.15.103:0/3861387105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 msgr2=0x5576bfbf9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfbf9db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25e0010420 tx=0x7f25e00108e0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f25d00a34c0 0x7f25d00a5980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25e0020000 tx=0x7f25e001a000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:23.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 -- 172.21.15.103:0/3861387105 shutdown_connections 2024-09-19T07:38:23.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfc06fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc04670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3861387105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfbf9db0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 -- 172.21.15.103:0/3861387105 >> 172.21.15.103:0/3861387105 conn(0x5576bfbf5f60 msgr2=0x5576bfbf6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 -- 172.21.15.103:0/3861387105 shutdown_connections 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 -- 172.21.15.103:0/3861387105 wait complete. 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 -- start start 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.031+0000 7f25ef36e740 1 Processor -- start 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc04460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbfa2f0 0x5576bfc017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfc04bb0 0x5576bfc01cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25ef36e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc07780 con 0x5576bfbf99d0 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25ef36e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb1af20 con 0x5576bfbfa2f0 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25ef36e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc07900 con 0x5576bfc04bb0 2024-09-19T07:38:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc04460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25ef36e740 10 client.0 _ll_register_callbacks cb 0x5576bfae4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc04460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55012/0 (socket says 172.21.15.103:55012) 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 -- 172.21.15.103:0/1177742368 learned_addr learned my addr 172.21.15.103:0/1177742368 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbfa2f0 0x5576bfc017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfc04bb0 0x5576bfc01cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 -- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfc04bb0 msgr2=0x5576bfc01cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfc04bb0 0x5576bfc01cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 -- 172.21.15.103:0/1177742368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbfa2f0 msgr2=0x5576bfc017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbfa2f0 0x5576bfc017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 -- 172.21.15.103:0/1177742368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bf8d12b0 con 0x5576bfbf99d0 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbfa2f0 0x5576bfc017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25e659c640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc04460 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25d8012700 tx=0x7f25d8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc3c02d79f09dba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:23.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.032+0000 7f25cdffb640 1 -- 172.21.15.103:0/1177742368 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8004260 con 0x5576bfbf99d0 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25cdffb640 1 -- 172.21.15.103:0/1177742368 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f25d8004400 con 0x5576bfbf99d0 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e001c470 con 0x5576bfbf99d0 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25cdffb640 1 -- 172.21.15.103:0/1177742368 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25d80119a0 con 0x5576bfbf99d0 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 10 client.15438 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576bfc13100 con 0x5576bfbf99d0 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25cdffb640 1 -- 172.21.15.103:0/1177742368 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25e001c470 con 0x5576bfbf99d0 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25cdffb640 1 -- 172.21.15.103:0/1177742368 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25d800f440 con 0x5576bfbf99d0 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 10 client.15438 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 10 client.15438 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 20 client.15438 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc15740 0x5576bfc16bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 4 client.15438 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576bfb6d600 con 0x5576bfc15740 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.033+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc15740 0x5576bfc16bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.034+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc15740 0x5576bfc16bb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c358400ea06000f server_cookie=1a108e8181b5d252 in_seq=0 out_seq=0 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.034+0000 7f25cdffb640 10 client.15438 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.034+0000 7f25cdffb640 1 -- 172.21.15.103:0/1177742368 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576bfb6d600 con 0x5576bfc15740 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.034+0000 7f25cdffb640 10 client.15438 handle_command_reply: tid=0 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.034+0000 7f25ef36e740 1 client.15438 shutdown 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 msgr2=0x5576bfc04460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc04460 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25d8012700 tx=0x7f25d8012bc0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 shutdown_connections 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc15740 0x5576bfc16bb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfc04bb0 0x5576bfc01cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25cdffb640 0 client.15438 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:23.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbfa2f0 0x5576bfc017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1177742368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc04460 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 >> 172.21.15.103:0/1177742368 conn(0x5576bfbf5f60 msgr2=0x5576bfbfbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:23.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 shutdown_connections 2024-09-19T07:38:23.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 1 -- 172.21.15.103:0/1177742368 wait complete. 2024-09-19T07:38:23.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.035+0000 7f25ef36e740 20 client.15438 trim_cache size 0 max 16384 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 Processor -- start 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 -- start start 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfc16bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfbf4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc14ef0 con 0x5576bfc04bb0 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc155c0 con 0x5576bfbf99d0 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25ef36e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc15c90 con 0x5576bfbfa2f0 2024-09-19T07:38:23.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfbf4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfbf4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55028/0 (socket says 172.21.15.103:55028) 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3987677170 learned_addr learned my addr 172.21.15.103:0/3987677170 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.053+0000 7f25e659c640 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfc16bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc00080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3987677170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 msgr2=0x5576bfc00080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc00080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3987677170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 msgr2=0x5576bfc16bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfc16bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e6d9d640 1 -- 172.21.15.103:0/3987677170 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bfc00760 con 0x5576bfc04bb0 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e659c640 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfc16bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfbf4ef0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f25e0010420 tx=0x7f25e00134c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2ba69326dd66b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:23.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25cffff640 1 -- 172.21.15.103:0/3987677170 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0023040 con 0x5576bfc04bb0 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25cffff640 1 -- 172.21.15.103:0/3987677170 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25e0009bb0 con 0x5576bfc04bb0 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.054+0000 7f25cffff640 1 -- 172.21.15.103:0/3987677170 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0027ba0 con 0x5576bfc04bb0 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 -- 172.21.15.103:0/3987677170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 msgr2=0x5576bfbf4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfbf4ef0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f25e0010420 tx=0x7f25e00134c0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 -- 172.21.15.103:0/3987677170 shutdown_connections 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfc04bb0 0x5576bfbf4ef0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 --2- 172.21.15.103:0/3987677170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfbf99d0 0x5576bfc16bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 -- 172.21.15.103:0/3987677170 >> 172.21.15.103:0/3987677170 conn(0x5576bfbf5f60 msgr2=0x5576bfbf47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 -- 172.21.15.103:0/3987677170 shutdown_connections 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 -- 172.21.15.103:0/3987677170 wait complete. 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.055+0000 7f25ef36e740 1 -- start start 2024-09-19T07:38:23.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 1 Processor -- start 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc1a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc1a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc04bb0 0x5576bfc7de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc16a30 con 0x5576bfbf99d0 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc16360 con 0x5576bfc04bb0 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc17100 con 0x5576bfbfa2f0 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25ef36e740 10 client.0 _ll_register_callbacks cb 0x7f25d80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc1a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc1a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55040/0 (socket says 172.21.15.103:55040) 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc04bb0 0x5576bfc7de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 -- 172.21.15.103:0/1150986261 learned_addr learned my addr 172.21.15.103:0/1150986261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc1a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 -- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 msgr2=0x5576bfc1a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc1a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 -- 172.21.15.103:0/1150986261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc04bb0 msgr2=0x5576bfc7de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc04bb0 0x5576bfc7de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 -- 172.21.15.103:0/1150986261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bfc7e400 con 0x5576bfbf99d0 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc1a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e6d9d640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc04bb0 0x5576bfc7de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25e659c640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc1a2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ada95109da3010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.056+0000 7f25cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25cdffb640 1 -- 172.21.15.103:0/1150986261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8029040 con 0x5576bfbf99d0 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25cdffb640 1 -- 172.21.15.103:0/1150986261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f25d8005bb0 con 0x5576bfbf99d0 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e0009bb0 con 0x5576bfbf99d0 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25cdffb640 1 -- 172.21.15.103:0/1150986261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25d803c050 con 0x5576bfbf99d0 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:23.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576bfc7f6e0 con 0x5576bfbf99d0 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25cdffb640 1 -- 172.21.15.103:0/1150986261 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25e0009bb0 con 0x5576bfbf99d0 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25cdffb640 1 -- 172.21.15.103:0/1150986261 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25d8037030 con 0x5576bfbf99d0 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 10 client.15444 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 20 client.15444 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc84310 0x5576bfc83020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 4 client.15444 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.057+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "status"}) -- 0x5576bfb67b10 con 0x5576bfc84310 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc84310 0x5576bfc83020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25e5d9b640 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc84310 0x5576bfc83020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5c968f33cf00a9c server_cookie=6a67f5ee74e7d7d5 in_seq=0 out_seq=0 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25cdffb640 10 client.15444 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:23.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25cdffb640 1 -- 172.21.15.103:0/1150986261 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5576bfb67b10 con 0x5576bfc84310 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25cdffb640 10 client.15444 handle_command_reply: tid=0 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 client.15444 shutdown 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 msgr2=0x5576bfc1a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc1a2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d8009ea0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 shutdown_connections 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5576bfc84310 0x5576bfc83020 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc04bb0 0x5576bfc7de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576bfbfa2f0 0x5576bfc1a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 --2- 172.21.15.103:0/1150986261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5576bfbf99d0 0x5576bfc1a2f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 >> 172.21.15.103:0/1150986261 conn(0x5576bfbf5f60 msgr2=0x5576bfbf47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25cdffb640 0 client.15444 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 shutdown_connections 2024-09-19T07:38:23.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.058+0000 7f25ef36e740 1 -- 172.21.15.103:0/1150986261 wait complete. 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ef36e740 20 client.15444 trim_cache size 0 max 16384 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 4, 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24448, 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 124.160361847, 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T07:38:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 85, 2024-09-19T07:38:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:38:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 160.76816424399999 2024-09-19T07:38:23.068 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:23.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f25d00a34c0 msgr2=0x7f25d00a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f25d00a34c0 0x7f25d00a5980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25e0020000 tx=0x7f25e001a000 comp rx=0 tx=0).stop 2024-09-19T07:38:23.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 msgr2=0x7f25e8198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:23.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8198b90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f25dc005940 tx=0x7f25dc004270 comp rx=0 tx=0).stop 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 shutdown_connections 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f25d00a34c0 0x7f25d00a5980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e8108290 0x7f25e819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25e8103740 0x7f25e8198b90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 --2- 172.21.15.103:0/699226037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e8102d90 0x7f25e8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 >> 172.21.15.103:0/699226037 conn(0x7f25e80fc7d0 msgr2=0x7f25e8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 shutdown_connections 2024-09-19T07:38:23.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:23.059+0000 7f25ecf95640 1 -- 172.21.15.103:0/699226037 wait complete. 2024-09-19T07:38:23.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:23 smithi174 ceph-mon[35370]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:38:23.693 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:23 smithi103 ceph-mon[31201]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:38:23.795 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-19T07:38:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:23 smithi134 ceph-mon[35414]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-19T07:38:24.058 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:24.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 -- 172.21.15.103:0/3270623130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 msgr2=0x7fdccc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 --2- 172.21.15.103:0/3270623130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc106e60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009930 tx=0x7fdcc002a420 comp rx=0 tx=0).stop 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 -- 172.21.15.103:0/3270623130 shutdown_connections 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 --2- 172.21.15.103:0/3270623130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdccc101420 0x7fdccc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 --2- 172.21.15.103:0/3270623130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdccc100a80 0x7fdccc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 --2- 172.21.15.103:0/3270623130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc106e60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 -- 172.21.15.103:0/3270623130 >> 172.21.15.103:0/3270623130 conn(0x7fdccc0fc7a0 msgr2=0x7fdccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 -- 172.21.15.103:0/3270623130 shutdown_connections 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 -- 172.21.15.103:0/3270623130 wait complete. 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.541+0000 7fdcd1376640 1 Processor -- start 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcd1376640 1 -- start start 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcd1376640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdccc100a80 0x7fdccc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcd1376640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdccc101420 0x7fdccc197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcd1376640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcd1376640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc110ea0 con 0x7fdccc106a80 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcd1376640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc110d20 con 0x7fdccc101420 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcd1376640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc111020 con 0x7fdccc100a80 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc19bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55046/0 (socket says 172.21.15.103:55046) 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 -- 172.21.15.103:0/1835914230 learned_addr learned my addr 172.21.15.103:0/1835914230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdccc101420 0x7fdccc197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccaffd640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdccc100a80 0x7fdccc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 -- 172.21.15.103:0/1835914230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdccc100a80 msgr2=0x7fdccc197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdccc100a80 0x7fdccc197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 -- 172.21.15.103:0/1835914230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdccc101420 msgr2=0x7fdccc197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdccc101420 0x7fdccc197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 -- 172.21.15.103:0/1835914230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc19c2b0 con 0x7fdccc106a80 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.542+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc19bbd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400b970 tx=0x7fdcc400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.543+0000 7fdcb3fff640 1 -- 172.21.15.103:0/1835914230 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4019070 con 0x7fdccc106a80 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.543+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc19c5a0 con 0x7fdccc106a80 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.543+0000 7fdcb3fff640 1 -- 172.21.15.103:0/1835914230 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc4014440 con 0x7fdccc106a80 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.543+0000 7fdcb3fff640 1 -- 172.21.15.103:0/1835914230 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40075d0 con 0x7fdccc106a80 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.543+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc06a790 con 0x7fdccc106a80 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.545+0000 7fdcb3fff640 1 -- 172.21.15.103:0/1835914230 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fdcc401c070 con 0x7fdccc106a80 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.545+0000 7fdcd374f740 1 Processor -- start 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.545+0000 7fdcd374f740 1 -- start start 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.545+0000 7fdcb3fff640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdcb40a3440 0x7fdcb40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f61db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f622f0 0x5604e5f6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdcb40a3440 0x7fdcb40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcb3fff640 1 -- 172.21.15.103:0/1835914230 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdcc40cf3f0 con 0x7fdccc106a80 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f61db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f6cbb0 0x5604e5f6efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f61db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55054/0 (socket says 172.21.15.103:55054) 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 -- 172.21.15.103:0/791991442 learned_addr learned my addr 172.21.15.103:0/791991442 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604e5ec8090 con 0x5604e5f619d0 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604e5e6fa70 con 0x5604e5f6cbb0 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604e5ec4050 con 0x5604e5f622f0 2024-09-19T07:38:24.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f622f0 0x5604e5f6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f6cbb0 0x5604e5f6efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdcb40a3440 0x7fdcb40a5900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0058c0 tx=0x7fdcbc00a770 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 -- 172.21.15.103:0/791991442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f622f0 msgr2=0x5604e5f6c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f622f0 0x5604e5f6c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 -- 172.21.15.103:0/791991442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f6cbb0 msgr2=0x5604e5f6efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f6cbb0 0x5604e5f6efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 -- 172.21.15.103:0/791991442 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e5e7c130 con 0x5604e5f619d0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.546+0000 7fdccaffd640 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f61db0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0005ab0 tx=0x7fdcc0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81bcca231c3bca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcb17fa640 1 -- 172.21.15.103:0/791991442 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc003f070 con 0x5604e5f619d0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcb17fa640 1 -- 172.21.15.103:0/791991442 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0031a20 con 0x5604e5f619d0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcb17fa640 1 -- 172.21.15.103:0/791991442 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0035070 con 0x5604e5f619d0 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 msgr2=0x5604e5f61db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f61db0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0005ab0 tx=0x7fdcc0031630 comp rx=0 tx=0).stop 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 shutdown_connections 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f6cbb0 0x5604e5f6efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f622f0 0x5604e5f6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 --2- 172.21.15.103:0/791991442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f61db0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 >> 172.21.15.103:0/791991442 conn(0x5604e5f5df60 msgr2=0x5604e5f5e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 shutdown_connections 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.547+0000 7fdcd374f740 1 -- 172.21.15.103:0/791991442 wait complete. 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 -- start start 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 Processor -- start 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f619d0 0x5604e5f6c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f622f0 0x5604e5f69710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f6f780 con 0x5604e5f622f0 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604e5e82f20 con 0x5604e5f619d0 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f6f900 con 0x5604e5f6cbb0 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcd374f740 10 client.0 _ll_register_callbacks cb 0x5604e5e4c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f622f0 0x5604e5f69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f622f0 0x5604e5f69710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55060/0 (socket says 172.21.15.103:55060) 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 -- 172.21.15.103:0/278409360 learned_addr learned my addr 172.21.15.103:0/278409360 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdccaffd640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f619d0 0x5604e5f6c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 -- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 msgr2=0x5604e5f69c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 -- 172.21.15.103:0/278409360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f619d0 msgr2=0x5604e5f6c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f619d0 0x5604e5f6c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdcca7fc640 1 -- 172.21.15.103:0/278409360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e5c392b0 con 0x5604e5f622f0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdccaffd640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f619d0 0x5604e5f6c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.548+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f622f0 0x5604e5f69710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc012e00 tx=0x7fdcbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70327848c225db6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcaeff5640 1 -- 172.21.15.103:0/278409360 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc02b060 con 0x5604e5f622f0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc0031a20 con 0x5604e5f622f0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcaeff5640 1 -- 172.21.15.103:0/278409360 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdcbc0263f0 con 0x5604e5f622f0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcaeff5640 1 -- 172.21.15.103:0/278409360 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc018070 con 0x5604e5f622f0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 10 client.15456 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604e5f7b060 con 0x5604e5f622f0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcaeff5640 1 -- 172.21.15.103:0/278409360 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0031a20 con 0x5604e5f622f0 2024-09-19T07:38:24.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcaeff5640 1 -- 172.21.15.103:0/278409360 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdcbc025070 con 0x5604e5f622f0 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 10 client.15456 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 10 client.15456 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 20 client.15456 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5f7d6c0 0x5604e5f7eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 4 client.15456 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.549+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604e5ed5600 con 0x5604e5f7d6c0 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.550+0000 7fdccaffd640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5f7d6c0 0x5604e5f7eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.550+0000 7fdcaeff5640 10 client.15456 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.550+0000 7fdccaffd640 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5f7d6c0 0x5604e5f7eb30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=315bef865025c64f server_cookie=950e788e7de22921 in_seq=0 out_seq=0 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcaeff5640 1 -- 172.21.15.103:0/278409360 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604e5ed5600 con 0x5604e5f7d6c0 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcaeff5640 10 client.15456 handle_command_reply: tid=0 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 client.15456 shutdown 2024-09-19T07:38:24.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f622f0 msgr2=0x5604e5f69710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f622f0 0x5604e5f69710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc012e00 tx=0x7fdcbc004730 comp rx=0 tx=0).stop 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 shutdown_connections 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5f7d6c0 0x5604e5f7eb30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f622f0 0x5604e5f69710 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcaeff5640 0 client.15456 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 --2- 172.21.15.103:0/278409360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f619d0 0x5604e5f6c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 >> 172.21.15.103:0/278409360 conn(0x5604e5f5df60 msgr2=0x5604e5f63c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 shutdown_connections 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 1 -- 172.21.15.103:0/278409360 wait complete. 2024-09-19T07:38:24.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.551+0000 7fdcd374f740 20 client.15456 trim_cache size 0 max 16384 2024-09-19T07:38:24.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 Processor -- start 2024-09-19T07:38:24.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 -- start start 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f7eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f7ce70 con 0x5604e5f619d0 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f7d540 con 0x5604e5f622f0 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcd374f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f7dc10 con 0x5604e5f6cbb0 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f7eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f7eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55062/0 (socket says 172.21.15.103:55062) 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 -- 172.21.15.103:0/1403815117 learned_addr learned my addr 172.21.15.103:0/1403815117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 -- 172.21.15.103:0/1403815117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 msgr2=0x5604e5f69c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 -- 172.21.15.103:0/1403815117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 msgr2=0x5604e5f696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccaffd640 1 -- 172.21.15.103:0/1403815117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e5f5dd90 con 0x5604e5f619d0 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.569+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:24.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdccaffd640 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f7eb30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0031550 tx=0x7fdcc0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5b033422f32c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcb0ff9640 1 -- 172.21.15.103:0/1403815117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0035050 con 0x5604e5f619d0 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcb0ff9640 1 -- 172.21.15.103:0/1403815117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0031720 con 0x5604e5f619d0 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcb0ff9640 1 -- 172.21.15.103:0/1403815117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc003f040 con 0x5604e5f619d0 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 -- 172.21.15.103:0/1403815117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 msgr2=0x5604e5f7eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f7eb30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0031550 tx=0x7fdcc0005fd0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 -- 172.21.15.103:0/1403815117 shutdown_connections 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f6cbb0 0x5604e5f69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 --2- 172.21.15.103:0/1403815117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f619d0 0x5604e5f7eb30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 -- 172.21.15.103:0/1403815117 >> 172.21.15.103:0/1403815117 conn(0x5604e5f5df60 msgr2=0x5604e5f58ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.570+0000 7fdcd374f740 1 -- 172.21.15.103:0/1403815117 shutdown_connections 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 -- 172.21.15.103:0/1403815117 wait complete. 2024-09-19T07:38:24.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 -- start start 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 Processor -- start 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f619d0 0x5604e5f87640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f6cbb0 0x5604e5f81f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f7e9b0 con 0x5604e5f6cbb0 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f7e2e0 con 0x5604e5f622f0 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604e5f7f080 con 0x5604e5f619d0 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcd374f740 10 client.0 _ll_register_callbacks cb 0x7fdcbc012ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f6cbb0 0x5604e5f81f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f6cbb0 0x5604e5f81f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55066/0 (socket says 172.21.15.103:55066) 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdccb7fe640 1 -- 172.21.15.103:0/4278131991 learned_addr learned my addr 172.21.15.103:0/4278131991 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.571+0000 7fdccaffd640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f619d0 0x5604e5f87640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdccb7fe640 1 -- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f619d0 msgr2=0x5604e5f87640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f619d0 0x5604e5f87640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdccb7fe640 1 -- 172.21.15.103:0/4278131991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 msgr2=0x5604e5f818c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f818c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdccb7fe640 1 -- 172.21.15.103:0/4278131991 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e5fe5dc0 con 0x5604e5f6cbb0 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcca7fc640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdccb7fe640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f6cbb0 0x5604e5f81f00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdcc409b610 tx=0x7fdcc409be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade356e49930cb1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcaeff5640 1 -- 172.21.15.103:0/4278131991 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40ccd40 con 0x5604e5f6cbb0 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcaeff5640 1 -- 172.21.15.103:0/4278131991 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdcc40ccee0 con 0x5604e5f6cbb0 2024-09-19T07:38:24.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc0031720 con 0x5604e5f6cbb0 2024-09-19T07:38:24.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcaeff5640 1 -- 172.21.15.103:0/4278131991 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcc40a3e60 con 0x5604e5f6cbb0 2024-09-19T07:38:24.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:24.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5604e5fe6f20 con 0x5604e5f6cbb0 2024-09-19T07:38:24.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcaeff5640 1 -- 172.21.15.103:0/4278131991 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0031720 con 0x5604e5f6cbb0 2024-09-19T07:38:24.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcaeff5640 1 -- 172.21.15.103:0/4278131991 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdcc409aad0 con 0x5604e5f6cbb0 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 10 client.15462 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 20 client.15462 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5febb40 0x5604e5fea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 4 client.15462 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.572+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "status"}) -- 0x5604e5ecfb10 con 0x5604e5febb40 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.573+0000 7fdccaffd640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5febb40 0x5604e5fea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.573+0000 7fdccaffd640 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5febb40 0x5604e5fea850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=111c4d036d9760b7 server_cookie=73450a25a627ddd2 in_seq=0 out_seq=0 2024-09-19T07:38:24.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.573+0000 7fdcaeff5640 10 client.15462 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.573+0000 7fdcaeff5640 1 -- 172.21.15.103:0/4278131991 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5604e5ecfb10 con 0x5604e5febb40 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.573+0000 7fdcaeff5640 10 client.15462 handle_command_reply: tid=0 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.573+0000 7fdcd374f740 1 client.15462 shutdown 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f6cbb0 msgr2=0x5604e5f81f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f6cbb0 0x5604e5f81f00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdcc409b610 tx=0x7fdcc409be90 comp rx=0 tx=0).stop 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 shutdown_connections 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5604e5febb40 0x5604e5fea850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604e5f6cbb0 0x5604e5f81f00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604e5f622f0 0x5604e5f818c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 --2- 172.21.15.103:0/4278131991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604e5f619d0 0x5604e5f87640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 >> 172.21.15.103:0/4278131991 conn(0x5604e5f5df60 msgr2=0x5604e5f696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcaeff5640 0 client.15462 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 shutdown_connections 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 1 -- 172.21.15.103:0/4278131991 wait complete. 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd374f740 20 client.15462 trim_cache size 0 max 16384 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 3, 2024-09-19T07:38:24.584 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24454, 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 126.81730457899999, 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 85, 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 159.73446829900001 2024-09-19T07:38:24.585 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdcb40a3440 msgr2=0x7fdcb40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdcb40a3440 0x7fdcb40a5900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0058c0 tx=0x7fdcbc00a770 comp rx=0 tx=0).stop 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 msgr2=0x7fdccc19bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc19bbd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400b970 tx=0x7fdcc400bd70 comp rx=0 tx=0).stop 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 shutdown_connections 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdcb40a3440 0x7fdcb40a5900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdccc106a80 0x7fdccc19bbd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdccc101420 0x7fdccc197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 --2- 172.21.15.103:0/1835914230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdccc100a80 0x7fdccc197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:24.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.574+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 >> 172.21.15.103:0/1835914230 conn(0x7fdccc0fc7a0 msgr2=0x7fdccc0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:24.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.575+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 shutdown_connections 2024-09-19T07:38:24.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:24.575+0000 7fdcd1376640 1 -- 172.21.15.103:0/1835914230 wait complete. 2024-09-19T07:38:25.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T07:38:25.639 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:25 smithi134 ceph-mon[35414]: pgmap v16: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 82 KiB/s wr, 16 op/s 2024-09-19T07:38:25.640 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:25.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:25 smithi174 ceph-mon[35370]: pgmap v16: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 82 KiB/s wr, 16 op/s 2024-09-19T07:38:25.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:25 smithi103 ceph-mon[31201]: pgmap v16: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 82 KiB/s wr, 16 op/s 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 -- 172.21.15.103:0/2767344138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c106f10 msgr2=0x7ffb6c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 --2- 172.21.15.103:0/2767344138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c106f10 0x7ffb6c10ea90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009ca0 tx=0x7ffb6402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 -- 172.21.15.103:0/2767344138 shutdown_connections 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 --2- 172.21.15.103:0/2767344138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c106f10 0x7ffb6c10ea90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 --2- 172.21.15.103:0/2767344138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb6c102790 0x7ffb6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 --2- 172.21.15.103:0/2767344138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c101de0 0x7ffb6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 -- 172.21.15.103:0/2767344138 >> 172.21.15.103:0/2767344138 conn(0x7ffb6c0fd8c0 msgr2=0x7ffb6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 -- 172.21.15.103:0/2767344138 shutdown_connections 2024-09-19T07:38:26.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.109+0000 7ffb714d7640 1 -- 172.21.15.103:0/2767344138 wait complete. 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 Processor -- start 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 -- start start 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c101de0 0x7ffb6c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb6c102790 0x7ffb6c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c106f10 0x7ffb6c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c111f90 con 0x7ffb6c101de0 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c111e10 con 0x7ffb6c106f10 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb714d7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c112110 con 0x7ffb6c102790 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c101de0 0x7ffb6c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c101de0 0x7ffb6c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55088/0 (socket says 172.21.15.103:55088) 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb6affd640 1 -- 172.21.15.103:0/3712546622 learned_addr learned my addr 172.21.15.103:0/3712546622 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:26.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.110+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c106f10 0x7ffb6c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb6c102790 0x7ffb6c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6affd640 1 -- 172.21.15.103:0/3712546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb6c102790 msgr2=0x7ffb6c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6affd640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb6c102790 0x7ffb6c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6affd640 1 -- 172.21.15.103:0/3712546622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c106f10 msgr2=0x7ffb6c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6affd640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c106f10 0x7ffb6c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6affd640 1 -- 172.21.15.103:0/3712546622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c19d400 con 0x7ffb6c101de0 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb6c102790 0x7ffb6c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb6affd640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c101de0 0x7ffb6c198450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9f0 tx=0x7ffb5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb53fff640 1 -- 172.21.15.103:0/3712546622 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00cd80 con 0x7ffb6c101de0 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb53fff640 1 -- 172.21.15.103:0/3712546622 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c010910 con 0x7ffb6c101de0 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c19d6f0 con 0x7ffb6c101de0 2024-09-19T07:38:26.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb53fff640 1 -- 172.21.15.103:0/3712546622 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c018990 con 0x7ffb6c101de0 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.111+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c1a4fd0 con 0x7ffb6c101de0 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.113+0000 7ffb53fff640 1 -- 172.21.15.103:0/3712546622 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7ffb5c010ac0 con 0x7ffb6c101de0 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.113+0000 7ffb738b0740 1 Processor -- start 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb738b0740 1 -- start start 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b9721d9d0 0x555b9721ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97228670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b97228bb0 0x555b9722afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb738b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b97184090 con 0x555b9721d9d0 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb738b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555b9712ba70 con 0x555b9721e2f0 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb738b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b97180050 con 0x555b97228bb0 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b9721d9d0 0x555b9721ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b9721d9d0 0x555b9721ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55094/0 (socket says 172.21.15.103:55094) 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1530149439 learned_addr learned my addr 172.21.15.103:0/1530149439 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6affd640 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97228670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b97228bb0 0x555b9722afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb53fff640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb540a3590 0x7ffb540a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1530149439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b97228bb0 msgr2=0x555b9722afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b97228bb0 0x555b9722afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1530149439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 msgr2=0x555b97228670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97228670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb53fff640 1 -- 172.21.15.103:0/3712546622 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ffb5c0d00d0 con 0x7ffb6c101de0 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1530149439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b97138130 con 0x555b9721d9d0 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.114+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb540a3590 0x7ffb540a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.115+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b9721d9d0 0x555b9721ddb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009ca0 tx=0x7ffb64009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ae9a8e079564b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.115+0000 7ffb517fa640 1 -- 172.21.15.103:0/1530149439 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64042040 con 0x555b9721d9d0 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.115+0000 7ffb517fa640 1 -- 172.21.15.103:0/1530149439 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb64036040 con 0x555b9721d9d0 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.115+0000 7ffb517fa640 1 -- 172.21.15.103:0/1530149439 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6402fa30 con 0x555b9721d9d0 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.115+0000 7ffb738b0740 1 -- 172.21.15.103:0/1530149439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b9721d9d0 msgr2=0x555b9721ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.115+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb540a3590 0x7ffb540a5a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffb6404d000 tx=0x7ffb6404e000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.115+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b9721d9d0 0x555b9721ddb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009ca0 tx=0x7ffb64009990 comp rx=0 tx=0).stop 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- 172.21.15.103:0/1530149439 shutdown_connections 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b97228bb0 0x555b9722afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97228670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1530149439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b9721d9d0 0x555b9721ddb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- 172.21.15.103:0/1530149439 >> 172.21.15.103:0/1530149439 conn(0x555b97219f60 msgr2=0x555b9721a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- 172.21.15.103:0/1530149439 shutdown_connections 2024-09-19T07:38:26.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- 172.21.15.103:0/1530149439 wait complete. 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- start start 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 Processor -- start 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b97228420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97225770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b97225cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b9722b780 con 0x555b97228bb0 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555b9713ef20 con 0x555b9721e2f0 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b9722b900 con 0x555b9721d9d0 2024-09-19T07:38:26.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb738b0740 10 client.0 _ll_register_callbacks cb 0x555b97108030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b97225cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb6a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97225770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b97225cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55100/0 (socket says 172.21.15.103:55100) 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.116+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1182159257 learned_addr learned my addr 172.21.15.103:0/1182159257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6affd640 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b97228420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1182159257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 msgr2=0x555b97228420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b97228420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 msgr2=0x555b97225770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97225770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/1182159257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b96ef52b0 con 0x555b97228bb0 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97225770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b97225cb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb6402ade0 tx=0x7ffb6402f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3ed109c524a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb4eff5640 1 -- 172.21.15.103:0/1182159257 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64042050 con 0x555b97228bb0 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb4eff5640 1 -- 172.21.15.103:0/1182159257 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ffb6402fa30 con 0x555b97228bb0 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb64036040 con 0x555b97228bb0 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb4eff5640 1 -- 172.21.15.103:0/1182159257 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb64004000 con 0x555b97228bb0 2024-09-19T07:38:26.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb738b0740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b9722b780 con 0x555b97228bb0 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb4eff5640 1 -- 172.21.15.103:0/1182159257 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb64036040 con 0x555b97228bb0 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.117+0000 7ffb4eff5640 1 -- 172.21.15.103:0/1182159257 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ffb640309e0 con 0x555b97228bb0 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb738b0740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb738b0740 10 client.15474 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb738b0740 20 client.15474 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b97239370 0x555b9723a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb738b0740 4 client.15474 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b97191600 con 0x555b97239370 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb6affd640 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b97239370 0x555b9723a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb4eff5640 10 client.15474 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.118+0000 7ffb6affd640 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b97239370 0x555b9723a7e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9634cccf7013c819 server_cookie=9430c52bcb00e645 in_seq=0 out_seq=0 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb4eff5640 1 -- 172.21.15.103:0/1182159257 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b97191600 con 0x555b97239370 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb4eff5640 10 client.15474 handle_command_reply: tid=0 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 client.15474 shutdown 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 msgr2=0x555b97225cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b97225cb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb6402ade0 tx=0x7ffb6402f750 comp rx=0 tx=0).stop 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 shutdown_connections 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b97239370 0x555b9723a7e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b97225cb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b97225770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 --2- 172.21.15.103:0/1182159257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b97228420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 >> 172.21.15.103:0/1182159257 conn(0x555b97219f60 msgr2=0x555b9721fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb4eff5640 0 client.15474 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 shutdown_connections 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.119+0000 7ffb738b0740 1 -- 172.21.15.103:0/1182159257 wait complete. 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.120+0000 7ffb738b0740 20 client.15474 trim_cache size 0 max 16384 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 Processor -- start 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 -- start start 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721d9d0 0x555b97239a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721e2f0 0x555b97214ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b972275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b97238b20 con 0x555b97228bb0 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555b972391f0 con 0x555b9721d9d0 2024-09-19T07:38:26.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb738b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b972398c0 con 0x555b9721e2f0 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b972275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b972275d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55112/0 (socket says 172.21.15.103:55112) 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/2588663109 learned_addr learned my addr 172.21.15.103:0/2588663109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6affd640 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721d9d0 0x555b97239a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721e2f0 0x555b97214ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/2588663109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721e2f0 msgr2=0x555b97214ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721e2f0 0x555b97214ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/2588663109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721d9d0 msgr2=0x555b97239a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721d9d0 0x555b97239a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.137+0000 7ffb6b7fe640 1 -- 172.21.15.103:0/2588663109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b97224740 con 0x555b97228bb0 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b972275d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffb64030e50 tx=0x7ffb64009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a47e26881a39a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:26.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb50ff9640 1 -- 172.21.15.103:0/2588663109 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64042040 con 0x555b97228bb0 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb50ff9640 1 -- 172.21.15.103:0/2588663109 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb64004840 con 0x555b97228bb0 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb50ff9640 1 -- 172.21.15.103:0/2588663109 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64030820 con 0x555b97228bb0 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 -- 172.21.15.103:0/2588663109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 msgr2=0x555b972275d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b972275d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffb64030e50 tx=0x7ffb64009b20 comp rx=0 tx=0).stop 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 -- 172.21.15.103:0/2588663109 shutdown_connections 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b972275d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721e2f0 0x555b97214ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 --2- 172.21.15.103:0/2588663109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721d9d0 0x555b97239a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 -- 172.21.15.103:0/2588663109 >> 172.21.15.103:0/2588663109 conn(0x555b97219f60 msgr2=0x555b97226ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.138+0000 7ffb738b0740 1 -- 172.21.15.103:0/2588663109 shutdown_connections 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 -- 172.21.15.103:0/2588663109 wait complete. 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 -- start start 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 Processor -- start 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b9723da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b9723dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b9723e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b9723a660 con 0x555b97228bb0 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555b97239f90 con 0x555b9721e2f0 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b9723ad30 con 0x555b9721d9d0 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb738b0740 10 client.0 _ll_register_callbacks cb 0x7ffb5c0cd6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb6a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b9723dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb6a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b9723dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45926/0 (socket says 172.21.15.103:45926) 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb6a7fc640 1 -- 172.21.15.103:0/3238233821 learned_addr learned my addr 172.21.15.103:0/3238233821 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb6b7fe640 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b9723e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.139+0000 7ffb6affd640 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b9723da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb6a7fc640 1 -- 172.21.15.103:0/3238233821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 msgr2=0x555b9723da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b9723da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb6a7fc640 1 -- 172.21.15.103:0/3238233821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 msgr2=0x555b9723e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b9723e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb6a7fc640 1 -- 172.21.15.103:0/3238233821 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b972a1c10 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb6a7fc640 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b9723dfa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0fdce0 tx=0x7ffb60014dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105332d04b0db42b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb4eff5640 1 -- 172.21.15.103:0/3238233821 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60012bc0 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b9718bb10 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb4eff5640 1 -- 172.21.15.103:0/3238233821 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ffb60012d60 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb4eff5640 1 -- 172.21.15.103:0/3238233821 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffb60027450 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb738b0740 10 client.25073 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b972a2fb0 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb4eff5640 1 -- 172.21.15.103:0/3238233821 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555b9718bb10 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.140+0000 7ffb4eff5640 1 -- 172.21.15.103:0/3238233821 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ffb600165f0 con 0x555b9721e2f0 2024-09-19T07:38:26.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 10 client.25073 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 10 client.25073 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 20 client.25073 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b972a7bd0 0x555b972a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 4 client.25073 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "status"}) -- 0x555b971080e0 con 0x555b972a7bd0 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb6affd640 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b972a7bd0 0x555b972a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb6affd640 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b972a7bd0 0x555b972a68e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51814ada38ae8b37 server_cookie=d6c0c8f7ba946c76 in_seq=0 out_seq=0 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb4eff5640 10 client.25073 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb4eff5640 1 -- 172.21.15.103:0/3238233821 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555b971080e0 con 0x555b972a7bd0 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb4eff5640 10 client.25073 handle_command_reply: tid=0 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 client.25073 shutdown 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 msgr2=0x555b9723dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b9723dfa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0fdce0 tx=0x7ffb60014dd0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 shutdown_connections 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x555b972a7bd0 0x555b972a68e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b97228bb0 0x555b9723e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b9721e2f0 0x555b9723dfa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb738b0740 1 --2- 172.21.15.103:0/3238233821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b9721d9d0 0x555b9723da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.141+0000 7ffb4eff5640 0 client.25073 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 >> 172.21.15.103:0/3238233821 conn(0x555b97219f60 msgr2=0x555b97225850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 shutdown_connections 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb738b0740 1 -- 172.21.15.103:0/3238233821 wait complete. 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb738b0740 20 client.25073 trim_cache size 0 max 16384 2024-09-19T07:38:26.163 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24455, 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 129.391660186, 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 85, 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 158.67911331799999 2024-09-19T07:38:26.164 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb540a3590 msgr2=0x7ffb540a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb540a3590 0x7ffb540a5a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffb6404d000 tx=0x7ffb6404e000 comp rx=0 tx=0).stop 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c101de0 msgr2=0x7ffb6c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c101de0 0x7ffb6c198450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9f0 tx=0x7ffb5c00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 shutdown_connections 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb540a3590 0x7ffb540a5a50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c106f10 0x7ffb6c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb6c102790 0x7ffb6c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 --2- 172.21.15.103:0/3712546622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb6c101de0 0x7ffb6c198450 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:26.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 >> 172.21.15.103:0/3712546622 conn(0x7ffb6c0fd8c0 msgr2=0x7ffb6c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:26.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 shutdown_connections 2024-09-19T07:38:26.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:26.142+0000 7ffb714d7640 1 -- 172.21.15.103:0/3712546622 wait complete. 2024-09-19T07:38:26.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-19T07:38:27.058 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:27.448 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:27 smithi103 ceph-mon[31201]: pgmap v17: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-09-19T07:38:27.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 -- 172.21.15.103:0/1349505194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4101de0 msgr2=0x7f6ec41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/1349505194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4101de0 0x7f6ec41021c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 -- 172.21.15.103:0/1349505194 shutdown_connections 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/1349505194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec4106f10 0x7f6ec410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/1349505194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec4102790 0x7f6ec41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/1349505194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4101de0 0x7f6ec41021c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 -- 172.21.15.103:0/1349505194 >> 172.21.15.103:0/1349505194 conn(0x7f6ec40fd8c0 msgr2=0x7f6ec40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 -- 172.21.15.103:0/1349505194 shutdown_connections 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 -- 172.21.15.103:0/1349505194 wait complete. 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.528+0000 7f6ec9be5640 1 Processor -- start 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec9be5640 1 -- start start 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec9be5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec4101de0 0x7f6ec41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec9be5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec4102790 0x7f6ec4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec9be5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4106f10 0x7f6ec419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec9be5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec4111f70 con 0x7f6ec4106f10 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec9be5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec4111df0 con 0x7f6ec4101de0 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec9be5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41120f0 con 0x7f6ec4102790 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4106f10 0x7f6ec419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4106f10 0x7f6ec419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55128/0 (socket says 172.21.15.103:55128) 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 -- 172.21.15.103:0/725347643 learned_addr learned my addr 172.21.15.103:0/725347643 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec4102790 0x7f6ec4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec4101de0 0x7f6ec41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 -- 172.21.15.103:0/725347643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec4102790 msgr2=0x7f6ec4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec4102790 0x7f6ec4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 -- 172.21.15.103:0/725347643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec4101de0 msgr2=0x7f6ec41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec4101de0 0x7f6ec41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 -- 172.21.15.103:0/725347643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec419d380 con 0x7f6ec4106f10 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec4101de0 0x7f6ec41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:27.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.529+0000 7f6ec3fff640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4106f10 0x7f6ec419cca0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c970 tx=0x7f6ebc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.530+0000 7f6ec0ff9640 1 -- 172.21.15.103:0/725347643 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc007bb0 con 0x7f6ec4106f10 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.530+0000 7f6ec0ff9640 1 -- 172.21.15.103:0/725347643 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc007d50 con 0x7f6ec4106f10 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.530+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec419d670 con 0x7f6ec4106f10 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.530+0000 7f6ec0ff9640 1 -- 172.21.15.103:0/725347643 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc018800 con 0x7f6ec4106f10 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.530+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec41a4f50 con 0x7f6ec4106f10 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.532+0000 7f6ec0ff9640 1 -- 172.21.15.103:0/725347643 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6ebc00fa50 con 0x7f6ec4106f10 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 Processor -- start 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 -- start start 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b2dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b2eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c0b245090 con 0x561c0b2de8b0 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c0b1eca70 con 0x561c0b2edfc0 2024-09-19T07:38:27.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c0b241050 con 0x561c0b2df1d0 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ec0ff9640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6eac0a3440 0x7f6eac0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ec2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b2dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ec2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b2dec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55140/0 (socket says 172.21.15.103:55140) 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/393952569 learned_addr learned my addr 172.21.15.103:0/393952569 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ec3fff640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6eac0a3440 0x7f6eac0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.533+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b2eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/393952569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 msgr2=0x561c0b2eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec0ff9640 1 -- 172.21.15.103:0/725347643 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6ebc0cff40 con 0x7f6ec4106f10 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b2eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/393952569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 msgr2=0x561c0b2f03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2f03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/393952569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0b1f9130 con 0x561c0b2de8b0 2024-09-19T07:38:27.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b2dec90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800cfc0 tx=0x7f6eb800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1642f6b23398ff96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ec3fff640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6eac0a3440 0x7f6eac0a5900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cd640 tx=0x7f6ebc01baf0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ea9ffb640 1 -- 172.21.15.103:0/393952569 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8018030 con 0x561c0b2de8b0 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ea9ffb640 1 -- 172.21.15.103:0/393952569 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb8005500 con 0x561c0b2de8b0 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.534+0000 7f6ea9ffb640 1 -- 172.21.15.103:0/393952569 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800d7b0 con 0x561c0b2de8b0 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/393952569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 msgr2=0x561c0b2dec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b2dec90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800cfc0 tx=0x7f6eb800c5a0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/393952569 shutdown_connections 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b2eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/393952569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b2dec90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/393952569 >> 172.21.15.103:0/393952569 conn(0x561c0b2daf60 msgr2=0x561c0b2db340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:27.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/393952569 shutdown_connections 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/393952569 wait complete. 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- start start 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 Processor -- start 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2de8b0 0x561c0b2ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c0b2f0b90 con 0x561c0b2df1d0 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c0b1fff20 con 0x561c0b2edfc0 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.535+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c0b2f0d10 con 0x561c0b2de8b0 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ecbfbe740 10 client.0 _ll_register_callbacks cb 0x561c0b1c9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2eab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55142/0 (socket says 172.21.15.103:55142) 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3285402175 learned_addr learned my addr 172.21.15.103:0/3285402175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2de8b0 0x561c0b2ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2de8b0 msgr2=0x561c0b2ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2de8b0 0x561c0b2ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3285402175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 msgr2=0x561c0b2eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3285402175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0afb62b0 con 0x561c0b2df1d0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2eab50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4035000 tx=0x7f6eb40039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f7baa308211b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/3285402175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4031650 con 0x561c0b2df1d0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/3285402175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6eb40317f0 con 0x561c0b2df1d0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb8005500 con 0x561c0b2df1d0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/3285402175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6eb4004040 con 0x561c0b2df1d0 2024-09-19T07:38:27.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ecbfbe740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.536+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c0b2fc490 con 0x561c0b2df1d0 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/3285402175 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb8005500 con 0x561c0b2df1d0 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/3285402175 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6eb4043070 con 0x561c0b2df1d0 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ecbfbe740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ecbfbe740 10 client.15492 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ecbfbe740 20 client.15492 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b2feaf0 0x561c0b2fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ecbfbe740 4 client.15492 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c0b252600 con 0x561c0b2feaf0 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.537+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b2feaf0 0x561c0b2fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.538+0000 7f6ea2ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:27.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.538+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b2feaf0 0x561c0b2fff60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=674a8c8a817931bb server_cookie=bb41a2ec375aeb22 in_seq=0 out_seq=0 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.538+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/3285402175 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c0b252600 con 0x561c0b2feaf0 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.538+0000 7f6ea2ffd640 10 client.15492 handle_command_reply: tid=0 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.538+0000 7f6ecbfbe740 1 client.15492 shutdown 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 msgr2=0x561c0b2eab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2eab50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4035000 tx=0x7f6eb40039f0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 shutdown_connections 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b2feaf0 0x561c0b2fff60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b2eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2eab50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ea2ffd640 0 client.15492 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3285402175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2de8b0 0x561c0b2ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 >> 172.21.15.103:0/3285402175 conn(0x561c0b2daf60 msgr2=0x561c0b2e0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:27.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 shutdown_connections 2024-09-19T07:38:27.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3285402175 wait complete. 2024-09-19T07:38:27.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.539+0000 7f6ecbfbe740 20 client.15492 trim_cache size 0 max 16384 2024-09-19T07:38:27.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.556+0000 7f6ecbfbe740 1 Processor -- start 2024-09-19T07:38:27.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.556+0000 7f6ecbfbe740 1 -- start start 2024-09-19T07:38:27.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2de8b0 0x561c0b2fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2edfc0 0x561c0b2e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c0b2fe2a0 con 0x561c0b2df1d0 2024-09-19T07:38:27.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c0b2fe970 con 0x561c0b2de8b0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c0b2ff040 con 0x561c0b2edfc0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2e6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55158/0 (socket says 172.21.15.103:55158) 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3060346135 learned_addr learned my addr 172.21.15.103:0/3060346135 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2de8b0 0x561c0b2fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3060346135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2edfc0 msgr2=0x561c0b2e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2edfc0 0x561c0b2e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2edfc0 0x561c0b2e7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3060346135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2de8b0 msgr2=0x561c0b2fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2de8b0 0x561c0b2fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 -- 172.21.15.103:0/3060346135 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0b2df000 con 0x561c0b2df1d0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2e6130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402fa20 tx=0x7f6eb402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe15a4e685dabea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ea97fa640 1 -- 172.21.15.103:0/3060346135 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb402f610 con 0x561c0b2df1d0 2024-09-19T07:38:27.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ea97fa640 1 -- 172.21.15.103:0/3060346135 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb4036040 con 0x561c0b2df1d0 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.557+0000 7f6ea97fa640 1 -- 172.21.15.103:0/3060346135 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4004460 con 0x561c0b2df1d0 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3060346135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 msgr2=0x561c0b2e6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2e6130 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402fa20 tx=0x7f6eb402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3060346135 shutdown_connections 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2edfc0 0x561c0b2e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2df1d0 0x561c0b2e6130 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/3060346135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2de8b0 0x561c0b2fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3060346135 >> 172.21.15.103:0/3060346135 conn(0x561c0b2daf60 msgr2=0x561c0b2e5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3060346135 shutdown_connections 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/3060346135 wait complete. 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.558+0000 7f6ecbfbe740 1 -- start start 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 1 Processor -- start 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b3084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b308a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b3018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c0b2ffde0 con 0x561c0b2de8b0 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c0b2ff710 con 0x561c0b2edfc0 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c0b3004b0 con 0x561c0b2df1d0 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ecbfbe740 10 client.0 _ll_register_callbacks cb 0x7f6eb4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b3084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b3084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55160/0 (socket says 172.21.15.103:55160) 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/1737199239 learned_addr learned my addr 172.21.15.103:0/1737199239 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b308a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b3018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 msgr2=0x561c0b308a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b308a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/1737199239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 msgr2=0x561c0b3018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:27.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b3018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec2ffd640 1 -- 172.21.15.103:0/1737199239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0b3020d0 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.559+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b308a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ec2ffd640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b3084f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8012de0 tx=0x7f6eb800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792852c12802b4f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/1737199239 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8018040 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/1737199239 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6eb80047c0 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb4036040 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/1737199239 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6eb8038050 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ecbfbe740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c0b306bb0 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/1737199239 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb4036040 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/1737199239 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6eb8005240 con 0x561c0b2de8b0 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ecbfbe740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ecbfbe740 10 client.15498 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-19T07:38:27.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.560+0000 7f6ecbfbe740 20 client.15498 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b36d7e0 0x561c0b36c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ecbfbe740 4 client.15498 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "status"}) -- 0x561c0b24cb10 con 0x561c0b36d7e0 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b36d7e0 0x561c0b36c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ea2ffd640 10 client.15498 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ec37fe640 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b36d7e0 0x561c0b36c4f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d797b6aaf46d469 server_cookie=789ac3c48eebe841 in_seq=0 out_seq=0 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ea2ffd640 1 -- 172.21.15.103:0/1737199239 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561c0b24cb10 con 0x561c0b36d7e0 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ea2ffd640 10 client.15498 handle_command_reply: tid=0 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.561+0000 7f6ecbfbe740 1 client.15498 shutdown 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 msgr2=0x561c0b3084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b3084f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8012de0 tx=0x7f6eb800ce60 comp rx=0 tx=0).stop 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 shutdown_connections 2024-09-19T07:38:27.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x561c0b36d7e0 0x561c0b36c4f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c0b2edfc0 0x561c0b3018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c0b2df1d0 0x561c0b308a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 --2- 172.21.15.103:0/1737199239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c0b2de8b0 0x561c0b3084f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 >> 172.21.15.103:0/1737199239 conn(0x561c0b2daf60 msgr2=0x561c0b2ec100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ea2ffd640 0 client.15498 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 shutdown_connections 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 1 -- 172.21.15.103:0/1737199239 wait complete. 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ecbfbe740 20 client.15498 trim_cache size 0 max 16384 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:27.569 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24466, 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 131.818911837, 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 85, 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 157.61338252600001 2024-09-19T07:38:27.570 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6eac0a3440 msgr2=0x7f6eac0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6eac0a3440 0x7f6eac0a5900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cd640 tx=0x7f6ebc01baf0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4106f10 msgr2=0x7f6ec419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4106f10 0x7f6ec419cca0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c970 tx=0x7f6ebc00ce30 comp rx=0 tx=0).stop 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 shutdown_connections 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6eac0a3440 0x7f6eac0a5900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec4106f10 0x7f6ec419cca0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec4102790 0x7f6ec4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 --2- 172.21.15.103:0/725347643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec4101de0 0x7f6ec41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:27.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.562+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 >> 172.21.15.103:0/725347643 conn(0x7f6ec40fd8c0 msgr2=0x7f6ec40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:27.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.563+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 shutdown_connections 2024-09-19T07:38:27.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:27.563+0000 7f6ec9be5640 1 -- 172.21.15.103:0/725347643 wait complete. 2024-09-19T07:38:27.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:27 smithi174 ceph-mon[35370]: pgmap v17: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-09-19T07:38:27.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:27 smithi134 ceph-mon[35414]: pgmap v17: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-09-19T07:38:28.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T07:38:28.525 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:28.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 -- 172.21.15.103:0/2211210232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c106a80 msgr2=0x7f336c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 --2- 172.21.15.103:0/2211210232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c106a80 0x7f336c106e60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3360009930 tx=0x7f336002a420 comp rx=0 tx=0).stop 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 -- 172.21.15.103:0/2211210232 shutdown_connections 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 --2- 172.21.15.103:0/2211210232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f336c101420 0x7f336c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 --2- 172.21.15.103:0/2211210232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f336c100a80 0x7f336c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 --2- 172.21.15.103:0/2211210232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c106a80 0x7f336c106e60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 -- 172.21.15.103:0/2211210232 >> 172.21.15.103:0/2211210232 conn(0x7f336c0fc7a0 msgr2=0x7f336c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 -- 172.21.15.103:0/2211210232 shutdown_connections 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 -- 172.21.15.103:0/2211210232 wait complete. 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.964+0000 7f337429a640 1 Processor -- start 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f337429a640 1 -- start start 2024-09-19T07:38:28.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f337429a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f336c100a80 0x7f336c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f337429a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c101420 0x7f336c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f337429a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f336c106a80 0x7f336c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f337429a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f336c110e90 con 0x7f336c101420 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f337429a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f336c110d10 con 0x7f336c100a80 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f337429a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f336c111010 con 0x7f336c106a80 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c101420 0x7f336c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c101420 0x7f336c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56236/0 (socket says 172.21.15.103:56236) 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 -- 172.21.15.103:0/2604553345 learned_addr learned my addr 172.21.15.103:0/2604553345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3372036640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f336c100a80 0x7f336c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3372837640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f336c106a80 0x7f336c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 -- 172.21.15.103:0/2604553345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f336c106a80 msgr2=0x7f336c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:28.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f336c106a80 0x7f336c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 -- 172.21.15.103:0/2604553345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f336c100a80 msgr2=0x7f336c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f336c100a80 0x7f336c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 -- 172.21.15.103:0/2604553345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c19d410 con 0x7f336c101420 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.965+0000 7f3371835640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c101420 0x7f336c1989a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f336800e9f0 tx=0x7f336800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.966+0000 7f335f7fe640 1 -- 172.21.15.103:0/2604553345 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336800cd80 con 0x7f336c101420 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.966+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c19d700 con 0x7f336c101420 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.966+0000 7f335f7fe640 1 -- 172.21.15.103:0/2604553345 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3368010910 con 0x7f336c101420 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.966+0000 7f335f7fe640 1 -- 172.21.15.103:0/2604553345 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680188f0 con 0x7f336c101420 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.966+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c1a4fe0 con 0x7f336c101420 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.968+0000 7f335f7fe640 1 -- 172.21.15.103:0/2604553345 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f3368010ac0 con 0x7f336c101420 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.968+0000 7f3376673740 1 Processor -- start 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.968+0000 7f3376673740 1 -- start start 2024-09-19T07:38:28.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3376673740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005379d0 0x560200537db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3376673740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005382f0 0x560200542670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3376673740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x560200544fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3376673740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56020049e090 con 0x5602005379d0 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3376673740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560200445a70 con 0x560200542bb0 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3376673740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56020049a050 con 0x5602005382f0 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372837640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005379d0 0x560200537db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005382f0 0x560200542670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005382f0 0x560200542670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56852/0 (socket says 172.21.15.103:56852) 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372837640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005379d0 0x560200537db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56246/0 (socket says 172.21.15.103:56246) 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 -- 172.21.15.103:0/966850990 learned_addr learned my addr 172.21.15.103:0/966850990 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3371835640 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x560200544fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 -- 172.21.15.103:0/966850990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 msgr2=0x560200544fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x560200544fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 -- 172.21.15.103:0/966850990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005379d0 msgr2=0x560200537db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005379d0 0x560200537db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 -- 172.21.15.103:0/966850990 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560200452130 con 0x5602005382f0 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3371835640 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x560200544fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372837640 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005379d0 0x560200537db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f335f7fe640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33580a3440 0x7f33580a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372837640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33580a3440 0x7f33580a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.969+0000 7f3372036640 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005382f0 0x560200542670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3360009930 tx=0x7f3360005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f55a28bc117f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.970+0000 7f335f7fe640 1 -- 172.21.15.103:0/2604553345 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f33680d0270 con 0x7f336c101420 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.970+0000 7f335cdea640 1 -- 172.21.15.103:0/966850990 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360035040 con 0x5602005382f0 2024-09-19T07:38:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.970+0000 7f335cdea640 1 -- 172.21.15.103:0/966850990 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3360005520 con 0x5602005382f0 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.970+0000 7f335cdea640 1 -- 172.21.15.103:0/966850990 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33600035f0 con 0x5602005382f0 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.970+0000 7f3376673740 1 -- 172.21.15.103:0/966850990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005382f0 msgr2=0x560200542670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.970+0000 7f3376673740 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005382f0 0x560200542670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3360009930 tx=0x7f3360005de0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.970+0000 7f3372837640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33580a3440 0x7f33580a5900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3364009fd0 tx=0x7f3364009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- 172.21.15.103:0/966850990 shutdown_connections 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x560200544fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005382f0 0x560200542670 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 --2- 172.21.15.103:0/966850990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005379d0 0x560200537db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- 172.21.15.103:0/966850990 >> 172.21.15.103:0/966850990 conn(0x560200533f60 msgr2=0x560200534340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- 172.21.15.103:0/966850990 shutdown_connections 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- 172.21.15.103:0/966850990 wait complete. 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- start start 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 Processor -- start 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x560200542430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x56020053fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560200545780 con 0x5602005382f0 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560200458f20 con 0x560200542bb0 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560200545900 con 0x5602005379d0 2024-09-19T07:38:28.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3376673740 10 client.0 _ll_register_callbacks cb 0x560200422030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3371835640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3372036640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x560200542430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3371835640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56258/0 (socket says 172.21.15.103:56258) 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3371835640 1 -- 172.21.15.103:0/2271890656 learned_addr learned my addr 172.21.15.103:0/2271890656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.971+0000 7f3372837640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x56020053fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3372036640 1 -- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 msgr2=0x56020053fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3372036640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x56020053fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3372036640 1 -- 172.21.15.103:0/2271890656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 msgr2=0x56020053f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3372036640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3372036640 1 -- 172.21.15.103:0/2271890656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56020020f2b0 con 0x5602005379d0 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3372837640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x56020053fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3371835640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3372036640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x560200542430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f336002a940 tx=0x7f3360004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da1d8ee82cc7924 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3355ffb640 1 -- 172.21.15.103:0/2271890656 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360052060 con 0x5602005379d0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3360005520 con 0x5602005379d0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3355ffb640 1 -- 172.21.15.103:0/2271890656 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3360004390 con 0x5602005379d0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3355ffb640 1 -- 172.21.15.103:0/2271890656 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3360035070 con 0x5602005379d0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3376673740 10 client.25081 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.972+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560200545780 con 0x5602005379d0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3355ffb640 1 -- 172.21.15.103:0/2271890656 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3360005520 con 0x5602005379d0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3355ffb640 1 -- 172.21.15.103:0/2271890656 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f33600039b0 con 0x5602005379d0 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3376673740 10 client.25081 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3376673740 10 client.25081 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3376673740 20 client.25081 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3376673740 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560200553220 0x560200554690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3376673740 4 client.25081 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602004ab600 con 0x560200553220 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3371835640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560200553220 0x560200554690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3371835640 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560200553220 0x560200554690 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9880a9cfec4d801 server_cookie=a8baf97873b78099 in_seq=0 out_seq=0 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.973+0000 7f3355ffb640 10 client.25081 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3355ffb640 1 -- 172.21.15.103:0/2271890656 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602004ab600 con 0x560200553220 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3355ffb640 10 client.25081 handle_command_reply: tid=0 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 client.25081 shutdown 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 msgr2=0x560200542430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x560200542430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f336002a940 tx=0x7f3360004a80 comp rx=0 tx=0).stop 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 shutdown_connections 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560200553220 0x560200554690 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560200542bb0 0x56020053fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3355ffb640 0 client.25081 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 --2- 172.21.15.103:0/2271890656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x560200542430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 >> 172.21.15.103:0/2271890656 conn(0x560200533f60 msgr2=0x560200539c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 shutdown_connections 2024-09-19T07:38:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.977+0000 7f3376673740 1 -- 172.21.15.103:0/2271890656 wait complete. 2024-09-19T07:38:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.978+0000 7f3376673740 20 client.25081 trim_cache size 0 max 16384 2024-09-19T07:38:29.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 Processor -- start 2024-09-19T07:38:29.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 -- start start 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005379d0 0x56020053c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560200542bb0 0x56020053a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602005529d0 con 0x5602005382f0 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5602005530a0 con 0x5602005379d0 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3376673740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560200553770 con 0x560200542bb0 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3371835640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3371835640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56266/0 (socket says 172.21.15.103:56266) 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3371835640 1 -- 172.21.15.103:0/1829547475 learned_addr learned my addr 172.21.15.103:0/1829547475 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3372837640 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560200542bb0 0x56020053a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.996+0000 7f3372036640 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005379d0 0x56020053c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:29.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3371835640 1 -- 172.21.15.103:0/1829547475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560200542bb0 msgr2=0x56020053a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3371835640 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560200542bb0 0x56020053a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3371835640 1 -- 172.21.15.103:0/1829547475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005379d0 msgr2=0x56020053c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3371835640 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005379d0 0x56020053c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3371835640 1 -- 172.21.15.103:0/1829547475 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560200533d90 con 0x5602005382f0 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3372036640 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005379d0 0x56020053c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3371835640 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053cf20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f336809c1a0 tx=0x7f336801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff0e43f006edb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3357fff640 1 -- 172.21.15.103:0/1829547475 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336809be00 con 0x5602005382f0 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3357fff640 1 -- 172.21.15.103:0/1829547475 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336809c4a0 con 0x5602005382f0 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3357fff640 1 -- 172.21.15.103:0/1829547475 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680cdbc0 con 0x5602005382f0 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3376673740 1 -- 172.21.15.103:0/1829547475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 msgr2=0x56020053cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.997+0000 7f3376673740 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053cf20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f336809c1a0 tx=0x7f336801f000 comp rx=0 tx=0).stop 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- 172.21.15.103:0/1829547475 shutdown_connections 2024-09-19T07:38:29.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560200542bb0 0x56020053a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602005382f0 0x56020053cf20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 --2- 172.21.15.103:0/1829547475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005379d0 0x56020053c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- 172.21.15.103:0/1829547475 >> 172.21.15.103:0/1829547475 conn(0x560200533f60 msgr2=0x560200552210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- 172.21.15.103:0/1829547475 shutdown_connections 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- 172.21.15.103:0/1829547475 wait complete. 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- start start 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 Processor -- start 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x56020055cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005382f0 0x56020055d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560200542bb0 0x560200555e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560200554510 con 0x560200542bb0 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560200553e40 con 0x5602005382f0 2024-09-19T07:38:29.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560200554be0 con 0x5602005379d0 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3376673740 10 client.0 _ll_register_callbacks cb 0x7f3364009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3372036640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x56020055cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3372837640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560200542bb0 0x560200555e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3372036640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x56020055cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56886/0 (socket says 172.21.15.103:56886) 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3372036640 1 -- 172.21.15.103:0/3881091054 learned_addr learned my addr 172.21.15.103:0/3881091054 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.998+0000 7f3371835640 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005382f0 0x56020055d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3372036640 1 -- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005382f0 msgr2=0x56020055d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3372036640 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005382f0 0x56020055d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3372036640 1 -- 172.21.15.103:0/3881091054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560200542bb0 msgr2=0x560200555e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3372036640 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560200542bb0 0x560200555e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3372036640 1 -- 172.21.15.103:0/3881091054 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560200556640 con 0x5602005379d0 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3372036640 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x56020055cae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3360000c00 tx=0x7f336002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d5fcb1d22c318e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:29.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:29.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3355ffb640 1 -- 172.21.15.103:0/3881091054 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360052070 con 0x5602005379d0 2024-09-19T07:38:29.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336809c4a0 con 0x5602005379d0 2024-09-19T07:38:29.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3355ffb640 1 -- 172.21.15.103:0/3881091054 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3360004390 con 0x5602005379d0 2024-09-19T07:38:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3355ffb640 1 -- 172.21.15.103:0/3881091054 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33600048d0 con 0x5602005379d0 2024-09-19T07:38:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3376673740 10 client.25087 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:29.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56020055b040 con 0x5602005379d0 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:28.999+0000 7f3355ffb640 1 -- 172.21.15.103:0/3881091054 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336809c4a0 con 0x5602005379d0 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3355ffb640 1 -- 172.21.15.103:0/3881091054 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f336003c8b0 con 0x5602005379d0 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3376673740 10 client.25087 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3376673740 10 client.25087 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3376673740 20 client.25087 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3376673740 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5602005c1e60 0x5602005c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3376673740 4 client.25087 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "status"}) -- 0x5602004a5b10 con 0x5602005c1e60 2024-09-19T07:38:29.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3371835640 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5602005c1e60 0x5602005c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3355ffb640 10 client.25087 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.000+0000 7f3371835640 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5602005c1e60 0x5602005c0b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f011ff551ae015f5 server_cookie=ad9afacb7d256097 in_seq=0 out_seq=0 2024-09-19T07:38:29.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3355ffb640 1 -- 172.21.15.103:0/3881091054 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5602004a5b10 con 0x5602005c1e60 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3355ffb640 10 client.25087 handle_command_reply: tid=0 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 client.25087 shutdown 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 msgr2=0x56020055cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x56020055cae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3360000c00 tx=0x7f336002a420 comp rx=0 tx=0).stop 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 shutdown_connections 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5602005c1e60 0x5602005c0b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560200542bb0 0x560200555e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3355ffb640 0 client.25087 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602005382f0 0x56020055d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 --2- 172.21.15.103:0/3881091054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602005379d0 0x56020055cae0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 >> 172.21.15.103:0/3881091054 conn(0x560200533f60 msgr2=0x56020053a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 shutdown_connections 2024-09-19T07:38:29.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 1 -- 172.21.15.103:0/3881091054 wait complete. 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.003+0000 7f3376673740 20 client.25087 trim_cache size 0 max 16384 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "9fd714d2-7658-11ef-bceb-c7b262605968", 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24494, 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-19T07:38:29.013 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-19T07:38:29.014 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 140.15897358399999, 2024-09-19T07:38:29.014 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-19T07:38:29.014 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 85, 2024-09-19T07:38:29.014 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-19T07:38:29.014 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 150.495516348 2024-09-19T07:38:29.014 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33580a3440 msgr2=0x7f33580a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33580a3440 0x7f33580a5900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3364009fd0 tx=0x7f3364009500 comp rx=0 tx=0).stop 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c101420 msgr2=0x7f336c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c101420 0x7f336c1989a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f336800e9f0 tx=0x7f336800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 shutdown_connections 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33580a3440 0x7f33580a5900 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f336c106a80 0x7f336c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f336c101420 0x7f336c1989a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 --2- 172.21.15.103:0/2604553345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f336c100a80 0x7f336c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 >> 172.21.15.103:0/2604553345 conn(0x7f336c0fc7a0 msgr2=0x7f336c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 shutdown_connections 2024-09-19T07:38:29.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:29.004+0000 7f337429a640 1 -- 172.21.15.103:0/2604553345 wait complete. 2024-09-19T07:38:29.118 INFO:teuthology.orchestra.run.smithi103.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-19T07:38:29.121 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-19T07:38:29.161 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/: 2024-09-19T07:38:29.161 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 .. 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 data 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data: 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:total 35 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 . 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 .. 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 base 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 19 07:38 global 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 log 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_commit_ts 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_dynshmem 2024-09-19T07:38:29.162 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 19 07:38 pg_hba.conf 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 19 07:38 pg_ident.conf 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 pg_logical 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 pg_multixact 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_notify 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_replslot 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_serial 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_snapshots 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_stat 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_stat_tmp 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 pg_subtrans 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_tblspc 2024-09-19T07:38:29.163 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_twophase 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 pg_wal 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 pg_xact 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 19 07:38 postgresql.auto.conf 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 19 07:38 postgresql.conf 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base: 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 . 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 1 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 13493 2024-09-19T07:38:29.164 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 13494 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:total 7777 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 . 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 .. 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 112 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 113 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 1247 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1247_fsm 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1247_vm 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 07:38 1249 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1249_fsm 2024-09-19T07:38:29.165 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1249_vm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 07:38 1255 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1255_fsm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1255_vm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 07:38 1259 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1259_fsm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1259_vm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 07:38 13344 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13344_fsm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13344_vm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13346 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13348 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13349_fsm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349_vm 2024-09-19T07:38:29.166 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13351 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13353 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13354_fsm 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354_vm 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13356 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13358 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13359_fsm 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359_vm 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13361 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13363 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1417 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1418 2024-09-19T07:38:29.167 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 174 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 175 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2187 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2224 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2328 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2336 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2337 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2579 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2600 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2600_fsm 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2600_vm 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2601_fsm 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601_vm 2024-09-19T07:38:29.168 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2602 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2602_fsm 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2602_vm 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2603 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2603_fsm 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2603_vm 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2604 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2605 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2605_fsm 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2605_vm 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2606_fsm 2024-09-19T07:38:29.169 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606_vm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2607 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2607_fsm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2607_vm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 07:38 2608 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2608_fsm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2608_vm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 07:38 2609 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2609_fsm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2609_vm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2610 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2610_fsm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2610_vm 2024-09-19T07:38:29.170 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2611 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2612_fsm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612_vm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2613 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2615_fsm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615_vm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2616 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2616_fsm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2616_vm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 07:38 2617 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2617_fsm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2617_vm 2024-09-19T07:38:29.171 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 07:38 2618 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2618_fsm 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2618_vm 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 07:38 2619 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2619_fsm 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2619_vm 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2620 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2650 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2651 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2652 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2653 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2654 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2655 2024-09-19T07:38:29.172 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2656 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2657 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 07:38 2658 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 2659 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2660 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2661 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2662 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2663 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2664 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2665 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2666 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2667 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2668 2024-09-19T07:38:29.173 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2669 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2670 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 07:38 2673 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 07:38 2674 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 07:38 2675 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2678 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2679 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2680 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2681 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2682 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2683 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2684 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2685 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2686 2024-09-19T07:38:29.174 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2687 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2688 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2689 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 07:38 2690 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 07:38 2691 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2692 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2693 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2696 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2699 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2701 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2702 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2703 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2704 2024-09-19T07:38:29.175 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2753 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2753_fsm 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2753_vm 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2754 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2755 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2756 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2757 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2830 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2831 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2832 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2833 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2834 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2835 2024-09-19T07:38:29.176 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2836 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2837 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 07:38 2838 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2838_fsm 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2838_vm 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2839 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840_fsm 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2840_vm 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2841 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2995 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2996 2024-09-19T07:38:29.177 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3079_fsm 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079_vm 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3080 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3081 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3085 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3118 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3119 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 07:38 3164 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3256 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3257 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3258 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3350 2024-09-19T07:38:29.178 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3351 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3379 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3380 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3381 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3394 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3394_fsm 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3394_vm 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3395 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3429 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3430 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3431 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3433 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3439 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3440 2024-09-19T07:38:29.179 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3455 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 07:38 3456 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3456_fsm 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3456_vm 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3466 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3467 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3468 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3501 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3502 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3503 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3534 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541 2024-09-19T07:38:29.180 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3541_fsm 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541_vm 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3542 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3574 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3575 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3576 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3596 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3597 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3598 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3599 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3600_fsm 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600_vm 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601 2024-09-19T07:38:29.181 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3601_fsm 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601_vm 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3602_fsm 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602_vm 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603_fsm 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3603_vm 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3604 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3605 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3606 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3607 2024-09-19T07:38:29.182 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3608 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 3609 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3712 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3764_fsm 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764_vm 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3766 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3767 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3997 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4143 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4144 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4145 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4146 2024-09-19T07:38:29.183 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4147 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4148 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4149 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4150 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4151 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4152 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4153 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4154 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4155 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4156 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4157 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4158 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4159 2024-09-19T07:38:29.184 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4160 2024-09-19T07:38:29.185 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4161 2024-09-19T07:38:29.185 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4162 2024-09-19T07:38:29.185 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4163 2024-09-19T07:38:29.185 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4164 2024-09-19T07:38:29.185 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4165 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4166 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4167 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4168 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4169 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4170 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4171 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4172 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4173 2024-09-19T07:38:29.188 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4174 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 5002 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 548 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 549 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6102 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6104 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6106 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6110 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6111 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6112 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6113 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6117 2024-09-19T07:38:29.189 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 826 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 827 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 828 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:total 7777 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 . 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 .. 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 112 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 113 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 1247 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1247_fsm 2024-09-19T07:38:29.190 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1247_vm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 07:38 1249 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1249_fsm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1249_vm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 07:38 1255 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1255_fsm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1255_vm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 07:38 1259 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1259_fsm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1259_vm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 07:38 13344 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13344_fsm 2024-09-19T07:38:29.191 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13344_vm 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13346 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13348 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13349_fsm 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349_vm 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13351 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13353 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13354_fsm 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354_vm 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13356 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13358 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359 2024-09-19T07:38:29.192 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13359_fsm 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359_vm 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13361 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13363 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1417 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1418 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 174 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 175 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2187 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2224 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2328 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2336 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2337 2024-09-19T07:38:29.193 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2579 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2600 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2600_fsm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2600_vm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2601_fsm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601_vm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2602 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2602_fsm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2602_vm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2603 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2603_fsm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2603_vm 2024-09-19T07:38:29.194 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2604 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2605 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2605_fsm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2605_vm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2606_fsm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606_vm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2607 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2607_fsm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2607_vm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 07:38 2608 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2608_fsm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2608_vm 2024-09-19T07:38:29.195 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 07:38 2609 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2609_fsm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2609_vm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2610 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2610_fsm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2610_vm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2611 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2612_fsm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612_vm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2613 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2615_fsm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615_vm 2024-09-19T07:38:29.196 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2616 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2616_fsm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2616_vm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 07:38 2617 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2617_fsm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2617_vm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 07:38 2618 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2618_fsm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2618_vm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 07:38 2619 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2619_fsm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2619_vm 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2620 2024-09-19T07:38:29.197 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2650 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2651 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2652 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2653 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2654 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2655 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2656 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2657 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 07:38 2658 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 2659 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2660 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2661 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2662 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2663 2024-09-19T07:38:29.198 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2664 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2665 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2666 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2667 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2668 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2669 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2670 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 07:38 2673 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 07:38 2674 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 07:38 2675 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2678 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2679 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2680 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2681 2024-09-19T07:38:29.199 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2682 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2683 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2684 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2685 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2686 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2687 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2688 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2689 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 07:38 2690 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 07:38 2691 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2692 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2693 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2696 2024-09-19T07:38:29.200 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2699 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2701 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2702 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2703 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2704 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2753 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2753_fsm 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2753_vm 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2754 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2755 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2756 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2757 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2830 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2831 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2832 2024-09-19T07:38:29.201 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2833 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2834 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2835 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2836 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2837 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 07:38 2838 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2838_fsm 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2838_vm 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2839 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840_fsm 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2840_vm 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2841 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2995 2024-09-19T07:38:29.202 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2996 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3079_fsm 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079_vm 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3080 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3081 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3085 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3118 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3119 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 07:38 3164 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3256 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3257 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3258 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3350 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3351 2024-09-19T07:38:29.203 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3379 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3380 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3381 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3394 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3394_fsm 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3394_vm 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3395 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3429 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3430 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3431 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3433 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3439 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3440 2024-09-19T07:38:29.204 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3455 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 07:38 3456 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3456_fsm 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3456_vm 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3466 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3467 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3468 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3501 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3502 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3503 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3534 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3541_fsm 2024-09-19T07:38:29.205 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541_vm 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3542 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3574 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3575 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3576 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3596 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3597 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3598 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3599 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3600_fsm 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600_vm 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3601_fsm 2024-09-19T07:38:29.206 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601_vm 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3602_fsm 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602_vm 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603_fsm 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3603_vm 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3604 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3605 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3606 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3607 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3608 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 3609 2024-09-19T07:38:29.207 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3712 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3764_fsm 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764_vm 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3766 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3767 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3997 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4143 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4144 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4145 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4146 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4147 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4148 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4149 2024-09-19T07:38:29.208 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4150 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4151 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4152 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4153 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4154 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4155 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4156 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4157 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4158 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4159 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4160 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4161 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4162 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4163 2024-09-19T07:38:29.209 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4164 2024-09-19T07:38:29.213 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4165 2024-09-19T07:38:29.213 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4166 2024-09-19T07:38:29.213 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4167 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4168 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4169 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4170 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4171 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4172 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4173 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4174 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 5002 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 548 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 549 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6102 2024-09-19T07:38:29.214 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6104 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6106 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6110 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6111 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6112 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6113 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6117 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 826 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 827 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 828 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.215 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:total 7777 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 . 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 .. 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 112 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 113 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 1247 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1247_fsm 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1247_vm 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 07:38 1249 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1249_fsm 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1249_vm 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 07:38 1255 2024-09-19T07:38:29.216 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1255_fsm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1255_vm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 07:38 1259 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1259_fsm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1259_vm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 07:38 13344 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13344_fsm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13344_vm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13346 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13348 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13349_fsm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349_vm 2024-09-19T07:38:29.217 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13351 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13353 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13354_fsm 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354_vm 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13356 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13358 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13359_fsm 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359_vm 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13361 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13363 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1417 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1418 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 174 2024-09-19T07:38:29.218 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 175 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2187 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2224 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2328 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2336 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2337 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2579 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2600 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2600_fsm 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2600_vm 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2601_fsm 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601_vm 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2602 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2602_fsm 2024-09-19T07:38:29.219 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2602_vm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2603 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2603_fsm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2603_vm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2604 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2605 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2605_fsm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2605_vm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2606_fsm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606_vm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2607 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2607_fsm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2607_vm 2024-09-19T07:38:29.220 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 07:38 2608 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2608_fsm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2608_vm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 07:38 2609 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2609_fsm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2609_vm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2610 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2610_fsm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2610_vm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2611 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2612_fsm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612_vm 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2613 2024-09-19T07:38:29.221 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2615_fsm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615_vm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2616 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2616_fsm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2616_vm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 07:38 2617 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2617_fsm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2617_vm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 07:38 2618 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2618_fsm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2618_vm 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 07:38 2619 2024-09-19T07:38:29.222 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2619_fsm 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2619_vm 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2620 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2650 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2651 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2652 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2653 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2654 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2655 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2656 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2657 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 07:38 2658 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 2659 2024-09-19T07:38:29.223 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2660 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2661 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2662 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2663 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2664 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2665 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2666 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2667 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2668 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2669 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2670 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 07:38 2673 2024-09-19T07:38:29.224 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 07:38 2674 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 07:38 2675 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2678 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2679 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2680 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2681 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2682 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2683 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2684 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2685 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2686 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2687 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2688 2024-09-19T07:38:29.225 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2689 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 07:38 2690 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 07:38 2691 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2692 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2693 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2696 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2699 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2701 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2702 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2703 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2704 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2753 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2753_fsm 2024-09-19T07:38:29.226 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2753_vm 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2754 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2755 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2756 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2757 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2830 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2831 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2832 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2833 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2834 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2835 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2836 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2837 2024-09-19T07:38:29.227 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 07:38 2838 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2838_fsm 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2838_vm 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2839 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840_fsm 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2840_vm 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2841 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2995 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2996 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3079_fsm 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079_vm 2024-09-19T07:38:29.228 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3080 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3081 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3085 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3118 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3119 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 07:38 3164 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3256 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3257 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3258 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3350 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3351 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3379 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3380 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3381 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3394 2024-09-19T07:38:29.229 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3394_fsm 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3394_vm 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3395 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3429 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3430 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3431 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3433 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3439 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3440 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3455 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 07:38 3456 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3456_fsm 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3456_vm 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3466 2024-09-19T07:38:29.230 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3467 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3468 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3501 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3502 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3503 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3534 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3541_fsm 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541_vm 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3542 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3574 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3575 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3576 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3596 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3597 2024-09-19T07:38:29.231 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3598 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3599 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3600_fsm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600_vm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3601_fsm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601_vm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3602_fsm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602_vm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603_fsm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3603_vm 2024-09-19T07:38:29.232 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3604 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3605 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3606 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3607 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3608 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 3609 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3712 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3764_fsm 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764_vm 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3766 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3767 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3997 2024-09-19T07:38:29.233 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4143 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4144 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4145 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4146 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4147 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4148 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4149 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4150 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4151 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4152 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4153 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4154 2024-09-19T07:38:29.234 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4155 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4156 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4157 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4158 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4159 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4160 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4161 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4162 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4163 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4164 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4165 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4166 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4167 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4168 2024-09-19T07:38:29.235 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4169 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4170 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4171 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4172 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4173 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4174 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 5002 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 548 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 549 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6102 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6104 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6106 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6110 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6111 2024-09-19T07:38:29.236 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6112 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6113 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6117 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 826 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 827 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 828 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/global: 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:total 537 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 19 07:38 . 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1213 2024-09-19T07:38:29.237 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1213_fsm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1213_vm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1214 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1214_fsm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1214_vm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 1232 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 1233 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1260 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1260_fsm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1260_vm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1261 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1261_fsm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1261_vm 2024-09-19T07:38:29.238 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1262 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1262_fsm 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1262_vm 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2396 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2396_fsm 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2396_vm 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2397 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2671 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2672 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2676 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2677 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2694 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2695 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2697 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2698 2024-09-19T07:38:29.239 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2846 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2847 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2964 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2965 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2966 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2967 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3592 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3593 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4060 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4061 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4175 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4176 2024-09-19T07:38:29.240 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4177 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4178 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4181 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4182 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4183 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4184 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4185 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4186 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6000 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6001 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6002 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6100 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6114 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6115 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 pg_control 2024-09-19T07:38:29.241 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/log: 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.242 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:total 1 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 . 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 mappings 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 07:38 replorigin_checkpoint 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 snapshots 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 .. 2024-09-19T07:38:29.243 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 .. 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 . 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 members 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 offsets 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:total 8 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 .. 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 0000 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:total 8 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 .. 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 0000 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-19T07:38:29.245 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.246 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.247 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:total 8 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 0000 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.248 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:total 16384 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 . 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:38 000000010000000000000001 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 archive_status 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-19T07:38:29.249 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 .. 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout:total 8 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 19 07:38 .. 2024-09-19T07:38:29.250 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 0000 2024-09-19T07:38:29.250 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-19T07:38:29.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-19T07:38:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:dropping old tables... 2024-09-19T07:38:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-19T07:38:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-19T07:38:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-19T07:38:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-19T07:38:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:creating tables... 2024-09-19T07:38:29.479 INFO:teuthology.orchestra.run.smithi103.stderr:generating data (client-side)... 2024-09-19T07:38:29.553 INFO:teuthology.orchestra.run.smithi103.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.80 s) 2024-09-19T07:38:29.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:29 smithi174 ceph-mon[35370]: pgmap v18: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 83 KiB/s wr, 12 op/s 2024-09-19T07:38:29.713 INFO:teuthology.orchestra.run.smithi103.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.22 s, remaining 3.27 s) 2024-09-19T07:38:29.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:29 smithi103 ceph-mon[31201]: pgmap v18: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 83 KiB/s wr, 12 op/s 2024-09-19T07:38:29.789 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T07:38:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:29 smithi134 ceph-mon[35414]: pgmap v18: 97 pgs: 97 active+clean; 2.2 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 83 KiB/s wr, 12 op/s 2024-09-19T07:38:29.852 INFO:teuthology.orchestra.run.smithi103.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.36 s, remaining 3.46 s) 2024-09-19T07:38:29.930 INFO:teuthology.orchestra.run.smithi103.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.44 s, remaining 3.05 s) 2024-09-19T07:38:30.064 INFO:teuthology.orchestra.run.smithi103.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.57 s, remaining 3.07 s) 2024-09-19T07:38:30.160 INFO:teuthology.orchestra.run.smithi103.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.67 s, remaining 2.88 s) 2024-09-19T07:38:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.80 s, remaining 2.86 s) 2024-09-19T07:38:30.464 INFO:teuthology.orchestra.run.smithi103.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.97 s, remaining 2.91 s) 2024-09-19T07:38:30.555 INFO:teuthology.orchestra.run.smithi103.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.06 s, remaining 2.71 s) 2024-09-19T07:38:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.19 s, remaining 2.62 s) 2024-09-19T07:38:30.848 INFO:teuthology.orchestra.run.smithi103.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.35 s, remaining 2.58 s) 2024-09-19T07:38:30.946 INFO:teuthology.orchestra.run.smithi103.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.45 s, remaining 2.42 s) 2024-09-19T07:38:31.071 INFO:teuthology.orchestra.run.smithi103.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.58 s, remaining 2.30 s) 2024-09-19T07:38:31.179 INFO:teuthology.orchestra.run.smithi103.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.68 s, remaining 2.17 s) 2024-09-19T07:38:31.328 INFO:teuthology.orchestra.run.smithi103.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.83 s, remaining 2.08 s) 2024-09-19T07:38:31.558 INFO:teuthology.orchestra.run.smithi103.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.06 s, remaining 2.06 s) 2024-09-19T07:38:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:31 smithi174 ceph-mon[35370]: pgmap v19: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 3.7 MiB/s wr, 118 op/s 2024-09-19T07:38:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.21 s, remaining 1.95 s) 2024-09-19T07:38:31.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:31 smithi103 ceph-mon[31201]: pgmap v19: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 3.7 MiB/s wr, 118 op/s 2024-09-19T07:38:31.785 INFO:teuthology.orchestra.run.smithi103.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.29 s, remaining 1.78 s) 2024-09-19T07:38:31.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:31 smithi134 ceph-mon[35414]: pgmap v19: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 3.7 MiB/s wr, 118 op/s 2024-09-19T07:38:31.931 INFO:teuthology.orchestra.run.smithi103.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.44 s, remaining 1.67 s) 2024-09-19T07:38:32.015 INFO:teuthology.orchestra.run.smithi103.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.52 s, remaining 1.51 s) 2024-09-19T07:38:32.151 INFO:teuthology.orchestra.run.smithi103.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.66 s, remaining 1.39 s) 2024-09-19T07:38:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.82 s, remaining 1.28 s) 2024-09-19T07:38:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.94 s, remaining 1.15 s) 2024-09-19T07:38:32.565 INFO:teuthology.orchestra.run.smithi103.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.07 s, remaining 1.02 s) 2024-09-19T07:38:32.728 INFO:teuthology.orchestra.run.smithi103.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.23 s, remaining 0.91 s) 2024-09-19T07:38:32.829 INFO:teuthology.orchestra.run.smithi103.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.33 s, remaining 0.77 s) 2024-09-19T07:38:32.959 INFO:teuthology.orchestra.run.smithi103.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.46 s, remaining 0.64 s) 2024-09-19T07:38:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.57 s, remaining 0.51 s) 2024-09-19T07:38:33.208 INFO:teuthology.orchestra.run.smithi103.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.71 s, remaining 0.38 s) 2024-09-19T07:38:33.342 INFO:teuthology.orchestra.run.smithi103.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.85 s, remaining 0.26 s) 2024-09-19T07:38:33.440 INFO:teuthology.orchestra.run.smithi103.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.95 s, remaining 0.13 s) 2024-09-19T07:38:33.591 INFO:teuthology.orchestra.run.smithi103.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.10 s, remaining 0.00 s) 2024-09-19T07:38:33.634 INFO:teuthology.orchestra.run.smithi103.stderr:vacuuming... 2024-09-19T07:38:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:33 smithi174 ceph-mon[35370]: pgmap v20: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 3.7 MiB/s wr, 114 op/s 2024-09-19T07:38:33.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:33 smithi103 ceph-mon[31201]: pgmap v20: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 3.7 MiB/s wr, 114 op/s 2024-09-19T07:38:33.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:33 smithi134 ceph-mon[35414]: pgmap v20: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 3.7 MiB/s wr, 114 op/s 2024-09-19T07:38:34.222 INFO:teuthology.orchestra.run.smithi103.stderr:creating primary keys... 2024-09-19T07:38:35.409 INFO:teuthology.orchestra.run.smithi103.stderr:done in 5.94 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.15 s, vacuum 0.59 s, primary keys 1.19 s). 2024-09-19T07:38:35.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-19T07:38:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:starting vacuum...end. 2024-09-19T07:38:35.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:35 smithi174 ceph-mon[35370]: pgmap v21: 97 pgs: 97 active+clean; 321 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 49 MiB/s wr, 150 op/s 2024-09-19T07:38:35.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:35 smithi103 ceph-mon[31201]: pgmap v21: 97 pgs: 97 active+clean; 321 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 49 MiB/s wr, 150 op/s 2024-09-19T07:38:35.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:35 smithi134 ceph-mon[35414]: pgmap v21: 97 pgs: 97 active+clean; 321 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 49 MiB/s wr, 150 op/s 2024-09-19T07:38:37.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:37 smithi174 ceph-mon[35370]: pgmap v22: 97 pgs: 97 active+clean; 381 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 59 MiB/s wr, 145 op/s 2024-09-19T07:38:37.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:37 smithi103 ceph-mon[31201]: pgmap v22: 97 pgs: 97 active+clean; 381 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 59 MiB/s wr, 145 op/s 2024-09-19T07:38:37.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:37 smithi134 ceph-mon[35414]: pgmap v22: 97 pgs: 97 active+clean; 381 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 59 MiB/s wr, 145 op/s 2024-09-19T07:38:38.733 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:38:39.068 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:39.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:39 smithi103 ceph-mon[31201]: pgmap v23: 97 pgs: 97 active+clean; 381 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 59 MiB/s wr, 145 op/s 2024-09-19T07:38:39.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.637+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3405547029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab8109b90 msgr2=0x7f5ab81158a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.637+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3405547029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab8109b90 0x7f5ab81158a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000bd70 tx=0x7f5ab002c440 comp rx=0 tx=0).stop 2024-09-19T07:38:39.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.638+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3405547029 shutdown_connections 2024-09-19T07:38:39.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.638+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3405547029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab8109b90 0x7f5ab81158a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.638+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3405547029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab81091f0 0x7f5ab8109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.638+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3405547029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab810eb10 0x7f5ab810eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.638+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3405547029 >> 172.21.15.103:0/3405547029 conn(0x7f5ab806e2d0 msgr2=0x7f5ab806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:39.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.638+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3405547029 shutdown_connections 2024-09-19T07:38:39.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.638+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3405547029 wait complete. 2024-09-19T07:38:39.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 Processor -- start 2024-09-19T07:38:39.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 -- start start 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab81091f0 0x7f5ab81143e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8109b90 0x7f5ab8114920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab810eb10 0x7f5ab81acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8117be0 con 0x7f5ab8109b90 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8117a60 con 0x7f5ab81091f0 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.639+0000 7f5abdb1d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8117d60 con 0x7f5ab810eb10 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab81091f0 0x7f5ab81143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8109b90 0x7f5ab8114920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab81091f0 0x7f5ab81143e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42564/0 (socket says 172.21.15.103:42564) 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8109b90 0x7f5ab8114920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45620/0 (socket says 172.21.15.103:45620) 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 -- 172.21.15.103:0/3835888179 learned_addr learned my addr 172.21.15.103:0/3835888179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:39.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab7fff640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab810eb10 0x7f5ab81acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 -- 172.21.15.103:0/3835888179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab810eb10 msgr2=0x7f5ab81acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab810eb10 0x7f5ab81acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 -- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8109b90 msgr2=0x7f5ab8114920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8109b90 0x7f5ab8114920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 -- 172.21.15.103:0/3835888179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab81ad360 con 0x7f5ab81091f0 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab81091f0 0x7f5ab81143e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000e970 tx=0x7f5aa000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab4ff9640 1 -- 172.21.15.103:0/3835888179 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000ccc0 con 0x7f5ab81091f0 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab4ff9640 1 -- 172.21.15.103:0/3835888179 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa0010910 con 0x7f5ab81091f0 2024-09-19T07:38:39.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.640+0000 7f5ab4ff9640 1 -- 172.21.15.103:0/3835888179 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0018880 con 0x7f5ab81091f0 2024-09-19T07:38:39.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.641+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab81ad650 con 0x7f5ab81091f0 2024-09-19T07:38:39.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.641+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab81b3d50 con 0x7f5ab81091f0 2024-09-19T07:38:39.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.644+0000 7f5ab4ff9640 1 -- 172.21.15.103:0/3835888179 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f5aa0010ac0 con 0x7f5ab81091f0 2024-09-19T07:38:39.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.646+0000 7f5ab4ff9640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3510 0x7f5aa40a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 Processor -- start 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3510 0x7f5aa40a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab4ff9640 1 -- 172.21.15.103:0/3835888179 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5aa00d01b0 con 0x7f5ab81091f0 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 -- start start 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cd8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6ce5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5633e6c3e2e0 con 0x5633e6cd8dc0 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5633e6be5ce0 con 0x5633e6cd84a0 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5abfef6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633e6c3a2a0 con 0x5633e6ce3680 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3510 0x7f5aa40a59d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ab806e7c0 tx=0x7f5aa8010040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6ce5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45636/0 (socket says 172.21.15.103:45636) 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/945817823 learned_addr learned my addr 172.21.15.103:0/945817823 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.647+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cd8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.648+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/945817823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 msgr2=0x5633e6ce5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.648+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6ce5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.648+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/945817823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 msgr2=0x5633e6cd8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.648+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cd8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.648+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/945817823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e6bca440 con 0x5633e6cd8dc0 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.648+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cd8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.648+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce3140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005c80 tx=0x7f5aa8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90953e5e2a108105 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:39.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.649+0000 7f5aadffb640 1 -- 172.21.15.103:0/945817823 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa801b190 con 0x5633e6cd8dc0 2024-09-19T07:38:39.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.649+0000 7f5aadffb640 1 -- 172.21.15.103:0/945817823 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8005020 con 0x5633e6cd8dc0 2024-09-19T07:38:39.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.650+0000 7f5aadffb640 1 -- 172.21.15.103:0/945817823 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004840 con 0x5633e6cd8dc0 2024-09-19T07:38:39.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 -- 172.21.15.103:0/945817823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 msgr2=0x5633e6ce3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce3140 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005c80 tx=0x7f5aa8014000 comp rx=0 tx=0).stop 2024-09-19T07:38:39.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 -- 172.21.15.103:0/945817823 shutdown_connections 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6ce5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce3140 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 --2- 172.21.15.103:0/945817823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cd8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 -- 172.21.15.103:0/945817823 >> 172.21.15.103:0/945817823 conn(0x5633e6cd4a30 msgr2=0x5633e6cd4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 -- 172.21.15.103:0/945817823 shutdown_connections 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 -- 172.21.15.103:0/945817823 wait complete. 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.651+0000 7f5abfef6740 1 -- start start 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 1 Processor -- start 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd84a0 0x5633e6ce2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6ce3680 0x5633e6ce07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5633e6ce62f0 con 0x5633e6ce3680 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5633e6bf9050 con 0x5633e6cd8dc0 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633e6ce6470 con 0x5633e6cd84a0 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5abfef6740 10 client.0 _ll_register_callbacks cb 0x5633e6c39160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42578/0 (socket says 172.21.15.103:42578) 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/2330603901 learned_addr learned my addr 172.21.15.103:0/2330603901 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd84a0 0x5633e6ce2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.652+0000 7f5ab7fff640 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6ce3680 0x5633e6ce07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.653+0000 7f5ab77fe640 1 -- 172.21.15.103:0/2330603901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd8dc0 msgr2=0x5633e6ce0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.653+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.653+0000 7f5ab77fe640 1 -- 172.21.15.103:0/2330603901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6ce3680 msgr2=0x5633e6ce07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.653+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6ce3680 0x5633e6ce07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.653+0000 7f5ab77fe640 1 -- 172.21.15.103:0/2330603901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e69af3c0 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.654+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd84a0 0x5633e6ce2fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00cd850 tx=0x7f5aa009bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3956f8de14f0a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.654+0000 7f5a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.654+0000 7f5a96ffd640 1 -- 172.21.15.103:0/2330603901 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa009b980 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5a96ffd640 1 -- 172.21.15.103:0/2330603901 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5aa0010430 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa8005020 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5a96ffd640 1 -- 172.21.15.103:0/2330603901 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5aa00cdd10 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 10 client.25099 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633e6cf1c10 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5a96ffd640 1 -- 172.21.15.103:0/2330603901 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8005020 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5a96ffd640 1 -- 172.21.15.103:0/2330603901 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5aa0098180 con 0x5633e6cd84a0 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 10 client.25099 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 10 client.25099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:39.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 20 client.25099 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:39.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6cf5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 4 client.25099 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:39.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.655+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633e6c4b850 con 0x5633e6cf3020 2024-09-19T07:38:39.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.656+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6cf5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.656+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6cf5410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a3ce6f1ffe2789 server_cookie=2663de003378d6ad in_seq=0 out_seq=0 2024-09-19T07:38:39.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.656+0000 7f5a96ffd640 10 client.25099 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:39.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.657+0000 7f5a96ffd640 1 -- 172.21.15.103:0/2330603901 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633e6c4b850 con 0x5633e6cf3020 2024-09-19T07:38:39.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.657+0000 7f5a96ffd640 10 client.25099 handle_command_reply: tid=0 2024-09-19T07:38:39.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 client.25099 shutdown 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd84a0 msgr2=0x5633e6ce2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd84a0 0x5633e6ce2fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00cd850 tx=0x7f5aa009bd80 comp rx=0 tx=0).stop 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 shutdown_connections 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6cf5410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5a96ffd640 0 client.25099 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6ce3680 0x5633e6ce07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 --2- 172.21.15.103:0/2330603901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd84a0 0x5633e6ce2fc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 >> 172.21.15.103:0/2330603901 conn(0x5633e6cd4a30 msgr2=0x5633e6cd3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 shutdown_connections 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 1 -- 172.21.15.103:0/2330603901 wait complete. 2024-09-19T07:38:39.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.658+0000 7f5abfef6740 20 client.25099 trim_cache size 0 max 16384 2024-09-19T07:38:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:39 smithi174 ceph-mon[35370]: pgmap v23: 97 pgs: 97 active+clean; 381 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 59 MiB/s wr, 145 op/s 2024-09-19T07:38:39.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 Processor -- start 2024-09-19T07:38:39.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 -- start start 2024-09-19T07:38:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6cfac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6cfb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81bc7c0 con 0x5633e6cd8dc0 2024-09-19T07:38:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81be300 con 0x5633e6cd84a0 2024-09-19T07:38:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5abfef6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81bce90 con 0x5633e6ce3680 2024-09-19T07:38:39.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.687+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6cfac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6cfac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45658/0 (socket says 172.21.15.103:45658) 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/4226361864 learned_addr learned my addr 172.21.15.103:0/4226361864 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab7fff640 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6cfb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/4226361864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 msgr2=0x5633e6cfb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6cfb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/4226361864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 msgr2=0x5633e6cef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/4226361864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e6cd9a20 con 0x5633e6cd8dc0 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab7fff640 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6cfb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6cfac90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000c00 tx=0x7f5aa801df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c401ace1840e0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5aad7fa640 1 -- 172.21.15.103:0/4226361864 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80041f0 con 0x5633e6cd8dc0 2024-09-19T07:38:39.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5aad7fa640 1 -- 172.21.15.103:0/4226361864 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8004390 con 0x5633e6cd8dc0 2024-09-19T07:38:39.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.688+0000 7f5aad7fa640 1 -- 172.21.15.103:0/4226361864 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80049f0 con 0x5633e6cd8dc0 2024-09-19T07:38:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 -- 172.21.15.103:0/4226361864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 msgr2=0x5633e6cfac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6cfac90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000c00 tx=0x7f5aa801df90 comp rx=0 tx=0).stop 2024-09-19T07:38:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 -- 172.21.15.103:0/4226361864 shutdown_connections 2024-09-19T07:38:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6ce3680 0x5633e6cfb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd8dc0 0x5633e6cfac90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 --2- 172.21.15.103:0/4226361864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6cd84a0 0x5633e6cef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 -- 172.21.15.103:0/4226361864 >> 172.21.15.103:0/4226361864 conn(0x5633e6cd4a30 msgr2=0x5633e6cda760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 -- 172.21.15.103:0/4226361864 shutdown_connections 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.689+0000 7f5abfef6740 1 -- 172.21.15.103:0/4226361864 wait complete. 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 -- start start 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 Processor -- start 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd84a0 0x5633e6ce7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6ce3680 0x5633e6ce7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81bdc30 con 0x5633e6cd84a0 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81bd560 con 0x5633e6ce3680 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab81ba7a0 con 0x5633e6cd8dc0 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5abfef6740 10 client.0 _ll_register_callbacks cb 0x7f5aa00cd6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50706/0 (socket says 172.21.15.103:50706) 2024-09-19T07:38:39.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd84a0 0x5633e6ce7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/1279723054 learned_addr learned my addr 172.21.15.103:0/1279723054 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.690+0000 7f5ab7fff640 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6ce3680 0x5633e6ce7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.691+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6ce3680 msgr2=0x5633e6ce7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.691+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6ce3680 0x5633e6ce7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.691+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/1279723054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd84a0 msgr2=0x5633e6ce7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.691+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd84a0 0x5633e6ce7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.691+0000 7f5ab6ffd640 1 -- 172.21.15.103:0/1279723054 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e6d5cc60 con 0x5633e6cd8dc0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.691+0000 7f5ab6ffd640 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce7880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800aa60 tx=0x7f5aa800add0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8151c7e83bafd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5a96ffd640 1 -- 172.21.15.103:0/1279723054 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004650 con 0x5633e6cd8dc0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5a96ffd640 1 -- 172.21.15.103:0/1279723054 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5aa8004970 con 0x5633e6cd8dc0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa8004390 con 0x5633e6cd8dc0 2024-09-19T07:38:39.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5a96ffd640 1 -- 172.21.15.103:0/1279723054 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5aa80035f0 con 0x5633e6cd8dc0 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5abfef6740 10 client.25102 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633e6d5dde0 con 0x5633e6cd8dc0 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.692+0000 7f5a96ffd640 1 -- 172.21.15.103:0/1279723054 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8025470 con 0x5633e6cd8dc0 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5a96ffd640 1 -- 172.21.15.103:0/1279723054 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5aa8018070 con 0x5633e6cd8dc0 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5abfef6740 10 client.25102 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5abfef6740 10 client.25102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5abfef6740 20 client.25102 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5abfef6740 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6d60130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5abfef6740 4 client.25102 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5aa8004390 con 0x5633e6cf3020 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6d60130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5a96ffd640 10 client.25102 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.693+0000 7f5ab77fe640 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6d60130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d0ecba5787df464 server_cookie=d388d7f22c12ab6c in_seq=0 out_seq=0 2024-09-19T07:38:39.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5a96ffd640 1 -- 172.21.15.103:0/1279723054 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5aa8004390 con 0x5633e6cf3020 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5a96ffd640 10 client.25102 handle_command_reply: tid=0 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 client.25102 shutdown 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd8dc0 msgr2=0x5633e6ce7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce7880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800aa60 tx=0x7f5aa800add0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 shutdown_connections 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5633e6cf3020 0x5633e6d60130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633e6ce3680 0x5633e6ce7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5a96ffd640 0 client.25102 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633e6cd8dc0 0x5633e6ce7880 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 --2- 172.21.15.103:0/1279723054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5633e6cd84a0 0x5633e6ce7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 >> 172.21.15.103:0/1279723054 conn(0x5633e6cd4a30 msgr2=0x5633e6cd3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 shutdown_connections 2024-09-19T07:38:39.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 1 -- 172.21.15.103:0/1279723054 wait complete. 2024-09-19T07:38:39.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.694+0000 7f5abfef6740 20 client.25102 trim_cache size 0 max 16384 2024-09-19T07:38:39.699 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:39.699 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:38:39.699 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:38:39.699 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3510 msgr2=0x7f5aa40a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3510 0x7f5aa40a59d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ab806e7c0 tx=0x7f5aa8010040 comp rx=0 tx=0).stop 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab81091f0 msgr2=0x7f5ab81143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab81091f0 0x7f5ab81143e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000e970 tx=0x7f5aa000ed70 comp rx=0 tx=0).stop 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 shutdown_connections 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3510 0x7f5aa40a59d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab810eb10 0x7f5ab81acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ab8109b90 0x7f5ab8114920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 --2- 172.21.15.103:0/3835888179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab81091f0 0x7f5ab81143e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:39.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 >> 172.21.15.103:0/3835888179 conn(0x7f5ab806e2d0 msgr2=0x7f5ab8073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:39.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.695+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 shutdown_connections 2024-09-19T07:38:39.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:39.696+0000 7f5abdb1d640 1 -- 172.21.15.103:0/3835888179 wait complete. 2024-09-19T07:38:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:39 smithi134 ceph-mon[35414]: pgmap v23: 97 pgs: 97 active+clean; 381 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 59 MiB/s wr, 145 op/s 2024-09-19T07:38:40.417 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:38:40.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:38:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 5.0 s, 3619.6 tps, lat 27.131 ms stddev 28.190 2024-09-19T07:38:40.781 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:41.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/1212067735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 msgr2=0x7f6bd010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:41.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/1212067735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd010da40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009ca0 tx=0x7f6bc802a950 comp rx=0 tx=0).stop 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/1212067735 shutdown_connections 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/1212067735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd010da40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/1212067735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0102570 0x7f6bd01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/1212067735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0100e20 0x7f6bd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/1212067735 >> 172.21.15.103:0/1212067735 conn(0x7f6bd00fc7d0 msgr2=0x7f6bd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/1212067735 shutdown_connections 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.349+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/1212067735 wait complete. 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.350+0000 7f6bd5bf6640 1 Processor -- start 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.350+0000 7f6bd5bf6640 1 -- start start 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.350+0000 7f6bd5bf6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.350+0000 7f6bd5bf6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0102570 0x7f6bd01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.350+0000 7f6bd5bf6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0102570 0x7f6bd01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0102570 0x7f6bd01989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42618/0 (socket says 172.21.15.103:42618) 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bceffd640 1 -- 172.21.15.103:0/4113866805 learned_addr learned my addr 172.21.15.103:0/4113866805 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0110fd0 con 0x7f6bd0102ef0 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0110e50 con 0x7f6bd0102570 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0111150 con 0x7f6bd0100e20 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bcffff640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.351+0000 7f6bceffd640 1 -- 172.21.15.103:0/4113866805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 msgr2=0x7f6bd0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bceffd640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bceffd640 1 -- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 msgr2=0x7f6bd019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bceffd640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bceffd640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd019d410 con 0x7f6bd0102570 2024-09-19T07:38:41.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bcf7fe640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:41.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bcffff640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bceffd640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0102570 0x7f6bd01989a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400d950 tx=0x7f6bc400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bccff9640 1 -- 172.21.15.103:0/4113866805 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4017070 con 0x7f6bd0102570 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bccff9640 1 -- 172.21.15.103:0/4113866805 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc4009c30 con 0x7f6bd0102570 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd019d700 con 0x7f6bd0102570 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bccff9640 1 -- 172.21.15.103:0/4113866805 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400f6b0 con 0x7f6bd0102570 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.352+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd01a4fe0 con 0x7f6bd0102570 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.353+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd0101200 con 0x7f6bd0102570 2024-09-19T07:38:41.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.355+0000 7f6bccff9640 1 -- 172.21.15.103:0/4113866805 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6bc40109d0 con 0x7f6bd0102570 2024-09-19T07:38:41.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.356+0000 7f6bccff9640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6bb80a3540 0x7f6bb80a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:41.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.356+0000 7f6bccff9640 1 -- 172.21.15.103:0/4113866805 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6bc40cfe50 con 0x7f6bd0102570 2024-09-19T07:38:41.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.356+0000 7f6bcf7fe640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6bb80a3540 0x7f6bb80a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:41.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.357+0000 7f6bcf7fe640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6bb80a3540 0x7f6bb80a5a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00095f0 tx=0x7f6bc0005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:41.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.360+0000 7f6bccff9640 1 -- 172.21.15.103:0/4113866805 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6bc4009de0 con 0x7f6bd0102570 2024-09-19T07:38:41.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.666+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bd010a0a0 con 0x7f6bd0102570 2024-09-19T07:38:41.667 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:41 smithi103 ceph-mon[31201]: pgmap v24: 97 pgs: 97 active+clean; 813 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 77 MiB/s wr, 284 op/s 2024-09-19T07:38:41.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.667+0000 7f6bccff9640 1 -- 172.21.15.103:0/4113866805 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11991 (secure 0 0 0) 0x7f6bc409c670 con 0x7f6bd0102570 2024-09-19T07:38:41.668 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:41.668 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-19T07:37:57:172446+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:37:57.122252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:38:41.668 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-19T07:38:41.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.671+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6bb80a3540 msgr2=0x7f6bb80a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:41.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.671+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6bb80a3540 0x7f6bb80a5a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00095f0 tx=0x7f6bc0005f90 comp rx=0 tx=0).stop 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.671+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0102570 msgr2=0x7f6bd01989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.671+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0102570 0x7f6bd01989a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400d950 tx=0x7f6bc400de10 comp rx=0 tx=0).stop 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 shutdown_connections 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6bb80a3540 0x7f6bb80a5a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6bd0102ef0 0x7f6bd019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bd0102570 0x7f6bd01989a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 --2- 172.21.15.103:0/4113866805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bd0100e20 0x7f6bd0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 >> 172.21.15.103:0/4113866805 conn(0x7f6bd00fc7d0 msgr2=0x7f6bd006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 shutdown_connections 2024-09-19T07:38:41.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:41.672+0000 7f6bd5bf6640 1 -- 172.21.15.103:0/4113866805 wait complete. 2024-09-19T07:38:41.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:41 smithi134 ceph-mon[35414]: pgmap v24: 97 pgs: 97 active+clean; 813 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 77 MiB/s wr, 284 op/s 2024-09-19T07:38:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:41 smithi174 ceph-mon[35370]: pgmap v24: 97 pgs: 97 active+clean; 813 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 77 MiB/s wr, 284 op/s 2024-09-19T07:38:42.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:42 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4113866805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:42.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:42 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4113866805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:42.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:42 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4113866805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:43.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:38:43.546 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:43.707 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:43 smithi103 ceph-mon[31201]: pgmap v25: 97 pgs: 97 active+clean; 813 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 MiB/s wr, 177 op/s 2024-09-19T07:38:43.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:43 smithi134 ceph-mon[35414]: pgmap v25: 97 pgs: 97 active+clean; 813 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 MiB/s wr, 177 op/s 2024-09-19T07:38:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:43 smithi174 ceph-mon[35370]: pgmap v25: 97 pgs: 97 active+clean; 813 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 MiB/s wr, 177 op/s 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.130+0000 7f6464de2640 1 -- 172.21.15.103:0/1707151032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460106f00 msgr2=0x7f646010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.130+0000 7f6464de2640 1 --2- 172.21.15.103:0/1707151032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460106f00 0x7f646010ea80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6458009be0 tx=0x7f645802e840 comp rx=0 tx=0).stop 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.131+0000 7f6464de2640 1 -- 172.21.15.103:0/1707151032 shutdown_connections 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.131+0000 7f6464de2640 1 --2- 172.21.15.103:0/1707151032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460106f00 0x7f646010ea80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.131+0000 7f6464de2640 1 --2- 172.21.15.103:0/1707151032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6460102780 0x7f64601067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.131+0000 7f6464de2640 1 --2- 172.21.15.103:0/1707151032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 0x7f64601021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.131+0000 7f6464de2640 1 -- 172.21.15.103:0/1707151032 >> 172.21.15.103:0/1707151032 conn(0x7f64600fd8b0 msgr2=0x7f64600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.131+0000 7f6464de2640 1 -- 172.21.15.103:0/1707151032 shutdown_connections 2024-09-19T07:38:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.131+0000 7f6464de2640 1 -- 172.21.15.103:0/1707151032 wait complete. 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 Processor -- start 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 -- start start 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 0x7f6460196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460102780 0x7f6460196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6460106f00 0x7f646019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f646010fd50 con 0x7f6460101dd0 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f646010fbd0 con 0x7f6460102780 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6464de2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f646010fed0 con 0x7f6460106f00 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 0x7f6460196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 0x7f6460196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45710/0 (socket says 172.21.15.103:45710) 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645e59c640 1 -- 172.21.15.103:0/2077411845 learned_addr learned my addr 172.21.15.103:0/2077411845 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645ed9d640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6460106f00 0x7f646019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f6457fff640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460102780 0x7f6460196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645ed9d640 1 -- 172.21.15.103:0/2077411845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460102780 msgr2=0x7f6460196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645ed9d640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460102780 0x7f6460196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645ed9d640 1 -- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 msgr2=0x7f6460196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.132+0000 7f645ed9d640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 0x7f6460196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f645ed9d640 1 -- 172.21.15.103:0/2077411845 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646019b200 con 0x7f6460106f00 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f6457fff640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460102780 0x7f6460196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f645e59c640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 0x7f6460196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f645ed9d640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6460106f00 0x7f646019ab20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f645802ed60 tx=0x7f6458035940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f64577fe640 1 -- 172.21.15.103:0/2077411845 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645803b050 con 0x7f6460106f00 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646019b490 con 0x7f6460106f00 2024-09-19T07:38:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f64577fe640 1 -- 172.21.15.103:0/2077411845 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6458047070 con 0x7f6460106f00 2024-09-19T07:38:44.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64601a2dd0 con 0x7f6460106f00 2024-09-19T07:38:44.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.133+0000 7f64577fe640 1 -- 172.21.15.103:0/2077411845 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458042450 con 0x7f6460106f00 2024-09-19T07:38:44.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.137+0000 7f64671bb740 1 Processor -- start 2024-09-19T07:38:44.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.137+0000 7f64577fe640 1 -- 172.21.15.103:0/2077411845 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6458004040 con 0x7f6460106f00 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64671bb740 1 -- start start 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64671bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effa5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64671bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa55d0 0x5601effa5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64671bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effb0370 0x5601effb2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64671bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5601eff0b2f0 con 0x5601effa6b40 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64671bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5601efeb2cf0 con 0x5601effa55d0 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64671bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601eff072b0 con 0x5601effb0370 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f645e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa55d0 0x5601effa5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effa5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effa5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45722/0 (socket says 172.21.15.103:45722) 2024-09-19T07:38:44.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f645e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa55d0 0x5601effa5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42648/0 (socket says 172.21.15.103:42648) 2024-09-19T07:38:44.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 -- 172.21.15.103:0/2737770514 learned_addr learned my addr 172.21.15.103:0/2737770514 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:44.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effb0370 0x5601effb2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f64577fe640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f644c0a34c0 0x7f644c0a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 -- 172.21.15.103:0/2737770514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effb0370 msgr2=0x5601effb2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effb0370 0x5601effb2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 -- 172.21.15.103:0/2737770514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa55d0 msgr2=0x5601effa5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.138+0000 7f6457fff640 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa55d0 0x5601effa5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.139+0000 7f6457fff640 1 -- 172.21.15.103:0/2737770514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601efe973a0 con 0x5601effa6b40 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.139+0000 7f64577fe640 1 -- 172.21.15.103:0/2077411845 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f64580f4730 con 0x7f6460106f00 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.139+0000 7f645e59c640 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa55d0 0x5601effa5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:44.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.139+0000 7f645ed9d640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f644c0a34c0 0x7f644c0a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.139+0000 7f6457fff640 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effa5090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6448012770 tx=0x7f64480096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6fca2898c530b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.140+0000 7f6454ff9640 1 -- 172.21.15.103:0/2737770514 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448012cf0 con 0x5601effa6b40 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.140+0000 7f645ed9d640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f644c0a34c0 0x7f644c0a5980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64580f1e30 tx=0x7f6458009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.140+0000 7f6454ff9640 1 -- 172.21.15.103:0/2737770514 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6448010cb0 con 0x5601effa6b40 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.140+0000 7f6454ff9640 1 -- 172.21.15.103:0/2737770514 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644801ec40 con 0x5601effa6b40 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.140+0000 7f64671bb740 1 -- 172.21.15.103:0/2737770514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 msgr2=0x5601effa5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.140+0000 7f64671bb740 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effa5090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6448012770 tx=0x7f64480096a0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- 172.21.15.103:0/2737770514 shutdown_connections 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effb0370 0x5601effb2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa55d0 0x5601effa5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 --2- 172.21.15.103:0/2737770514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effa5090 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- 172.21.15.103:0/2737770514 >> 172.21.15.103:0/2737770514 conn(0x5601eff6e310 msgr2=0x5601effa1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- 172.21.15.103:0/2737770514 shutdown_connections 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- 172.21.15.103:0/2737770514 wait complete. 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- start start 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 Processor -- start 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effafda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa6b40 0x5601effad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effb0370 0x5601effad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5601effbb790 con 0x5601effb0370 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5601efec6060 con 0x5601effa6b40 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601effbb910 con 0x5601effa55d0 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.141+0000 7f64671bb740 10 client.0 _ll_register_callbacks cb 0x5601efc198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:44.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effafda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effafda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50760/0 (socket says 172.21.15.103:50760) 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 -- 172.21.15.103:0/1751962439 learned_addr learned my addr 172.21.15.103:0/1751962439 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effb0370 0x5601effad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f645e59c640 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa6b40 0x5601effad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 -- 172.21.15.103:0/1751962439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa6b40 msgr2=0x5601effad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa6b40 0x5601effad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 -- 172.21.15.103:0/1751962439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effb0370 msgr2=0x5601effad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effb0370 0x5601effad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 -- 172.21.15.103:0/1751962439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601efc7c3c0 con 0x5601effa55d0 2024-09-19T07:38:44.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6457fff640 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effafda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6448009860 tx=0x7f644801b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d61f1fdfd62038 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:44.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:44.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.142+0000 7f6445ffb640 1 -- 172.21.15.103:0/1751962439 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64480109b0 con 0x5601effa55d0 2024-09-19T07:38:44.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.143+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6448010cb0 con 0x5601effa55d0 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.143+0000 7f6445ffb640 1 -- 172.21.15.103:0/1751962439 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f644801e9e0 con 0x5601effa55d0 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.143+0000 7f6445ffb640 1 -- 172.21.15.103:0/1751962439 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f644801ebe0 con 0x5601effa55d0 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.143+0000 7f64671bb740 10 client.25120 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.143+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601effc7310 con 0x5601effa55d0 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f6445ffb640 1 -- 172.21.15.103:0/1751962439 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644801ed80 con 0x5601effa55d0 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f6445ffb640 1 -- 172.21.15.103:0/1751962439 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f644802a040 con 0x5601effa55d0 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f64671bb740 10 client.25120 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f64671bb740 10 client.25120 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f64671bb740 20 client.25120 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f64671bb740 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601effc8b70 0x5601effcaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f64671bb740 4 client.25120 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.144+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6448010cb0 con 0x5601effc8b70 2024-09-19T07:38:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.145+0000 7f645e59c640 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601effc8b70 0x5601effcaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.145+0000 7f645e59c640 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601effc8b70 0x5601effcaf90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f075fdf66314558b server_cookie=cb0b0e0959644c02 in_seq=0 out_seq=0 2024-09-19T07:38:44.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.146+0000 7f6445ffb640 10 client.25120 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.149+0000 7f6445ffb640 1 -- 172.21.15.103:0/1751962439 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6448010cb0 con 0x5601effc8b70 2024-09-19T07:38:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.149+0000 7f6445ffb640 10 client.25120 handle_command_reply: tid=0 2024-09-19T07:38:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.150+0000 7f64671bb740 1 client.25120 shutdown 2024-09-19T07:38:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.150+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 msgr2=0x5601effafda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.150+0000 7f64671bb740 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effafda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6448009860 tx=0x7f644801b040 comp rx=0 tx=0).stop 2024-09-19T07:38:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.150+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 shutdown_connections 2024-09-19T07:38:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.150+0000 7f64671bb740 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601effc8b70 0x5601effcaf90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.150+0000 7f64671bb740 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effb0370 0x5601effad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.151+0000 7f64671bb740 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effa6b40 0x5601effad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.151+0000 7f64671bb740 1 --2- 172.21.15.103:0/1751962439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effafda0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.151+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 >> 172.21.15.103:0/1751962439 conn(0x5601eff6e310 msgr2=0x5601effa7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:44.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.151+0000 7f6445ffb640 0 client.25120 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.152+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 shutdown_connections 2024-09-19T07:38:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.152+0000 7f64671bb740 1 -- 172.21.15.103:0/1751962439 wait complete. 2024-09-19T07:38:44.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.152+0000 7f64671bb740 20 client.25120 trim_cache size 0 max 16384 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 Processor -- start 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 -- start start 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effa6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601efface50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f644000e810 con 0x5601effa6b40 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6440010350 con 0x5601effb0370 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f64671bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f644000eee0 con 0x5601effa55d0 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f645e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f645e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effac910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45746/0 (socket says 172.21.15.103:45746) 2024-09-19T07:38:44.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f645e59c640 1 -- 172.21.15.103:0/1029316009 learned_addr learned my addr 172.21.15.103:0/1029316009 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:44.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f6457fff640 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effa6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.174+0000 7f6457fff640 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601efface50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6457fff640 1 -- 172.21.15.103:0/1029316009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 msgr2=0x5601efface50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6457fff640 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601efface50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6457fff640 1 -- 172.21.15.103:0/1029316009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 msgr2=0x5601effac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6457fff640 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6457fff640 1 -- 172.21.15.103:0/1029316009 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601effa14f0 con 0x5601effa55d0 2024-09-19T07:38:44.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6457fff640 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effa6300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6448003e90 tx=0x7f6448000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a4d48c9a6bbb27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:44.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6447fff640 1 -- 172.21.15.103:0/1029316009 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644801e450 con 0x5601effa55d0 2024-09-19T07:38:44.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6447fff640 1 -- 172.21.15.103:0/1029316009 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644801e8c0 con 0x5601effa55d0 2024-09-19T07:38:44.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.175+0000 7f6447fff640 1 -- 172.21.15.103:0/1029316009 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448005340 con 0x5601effa55d0 2024-09-19T07:38:44.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 -- 172.21.15.103:0/1029316009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 msgr2=0x5601effa6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effa6300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6448003e90 tx=0x7f6448000bc0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 -- 172.21.15.103:0/1029316009 shutdown_connections 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601efface50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 --2- 172.21.15.103:0/1029316009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effa6300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 -- 172.21.15.103:0/1029316009 >> 172.21.15.103:0/1029316009 conn(0x5601eff6e310 msgr2=0x5601effa1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 -- 172.21.15.103:0/1029316009 shutdown_connections 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.176+0000 7f64671bb740 1 -- 172.21.15.103:0/1029316009 wait complete. 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f64671bb740 1 -- start start 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f64671bb740 1 Processor -- start 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f64671bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effc1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f64671bb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effbbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f6457fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effc1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f6457fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effc1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50768/0 (socket says 172.21.15.103:50768) 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f6457fff640 1 -- 172.21.15.103:0/91216969 learned_addr learned my addr 172.21.15.103:0/91216969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f64671bb740 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601effbc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f644000fc80 con 0x5601effa6b40 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.177+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f644000f5b0 con 0x5601effb0370 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f644000c7f0 con 0x5601effa55d0 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f64671bb740 10 client.0 _ll_register_callbacks cb 0x7f6450009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f645e59c640 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effbbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6457fff640 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601effbc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6457fff640 1 -- 172.21.15.103:0/91216969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 msgr2=0x5601effbc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6457fff640 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601effbc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6457fff640 1 -- 172.21.15.103:0/91216969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 msgr2=0x5601effbbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:44.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6457fff640 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effbbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6457fff640 1 -- 172.21.15.103:0/91216969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601effbc7b0 con 0x5601effa55d0 2024-09-19T07:38:44.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6457fff640 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effc1200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64480096e0 tx=0x7f6448009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35068f9faf651df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:44.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.178+0000 7f6445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:44.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.179+0000 7f6445ffb640 1 -- 172.21.15.103:0/91216969 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448017040 con 0x5601effa55d0 2024-09-19T07:38:44.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.180+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644801e8c0 con 0x5601effa55d0 2024-09-19T07:38:44.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.180+0000 7f6445ffb640 1 -- 172.21.15.103:0/91216969 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6448012040 con 0x5601effa55d0 2024-09-19T07:38:44.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.180+0000 7f6445ffb640 1 -- 172.21.15.103:0/91216969 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6448047050 con 0x5601effa55d0 2024-09-19T07:38:44.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.180+0000 7f64671bb740 10 client.25126 fetch_fsmap learned FSMap version 27 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.180+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601f0032690 con 0x5601effa55d0 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.181+0000 7f6445ffb640 1 -- 172.21.15.103:0/91216969 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64480035f0 con 0x5601effa55d0 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.181+0000 7f6445ffb640 1 -- 172.21.15.103:0/91216969 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f644802c470 con 0x5601effa55d0 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.181+0000 7f64671bb740 10 client.25126 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.181+0000 7f64671bb740 10 client.25126 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.181+0000 7f64671bb740 20 client.25126 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.181+0000 7f64671bb740 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601f0032cd0 0x5601f00350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.182+0000 7f64671bb740 4 client.25126 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.182+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f644801e8c0 con 0x5601f0032cd0 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.182+0000 7f645e59c640 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601f0032cd0 0x5601f00350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:44.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f645e59c640 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601f0032cd0 0x5601f00350f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f05b4f7c341471a server_cookie=df99ceccd74c241b in_seq=0 out_seq=0 2024-09-19T07:38:44.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f6445ffb640 10 client.25126 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f6445ffb640 1 -- 172.21.15.103:0/91216969 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f644801e8c0 con 0x5601f0032cd0 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f6445ffb640 10 client.25126 handle_command_reply: tid=0 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 client.25126 shutdown 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 msgr2=0x5601effc1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effc1200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64480096e0 tx=0x7f6448009670 comp rx=0 tx=0).stop 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 shutdown_connections 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5601f0032cd0 0x5601f00350f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601effb0370 0x5601effbc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601effa6b40 0x5601effbbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 --2- 172.21.15.103:0/91216969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601effa55d0 0x5601effc1200 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 >> 172.21.15.103:0/91216969 conn(0x5601eff6e310 msgr2=0x5601effa1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:44.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.183+0000 7f6445ffb640 0 client.25126 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:38:44.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.184+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 shutdown_connections 2024-09-19T07:38:44.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.184+0000 7f64671bb740 1 -- 172.21.15.103:0/91216969 wait complete. 2024-09-19T07:38:44.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.184+0000 7f64671bb740 20 client.25126 trim_cache size 0 max 16384 2024-09-19T07:38:44.186 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f644c0a34c0 msgr2=0x7f644c0a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f644c0a34c0 0x7f644c0a5980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64580f1e30 tx=0x7f6458009500 comp rx=0 tx=0).stop 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6460106f00 msgr2=0x7f646019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6460106f00 0x7f646019ab20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f645802ed60 tx=0x7f6458035940 comp rx=0 tx=0).stop 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 shutdown_connections 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f644c0a34c0 0x7f644c0a5980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6460106f00 0x7f646019ab20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6460102780 0x7f6460196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 --2- 172.21.15.103:0/2077411845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6460101dd0 0x7f6460196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:44.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 >> 172.21.15.103:0/2077411845 conn(0x7f64600fd8b0 msgr2=0x7f64600feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:44.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 shutdown_connections 2024-09-19T07:38:44.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:44.185+0000 7f6464de2640 1 -- 172.21.15.103:0/2077411845 wait complete. 2024-09-19T07:38:44.790 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T07:38:44.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:38:44.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:38:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 10.0 s, 3782.6 tps, lat 26.413 ms stddev 27.732 2024-09-19T07:38:45.579 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:45 smithi103 ceph-mon[31201]: pgmap v26: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 MiB/s wr, 501 op/s 2024-09-19T07:38:45.579 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:45 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:38:45.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 -- 172.21.15.103:0/2049663843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 msgr2=0x7f691c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:45.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 --2- 172.21.15.103:0/2049663843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c1021d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f690c009900 tx=0x7f690c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:38:45.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 -- 172.21.15.103:0/2049663843 shutdown_connections 2024-09-19T07:38:45.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 --2- 172.21.15.103:0/2049663843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f691c106f20 0x7f691c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:45.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 --2- 172.21.15.103:0/2049663843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f691c1027a0 0x7f691c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:45.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 --2- 172.21.15.103:0/2049663843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c1021d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:45.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 -- 172.21.15.103:0/2049663843 >> 172.21.15.103:0/2049663843 conn(0x7f691c0fd8f0 msgr2=0x7f691c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:45.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 -- 172.21.15.103:0/2049663843 shutdown_connections 2024-09-19T07:38:45.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.691+0000 7f6921578640 1 -- 172.21.15.103:0/2049663843 wait complete. 2024-09-19T07:38:45.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.692+0000 7f6921578640 1 Processor -- start 2024-09-19T07:38:45.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.692+0000 7f6921578640 1 -- start start 2024-09-19T07:38:45.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.692+0000 7f6921578640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c117320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:45.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.692+0000 7f6921578640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f691c1027a0 0x7f691c117860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:45.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.692+0000 7f6921578640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f691c106f20 0x7f691c1115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:45.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.693+0000 7f691affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c117320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:45.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.693+0000 7f691affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c117320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50796/0 (socket says 172.21.15.103:50796) 2024-09-19T07:38:45.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.693+0000 7f691affd640 1 -- 172.21.15.103:0/3292804190 learned_addr learned my addr 172.21.15.103:0/3292804190 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:45.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.693+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10fde0 con 0x7f691c106f20 2024-09-19T07:38:45.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.693+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10fc60 con 0x7f691c1027a0 2024-09-19T07:38:45.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.693+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10ff60 con 0x7f691c101df0 2024-09-19T07:38:45.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f6913fff640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f691c1027a0 0x7f691c117860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:45.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f691affd640 1 -- 172.21.15.103:0/3292804190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f691c1027a0 msgr2=0x7f691c117860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:45.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f691affd640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f691c1027a0 0x7f691c117860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:45.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f691affd640 1 -- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f691c106f20 msgr2=0x7f691c1115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:45.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f691affd640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f691c106f20 0x7f691c1115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:45.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f691affd640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c111e90 con 0x7f691c101df0 2024-09-19T07:38:45.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f6913fff640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f691c1027a0 0x7f691c117860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:45.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.694+0000 7f691affd640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c117320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f690c005ab0 tx=0x7f690c003bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:45.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.695+0000 7f6918ff9640 1 -- 172.21.15.103:0/3292804190 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c03f070 con 0x7f691c101df0 2024-09-19T07:38:45.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.695+0000 7f6918ff9640 1 -- 172.21.15.103:0/3292804190 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f690c03b460 con 0x7f691c101df0 2024-09-19T07:38:45.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.695+0000 7f6918ff9640 1 -- 172.21.15.103:0/3292804190 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c035030 con 0x7f691c101df0 2024-09-19T07:38:45.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.695+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1120c0 con 0x7f691c101df0 2024-09-19T07:38:45.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.695+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1b3d20 con 0x7f691c101df0 2024-09-19T07:38:45.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.697+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c1032d0 con 0x7f691c101df0 2024-09-19T07:38:45.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.698+0000 7f6918ff9640 1 -- 172.21.15.103:0/3292804190 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f690c042070 con 0x7f691c101df0 2024-09-19T07:38:45.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.699+0000 7f6918ff9640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f69080a35d0 0x7f69080a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:45.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.699+0000 7f6918ff9640 1 -- 172.21.15.103:0/3292804190 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f690c0f0720 con 0x7f691c101df0 2024-09-19T07:38:45.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.699+0000 7f6913fff640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f69080a35d0 0x7f69080a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:45.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.700+0000 7f6913fff640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f69080a35d0 0x7f69080a5a90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f691c112b60 tx=0x7f6904005d30 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:45.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:45.703+0000 7f6918ff9640 1 -- 172.21.15.103:0/3292804190 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f690c0bcf40 con 0x7f691c101df0 2024-09-19T07:38:45.750 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:45.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:45 smithi134 ceph-mon[35414]: pgmap v26: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 MiB/s wr, 501 op/s 2024-09-19T07:38:45.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:45 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:38:45.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:45 smithi174 ceph-mon[35370]: pgmap v26: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 MiB/s wr, 501 op/s 2024-09-19T07:38:45.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:45 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:38:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.039+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f691c1021d0 con 0x7f691c101df0 2024-09-19T07:38:46.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.040+0000 7f6918ff9640 1 -- 172.21.15.103:0/3292804190 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11994 (secure 0 0 0) 0x7f690c0bc840 con 0x7f691c101df0 2024-09-19T07:38:46.041 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:46.041 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":28,"btime":"2024-09-19T07:38:44:429284+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:38:43.516586+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:38:46.042 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 28 2024-09-19T07:38:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.044+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f69080a35d0 msgr2=0x7f69080a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.044+0000 7f6921578640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f69080a35d0 0x7f69080a5a90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f691c112b60 tx=0x7f6904005d30 comp rx=0 tx=0).stop 2024-09-19T07:38:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.044+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 msgr2=0x7f691c117320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.044+0000 7f6921578640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c117320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f690c005ab0 tx=0x7f690c003bb0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 shutdown_connections 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f69080a35d0 0x7f69080a5a90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f691c106f20 0x7f691c1115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f691c1027a0 0x7f691c117860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 --2- 172.21.15.103:0/3292804190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f691c101df0 0x7f691c117320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 >> 172.21.15.103:0/3292804190 conn(0x7f691c0fd8f0 msgr2=0x7f691c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 shutdown_connections 2024-09-19T07:38:46.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.046+0000 7f6921578640 1 -- 172.21.15.103:0/3292804190 wait complete. 2024-09-19T07:38:46.502 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:46 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3292804190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:46.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.552+0000 7f0fe74c5640 1 -- 172.21.15.103:0/915985277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 msgr2=0x7f0fe0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.552+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/915985277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe0106f60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009900 tx=0x7f0fdc02a410 comp rx=0 tx=0).stop 2024-09-19T07:38:46.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.552+0000 7f0fe74c5640 1 -- 172.21.15.103:0/915985277 shutdown_connections 2024-09-19T07:38:46.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.552+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/915985277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fe01005a0 0x7f0fe010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.552+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/915985277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fe00ffc00 0x7f0fe0100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.552+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/915985277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe0106f60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.552+0000 7f0fe74c5640 1 -- 172.21.15.103:0/915985277 >> 172.21.15.103:0/915985277 conn(0x7f0fe00fba90 msgr2=0x7f0fe00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:46.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.554+0000 7f0fe74c5640 1 -- 172.21.15.103:0/915985277 shutdown_connections 2024-09-19T07:38:46.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.554+0000 7f0fe74c5640 1 -- 172.21.15.103:0/915985277 wait complete. 2024-09-19T07:38:46.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.555+0000 7f0fe74c5640 1 Processor -- start 2024-09-19T07:38:46.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.555+0000 7f0fe74c5640 1 -- start start 2024-09-19T07:38:46.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe74c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fe00ffc00 0x7f0fe019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe74c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fe01005a0 0x7f0fe019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe74c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe74c5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0111fa0 con 0x7f0fe0106b80 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe74c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0111e20 con 0x7f0fe01005a0 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe74c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0112120 con 0x7f0fe00ffc00 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45784/0 (socket says 172.21.15.103:45784) 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 -- 172.21.15.103:0/3956268577 learned_addr learned my addr 172.21.15.103:0/3956268577 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5261640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fe00ffc00 0x7f0fe019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fe01005a0 0x7f0fe019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 -- 172.21.15.103:0/3956268577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fe00ffc00 msgr2=0x7f0fe019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fe00ffc00 0x7f0fe019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 -- 172.21.15.103:0/3956268577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fe01005a0 msgr2=0x7f0fe019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fe01005a0 0x7f0fe019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 -- 172.21.15.103:0/3956268577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe019f560 con 0x7f0fe0106b80 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fe01005a0 0x7f0fe019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5261640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fe00ffc00 0x7f0fe019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.556+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe019ee80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800ccb0 tx=0x7f0fd800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.557+0000 7f0fd67fc640 1 -- 172.21.15.103:0/3956268577 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8003970 con 0x7f0fe0106b80 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.557+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe019f850 con 0x7f0fe0106b80 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.557+0000 7f0fd67fc640 1 -- 172.21.15.103:0/3956268577 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fd8004030 con 0x7f0fe0106b80 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.557+0000 7f0fd67fc640 1 -- 172.21.15.103:0/3956268577 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800f780 con 0x7f0fe0106b80 2024-09-19T07:38:46.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.557+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe01a7130 con 0x7f0fe0106b80 2024-09-19T07:38:46.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.560+0000 7f0fd67fc640 1 -- 172.21.15.103:0/3956268577 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f0fd80041e0 con 0x7f0fe0106b80 2024-09-19T07:38:46.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 Processor -- start 2024-09-19T07:38:46.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 -- start start 2024-09-19T07:38:46.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fd67fc640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0fcc0a34c0 0x7f0fcc0a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9e88090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564aa9dee2f0 con 0x564aa9e89b40 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564aa9d95cf0 con 0x564aa9e93370 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe989e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564aa9dea2b0 con 0x564aa9e885d0 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fd67fc640 1 -- 172.21.15.103:0/3956268577 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0fd80cff40 con 0x7f0fe0106b80 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0fcc0a34c0 0x7f0fcc0a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe5261640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9e88090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe5261640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9e88090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45788/0 (socket says 172.21.15.103:45788) 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe5261640 1 -- 172.21.15.103:0/1489868544 learned_addr learned my addr 172.21.15.103:0/1489868544 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.561+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5a62640 1 -- 172.21.15.103:0/1489868544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 msgr2=0x564aa9e88a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e88a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5a62640 1 -- 172.21.15.103:0/1489868544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 msgr2=0x564aa9e88090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9e88090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5a62640 1 -- 172.21.15.103:0/1489868544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa9d7a3a0 con 0x564aa9e93370 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5261640 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9e88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0fcc0a34c0 0x7f0fcc0a5980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00058c0 tx=0x7f0fd000cc00 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e95760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80cd5e0 tx=0x7f0fd809bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b0d50618215543 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:46.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fcb7fe640 1 -- 172.21.15.103:0/1489868544 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd801d070 con 0x564aa9e93370 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fcb7fe640 1 -- 172.21.15.103:0/1489868544 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fd809c7c0 con 0x564aa9e93370 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.562+0000 7f0fcb7fe640 1 -- 172.21.15.103:0/1489868544 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd809a3c0 con 0x564aa9e93370 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 -- 172.21.15.103:0/1489868544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 msgr2=0x564aa9e95760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e95760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80cd5e0 tx=0x7f0fd809bc00 comp rx=0 tx=0).stop 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 -- 172.21.15.103:0/1489868544 shutdown_connections 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e95760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 --2- 172.21.15.103:0/1489868544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9e88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 -- 172.21.15.103:0/1489868544 >> 172.21.15.103:0/1489868544 conn(0x564aa9e51310 msgr2=0x564aa9e84c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 -- 172.21.15.103:0/1489868544 shutdown_connections 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.563+0000 7f0fe989e740 1 -- 172.21.15.103:0/1489868544 wait complete. 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.564+0000 7f0fe989e740 1 -- start start 2024-09-19T07:38:46.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.564+0000 7f0fe989e740 1 Processor -- start 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.565+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e885d0 0x564aa9e92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.565+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e89b40 0x564aa9e90040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.565+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e90650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe989e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564aa9e96070 con 0x564aa9e885d0 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe989e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564aa9da9060 con 0x564aa9e93370 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe989e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564aa9e961f0 con 0x564aa9e89b40 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe5261640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e885d0 0x564aa9e92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe5261640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e885d0 0x564aa9e92d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45794/0 (socket says 172.21.15.103:45794) 2024-09-19T07:38:46.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe5261640 1 -- 172.21.15.103:0/2585335406 learned_addr learned my addr 172.21.15.103:0/2585335406 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe989e740 10 client.0 _ll_register_callbacks cb 0x564aa9afc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e90650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.566+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e89b40 0x564aa9e90040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe5261640 1 -- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e89b40 msgr2=0x564aa9e90040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe5261640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e89b40 0x564aa9e90040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe5261640 1 -- 172.21.15.103:0/2585335406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 msgr2=0x564aa9e90650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe5261640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e90650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe5261640 1 -- 172.21.15.103:0/2585335406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa9b5f3c0 con 0x564aa9e885d0 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e90650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe5261640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e885d0 0x564aa9e92d90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc006010 tx=0x7f0fdc003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72cf7e4b5dd4467 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:46.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/2585335406 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc035040 con 0x564aa9e885d0 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/2585335406 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0fdc003c80 con 0x564aa9e885d0 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fd809c7c0 con 0x564aa9e885d0 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/2585335406 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0fdc005020 con 0x564aa9e885d0 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe989e740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.567+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564aa9ebb360 con 0x564aa9e885d0 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/2585335406 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fd809c7c0 con 0x564aa9e885d0 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/2585335406 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0fdc031540 con 0x564aa9e885d0 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fe989e740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fe989e740 10 client.15570 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fe989e740 20 client.15570 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fe989e740 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9ebc770 0x564aa9ebebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fe989e740 4 client.15570 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:46.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aa9dfb860 con 0x564aa9ebc770 2024-09-19T07:38:46.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.568+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9ebc770 0x564aa9ebebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.570+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9ebc770 0x564aa9ebebb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=645d328c33cfefa9 server_cookie=72633361d3f9bde8 in_seq=0 out_seq=0 2024-09-19T07:38:46.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.570+0000 7f0fc8ff9640 10 client.15570 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:46.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.571+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/2585335406 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564aa9dfb860 con 0x564aa9ebc770 2024-09-19T07:38:46.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.571+0000 7f0fc8ff9640 10 client.15570 handle_command_reply: tid=0 2024-09-19T07:38:46.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.571+0000 7f0fe989e740 1 client.15570 shutdown 2024-09-19T07:38:46.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e885d0 msgr2=0x564aa9e92d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e885d0 0x564aa9e92d90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc006010 tx=0x7f0fdc003890 comp rx=0 tx=0).stop 2024-09-19T07:38:46.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 shutdown_connections 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9ebc770 0x564aa9ebebb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9e90650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e89b40 0x564aa9e90040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 --2- 172.21.15.103:0/2585335406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e885d0 0x564aa9e92d90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 >> 172.21.15.103:0/2585335406 conn(0x564aa9e51310 msgr2=0x564aa9e8a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fc8ff9640 0 client.15570 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 shutdown_connections 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.572+0000 7f0fe989e740 1 -- 172.21.15.103:0/2585335406 wait complete. 2024-09-19T07:38:46.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.573+0000 7f0fe989e740 20 client.15570 trim_cache size 0 max 16384 2024-09-19T07:38:46.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 Processor -- start 2024-09-19T07:38:46.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 -- start start 2024-09-19T07:38:46.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e90010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ec4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9ec49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe006ff10 con 0x564aa9e89b40 2024-09-19T07:38:46.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0071ab0 con 0x564aa9e93370 2024-09-19T07:38:46.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe989e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe00705e0 con 0x564aa9e885d0 2024-09-19T07:38:46.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe5261640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e90010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe5261640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e90010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50836/0 (socket says 172.21.15.103:50836) 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe5261640 1 -- 172.21.15.103:0/4030405513 learned_addr learned my addr 172.21.15.103:0/4030405513 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.592+0000 7f0fe4a60640 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ec4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9ec49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fe5261640 1 -- 172.21.15.103:0/4030405513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 msgr2=0x564aa9ec49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fe5261640 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9ec49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fe5261640 1 -- 172.21.15.103:0/4030405513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 msgr2=0x564aa9ec4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fe5261640 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ec4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fe5261640 1 -- 172.21.15.103:0/4030405513 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa9e8b120 con 0x564aa9e885d0 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fe5261640 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e90010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc02a930 tx=0x7f0fdc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ba71f99039f356 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fcaffd640 1 -- 172.21.15.103:0/4030405513 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc035050 con 0x564aa9e885d0 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fcaffd640 1 -- 172.21.15.103:0/4030405513 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fdc031d30 con 0x564aa9e885d0 2024-09-19T07:38:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.593+0000 7f0fcaffd640 1 -- 172.21.15.103:0/4030405513 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc004750 con 0x564aa9e885d0 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.594+0000 7f0fe989e740 1 -- 172.21.15.103:0/4030405513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 msgr2=0x564aa9e90010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.594+0000 7f0fe989e740 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e90010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc02a930 tx=0x7f0fdc02a410 comp rx=0 tx=0).stop 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.594+0000 7f0fe989e740 1 -- 172.21.15.103:0/4030405513 shutdown_connections 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.594+0000 7f0fe989e740 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9ec49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.594+0000 7f0fe989e740 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ec4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.594+0000 7f0fe989e740 1 --2- 172.21.15.103:0/4030405513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9e90010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.594+0000 7f0fe989e740 1 -- 172.21.15.103:0/4030405513 >> 172.21.15.103:0/4030405513 conn(0x564aa9e51310 msgr2=0x564aa9e8f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.595+0000 7f0fe989e740 1 -- 172.21.15.103:0/4030405513 shutdown_connections 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.595+0000 7f0fe989e740 1 -- 172.21.15.103:0/4030405513 wait complete. 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.595+0000 7f0fe989e740 1 -- start start 2024-09-19T07:38:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 1 Processor -- start 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9ea01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ea0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9f26450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0071380 con 0x564aa9e89b40 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0070cb0 con 0x564aa9e93370 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe006df30 con 0x564aa9e885d0 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe989e740 10 client.0 _ll_register_callbacks cb 0x7f0fdc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe5261640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9ea01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe4a60640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ea0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe5a62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9f26450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe4a60640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ea0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45814/0 (socket says 172.21.15.103:45814) 2024-09-19T07:38:46.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.596+0000 7f0fe4a60640 1 -- 172.21.15.103:0/3636487068 learned_addr learned my addr 172.21.15.103:0/3636487068 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fe5a62640 1 -- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 msgr2=0x564aa9ea01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9ea01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fe5a62640 1 -- 172.21.15.103:0/3636487068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 msgr2=0x564aa9ea0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ea0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fe5a62640 1 -- 172.21.15.103:0/3636487068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa9f26a10 con 0x564aa9e93370 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fe5261640 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9ea01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fe5a62640 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9f26450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0fd809c050 tx=0x7f0fd8004930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67dbecba8fb79030 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/3636487068 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd801d040 con 0x564aa9e93370 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.597+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/3636487068 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0fd80850b0 con 0x564aa9e93370 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.598+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fdc031d30 con 0x564aa9e93370 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.598+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/3636487068 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0fd809a160 con 0x564aa9e93370 2024-09-19T07:38:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.598+0000 7f0fe989e740 10 client.25154 fetch_fsmap learned FSMap version 28 2024-09-19T07:38:46.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.598+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564aa9f27cd0 con 0x564aa9e93370 2024-09-19T07:38:46.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.598+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/3636487068 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fd809c0c0 con 0x564aa9e93370 2024-09-19T07:38:46.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.599+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/3636487068 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0fd8082600 con 0x564aa9e93370 2024-09-19T07:38:46.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.599+0000 7f0fe989e740 10 client.25154 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T07:38:46.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.599+0000 7f0fe989e740 10 client.25154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.599+0000 7f0fe989e740 20 client.25154 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.599+0000 7f0fe989e740 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9f27fb0 0x564aa9f2a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.600+0000 7f0fe989e740 4 client.25154 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.600+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0fdc031d30 con 0x564aa9f27fb0 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.600+0000 7f0fe5261640 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9f27fb0 0x564aa9f2a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.600+0000 7f0fe5261640 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9f27fb0 0x564aa9f2a3a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd188c96590d09df server_cookie=ab8d39b65b5dcf6a in_seq=0 out_seq=0 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.601+0000 7f0fc8ff9640 10 client.25154 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.601+0000 7f0fc8ff9640 1 -- 172.21.15.103:0/3636487068 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0fdc031d30 con 0x564aa9f27fb0 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.601+0000 7f0fc8ff9640 10 client.25154 handle_command_reply: tid=0 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.601+0000 7f0fe989e740 1 client.25154 shutdown 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 msgr2=0x564aa9f26450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9f26450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0fd809c050 tx=0x7f0fd8004930 comp rx=0 tx=0).stop 2024-09-19T07:38:46.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 shutdown_connections 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x564aa9f27fb0 0x564aa9f2a3a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564aa9e93370 0x564aa9f26450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564aa9e89b40 0x564aa9ea0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 --2- 172.21.15.103:0/3636487068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa9e885d0 0x564aa9ea01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 >> 172.21.15.103:0/3636487068 conn(0x564aa9e51310 msgr2=0x564aa9e8a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fc8ff9640 0 client.25154 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.602+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 shutdown_connections 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.603+0000 7f0fe989e740 1 -- 172.21.15.103:0/3636487068 wait complete. 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.603+0000 7f0fe989e740 20 client.25154 trim_cache size 0 max 16384 2024-09-19T07:38:46.606 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:38:46.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0fcc0a34c0 msgr2=0x7f0fcc0a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0fcc0a34c0 0x7f0fcc0a5980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00058c0 tx=0x7f0fd000cc00 comp rx=0 tx=0).stop 2024-09-19T07:38:46.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 msgr2=0x7f0fe019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:46.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe019ee80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800ccb0 tx=0x7f0fd800a620 comp rx=0 tx=0).stop 2024-09-19T07:38:46.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 shutdown_connections 2024-09-19T07:38:46.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0fcc0a34c0 0x7f0fcc0a5980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fe0106b80 0x7f0fe019ee80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fe01005a0 0x7f0fe019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 --2- 172.21.15.103:0/3956268577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fe00ffc00 0x7f0fe019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:46.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 >> 172.21.15.103:0/3956268577 conn(0x7f0fe00fba90 msgr2=0x7f0fe010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:46.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 shutdown_connections 2024-09-19T07:38:46.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:46.604+0000 7f0fe74c5640 1 -- 172.21.15.103:0/3956268577 wait complete. 2024-09-19T07:38:46.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:46 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3292804190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:46.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:46 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3292804190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:46.954 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T07:38:46.954 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T07:38:47.603 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:47 smithi103 ceph-mon[31201]: pgmap v27: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 465 op/s 2024-09-19T07:38:47.642 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:38:47.774 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:47.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:47 smithi134 ceph-mon[35414]: pgmap v27: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 465 op/s 2024-09-19T07:38:47.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:47 smithi174 ceph-mon[35370]: pgmap v27: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 465 op/s 2024-09-19T07:38:48.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 -- 172.21.15.103:0/1836688279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 msgr2=0x7f6b3c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:48.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/1836688279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 0x7f6b3c101e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009c10 tx=0x7f6b3402a6f0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 -- 172.21.15.103:0/1836688279 shutdown_connections 2024-09-19T07:38:48.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/1836688279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c102370 0x7f6b3c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/1836688279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 0x7f6b3c101e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/1836688279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c1079d0 0x7f6b3c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 -- 172.21.15.103:0/1836688279 >> 172.21.15.103:0/1836688279 conn(0x7f6b3c0fd710 msgr2=0x7f6b3c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:48.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 -- 172.21.15.103:0/1836688279 shutdown_connections 2024-09-19T07:38:48.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.724+0000 7f6b41fa1640 1 -- 172.21.15.103:0/1836688279 wait complete. 2024-09-19T07:38:48.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.725+0000 7f6b41fa1640 1 Processor -- start 2024-09-19T07:38:48.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.726+0000 7f6b41fa1640 1 -- start start 2024-09-19T07:38:48.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.726+0000 7f6b41fa1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 0x7f6b3c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b41fa1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c102370 0x7f6b3c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b41fa1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1079d0 0x7f6b3c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b41fa1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c111dd0 con 0x7f6b3c1079d0 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b41fa1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c111c50 con 0x7f6b3c102370 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b41fa1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c111f50 con 0x7f6b3c1019d0 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c102370 0x7f6b3c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c102370 0x7f6b3c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48924/0 (socket says 172.21.15.103:48924) 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3847946653 learned_addr learned my addr 172.21.15.103:0/3847946653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b40f9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 0x7f6b3c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3847946653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 msgr2=0x7f6b3c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:48.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 0x7f6b3c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1079d0 msgr2=0x7f6b3c19cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:38:48.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1079d0 0x7f6b3c19cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3847946653 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c19d2a0 con 0x7f6b3c102370 2024-09-19T07:38:48.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.727+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 0x7f6b3c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:48.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.729+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c102370 0x7f6b3c198830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b34000f80 tx=0x7f6b34030620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:48.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.730+0000 7f6b39ffb640 1 -- 172.21.15.103:0/3847946653 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34036040 con 0x7f6b3c102370 2024-09-19T07:38:48.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.730+0000 7f6b39ffb640 1 -- 172.21.15.103:0/3847946653 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b34031920 con 0x7f6b3c102370 2024-09-19T07:38:48.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.730+0000 7f6b39ffb640 1 -- 172.21.15.103:0/3847946653 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3403cc30 con 0x7f6b3c102370 2024-09-19T07:38:48.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.730+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c19d530 con 0x7f6b3c102370 2024-09-19T07:38:48.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.730+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c19d900 con 0x7f6b3c102370 2024-09-19T07:38:48.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.732+0000 7f6b39ffb640 1 -- 172.21.15.103:0/3847946653 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6b34003c90 con 0x7f6b3c102370 2024-09-19T07:38:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.733+0000 7f6b39ffb640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b200a3590 0x7f6b200a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.733+0000 7f6b455dc740 1 Processor -- start 2024-09-19T07:38:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.733+0000 7f6b455dc740 1 -- start start 2024-09-19T07:38:48.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.733+0000 7f6b39ffb640 1 -- 172.21.15.103:0/3847946653 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6b340f4910 con 0x7f6b3c102370 2024-09-19T07:38:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65522bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65522c140 0x55e65523edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65523f2f0 0x55e6552416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b455dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6551912a0 con 0x55e65522c140 2024-09-19T07:38:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b455dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e655138b10 con 0x55e65522b820 2024-09-19T07:38:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b455dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e65518d260 con 0x55e65523f2f0 2024-09-19T07:38:48.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65522c140 0x55e65523edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b417a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65523f2f0 0x55e6552416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65522c140 0x55e65523edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45476/0 (socket says 172.21.15.103:45476) 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b3bfff640 1 -- 172.21.15.103:0/890598118 learned_addr learned my addr 172.21.15.103:0/890598118 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b200a3590 0x7f6b200a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b417a0640 1 -- 172.21.15.103:0/890598118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 msgr2=0x55e65522bc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b417a0640 1 --2- 172.21.15.103:0/890598118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65522bc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b417a0640 1 -- 172.21.15.103:0/890598118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65522c140 msgr2=0x55e65523edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:48.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b417a0640 1 --2- 172.21.15.103:0/890598118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65522c140 0x55e65523edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.734+0000 7f6b417a0640 1 -- 172.21.15.103:0/890598118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e654f02b90 con 0x55e65523f2f0 2024-09-19T07:38:48.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.735+0000 7f6b417a0640 1 --2- 172.21.15.103:0/890598118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65523f2f0 0x55e6552416e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00a820 tx=0x7f6b2c00ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e3ecc3ec0b7635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:48.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.735+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b200a3590 0x7f6b200a5a50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800bf50 tx=0x7f6b2800b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:48.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.735+0000 7f6b32ffd640 1 -- 172.21.15.103:0/890598118 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00ccc0 con 0x55e65523f2f0 2024-09-19T07:38:48.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.735+0000 7f6b32ffd640 1 -- 172.21.15.103:0/890598118 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b2c0035f0 con 0x55e65523f2f0 2024-09-19T07:38:48.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.735+0000 7f6b32ffd640 1 -- 172.21.15.103:0/890598118 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00ce60 con 0x55e65523f2f0 2024-09-19T07:38:48.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.735+0000 7f6b455dc740 1 -- 172.21.15.103:0/890598118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65523f2f0 msgr2=0x55e6552416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:48.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.735+0000 7f6b455dc740 1 --2- 172.21.15.103:0/890598118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65523f2f0 0x55e6552416e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00a820 tx=0x7f6b2c00ace0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- 172.21.15.103:0/890598118 shutdown_connections 2024-09-19T07:38:48.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 --2- 172.21.15.103:0/890598118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65523f2f0 0x55e6552416e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 --2- 172.21.15.103:0/890598118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65522c140 0x55e65523edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 --2- 172.21.15.103:0/890598118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65522bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- 172.21.15.103:0/890598118 >> 172.21.15.103:0/890598118 conn(0x55e655227e90 msgr2=0x55e655228270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:48.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- 172.21.15.103:0/890598118 shutdown_connections 2024-09-19T07:38:48.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- 172.21.15.103:0/890598118 wait complete. 2024-09-19T07:38:48.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- start start 2024-09-19T07:38:48.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 Processor -- start 2024-09-19T07:38:48.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65523c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e65523c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e65523cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e655241e60 con 0x55e65523f2f0 2024-09-19T07:38:48.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e6551196e0 con 0x55e65522b820 2024-09-19T07:38:48.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e655241fe0 con 0x55e65522c140 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.736+0000 7f6b455dc740 10 client.0 _ll_register_callbacks cb 0x55e65513da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e65523c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e65523c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37344/0 (socket says 172.21.15.103:37344) 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 -- 172.21.15.103:0/4092679984 learned_addr learned my addr 172.21.15.103:0/4092679984 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b417a0640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e65523cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65523c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 -- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 msgr2=0x55e65523c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65523c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 -- 172.21.15.103:0/4092679984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 msgr2=0x55e65523cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e65523cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 -- 172.21.15.103:0/4092679984 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65524b4b0 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65523c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.737+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e65523c810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b34004af0 tx=0x7f6b340c0740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b01e4c6069e330f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.738+0000 7f6b1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.738+0000 7f6b1ffff640 1 -- 172.21.15.103:0/4092679984 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b340f2400 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.738+0000 7f6b1ffff640 1 -- 172.21.15.103:0/4092679984 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6b340c0da0 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.738+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b2c0035f0 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.738+0000 7f6b1ffff640 1 -- 172.21.15.103:0/4092679984 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b340c4ef0 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.738+0000 7f6b455dc740 10 client.25162 fetch_fsmap learned FSMap version 28 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.738+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e655241ec0 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b1ffff640 1 -- 172.21.15.103:0/4092679984 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b2c0035f0 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b1ffff640 1 -- 172.21.15.103:0/4092679984 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6b34004b40 con 0x55e65522c140 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b455dc740 10 client.25162 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T07:38:48.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b455dc740 10 client.25162 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b455dc740 20 client.25162 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b455dc740 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e65524f2e0 0x55e655251700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b455dc740 4 client.25162 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e65519e9d0 con 0x55e65524f2e0 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.739+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e65524f2e0 0x55e655251700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.740+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e65524f2e0 0x55e655251700 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11738b072b8b1893 server_cookie=e0d918ca1674412a in_seq=0 out_seq=0 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.740+0000 7f6b1ffff640 10 client.25162 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.741+0000 7f6b1ffff640 1 -- 172.21.15.103:0/4092679984 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e65519e9d0 con 0x55e65524f2e0 2024-09-19T07:38:48.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.741+0000 7f6b1ffff640 10 client.25162 handle_command_reply: tid=0 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.741+0000 7f6b455dc740 1 client.25162 shutdown 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.741+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 msgr2=0x55e65523c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.741+0000 7f6b455dc740 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e65523c810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b34004af0 tx=0x7f6b340c0740 comp rx=0 tx=0).stop 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 shutdown_connections 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b455dc740 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e65524f2e0 0x55e655251700 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b1ffff640 0 client.25162 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b455dc740 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e65523cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b455dc740 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e65523c810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b455dc740 1 --2- 172.21.15.103:0/4092679984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e65523c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 >> 172.21.15.103:0/4092679984 conn(0x55e655227e90 msgr2=0x55e655227000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:48.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.742+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 shutdown_connections 2024-09-19T07:38:48.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.743+0000 7f6b455dc740 1 -- 172.21.15.103:0/4092679984 wait complete. 2024-09-19T07:38:48.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.743+0000 7f6b455dc740 20 client.25162 trim_cache size 0 max 16384 2024-09-19T07:38:48.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.771+0000 7f6b455dc740 1 Processor -- start 2024-09-19T07:38:48.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.771+0000 7f6b455dc740 1 -- start start 2024-09-19T07:38:48.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522b820 0x55e655226ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522c140 0x55e655256fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655257510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b455dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1ab530 con 0x55e65523f2f0 2024-09-19T07:38:48.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522c140 0x55e655256fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522c140 0x55e655256fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48948/0 (socket says 172.21.15.103:48948) 2024-09-19T07:38:48.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3998181367 learned_addr learned my addr 172.21.15.103:0/3998181367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:48.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b455dc740 1 -- 172.21.15.103:0/3998181367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1ad0d0 con 0x55e65522c140 2024-09-19T07:38:48.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b417a0640 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655257510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.772+0000 7f6b455dc740 1 -- 172.21.15.103:0/3998181367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1abc00 con 0x55e65522b820 2024-09-19T07:38:48.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.773+0000 7f6b417a0640 1 -- 172.21.15.103:0/3998181367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522b820 msgr2=0x55e655226ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:38:48.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.773+0000 7f6b417a0640 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522b820 0x55e655226ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.773+0000 7f6b417a0640 1 -- 172.21.15.103:0/3998181367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522c140 msgr2=0x55e655256fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:48.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.773+0000 7f6b417a0640 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522c140 0x55e655256fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.773+0000 7f6b417a0640 1 -- 172.21.15.103:0/3998181367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65523b280 con 0x55e65523f2f0 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.773+0000 7f6b417a0640 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655257510 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c6a0 tx=0x7f6b2c005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810bc1da519ae6ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.774+0000 7f6b327fc640 1 -- 172.21.15.103:0/3998181367 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c029070 con 0x55e65523f2f0 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.774+0000 7f6b327fc640 1 -- 172.21.15.103:0/3998181367 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b2c00cd40 con 0x55e65523f2f0 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.774+0000 7f6b327fc640 1 -- 172.21.15.103:0/3998181367 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c018660 con 0x55e65523f2f0 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.774+0000 7f6b455dc740 1 -- 172.21.15.103:0/3998181367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 msgr2=0x55e655257510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.774+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655257510 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c6a0 tx=0x7f6b2c005690 comp rx=0 tx=0).stop 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.775+0000 7f6b455dc740 1 -- 172.21.15.103:0/3998181367 shutdown_connections 2024-09-19T07:38:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.775+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655257510 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.775+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522c140 0x55e655256fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.775+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3998181367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522b820 0x55e655226ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.775+0000 7f6b455dc740 1 -- 172.21.15.103:0/3998181367 >> 172.21.15.103:0/3998181367 conn(0x55e655227e90 msgr2=0x55e6552267e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:48.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.775+0000 7f6b455dc740 1 -- 172.21.15.103:0/3998181367 shutdown_connections 2024-09-19T07:38:48.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.775+0000 7f6b455dc740 1 -- 172.21.15.103:0/3998181367 wait complete. 2024-09-19T07:38:48.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.776+0000 7f6b455dc740 1 -- start start 2024-09-19T07:38:48.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.776+0000 7f6b455dc740 1 Processor -- start 2024-09-19T07:38:48.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e6552442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e655244800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b455dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655244d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b455dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1ac9a0 con 0x55e65523f2f0 2024-09-19T07:38:48.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b455dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1ac2d0 con 0x55e65522b820 2024-09-19T07:38:48.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b455dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1a9510 con 0x55e65522c140 2024-09-19T07:38:48.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b455dc740 10 client.0 _ll_register_callbacks cb 0x7f6b340c0910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:48.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b40f9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e6552442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e655244800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b417a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655244d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b417a0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655244d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45510/0 (socket says 172.21.15.103:45510) 2024-09-19T07:38:48.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.777+0000 7f6b417a0640 1 -- 172.21.15.103:0/3639028833 learned_addr learned my addr 172.21.15.103:0/3639028833 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:48.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3639028833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 msgr2=0x55e6552442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:48.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e6552442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3639028833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 msgr2=0x55e655244d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:48.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655244d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b3bfff640 1 -- 172.21.15.103:0/3639028833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6552ba280 con 0x55e65522c140 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b3bfff640 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e655244800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b340c50b0 tx=0x7f6b3402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3579ca2b47a96d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b1ffff640 1 -- 172.21.15.103:0/3639028833 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b340cd060 con 0x55e65522c140 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.778+0000 7f6b1ffff640 1 -- 172.21.15.103:0/3639028833 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6b340c0da0 con 0x55e65522c140 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.779+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b2c00cd40 con 0x55e65522c140 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.779+0000 7f6b1ffff640 1 -- 172.21.15.103:0/3639028833 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b340bcde0 con 0x55e65522c140 2024-09-19T07:38:48.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.779+0000 7f6b455dc740 10 client.25165 fetch_fsmap learned FSMap version 28 2024-09-19T07:38:48.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.779+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6552bb560 con 0x55e65522c140 2024-09-19T07:38:48.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.779+0000 7f6b1ffff640 1 -- 172.21.15.103:0/3639028833 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b2c00cd40 con 0x55e65522c140 2024-09-19T07:38:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b1ffff640 1 -- 172.21.15.103:0/3639028833 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6b340bc890 con 0x55e65522c140 2024-09-19T07:38:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b455dc740 10 client.25165 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T07:38:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b455dc740 10 client.25165 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:38:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b455dc740 20 client.25165 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e6552bbb90 0x55e6552bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b455dc740 4 client.25165 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T07:38:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x55e655198d20 con 0x55e6552bbb90 2024-09-19T07:38:48.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.780+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e6552bbb90 0x55e6552bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:48.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.781+0000 7f6b1ffff640 10 client.25165 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:48.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:48.781+0000 7f6b40f9f640 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e6552bbb90 0x55e6552bdfb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea8368170856da0 server_cookie=eaeb6a7b5f8dbd4e in_seq=0 out_seq=0 2024-09-19T07:38:48.790 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 -- 172.21.15.103:0/813006565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58102790 msgr2=0x7f6e581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/813006565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58102790 0x7f6e581067e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4802a410 comp rx=0 tx=0).stop 2024-09-19T07:38:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 -- 172.21.15.103:0/813006565 shutdown_connections 2024-09-19T07:38:49.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/813006565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e58106f10 0x7f6e5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/813006565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58102790 0x7f6e581067e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/813006565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e58101de0 0x7f6e581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 -- 172.21.15.103:0/813006565 >> 172.21.15.103:0/813006565 conn(0x7f6e580fd8c0 msgr2=0x7f6e580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 -- 172.21.15.103:0/813006565 shutdown_connections 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.319+0000 7f6e5c97e640 1 -- 172.21.15.103:0/813006565 wait complete. 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 Processor -- start 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 -- start start 2024-09-19T07:38:49.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58101de0 0x7f6e58198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e58102790 0x7f6e58198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e58106f10 0x7f6e5819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58111f30 con 0x7f6e58101de0 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e58111db0 con 0x7f6e58106f10 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5c97e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581120b0 con 0x7f6e58102790 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58101de0 0x7f6e58198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58101de0 0x7f6e58198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45526/0 (socket says 172.21.15.103:45526) 2024-09-19T07:38:49.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e5659c640 1 -- 172.21.15.103:0/1075825660 learned_addr learned my addr 172.21.15.103:0/1075825660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e58106f10 0x7f6e5819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.320+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e58102790 0x7f6e58198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5659c640 1 -- 172.21.15.103:0/1075825660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e58102790 msgr2=0x7f6e58198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5659c640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e58102790 0x7f6e58198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5659c640 1 -- 172.21.15.103:0/1075825660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e58106f10 msgr2=0x7f6e5819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5659c640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e58106f10 0x7f6e5819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5659c640 1 -- 172.21.15.103:0/1075825660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5819d3e0 con 0x7f6e58101de0 2024-09-19T07:38:49.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5659c640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58101de0 0x7f6e58198430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d950 tx=0x7f6e4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e4f7fe640 1 -- 172.21.15.103:0/1075825660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40017030 con 0x7f6e58101de0 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e4f7fe640 1 -- 172.21.15.103:0/1075825660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e40004020 con 0x7f6e58101de0 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5819d6d0 con 0x7f6e58101de0 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e4f7fe640 1 -- 172.21.15.103:0/1075825660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e400047a0 con 0x7f6e58101de0 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.321+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e581a4fb0 con 0x7f6e58101de0 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.323+0000 7f6e4f7fe640 1 -- 172.21.15.103:0/1075825660 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f6e400041d0 con 0x7f6e58101de0 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.323+0000 7f6e5ed57740 1 Processor -- start 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5ed57740 1 -- start start 2024-09-19T07:38:49.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb32b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aceea872f0 con 0x55aceeb30780 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aceea2ecf0 con 0x55aceeb215d0 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aceea832b0 con 0x55aceeb22b40 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb21090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37382/0 (socket says 172.21.15.103:37382) 2024-09-19T07:38:49.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 -- 172.21.15.103:0/1196176295 learned_addr learned my addr 172.21.15.103:0/1196176295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb32b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e4f7fe640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6e440a3440 0x7f6e440a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 -- 172.21.15.103:0/1196176295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 msgr2=0x55aceeb21a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb21a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 -- 172.21.15.103:0/1196176295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 msgr2=0x55aceeb32b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb32b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e56d9d640 1 -- 172.21.15.103:0/1196176295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aceea133a0 con 0x55aceeb22b40 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.324+0000 7f6e4f7fe640 1 -- 172.21.15.103:0/1075825660 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6e400cf7c0 con 0x7f6e58101de0 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.325+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6e440a3440 0x7f6e440a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.325+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb21090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e50011590 tx=0x7f6e50011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee9b58be7767603 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.325+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6e440a3440 0x7f6e440a5900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e50022000 tx=0x7f6e50023000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.325+0000 7f6e4cff9640 1 -- 172.21.15.103:0/1196176295 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50017050 con 0x55aceeb22b40 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.325+0000 7f6e4cff9640 1 -- 172.21.15.103:0/1196176295 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e5000f7a0 con 0x55aceeb22b40 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.325+0000 7f6e4cff9640 1 -- 172.21.15.103:0/1196176295 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5001a360 con 0x55aceeb22b40 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1196176295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 msgr2=0x55aceeb21090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb21090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e50011590 tx=0x7f6e50011a50 comp rx=0 tx=0).stop 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1196176295 shutdown_connections 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb32b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb21a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1196176295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb21090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1196176295 >> 172.21.15.103:0/1196176295 conn(0x55aceeaea310 msgr2=0x55aceeb1dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1196176295 shutdown_connections 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1196176295 wait complete. 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- start start 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 Processor -- start 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb22b40 0x55aceeb2d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb2dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aceeb33480 con 0x55aceeb22b40 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aceea42060 con 0x55aceeb215d0 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.326+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aceeb33600 con 0x55aceeb30780 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb22b40 0x55aceeb2d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb2dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb22b40 0x55aceeb2d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45540/0 (socket says 172.21.15.103:45540) 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb301c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48982/0 (socket says 172.21.15.103:48982) 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e5ed57740 10 client.0 _ll_register_callbacks cb 0x55acee7958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 -- 172.21.15.103:0/589109463 learned_addr learned my addr 172.21.15.103:0/589109463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 -- 172.21.15.103:0/589109463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 msgr2=0x55aceeb2dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb2dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 -- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 msgr2=0x55aceeb301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 -- 172.21.15.103:0/589109463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acee7f83c0 con 0x55aceeb22b40 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e5659c640 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb2dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb22b40 0x55aceeb2d470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6e4802a930 tx=0x7f6e48031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae38b4f5612aff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e3dffb640 1 -- 172.21.15.103:0/589109463 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e48035040 con 0x55aceeb22b40 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e3dffb640 1 -- 172.21.15.103:0/589109463 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6e48004240 con 0x55aceeb22b40 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aceea94860 con 0x55aceeb22b40 2024-09-19T07:38:49.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e3dffb640 1 -- 172.21.15.103:0/589109463 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e48004440 con 0x55aceeb22b40 2024-09-19T07:38:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e5ed57740 10 client.15600 fetch_fsmap learned FSMap version 28 2024-09-19T07:38:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.327+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aceeb3ef70 con 0x55aceeb22b40 2024-09-19T07:38:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e3dffb640 1 -- 172.21.15.103:0/589109463 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aceea94860 con 0x55aceeb22b40 2024-09-19T07:38:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e3dffb640 1 -- 172.21.15.103:0/589109463 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6e4803d550 con 0x55aceeb22b40 2024-09-19T07:38:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e5ed57740 10 client.15600 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T07:38:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e5ed57740 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:38:49.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e5ed57740 20 client.15600 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceeb40480 0x55aceeb428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e5ed57740 4 client.15600 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aceea8ed70 con 0x55aceeb40480 2024-09-19T07:38:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.328+0000 7f6e5659c640 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceeb40480 0x55aceeb428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.329+0000 7f6e5659c640 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceeb40480 0x55aceeb428c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=206c2f58bb83f32 server_cookie=3f941ad0aa8977ee in_seq=0 out_seq=0 2024-09-19T07:38:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.329+0000 7f6e3dffb640 10 client.15600 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:49.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.329+0000 7f6e3dffb640 1 -- 172.21.15.103:0/589109463 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aceea8ed70 con 0x55aceeb40480 2024-09-19T07:38:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.329+0000 7f6e3dffb640 10 client.15600 handle_command_reply: tid=0 2024-09-19T07:38:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.329+0000 7f6e5ed57740 1 client.15600 shutdown 2024-09-19T07:38:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.329+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb22b40 msgr2=0x55aceeb2d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb22b40 0x55aceeb2d470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6e4802a930 tx=0x7f6e48031850 comp rx=0 tx=0).stop 2024-09-19T07:38:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 shutdown_connections 2024-09-19T07:38:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceeb40480 0x55aceeb428c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb2dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb22b40 0x55aceeb2d470 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e3dffb640 0 client.15600 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/589109463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 >> 172.21.15.103:0/589109463 conn(0x55aceeaea310 msgr2=0x55aceeb23960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 shutdown_connections 2024-09-19T07:38:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 1 -- 172.21.15.103:0/589109463 wait complete. 2024-09-19T07:38:49.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.330+0000 7f6e5ed57740 20 client.15600 trim_cache size 0 max 16384 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.342+0000 7f6e5ed57740 1 Processor -- start 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5ed57740 1 -- start start 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb215d0 0x55aceeb2c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb22b40 0x55aceeb48190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ab620 con 0x55aceeb215d0 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ad1c0 con 0x55aceeb22b40 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581abcf0 con 0x55aceeb30780 2024-09-19T07:38:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb215d0 0x55aceeb2c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb215d0 0x55aceeb2c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45544/0 (socket says 172.21.15.103:45544) 2024-09-19T07:38:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 -- 172.21.15.103:0/1761433514 learned_addr learned my addr 172.21.15.103:0/1761433514 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb22b40 0x55aceeb48190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 -- 172.21.15.103:0/1761433514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 msgr2=0x55aceeb486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 -- 172.21.15.103:0/1761433514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb22b40 msgr2=0x55aceeb48190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb22b40 0x55aceeb48190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 -- 172.21.15.103:0/1761433514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aceeb24120 con 0x55aceeb215d0 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e5659c640 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb215d0 0x55aceeb2c450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6e40097620 tx=0x7f6e400d1cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1183629ccb712c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.343+0000 7f6e3ffff640 1 -- 172.21.15.103:0/1761433514 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e400afe60 con 0x55aceeb215d0 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e3ffff640 1 -- 172.21.15.103:0/1761433514 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4009c040 con 0x55aceeb215d0 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e3ffff640 1 -- 172.21.15.103:0/1761433514 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e400860b0 con 0x55aceeb215d0 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1761433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb215d0 msgr2=0x55aceeb2c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb215d0 0x55aceeb2c450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6e40097620 tx=0x7f6e400d1cd0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1761433514 shutdown_connections 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb30780 0x55aceeb486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb22b40 0x55aceeb48190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/1761433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb215d0 0x55aceeb2c450 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1761433514 >> 172.21.15.103:0/1761433514 conn(0x55aceeaea310 msgr2=0x55aceeb2bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1761433514 shutdown_connections 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.344+0000 7f6e5ed57740 1 -- 172.21.15.103:0/1761433514 wait complete. 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 -- start start 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 Processor -- start 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581aca90 con 0x55aceeb30780 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581ac3c0 con 0x55aceeb215d0 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e581a9600 con 0x55aceeb22b40 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5ed57740 10 client.0 _ll_register_callbacks cb 0x7f6e48005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37406/0 (socket says 172.21.15.103:37406) 2024-09-19T07:38:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e55d9b640 1 -- 172.21.15.103:0/3397023458 learned_addr learned my addr 172.21.15.103:0/3397023458 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5659c640 1 -- 172.21.15.103:0/3397023458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 msgr2=0x55aceeb340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5659c640 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5659c640 1 -- 172.21.15.103:0/3397023458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 msgr2=0x55aceeb34610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5659c640 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb34610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e5659c640 1 -- 172.21.15.103:0/3397023458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aceeb34bd0 con 0x55aceeb215d0 2024-09-19T07:38:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.345+0000 7f6e56d9d640 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5659c640 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb38da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e4009c1f0 tx=0x7f6e4009b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e25d7b314529b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:38:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e3dffb640 1 -- 172.21.15.103:0/3397023458 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e400ba060 con 0x55aceeb215d0 2024-09-19T07:38:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e3dffb640 1 -- 172.21.15.103:0/3397023458 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6e40097fb0 con 0x55aceeb215d0 2024-09-19T07:38:49.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4009c040 con 0x55aceeb215d0 2024-09-19T07:38:49.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e3dffb640 1 -- 172.21.15.103:0/3397023458 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e400afe60 con 0x55aceeb215d0 2024-09-19T07:38:49.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 10 client.25184 fetch_fsmap learned FSMap version 28 2024-09-19T07:38:49.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aceebaaf30 con 0x55aceeb215d0 2024-09-19T07:38:49.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e3dffb640 1 -- 172.21.15.103:0/3397023458 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e4009c040 con 0x55aceeb215d0 2024-09-19T07:38:49.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e3dffb640 1 -- 172.21.15.103:0/3397023458 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6e400cd110 con 0x55aceeb215d0 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 10 client.25184 fetch_fsmap finished waiting for FSMap version 28 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 10 client.25184 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 20 client.25184 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceebab6a0 0x55aceebadac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 4 client.25184 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.346+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aceea0b170 con 0x55aceebab6a0 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceebab6a0 0x55aceebadac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e55d9b640 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceebab6a0 0x55aceebadac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=880a93bb3ca9bc24 server_cookie=34731cbb10fab827 in_seq=0 out_seq=0 2024-09-19T07:38:49.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e3dffb640 10 client.25184 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:49.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e3dffb640 1 -- 172.21.15.103:0/3397023458 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aceea0b170 con 0x55aceebab6a0 2024-09-19T07:38:49.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e3dffb640 10 client.25184 handle_command_reply: tid=0 2024-09-19T07:38:49.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 client.25184 shutdown 2024-09-19T07:38:49.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 msgr2=0x55aceeb38da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb38da0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e4009c1f0 tx=0x7f6e4009b5a0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 shutdown_connections 2024-09-19T07:38:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aceebab6a0 0x55aceebadac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aceeb30780 0x55aceeb34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aceeb22b40 0x55aceeb340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e3dffb640 0 client.25184 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:38:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 --2- 172.21.15.103:0/3397023458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aceeb215d0 0x55aceeb38da0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.347+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 >> 172.21.15.103:0/3397023458 conn(0x55aceeaea310 msgr2=0x55aceeb2c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 shutdown_connections 2024-09-19T07:38:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5ed57740 1 -- 172.21.15.103:0/3397023458 wait complete. 2024-09-19T07:38:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5ed57740 20 client.25184 trim_cache size 0 max 16384 2024-09-19T07:38:49.364 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:38:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6e440a3440 msgr2=0x7f6e440a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6e440a3440 0x7f6e440a5900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e50022000 tx=0x7f6e50023000 comp rx=0 tx=0).stop 2024-09-19T07:38:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58101de0 msgr2=0x7f6e58198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58101de0 0x7f6e58198430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d950 tx=0x7f6e4000de10 comp rx=0 tx=0).stop 2024-09-19T07:38:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 shutdown_connections 2024-09-19T07:38:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6e440a3440 0x7f6e440a5900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e58106f10 0x7f6e5819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e58102790 0x7f6e58198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 --2- 172.21.15.103:0/1075825660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e58101de0 0x7f6e58198430 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 >> 172.21.15.103:0/1075825660 conn(0x7f6e580fd8c0 msgr2=0x7f6e580fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 shutdown_connections 2024-09-19T07:38:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.348+0000 7f6e5c97e640 1 -- 172.21.15.103:0/1075825660 wait complete. 2024-09-19T07:38:49.651 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:49 smithi103 ceph-mon[31201]: pgmap v28: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 462 op/s 2024-09-19T07:38:49.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:49 smithi134 ceph-mon[35414]: pgmap v28: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 462 op/s 2024-09-19T07:38:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:49 smithi174 ceph-mon[35370]: pgmap v28: 97 pgs: 97 active+clean; 923 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 462 op/s 2024-09-19T07:38:49.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b1ffff640 1 -- 172.21.15.103:0/3639028833 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e655198d20 con 0x55e6552bbb90 2024-09-19T07:38:49.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b1ffff640 10 client.25165 handle_command_reply: tid=0 2024-09-19T07:38:49.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 client.25165 shutdown 2024-09-19T07:38:49.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 msgr2=0x55e655244800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:49.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e655244800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b340c50b0 tx=0x7f6b3402a6f0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 shutdown_connections 2024-09-19T07:38:49.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e6552bbb90 0x55e6552bdfb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:49.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e65523f2f0 0x55e655244d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:50.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b1ffff640 0 client.25165 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:50.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e65522c140 0x55e655244800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:50.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 --2- 172.21.15.103:0/3639028833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e65522b820 0x55e6552442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:50.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 >> 172.21.15.103:0/3639028833 conn(0x55e655227e90 msgr2=0x55e65522dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:50.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 shutdown_connections 2024-09-19T07:38:50.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 1 -- 172.21.15.103:0/3639028833 wait complete. 2024-09-19T07:38:50.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.997+0000 7f6b455dc740 20 client.25165 trim_cache size 0 max 16384 2024-09-19T07:38:50.001 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:50.001 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 29, 2024-09-19T07:38:50.001 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:38:50.002 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 2, 2024-09-19T07:38:50.002 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:38:50.002 INFO:teuthology.orchestra.run.smithi103.stdout: "2c26da9c": { 2024-09-19T07:38:50.002 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 2, 2024-09-19T07:38:50.002 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:38:50.002 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:38:50.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:38:50.003 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:38:50.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:38:50.003 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:38:50.004 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 131.2, 2024-09-19T07:38:50.004 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:38:50.005 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:38:50.005 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:38:50.005 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:38:50.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:38:50.005 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:38:50.006 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:50.006 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:50.006 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:50.006 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:50.006 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:50.006 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:50.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b200a3590 msgr2=0x7f6b200a5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:50.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b200a3590 0x7f6b200a5a50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800bf50 tx=0x7f6b2800b390 comp rx=0 tx=0).stop 2024-09-19T07:38:50.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c102370 msgr2=0x7f6b3c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:50.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c102370 0x7f6b3c198830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b34000f80 tx=0x7f6b34030620 comp rx=0 tx=0).stop 2024-09-19T07:38:50.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 shutdown_connections 2024-09-19T07:38:50.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b200a3590 0x7f6b200a5a50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:50.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b3c1079d0 0x7f6b3c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:50.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b3c102370 0x7f6b3c198830 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:50.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 --2- 172.21.15.103:0/3847946653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c1019d0 0x7f6b3c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:50.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 >> 172.21.15.103:0/3847946653 conn(0x7f6b3c0fd710 msgr2=0x7f6b3c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:50.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 shutdown_connections 2024-09-19T07:38:50.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:49.998+0000 7f6b41fa1640 1 -- 172.21.15.103:0/3847946653 wait complete. 2024-09-19T07:38:50.468 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 15.0 s, 2436.6 tps, lat 27.265 ms stddev 27.631 2024-09-19T07:38:50.625 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:50 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:38:50.661 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2c26da9c', quiesce took 5.9 seconds. Will release after: 5.299999999999999 2024-09-19T07:38:50.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:38:50.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:50 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:38:50.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:50 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:38:50.969 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 -- 172.21.15.103:0/3238540960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 msgr2=0x7f5abc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 --2- 172.21.15.103:0/3238540960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc10ec30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009cd0 tx=0x7f5ab402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 -- 172.21.15.103:0/3238540960 shutdown_connections 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 --2- 172.21.15.103:0/3238540960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc10ec30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 --2- 172.21.15.103:0/3238540960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc103740 0x7f5abc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 --2- 172.21.15.103:0/3238540960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc102d90 0x7f5abc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 -- 172.21.15.103:0/3238540960 >> 172.21.15.103:0/3238540960 conn(0x7f5abc0fc7d0 msgr2=0x7f5abc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 -- 172.21.15.103:0/3238540960 shutdown_connections 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 -- 172.21.15.103:0/3238540960 wait complete. 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.460+0000 7f5ac1295640 1 Processor -- start 2024-09-19T07:38:51.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5ac1295640 1 -- start start 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5ac1295640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc102d90 0x7f5abc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5ac1295640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc103740 0x7f5abc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5ac1295640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5ac1295640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc112090 con 0x7f5abc108290 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5ac1295640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc111f10 con 0x7f5abc102d90 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5ac1295640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc112210 con 0x7f5abc103740 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45558/0 (socket says 172.21.15.103:45558) 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 -- 172.21.15.103:0/24484695 learned_addr learned my addr 172.21.15.103:0/24484695 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abaffd640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc102d90 0x7f5abc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc103740 0x7f5abc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 -- 172.21.15.103:0/24484695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc103740 msgr2=0x7f5abc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc103740 0x7f5abc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 -- 172.21.15.103:0/24484695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc102d90 msgr2=0x7f5abc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc102d90 0x7f5abc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 -- 172.21.15.103:0/24484695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5abc19d590 con 0x7f5abc108290 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.461+0000 7f5abb7fe640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc19ceb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005cc0 tx=0x7f5ab4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.462+0000 7f5aa3fff640 1 -- 172.21.15.103:0/24484695 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4004040 con 0x7f5abc108290 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.462+0000 7f5aa3fff640 1 -- 172.21.15.103:0/24484695 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab40046c0 con 0x7f5abc108290 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.462+0000 7f5aa3fff640 1 -- 172.21.15.103:0/24484695 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4046480 con 0x7f5abc108290 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.462+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5abc19d820 con 0x7f5abc108290 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.462+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5abc1a5160 con 0x7f5abc108290 2024-09-19T07:38:51.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.464+0000 7f5aa3fff640 1 -- 172.21.15.103:0/24484695 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f5ab40305a0 con 0x7f5abc108290 2024-09-19T07:38:51.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.464+0000 7f5ac366e740 1 Processor -- start 2024-09-19T07:38:51.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.464+0000 7f5ac366e740 1 -- start start 2024-09-19T07:38:51.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560683585b40 0x560683584090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x560683584a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56068358f370 0x560683591760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aa3fff640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3440 0x7f5aa40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5ac366e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5606834ea2f0 con 0x5606835845d0 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5ac366e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560683491cf0 con 0x560683585b40 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5ac366e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606834e62b0 con 0x56068358f370 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5abaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x560683584a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5abb7fe640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3440 0x7f5aa40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560683585b40 0x560683584090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5abaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x560683584a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45562/0 (socket says 172.21.15.103:45562) 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560683585b40 0x560683584090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:49022/0 (socket says 172.21.15.103:49022) 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5abaffd640 1 -- 172.21.15.103:0/3344597780 learned_addr learned my addr 172.21.15.103:0/3344597780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56068358f370 0x560683591760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aa3fff640 1 -- 172.21.15.103:0/24484695 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5ab402e030 con 0x7f5abc108290 2024-09-19T07:38:51.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 -- 172.21.15.103:0/3344597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56068358f370 msgr2=0x560683591760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56068358f370 0x560683591760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 -- 172.21.15.103:0/3344597780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 msgr2=0x560683584a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x560683584a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 -- 172.21.15.103:0/3344597780 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606834763a0 con 0x560683585b40 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560683585b40 0x560683584090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0feda0 tx=0x7f5ab000b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb759bc0deb69cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.465+0000 7f5abb7fe640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3440 0x7f5aa40a5900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40eed30 tx=0x7f5ab4088040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5aa17fa640 1 -- 172.21.15.103:0/3344597780 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0005020 con 0x560683585b40 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5aa17fa640 1 -- 172.21.15.103:0/3344597780 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab00047a0 con 0x560683585b40 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5aa17fa640 1 -- 172.21.15.103:0/3344597780 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000f6f0 con 0x560683585b40 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 -- 172.21.15.103:0/3344597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560683585b40 msgr2=0x560683584090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560683585b40 0x560683584090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0feda0 tx=0x7f5ab000b6e0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 -- 172.21.15.103:0/3344597780 shutdown_connections 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56068358f370 0x560683591760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x560683584a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3344597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560683585b40 0x560683584090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 -- 172.21.15.103:0/3344597780 >> 172.21.15.103:0/3344597780 conn(0x56068354d310 msgr2=0x560683580c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 -- 172.21.15.103:0/3344597780 shutdown_connections 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.466+0000 7f5ac366e740 1 -- 172.21.15.103:0/3344597780 wait complete. 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 -- start start 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 Processor -- start 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068358ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x56068358c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x56068358c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560683592070 con 0x56068358f370 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5606834a5060 con 0x5606835845d0 2024-09-19T07:38:51.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606835921f0 con 0x560683585b40 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5ac366e740 10 client.0 _ll_register_callbacks cb 0x5606831f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x56068358c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x56068358c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45574/0 (socket says 172.21.15.103:45574) 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5aba7fc640 1 -- 172.21.15.103:0/316263298 learned_addr learned my addr 172.21.15.103:0/316263298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5abaffd640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x56068358c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068358ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5aba7fc640 1 -- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 msgr2=0x56068358c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.467+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x56068358c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5aba7fc640 1 -- 172.21.15.103:0/316263298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 msgr2=0x56068358ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068358ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5aba7fc640 1 -- 172.21.15.103:0/316263298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56068325b3c0 con 0x56068358f370 2024-09-19T07:38:51.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5abaffd640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x56068358c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:51.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x56068358c600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0004ec0 tx=0x7f5ab001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9d7939ee8f206a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:51.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5a9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:51.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5a9eff5640 1 -- 172.21.15.103:0/316263298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0004260 con 0x56068358f370 2024-09-19T07:38:51.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5a9eff5640 1 -- 172.21.15.103:0/316263298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5ab0004400 con 0x56068358f370 2024-09-19T07:38:51.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab00047a0 con 0x56068358f370 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5a9eff5640 1 -- 172.21.15.103:0/316263298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ab0005050 con 0x56068358f370 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 10 client.15618 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56068359db10 con 0x56068358f370 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5a9eff5640 1 -- 172.21.15.103:0/316263298 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab00047a0 con 0x56068358f370 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5a9eff5640 1 -- 172.21.15.103:0/316263298 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f5ab0003d20 con 0x56068358f370 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 10 client.15618 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 20 client.15618 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56068359ee60 0x5606835a1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 4 client.15618 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.468+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606834f7860 con 0x56068359ee60 2024-09-19T07:38:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.469+0000 7f5abaffd640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56068359ee60 0x5606835a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.469+0000 7f5a9eff5640 10 client.15618 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.469+0000 7f5abaffd640 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56068359ee60 0x5606835a1250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b13c1242f8c41f3 server_cookie=799756cb972c04f in_seq=0 out_seq=0 2024-09-19T07:38:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.470+0000 7f5a9eff5640 1 -- 172.21.15.103:0/316263298 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606834f7860 con 0x56068359ee60 2024-09-19T07:38:51.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.470+0000 7f5a9eff5640 10 client.15618 handle_command_reply: tid=0 2024-09-19T07:38:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.470+0000 7f5ac366e740 1 client.15618 shutdown 2024-09-19T07:38:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 msgr2=0x56068358c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x56068358c600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0004ec0 tx=0x7f5ab001b040 comp rx=0 tx=0).stop 2024-09-19T07:38:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 shutdown_connections 2024-09-19T07:38:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56068359ee60 0x5606835a1250 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x56068358c600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x56068358c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 --2- 172.21.15.103:0/316263298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068358ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5a9eff5640 0 client.15618 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 >> 172.21.15.103:0/316263298 conn(0x56068354d310 msgr2=0x560683586930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 shutdown_connections 2024-09-19T07:38:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 1 -- 172.21.15.103:0/316263298 wait complete. 2024-09-19T07:38:51.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.471+0000 7f5ac366e740 20 client.15618 trim_cache size 0 max 16384 2024-09-19T07:38:51.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.483+0000 7f5ac366e740 1 Processor -- start 2024-09-19T07:38:51.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5ac366e740 1 -- start start 2024-09-19T07:38:51.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068359b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x5606835a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x5606835a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5ac366e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1ad8c0 con 0x56068358f370 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5ac366e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1af490 con 0x5606835845d0 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5ac366e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1adf90 con 0x560683585b40 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x5606835a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x5606835a6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45586/0 (socket says 172.21.15.103:45586) 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 -- 172.21.15.103:0/3495460396 learned_addr learned my addr 172.21.15.103:0/3495460396 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068359b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5abaffd640 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x5606835a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 -- 172.21.15.103:0/3495460396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 msgr2=0x5606835a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x5606835a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 -- 172.21.15.103:0/3495460396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 msgr2=0x56068359b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068359b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 -- 172.21.15.103:0/3495460396 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560683585760 con 0x56068358f370 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.484+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x5606835a6e80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000f690 tx=0x7f5ab000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef4605adff3c2e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5aa0ff9640 1 -- 172.21.15.103:0/3495460396 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000f910 con 0x56068358f370 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5aa0ff9640 1 -- 172.21.15.103:0/3495460396 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab0004260 con 0x56068358f370 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5aa0ff9640 1 -- 172.21.15.103:0/3495460396 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0004a30 con 0x56068358f370 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 -- 172.21.15.103:0/3495460396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 msgr2=0x5606835a6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x5606835a6e80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000f690 tx=0x7f5ab000f460 comp rx=0 tx=0).stop 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 -- 172.21.15.103:0/3495460396 shutdown_connections 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56068358f370 0x5606835a6e80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x5606835a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 --2- 172.21.15.103:0/3495460396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606835845d0 0x56068359b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 -- 172.21.15.103:0/3495460396 >> 172.21.15.103:0/3495460396 conn(0x56068354d310 msgr2=0x56068358bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:51.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 -- 172.21.15.103:0/3495460396 shutdown_connections 2024-09-19T07:38:51.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.485+0000 7f5ac366e740 1 -- 172.21.15.103:0/3495460396 wait complete. 2024-09-19T07:38:51.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 -- start start 2024-09-19T07:38:51.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 Processor -- start 2024-09-19T07:38:51.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x5606835924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x560683592a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068358f370 0x560683592f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1aed30 con 0x5606835845d0 2024-09-19T07:38:51.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1ae660 con 0x56068358f370 2024-09-19T07:38:51.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc1ab8c0 con 0x560683585b40 2024-09-19T07:38:51.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5ac366e740 10 client.0 _ll_register_callbacks cb 0x7f5aac00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:51.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x5606835924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x5606835924e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45604/0 (socket says 172.21.15.103:45604) 2024-09-19T07:38:51.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5abaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x560683592a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 -- 172.21.15.103:0/442223798 learned_addr learned my addr 172.21.15.103:0/442223798 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:51.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068358f370 0x560683592f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 -- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 msgr2=0x560683592a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x560683592a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 -- 172.21.15.103:0/442223798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068358f370 msgr2=0x560683592f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068358f370 0x560683592f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5aba7fc640 1 -- 172.21.15.103:0/442223798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560683607ba0 con 0x5606835845d0 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.486+0000 7f5abaffd640 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x560683592a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5aba7fc640 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x5606835924e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b520 tx=0x7f5ab000fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48722a225c90e4bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5a9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5a9eff5640 1 -- 172.21.15.103:0/442223798 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000fcf0 con 0x5606835845d0 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5a9eff5640 1 -- 172.21.15.103:0/442223798 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5ab00047b0 con 0x5606835845d0 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab0004260 con 0x5606835845d0 2024-09-19T07:38:51.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5a9eff5640 1 -- 172.21.15.103:0/442223798 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ab00049b0 con 0x5606835845d0 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 10 client.15624 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560683608e80 con 0x5606835845d0 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5a9eff5640 1 -- 172.21.15.103:0/442223798 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ab0004260 con 0x5606835845d0 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5a9eff5640 1 -- 172.21.15.103:0/442223798 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f5ab0003d30 con 0x5606835845d0 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 10 client.15624 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 10 client.15624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 20 client.15624 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560683609170 0x56068360b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 4 client.15624 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.487+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606834f1d70 con 0x560683609170 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5abaffd640 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560683609170 0x56068360b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:51.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5abaffd640 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560683609170 0x56068360b560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e5b7263b1f9b4b4 server_cookie=a2a2c51acdec3d2c in_seq=0 out_seq=0 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5a9eff5640 10 client.15624 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5a9eff5640 1 -- 172.21.15.103:0/442223798 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606834f1d70 con 0x560683609170 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5a9eff5640 10 client.15624 handle_command_reply: tid=0 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 client.15624 shutdown 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 msgr2=0x5606835924e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x5606835924e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b520 tx=0x7f5ab000fed0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 shutdown_connections 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560683609170 0x56068360b560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56068358f370 0x560683592f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560683585b40 0x560683592a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 --2- 172.21.15.103:0/442223798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606835845d0 0x5606835924e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.488+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 >> 172.21.15.103:0/442223798 conn(0x56068354d310 msgr2=0x56068358d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:51.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5a9eff5640 0 client.15624 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:38:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 shutdown_connections 2024-09-19T07:38:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac366e740 1 -- 172.21.15.103:0/442223798 wait complete. 2024-09-19T07:38:51.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac366e740 20 client.15624 trim_cache size 0 max 16384 2024-09-19T07:38:51.501 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:38:51.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3440 msgr2=0x7f5aa40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3440 0x7f5aa40a5900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40eed30 tx=0x7f5ab4088040 comp rx=0 tx=0).stop 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 msgr2=0x7f5abc19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc19ceb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005cc0 tx=0x7f5ab4003970 comp rx=0 tx=0).stop 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 shutdown_connections 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5aa40a3440 0x7f5aa40a5900 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc108290 0x7f5abc19ceb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc103740 0x7f5abc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 --2- 172.21.15.103:0/24484695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5abc102d90 0x7f5abc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 >> 172.21.15.103:0/24484695 conn(0x7f5abc0fc7d0 msgr2=0x7f5abc106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 shutdown_connections 2024-09-19T07:38:51.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:51.489+0000 7f5ac1295640 1 -- 172.21.15.103:0/24484695 wait complete. 2024-09-19T07:38:51.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:51 smithi103 ceph-mon[31201]: pgmap v29: 97 pgs: 97 active+clean; 934 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 26 MiB/s wr, 797 op/s 2024-09-19T07:38:51.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:51 smithi134 ceph-mon[35414]: pgmap v29: 97 pgs: 97 active+clean; 934 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 26 MiB/s wr, 797 op/s 2024-09-19T07:38:51.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:51 smithi174 ceph-mon[35370]: pgmap v29: 97 pgs: 97 active+clean; 934 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 26 MiB/s wr, 797 op/s 2024-09-19T07:38:52.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:38:52.485 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:53.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.938+0000 7fc2930ed640 1 -- 172.21.15.103:0/3837511991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c102790 msgr2=0x7fc28c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.938+0000 7fc2930ed640 1 --2- 172.21.15.103:0/3837511991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c102790 0x7fc28c1067e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc278009860 tx=0x7fc27802a340 comp rx=0 tx=0).stop 2024-09-19T07:38:53.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- 172.21.15.103:0/3837511991 shutdown_connections 2024-09-19T07:38:53.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 --2- 172.21.15.103:0/3837511991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc28c106f10 0x7fc28c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 --2- 172.21.15.103:0/3837511991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c102790 0x7fc28c1067e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 --2- 172.21.15.103:0/3837511991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc28c101de0 0x7fc28c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- 172.21.15.103:0/3837511991 >> 172.21.15.103:0/3837511991 conn(0x7fc28c0fd8c0 msgr2=0x7fc28c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- 172.21.15.103:0/3837511991 shutdown_connections 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- 172.21.15.103:0/3837511991 wait complete. 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 Processor -- start 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- start start 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc28c101de0 0x7fc28c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc28c102790 0x7fc28c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c106f10 0x7fc28c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c111f70 con 0x7fc28c106f10 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c111df0 con 0x7fc28c102790 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.939+0000 7fc2930ed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1120f0 con 0x7fc28c101de0 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c106f10 0x7fc28c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c106f10 0x7fc28c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45614/0 (socket says 172.21.15.103:45614) 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 -- 172.21.15.103:0/638174797 learned_addr learned my addr 172.21.15.103:0/638174797 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc28bfff640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc28c102790 0x7fc28c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc290e89640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc28c101de0 0x7fc28c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 -- 172.21.15.103:0/638174797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc28c101de0 msgr2=0x7fc28c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc28c101de0 0x7fc28c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 -- 172.21.15.103:0/638174797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc28c102790 msgr2=0x7fc28c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc28c102790 0x7fc28c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 -- 172.21.15.103:0/638174797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c19d440 con 0x7fc28c106f10 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc290e89640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc28c101de0 0x7fc28c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc29168a640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c106f10 0x7fc28c19cd60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc28400eaa0 tx=0x7fc28400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc289ffb640 1 -- 172.21.15.103:0/638174797 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28400cdb0 con 0x7fc28c106f10 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc289ffb640 1 -- 172.21.15.103:0/638174797 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc284005020 con 0x7fc28c106f10 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c19d730 con 0x7fc28c106f10 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc289ffb640 1 -- 172.21.15.103:0/638174797 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840116b0 con 0x7fc28c106f10 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.940+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1a5010 con 0x7fc28c106f10 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.943+0000 7fc289ffb640 1 -- 172.21.15.103:0/638174797 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fc284005560 con 0x7fc28c106f10 2024-09-19T07:38:53.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.943+0000 7fc2954c6740 1 Processor -- start 2024-09-19T07:38:53.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.943+0000 7fc2954c6740 1 -- start start 2024-09-19T07:38:53.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d019809090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019809a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d019816760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc2954c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d01976f2f0 con 0x55d0198095d0 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc2954c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d019716cf0 con 0x55d01980ab40 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc2954c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d01976b2b0 con 0x55d019814370 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019809a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019809a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45620/0 (socket says 172.21.15.103:45620) 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 -- 172.21.15.103:0/566060230 learned_addr learned my addr 172.21.15.103:0/566060230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc289ffb640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc27c0a3440 0x7fc27c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc29168a640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc27c0a3440 0x7fc27c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc28bfff640 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d019809090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 -- 172.21.15.103:0/566060230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 msgr2=0x55d019816760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d019816760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 -- 172.21.15.103:0/566060230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 msgr2=0x55d019809090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d019809090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 -- 172.21.15.103:0/566060230 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0196fb3a0 con 0x55d0198095d0 2024-09-19T07:38:53.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc289ffb640 1 -- 172.21.15.103:0/638174797 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc2840d0140 con 0x7fc28c106f10 2024-09-19T07:38:53.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc290e89640 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019809a10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc280009c30 tx=0x7fc280010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d4fb28ea1a663a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:53.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.944+0000 7fc276ffd640 1 -- 172.21.15.103:0/566060230 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280004020 con 0x55d0198095d0 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc29168a640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc27c0a3440 0x7fc27c0a5900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2840cd840 tx=0x7fc28406e040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc276ffd640 1 -- 172.21.15.103:0/566060230 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2800041c0 con 0x55d0198095d0 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc276ffd640 1 -- 172.21.15.103:0/566060230 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2800126b0 con 0x55d0198095d0 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 -- 172.21.15.103:0/566060230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 msgr2=0x55d019809a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019809a10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc280009c30 tx=0x7fc280010740 comp rx=0 tx=0).stop 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 -- 172.21.15.103:0/566060230 shutdown_connections 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d019816760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019809a10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 --2- 172.21.15.103:0/566060230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d019809090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 -- 172.21.15.103:0/566060230 >> 172.21.15.103:0/566060230 conn(0x55d0197d2310 msgr2=0x55d019805c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 -- 172.21.15.103:0/566060230 shutdown_connections 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.945+0000 7fc2954c6740 1 -- 172.21.15.103:0/566060230 wait complete. 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 -- start start 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 Processor -- start 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0198095d0 0x55d019813e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d01980ab40 0x55d0198110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d019814370 0x55d019811840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d019817070 con 0x55d019814370 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d01972a060 con 0x55d0198095d0 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0198171f0 con 0x55d01980ab40 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc2954c6740 10 client.0 _ll_register_callbacks cb 0x55d01947d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d019814370 0x55d019811840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d019814370 0x55d019811840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45632/0 (socket says 172.21.15.103:45632) 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc28bfff640 1 -- 172.21.15.103:0/3587364383 learned_addr learned my addr 172.21.15.103:0/3587364383 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0198095d0 0x55d019813e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.946+0000 7fc290e89640 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d01980ab40 0x55d0198110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc28bfff640 1 -- 172.21.15.103:0/3587364383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d01980ab40 msgr2=0x55d0198110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d01980ab40 0x55d0198110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc28bfff640 1 -- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0198095d0 msgr2=0x55d019813e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0198095d0 0x55d019813e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc28bfff640 1 -- 172.21.15.103:0/3587364383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0194e03c0 con 0x55d019814370 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d019814370 0x55d019811840 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc27802a380 tx=0x7fc2780056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9add58440828ba0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc26bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:53.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc26bfff640 1 -- 172.21.15.103:0/3587364383 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278046060 con 0x55d019814370 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc26bfff640 1 -- 172.21.15.103:0/3587364383 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc278030750 con 0x55d019814370 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2800041c0 con 0x55d019814370 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc26bfff640 1 -- 172.21.15.103:0/3587364383 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc278037040 con 0x55d019814370 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc2954c6740 10 client.15636 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d019822bc0 con 0x55d019814370 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc26bfff640 1 -- 172.21.15.103:0/3587364383 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2800041c0 con 0x55d019814370 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc26bfff640 1 -- 172.21.15.103:0/3587364383 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fc278032440 con 0x55d019814370 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc2954c6740 10 client.15636 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc2954c6740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.947+0000 7fc2954c6740 20 client.15636 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:53.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.948+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d019824080 0x55d019826470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.948+0000 7fc2954c6740 4 client.15636 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.948+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d01977c860 con 0x55d019824080 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.948+0000 7fc290e89640 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d019824080 0x55d019826470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.948+0000 7fc26bfff640 10 client.15636 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.948+0000 7fc290e89640 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d019824080 0x55d019826470 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8e75504536df43 server_cookie=38879bffbc07d549 in_seq=0 out_seq=0 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc26bfff640 1 -- 172.21.15.103:0/3587364383 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d01977c860 con 0x55d019824080 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc26bfff640 10 client.15636 handle_command_reply: tid=0 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 client.15636 shutdown 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d019814370 msgr2=0x55d019811840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d019814370 0x55d019811840 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc27802a380 tx=0x7fc2780056d0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 shutdown_connections 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d019824080 0x55d019826470 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d019814370 0x55d019811840 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d01980ab40 0x55d0198110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3587364383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0198095d0 0x55d019813e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc26bfff640 0 client.15636 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 >> 172.21.15.103:0/3587364383 conn(0x55d0197d2310 msgr2=0x55d01980b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 shutdown_connections 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 1 -- 172.21.15.103:0/3587364383 wait complete. 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.949+0000 7fc2954c6740 20 client.15636 trim_cache size 0 max 16384 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 Processor -- start 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 -- start start 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019800a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01980c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d01980c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1ad900 con 0x55d0198095d0 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1af4d0 con 0x55d01980ab40 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc2954c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1adfd0 con 0x55d019814370 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019800a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019800a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45642/0 (socket says 172.21.15.103:45642) 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 -- 172.21.15.103:0/3012616772 learned_addr learned my addr 172.21.15.103:0/3012616772 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 -- 172.21.15.103:0/3012616772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 msgr2=0x55d01980c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d01980c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 -- 172.21.15.103:0/3012616772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 msgr2=0x55d01980c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc290e89640 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01980c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01980c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc28bfff640 1 -- 172.21.15.103:0/3012616772 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d01980a300 con 0x55d0198095d0 2024-09-19T07:38:53.316 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:38:53.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.962+0000 7fc290e89640 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01980c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:53.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019800a30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc27802d020 tx=0x7fc278005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faf913b401338eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:53.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2767fc640 1 -- 172.21.15.103:0/3012616772 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278046040 con 0x55d0198095d0 2024-09-19T07:38:53.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2767fc640 1 -- 172.21.15.103:0/3012616772 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc278032440 con 0x55d0198095d0 2024-09-19T07:38:53.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2767fc640 1 -- 172.21.15.103:0/3012616772 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278003850 con 0x55d0198095d0 2024-09-19T07:38:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 -- 172.21.15.103:0/3012616772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 msgr2=0x55d019800a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019800a30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc27802d020 tx=0x7fc278005f10 comp rx=0 tx=0).stop 2024-09-19T07:38:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 -- 172.21.15.103:0/3012616772 shutdown_connections 2024-09-19T07:38:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d01980c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01980c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3012616772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d019800a30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 -- 172.21.15.103:0/3012616772 >> 172.21.15.103:0/3012616772 conn(0x55d0197d2310 msgr2=0x55d0198101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 -- 172.21.15.103:0/3012616772 shutdown_connections 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.963+0000 7fc2954c6740 1 -- 172.21.15.103:0/3012616772 wait complete. 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 -- start start 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 Processor -- start 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d01981c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01981cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d01980dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1aed70 con 0x55d0198095d0 2024-09-19T07:38:53.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1ae6a0 con 0x55d01980ab40 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1ab8e0 con 0x55d019814370 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc2954c6740 10 client.0 _ll_register_callbacks cb 0x7fc278035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d01981c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d01981c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45656/0 (socket says 172.21.15.103:45656) 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc28bfff640 1 -- 172.21.15.103:0/3732045212 learned_addr learned my addr 172.21.15.103:0/3732045212 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.964+0000 7fc290e89640 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01981cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc28bfff640 1 -- 172.21.15.103:0/3732045212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 msgr2=0x55d01980dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d01980dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc28bfff640 1 -- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 msgr2=0x55d01981cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01981cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc28bfff640 1 -- 172.21.15.103:0/3732045212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d01980e500 con 0x55d0198095d0 2024-09-19T07:38:53.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc28bfff640 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d01981c810 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc27802aa90 tx=0x7fc278032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4182aaa4cf623f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc26bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc26bfff640 1 -- 172.21.15.103:0/3732045212 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278046040 con 0x55d0198095d0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc26bfff640 1 -- 172.21.15.103:0/3732045212 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc2780308e0 con 0x55d0198095d0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc278032440 con 0x55d0198095d0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc26bfff640 1 -- 172.21.15.103:0/3732045212 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc278005040 con 0x55d0198095d0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 10 client.15642 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d01988e8e0 con 0x55d0198095d0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc26bfff640 1 -- 172.21.15.103:0/3732045212 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc278032440 con 0x55d0198095d0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc26bfff640 1 -- 172.21.15.103:0/3732045212 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fc278052020 con 0x55d0198095d0 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 10 client.15642 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:53.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 20 client.15642 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d01988ed80 0x55d0198911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 4 client.15642 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.965+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d019776d70 con 0x55d01988ed80 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.966+0000 7fc290e89640 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d01988ed80 0x55d0198911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.966+0000 7fc290e89640 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d01988ed80 0x55d0198911a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f069fa1fb9bc4f5 server_cookie=d731bc0a279d27c3 in_seq=0 out_seq=0 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.966+0000 7fc26bfff640 10 client.15642 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.966+0000 7fc26bfff640 1 -- 172.21.15.103:0/3732045212 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d019776d70 con 0x55d01988ed80 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.966+0000 7fc26bfff640 10 client.15642 handle_command_reply: tid=0 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.966+0000 7fc2954c6740 1 client.15642 shutdown 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 msgr2=0x55d01981c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d01981c810 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc27802aa90 tx=0x7fc278032740 comp rx=0 tx=0).stop 2024-09-19T07:38:53.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 shutdown_connections 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d01988ed80 0x55d0198911a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d019814370 0x55d01980dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d01980ab40 0x55d01981cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc26bfff640 0 client.15642 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 --2- 172.21.15.103:0/3732045212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0198095d0 0x55d01981c810 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 >> 172.21.15.103:0/3732045212 conn(0x55d0197d2310 msgr2=0x55d0198101c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 shutdown_connections 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 1 -- 172.21.15.103:0/3732045212 wait complete. 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2954c6740 20 client.15642 trim_cache size 0 max 16384 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc27c0a3440 msgr2=0x7fc27c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc27c0a3440 0x7fc27c0a5900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2840cd840 tx=0x7fc28406e040 comp rx=0 tx=0).stop 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c106f10 msgr2=0x7fc28c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c106f10 0x7fc28c19cd60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc28400eaa0 tx=0x7fc28400ef60 comp rx=0 tx=0).stop 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 shutdown_connections 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc27c0a3440 0x7fc27c0a5900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc28c106f10 0x7fc28c19cd60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc28c102790 0x7fc28c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 --2- 172.21.15.103:0/638174797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc28c101de0 0x7fc28c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 >> 172.21.15.103:0/638174797 conn(0x7fc28c0fd8c0 msgr2=0x7fc28c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 shutdown_connections 2024-09-19T07:38:53.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:52.967+0000 7fc2930ed640 1 -- 172.21.15.103:0/638174797 wait complete. 2024-09-19T07:38:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:53 smithi103 ceph-mon[31201]: pgmap v30: 97 pgs: 97 active+clean; 934 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-09-19T07:38:53.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:53 smithi134 ceph-mon[35414]: pgmap v30: 97 pgs: 97 active+clean; 934 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-09-19T07:38:53.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:53 smithi174 ceph-mon[35370]: pgmap v30: 97 pgs: 97 active+clean; 934 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.8 MiB/s wr, 658 op/s 2024-09-19T07:38:55.033 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:38:55.033 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:38:55.297 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:55.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 20.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:38:55.700 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:55 smithi103 ceph-mon[31201]: pgmap v31: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.9 MiB/s wr, 719 op/s 2024-09-19T07:38:55.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:55 smithi174 ceph-mon[35370]: pgmap v31: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.9 MiB/s wr, 719 op/s 2024-09-19T07:38:55.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:55 smithi134 ceph-mon[35414]: pgmap v31: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.9 MiB/s wr, 719 op/s 2024-09-19T07:38:55.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 -- 172.21.15.103:0/2382690773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102ef0 msgr2=0x7fb7b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2382690773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102ef0 0x7fb7b010da40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009ca0 tx=0x7fb7a802a950 comp rx=0 tx=0).stop 2024-09-19T07:38:55.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 -- 172.21.15.103:0/2382690773 shutdown_connections 2024-09-19T07:38:55.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2382690773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102ef0 0x7fb7b010da40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2382690773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7b0102570 0x7fb7b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2382690773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b0100e20 0x7fb7b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 -- 172.21.15.103:0/2382690773 >> 172.21.15.103:0/2382690773 conn(0x7fb7b00fc7d0 msgr2=0x7fb7b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 -- 172.21.15.103:0/2382690773 shutdown_connections 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 -- 172.21.15.103:0/2382690773 wait complete. 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.783+0000 7fb7b6295640 1 Processor -- start 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b6295640 1 -- start start 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b6295640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7b0100e20 0x7fb7b0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b6295640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102570 0x7fb7b01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b6295640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b0102ef0 0x7fb7b01a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b6295640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0110f80 con 0x7fb7b0102570 2024-09-19T07:38:55.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b6295640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0110e00 con 0x7fb7b0102ef0 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b6295640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0111100 con 0x7fb7b0100e20 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102570 0x7fb7b01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102570 0x7fb7b01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45664/0 (socket says 172.21.15.103:45664) 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 -- 172.21.15.103:0/2621794987 learned_addr learned my addr 172.21.15.103:0/2621794987 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7affff640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7b0100e20 0x7fb7b0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7b4832640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b0102ef0 0x7fb7b01a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 -- 172.21.15.103:0/2621794987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7b0100e20 msgr2=0x7fb7b0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7b0100e20 0x7fb7b0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 -- 172.21.15.103:0/2621794987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b0102ef0 msgr2=0x7fb7b01a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b0102ef0 0x7fb7b01a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 -- 172.21.15.103:0/2621794987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b01a4850 con 0x7fb7b0102570 2024-09-19T07:38:55.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.784+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102570 0x7fb7b01997d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d770 tx=0x7fb79800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.785+0000 7fb7ad7fa640 1 -- 172.21.15.103:0/2621794987 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798017070 con 0x7fb7b0102570 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.785+0000 7fb7ad7fa640 1 -- 172.21.15.103:0/2621794987 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb798005500 con 0x7fb7b0102570 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.785+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b01a4a20 con 0x7fb7b0102570 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.785+0000 7fb7ad7fa640 1 -- 172.21.15.103:0/2621794987 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798004e00 con 0x7fb7b0102570 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.785+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b01a4ee0 con 0x7fb7b0102570 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7ad7fa640 1 -- 172.21.15.103:0/2621794987 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fb798004fa0 con 0x7fb7b0102570 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 Processor -- start 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 -- start start 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bdca40 0x55d5f2bdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2bebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bec150 0x55d5f2bee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2b42280 con 0x55d5f2bdd360 2024-09-19T07:38:55.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2ae9b10 con 0x55d5f2bdca40 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b866e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2b3e240 con 0x55d5f2bec150 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2bebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2bebc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45674/0 (socket says 172.21.15.103:45674) 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7affff640 1 -- 172.21.15.103:0/3260103556 learned_addr learned my addr 172.21.15.103:0/3260103556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7b4832640 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bdca40 0x55d5f2bdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7ad7fa640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb79c0a3440 0x7fb79c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.788+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bec150 0x55d5f2bee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7affff640 1 -- 172.21.15.103:0/3260103556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bec150 msgr2=0x55d5f2bee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7affff640 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bec150 0x55d5f2bee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7b4832640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb79c0a3440 0x7fb79c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7affff640 1 -- 172.21.15.103:0/3260103556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bdca40 msgr2=0x55d5f2bdce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7affff640 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bdca40 0x55d5f2bdce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7affff640 1 -- 172.21.15.103:0/3260103556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f2af61f0 con 0x55d5f2bdd360 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bec150 0x55d5f2bee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7ad7fa640 1 -- 172.21.15.103:0/2621794987 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb7980cf980 con 0x7fb7b0102570 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7affff640 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2bebc10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000bb10 tx=0x7fb7a000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2599cf0094136b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7a67fc640 1 -- 172.21.15.103:0/3260103556 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0017420 con 0x55d5f2bdd360 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7a67fc640 1 -- 172.21.15.103:0/3260103556 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a0017aa0 con 0x55d5f2bdd360 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.789+0000 7fb7a67fc640 1 -- 172.21.15.103:0/3260103556 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000fb70 con 0x55d5f2bdd360 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 -- 172.21.15.103:0/3260103556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 msgr2=0x55d5f2bebc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b4832640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb79c0a3440 0x7fb79c0a5900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009430 tx=0x7fb7a802a950 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2bebc10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000bb10 tx=0x7fb7a000d340 comp rx=0 tx=0).stop 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 -- 172.21.15.103:0/3260103556 shutdown_connections 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bec150 0x55d5f2bee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2bebc10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3260103556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bdca40 0x55d5f2bdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 -- 172.21.15.103:0/3260103556 >> 172.21.15.103:0/3260103556 conn(0x55d5f2bd90f0 msgr2=0x55d5f2bd94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 -- 172.21.15.103:0/3260103556 shutdown_connections 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 -- 172.21.15.103:0/3260103556 wait complete. 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 -- start start 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.790+0000 7fb7b866e740 1 Processor -- start 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2be9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2be95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2be9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2beed20 con 0x55d5f2bdca40 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2afcf10 con 0x55d5f2bec150 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2beeea0 con 0x55d5f2bdd360 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 10 client.0 _ll_register_callbacks cb 0x55d5f2ac5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2be9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2be9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45688/0 (socket says 172.21.15.103:45688) 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 -- 172.21.15.103:0/3813310276 learned_addr learned my addr 172.21.15.103:0/3813310276 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2be95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b4832640 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2be9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 -- 172.21.15.103:0/3813310276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 msgr2=0x55d5f2be95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2be95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 -- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 msgr2=0x55d5f2be9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2be9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 -- 172.21.15.103:0/3813310276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f28b36a0 con 0x55d5f2bdca40 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7affff640 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2be9060 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0019040 tx=0x7fb7a000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e5e15ae0c62573 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb78f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb78f7fe640 1 -- 172.21.15.103:0/3813310276 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0026020 con 0x55d5f2bdca40 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb78f7fe640 1 -- 172.21.15.103:0/3813310276 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb7a0017620 con 0x55d5f2bdca40 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a0017aa0 con 0x55d5f2bdca40 2024-09-19T07:38:55.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb78f7fe640 1 -- 172.21.15.103:0/3813310276 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7a0031050 con 0x55d5f2bdca40 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 10 client.15654 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.791+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5f2bfa8f0 con 0x55d5f2bdca40 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb78f7fe640 1 -- 172.21.15.103:0/3813310276 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a0017aa0 con 0x55d5f2bdca40 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb78f7fe640 1 -- 172.21.15.103:0/3813310276 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fb7a0004770 con 0x55d5f2bdca40 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb7b866e740 10 client.15654 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb7b866e740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb7b866e740 20 client.15654 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2bfbd50 0x55d5f2bfe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb7b866e740 4 client.15654 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:55.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5f2b4f7f0 con 0x55d5f2bfbd50 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.792+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2bfbd50 0x55d5f2bfe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.793+0000 7fb78f7fe640 10 client.15654 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.793+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2bfbd50 0x55d5f2bfe140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2987f8b89a54d server_cookie=b5316bddaa7e4d93 in_seq=0 out_seq=0 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.793+0000 7fb78f7fe640 1 -- 172.21.15.103:0/3813310276 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5f2b4f7f0 con 0x55d5f2bfbd50 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.793+0000 7fb78f7fe640 10 client.15654 handle_command_reply: tid=0 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.793+0000 7fb7b866e740 1 client.15654 shutdown 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 msgr2=0x55d5f2be9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2be9060 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0019040 tx=0x7fb7a000f490 comp rx=0 tx=0).stop 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 shutdown_connections 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2bfbd50 0x55d5f2bfe140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2be9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2be95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb78f7fe640 0 client.15654 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 --2- 172.21.15.103:0/3813310276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2be9060 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 >> 172.21.15.103:0/3813310276 conn(0x55d5f2bd90f0 msgr2=0x55d5f2bdee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 shutdown_connections 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 1 -- 172.21.15.103:0/3813310276 wait complete. 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.794+0000 7fb7b866e740 20 client.15654 trim_cache size 0 max 16384 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 Processor -- start 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 -- start start 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdca40 0x55d5f2bd8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2c037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2c03d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ab640 con 0x55d5f2bdd360 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ad210 con 0x55d5f2bec150 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7b866e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01abd10 con 0x55d5f2bdca40 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2c037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2c037f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45702/0 (socket says 172.21.15.103:45702) 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 -- 172.21.15.103:0/745363511 learned_addr learned my addr 172.21.15.103:0/745363511 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7affff640 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdca40 0x55d5f2bd8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 -- 172.21.15.103:0/745363511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdca40 msgr2=0x55d5f2bd8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdca40 0x55d5f2bd8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 -- 172.21.15.103:0/745363511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 msgr2=0x55d5f2c03d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2c03d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 -- 172.21.15.103:0/745363511 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f2bd8b90 con 0x55d5f2bdd360 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2c037f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb7980cd6e0 tx=0x7fb798086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355e8f205093baf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:55.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.811+0000 7fb7a5ffb640 1 -- 172.21.15.103:0/745363511 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798020040 con 0x55d5f2bdd360 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7a5ffb640 1 -- 172.21.15.103:0/745363511 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79809feb0 con 0x55d5f2bdd360 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7a5ffb640 1 -- 172.21.15.103:0/745363511 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798098170 con 0x55d5f2bdd360 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 -- 172.21.15.103:0/745363511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 msgr2=0x55d5f2c037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2c037f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb7980cd6e0 tx=0x7fb798086040 comp rx=0 tx=0).stop 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 -- 172.21.15.103:0/745363511 shutdown_connections 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2c03d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdd360 0x55d5f2c037f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 --2- 172.21.15.103:0/745363511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdca40 0x55d5f2bd8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 -- 172.21.15.103:0/745363511 >> 172.21.15.103:0/745363511 conn(0x55d5f2bd90f0 msgr2=0x55d5f2bd7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 -- 172.21.15.103:0/745363511 shutdown_connections 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.812+0000 7fb7b866e740 1 -- 172.21.15.103:0/745363511 wait complete. 2024-09-19T07:38:55.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 -- start start 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 Processor -- start 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2befdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2bf0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2bf0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01acab0 con 0x55d5f2bdca40 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ac3e0 con 0x55d5f2bec150 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01a9620 con 0x55d5f2bdd360 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b866e740 10 client.0 _ll_register_callbacks cb 0x7fb7a8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2befdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7affff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2befdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45712/0 (socket says 172.21.15.103:45712) 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7affff640 1 -- 172.21.15.103:0/270520399 learned_addr learned my addr 172.21.15.103:0/270520399 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:55.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2bf0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7b4832640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2bf0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7affff640 1 -- 172.21.15.103:0/270520399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 msgr2=0x55d5f2bf0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7affff640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2bf0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.813+0000 7fb7affff640 1 -- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 msgr2=0x55d5f2bf0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7affff640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2bf0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7affff640 1 -- 172.21.15.103:0/270520399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f2c657c0 con 0x55d5f2bdca40 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b4832640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2bf0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7affff640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2befdd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0004b90 tx=0x7fb7a00048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90d71f7ff383868 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb78f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb78f7fe640 1 -- 172.21.15.103:0/270520399 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a003a060 con 0x55d5f2bdca40 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb78f7fe640 1 -- 172.21.15.103:0/270520399 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb7a0017830 con 0x55d5f2bdca40 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb79809feb0 con 0x55d5f2bdca40 2024-09-19T07:38:55.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb78f7fe640 1 -- 172.21.15.103:0/270520399 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7a0026040 con 0x55d5f2bdca40 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 10 client.15660 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5f2c66a40 con 0x55d5f2bdca40 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb78f7fe640 1 -- 172.21.15.103:0/270520399 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79809feb0 con 0x55d5f2bdca40 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb78f7fe640 1 -- 172.21.15.103:0/270520399 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fb7a0020570 con 0x55d5f2bdca40 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 10 client.15660 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 20 client.15660 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2c67100 0x55d5f2c69520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 4 client.15660 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.814+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5f2b49d00 con 0x55d5f2c67100 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.815+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2c67100 0x55d5f2c69520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:55.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.815+0000 7fb7af7fe640 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2c67100 0x55d5f2c69520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=883ae41e03c3a1f4 server_cookie=827b86616d7acc6c in_seq=0 out_seq=0 2024-09-19T07:38:55.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.815+0000 7fb78f7fe640 10 client.15660 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:55.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb78f7fe640 1 -- 172.21.15.103:0/270520399 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5f2b49d00 con 0x55d5f2c67100 2024-09-19T07:38:55.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb78f7fe640 10 client.15660 handle_command_reply: tid=0 2024-09-19T07:38:55.852 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "3473486d-6045-48ca-8ce0-1c28614627a4", 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 client.15660 shutdown 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 msgr2=0x55d5f2befdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2befdd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0004b90 tx=0x7fb7a00048d0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 shutdown_connections 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d5f2c67100 0x55d5f2c69520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5f2bec150 0x55d5f2bf0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb78f7fe640 0 client.15660 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5f2bdd360 0x55d5f2bf0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 --2- 172.21.15.103:0/270520399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5f2bdca40 0x55d5f2befdd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 >> 172.21.15.103:0/270520399 conn(0x55d5f2bd90f0 msgr2=0x55d5f2bd8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.816+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 shutdown_connections 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b866e740 1 -- 172.21.15.103:0/270520399 wait complete. 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b866e740 20 client.15660 trim_cache size 0 max 16384 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb79c0a3440 msgr2=0x7fb79c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb79c0a3440 0x7fb79c0a5900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009430 tx=0x7fb7a802a950 comp rx=0 tx=0).stop 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102570 msgr2=0x7fb7b01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:55.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102570 0x7fb7b01997d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d770 tx=0x7fb79800dc30 comp rx=0 tx=0).stop 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 shutdown_connections 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb79c0a3440 0x7fb79c0a5900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7b0102ef0 0x7fb7b01a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7b0102570 0x7fb7b01997d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 --2- 172.21.15.103:0/2621794987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7b0100e20 0x7fb7b0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 >> 172.21.15.103:0/2621794987 conn(0x7fb7b00fc7d0 msgr2=0x7fb7b01067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 shutdown_connections 2024-09-19T07:38:55.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:55.817+0000 7fb7b6295640 1 -- 172.21.15.103:0/2621794987 wait complete. 2024-09-19T07:38:55.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:38:56.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:38:56.648 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:56 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:38:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:56 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:56.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:56 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:38:56.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:56 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:56.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:56 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:38:56.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:56 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 -- 172.21.15.103:0/3180281610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44106a80 msgr2=0x7fef44106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 --2- 172.21.15.103:0/3180281610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44106a80 0x7fef44106e60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fef34009930 tx=0x7fef3402a420 comp rx=0 tx=0).stop 2024-09-19T07:38:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 -- 172.21.15.103:0/3180281610 shutdown_connections 2024-09-19T07:38:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 --2- 172.21.15.103:0/3180281610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef44101420 0x7fef4410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 --2- 172.21.15.103:0/3180281610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100a80 0x7fef44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 --2- 172.21.15.103:0/3180281610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44106a80 0x7fef44106e60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 -- 172.21.15.103:0/3180281610 >> 172.21.15.103:0/3180281610 conn(0x7fef440fc7a0 msgr2=0x7fef440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 -- 172.21.15.103:0/3180281610 shutdown_connections 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.323+0000 7fef4a07c640 1 -- 172.21.15.103:0/3180281610 wait complete. 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 Processor -- start 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 -- start start 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100a80 0x7fef44199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44101420 0x7fef44199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef44106a80 0x7fef441a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef44110e40 con 0x7fef44101420 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef44110cc0 con 0x7fef44106a80 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef4a07c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef44110fc0 con 0x7fef44100a80 2024-09-19T07:38:57.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44101420 0x7fef44199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44101420 0x7fef44199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45720/0 (socket says 172.21.15.103:45720) 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 -- 172.21.15.103:0/2852229892 learned_addr learned my addr 172.21.15.103:0/2852229892 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef437fe640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100a80 0x7fef44199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef43fff640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef44106a80 0x7fef441a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 -- 172.21.15.103:0/2852229892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100a80 msgr2=0x7fef44199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100a80 0x7fef44199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 -- 172.21.15.103:0/2852229892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef44106a80 msgr2=0x7fef441a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef44106a80 0x7fef441a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 -- 172.21.15.103:0/2852229892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef441a48d0 con 0x7fef44101420 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef437fe640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100a80 0x7fef44199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef43fff640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef44106a80 0x7fef441a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.324+0000 7fef3bfff640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44101420 0x7fef44199850 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00d950 tx=0x7fef2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:57.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.325+0000 7fef417fa640 1 -- 172.21.15.103:0/2852229892 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c017030 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.325+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef441a4aa0 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.325+0000 7fef417fa640 1 -- 172.21.15.103:0/2852229892 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c004020 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.325+0000 7fef417fa640 1 -- 172.21.15.103:0/2852229892 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c0047a0 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.325+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef441a4fe0 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.326+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef44100ee0 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.327+0000 7fef417fa640 1 -- 172.21.15.103:0/2852229892 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fef2c0041d0 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.327+0000 7fef417fa640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fef300a3440 0x7fef300a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.328+0000 7fef437fe640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fef300a3440 0x7fef300a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.328+0000 7fef417fa640 1 -- 172.21.15.103:0/2852229892 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fef2c0cfff0 con 0x7fef44101420 2024-09-19T07:38:57.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.328+0000 7fef437fe640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fef300a3440 0x7fef300a5900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef34000c00 tx=0x7fef34005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:57.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.331+0000 7fef417fa640 1 -- 172.21.15.103:0/2852229892 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fef2c09c870 con 0x7fef44101420 2024-09-19T07:38:57.494 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:57.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.546+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef44108ff0 con 0x7fef44101420 2024-09-19T07:38:57.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:57 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:38:57.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:57 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:57 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:38:57.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:57 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:38:57.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:57 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:57 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:57 smithi103 ceph-mon[31201]: pgmap v32: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.3 MiB/s wr, 395 op/s 2024-09-19T07:38:57.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.547+0000 7fef417fa640 1 -- 172.21.15.103:0/2852229892 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11996 (secure 0 0 0) 0x7fef2c09c170 con 0x7fef44101420 2024-09-19T07:38:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:38:57.549 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":29,"btime":"2024-09-19T07:38:49:444736+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:38:48.515812+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:38:57.549 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 29 2024-09-19T07:38:57.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fef300a3440 msgr2=0x7fef300a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fef300a3440 0x7fef300a5900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef34000c00 tx=0x7fef34005a20 comp rx=0 tx=0).stop 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44101420 msgr2=0x7fef44199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44101420 0x7fef44199850 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00d950 tx=0x7fef2c00de10 comp rx=0 tx=0).stop 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 shutdown_connections 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fef300a3440 0x7fef300a5900 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef44106a80 0x7fef441a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef44101420 0x7fef44199850 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 --2- 172.21.15.103:0/2852229892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100a80 0x7fef44199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.549+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 >> 172.21.15.103:0/2852229892 conn(0x7fef440fc7a0 msgr2=0x7fef440fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.550+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 shutdown_connections 2024-09-19T07:38:57.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:57.550+0000 7fef4a07c640 1 -- 172.21.15.103:0/2852229892 wait complete. 2024-09-19T07:38:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:57 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:38:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:57 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:57 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:38:57.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:57 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:38:57.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:57 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:57 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:57 smithi134 ceph-mon[35414]: pgmap v32: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.3 MiB/s wr, 395 op/s 2024-09-19T07:38:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:57 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:38:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:57 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:57 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:38:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:57 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:38:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:57 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:57 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:38:57.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:57 smithi174 ceph-mon[35370]: pgmap v32: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.3 MiB/s wr, 395 op/s 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 -- 172.21.15.103:0/1663683652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 msgr2=0x7fd83810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1663683652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83810eaa0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd830009ca0 tx=0x7fd83002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 -- 172.21.15.103:0/1663683652 shutdown_connections 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1663683652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83810eaa0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1663683652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8381027a0 0x7fd8381067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1663683652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd838101df0 0x7fd8381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 -- 172.21.15.103:0/1663683652 >> 172.21.15.103:0/1663683652 conn(0x7fd8380fd8b0 msgr2=0x7fd8380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 -- 172.21.15.103:0/1663683652 shutdown_connections 2024-09-19T07:38:58.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.190+0000 7fd8407cb640 1 -- 172.21.15.103:0/1663683652 wait complete. 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 Processor -- start 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 -- start start 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd838101df0 0x7fd8381983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8381027a0 0x7fd838198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd838111fa0 con 0x7fd838106f20 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd838111e20 con 0x7fd8381027a0 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd8407cb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd838112120 con 0x7fd838101df0 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34446/0 (socket says 172.21.15.103:34446) 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 -- 172.21.15.103:0/1215453636 learned_addr learned my addr 172.21.15.103:0/1215453636 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83e567640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd838101df0 0x7fd8381983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83dd66640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8381027a0 0x7fd838198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 -- 172.21.15.103:0/1215453636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd838101df0 msgr2=0x7fd8381983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd838101df0 0x7fd8381983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 -- 172.21.15.103:0/1215453636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8381027a0 msgr2=0x7fd838198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8381027a0 0x7fd838198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 -- 172.21.15.103:0/1215453636 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83819d380 con 0x7fd838106f20 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.191+0000 7fd83ed68640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83819cca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd830031a60 tx=0x7fd830031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.192+0000 7fd82f7fe640 1 -- 172.21.15.103:0/1215453636 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830003c30 con 0x7fd838106f20 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.192+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83819d610 con 0x7fd838106f20 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.192+0000 7fd82f7fe640 1 -- 172.21.15.103:0/1215453636 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd830003dd0 con 0x7fd838106f20 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.192+0000 7fd82f7fe640 1 -- 172.21.15.103:0/1215453636 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300047e0 con 0x7fd838106f20 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.192+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8381a4f50 con 0x7fd838106f20 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.194+0000 7fd82f7fe640 1 -- 172.21.15.103:0/1215453636 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7fd83002f790 con 0x7fd838106f20 2024-09-19T07:38:58.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.194+0000 7fd842ba4740 1 Processor -- start 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.194+0000 7fd842ba4740 1 -- start start 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd82f7fe640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd8240a3440 0x7fd8240a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344dfa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e05140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e07a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd842ba4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565344d602e0 con 0x565344dfadc0 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd842ba4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565344d07ce0 con 0x565344dfa4a0 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd842ba4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565344d5c2a0 con 0x565344e05680 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd8240a3440 0x7fd8240a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd82f7fe640 1 -- 172.21.15.103:0/1215453636 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd8300f1230 con 0x7fd838106f20 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e05140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e05140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34448/0 (socket says 172.21.15.103:34448) 2024-09-19T07:38:58.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 -- 172.21.15.103:0/3298245594 learned_addr learned my addr 172.21.15.103:0/3298245594 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83e567640 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344dfa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83ed68640 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e07a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 -- 172.21.15.103:0/3298245594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 msgr2=0x565344e07a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e07a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 -- 172.21.15.103:0/3298245594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 msgr2=0x565344dfa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344dfa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 -- 172.21.15.103:0/3298245594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565344cec440 con 0x565344dfadc0 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd8240a3440 0x7fd8240a5900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd828014670 tx=0x7fd82800a940 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:58.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.195+0000 7fd83dd66640 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e05140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd828006aa0 tx=0x7fd828019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9378c9276fdbbebb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd82cff9640 1 -- 172.21.15.103:0/3298245594 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828025070 con 0x565344dfadc0 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd82cff9640 1 -- 172.21.15.103:0/3298245594 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82801c070 con 0x565344dfadc0 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd82cff9640 1 -- 172.21.15.103:0/3298245594 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828020680 con 0x565344dfadc0 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 -- 172.21.15.103:0/3298245594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 msgr2=0x565344e05140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e05140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd828006aa0 tx=0x7fd828019000 comp rx=0 tx=0).stop 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 -- 172.21.15.103:0/3298245594 shutdown_connections 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e07a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e05140 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3298245594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344dfa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 -- 172.21.15.103:0/3298245594 >> 172.21.15.103:0/3298245594 conn(0x565344df6a30 msgr2=0x565344df6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 -- 172.21.15.103:0/3298245594 shutdown_connections 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.196+0000 7fd842ba4740 1 -- 172.21.15.103:0/3298245594 wait complete. 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 -- start start 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 Processor -- start 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e05060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565344e08250 con 0x565344e05680 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565344d1b050 con 0x565344dfadc0 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565344e083d0 con 0x565344dfa4a0 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd842ba4740 10 client.0 _ll_register_callbacks cb 0x565344d5b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34456/0 (socket says 172.21.15.103:34456) 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 -- 172.21.15.103:0/3051703566 learned_addr learned my addr 172.21.15.103:0/3051703566 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83e567640 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e05060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83dd66640 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 -- 172.21.15.103:0/3051703566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 msgr2=0x565344e05060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e05060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 -- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 msgr2=0x565344e02310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e02310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.197+0000 7fd83ed68640 1 -- 172.21.15.103:0/3051703566 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565344ad13c0 con 0x565344e05680 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd83ed68640 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e028e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd8300b9920 tx=0x7fd8300bd0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e609a6996f2fc95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd821ffb640 1 -- 172.21.15.103:0/3051703566 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300bd4c0 con 0x565344e05680 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd821ffb640 1 -- 172.21.15.103:0/3051703566 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd8300dd400 con 0x565344e05680 2024-09-19T07:38:58.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82801c070 con 0x565344e05680 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd821ffb640 1 -- 172.21.15.103:0/3051703566 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8300c1870 con 0x565344e05680 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 10 client.15678 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565344e08250 con 0x565344e05680 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd821ffb640 1 -- 172.21.15.103:0/3051703566 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82801c070 con 0x565344e05680 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd821ffb640 1 -- 172.21.15.103:0/3051703566 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fd8300a1a40 con 0x565344e05680 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 10 client.15678 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 20 client.15678 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 4 client.15678 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:38:58.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.198+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565344d6d850 con 0x565344e15060 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.199+0000 7fd83e567640 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.199+0000 7fd821ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.199+0000 7fd83e567640 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e174c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c69f1feed7b73d server_cookie=3d6fd2c87b3049c6 in_seq=0 out_seq=0 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd821ffb640 1 -- 172.21.15.103:0/3051703566 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565344d6d850 con 0x565344e15060 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd821ffb640 10 client.15678 handle_command_reply: tid=0 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 client.15678 shutdown 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 msgr2=0x565344e028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e028e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd8300b9920 tx=0x7fd8300bd0c0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 shutdown_connections 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e174c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd821ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:58.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e028e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e02310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 --2- 172.21.15.103:0/3051703566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e05060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 >> 172.21.15.103:0/3051703566 conn(0x565344df6a30 msgr2=0x565344df5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 shutdown_connections 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 1 -- 172.21.15.103:0/3051703566 wait complete. 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.200+0000 7fd842ba4740 20 client.15678 trim_cache size 0 max 16384 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 Processor -- start 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 -- start start 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e04280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e1cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e1d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381ad6c0 con 0x565344e05680 2024-09-19T07:38:58.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381af290 con 0x565344dfadc0 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd842ba4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381add90 con 0x565344dfa4a0 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e1d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e1d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34466/0 (socket says 172.21.15.103:34466) 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 -- 172.21.15.103:0/1858601086 learned_addr learned my addr 172.21.15.103:0/1858601086 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83e567640 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e04280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83dd66640 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e1cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 -- 172.21.15.103:0/1858601086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 msgr2=0x565344e04280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e04280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 -- 172.21.15.103:0/1858601086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 msgr2=0x565344e1cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e1cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 -- 172.21.15.103:0/1858601086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565344df76c0 con 0x565344e05680 2024-09-19T07:38:58.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd83ed68640 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e1d310 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd830006010 tx=0x7fd8300bd130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b438cafd2c0c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.217+0000 7fd823fff640 1 -- 172.21.15.103:0/1858601086 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300d6030 con 0x565344e05680 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd823fff640 1 -- 172.21.15.103:0/1858601086 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8300f1050 con 0x565344e05680 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd823fff640 1 -- 172.21.15.103:0/1858601086 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300eed70 con 0x565344e05680 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 -- 172.21.15.103:0/1858601086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 msgr2=0x565344e1d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e1d310 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd830006010 tx=0x7fd8300bd130 comp rx=0 tx=0).stop 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 -- 172.21.15.103:0/1858601086 shutdown_connections 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344e05680 0x565344e1d310 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfadc0 0x565344e1cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 --2- 172.21.15.103:0/1858601086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344dfa4a0 0x565344e04280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 -- 172.21.15.103:0/1858601086 >> 172.21.15.103:0/1858601086 conn(0x565344df6a30 msgr2=0x565344e03b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:58.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 -- 172.21.15.103:0/1858601086 shutdown_connections 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 -- 172.21.15.103:0/1858601086 wait complete. 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.218+0000 7fd842ba4740 1 -- start start 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 1 Processor -- start 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344e0b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e0bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381aeb30 con 0x565344dfadc0 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381ae460 con 0x565344dfa4a0 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8381ab6a0 con 0x565344e05680 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd842ba4740 10 client.0 _ll_register_callbacks cb 0x7fd830004000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e0bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e0bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34470/0 (socket says 172.21.15.103:34470) 2024-09-19T07:38:58.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 -- 172.21.15.103:0/2474443831 learned_addr learned my addr 172.21.15.103:0/2474443831 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83e567640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344e0b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83ed68640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e7edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 -- 172.21.15.103:0/2474443831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 msgr2=0x565344e7edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e7edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 -- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 msgr2=0x565344e0b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344e0b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83dd66640 1 -- 172.21.15.103:0/2474443831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565344e0c490 con 0x565344dfadc0 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.219+0000 7fd83ed68640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd83dd66640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e0bc00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd828004810 tx=0x7fd828018850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84693f17a1b796a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:38:58.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd821ffb640 1 -- 172.21.15.103:0/2474443831 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82802f070 con 0x565344dfadc0 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd821ffb640 1 -- 172.21.15.103:0/2474443831 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd828018a40 con 0x565344dfadc0 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8300f1050 con 0x565344dfadc0 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd821ffb640 1 -- 172.21.15.103:0/2474443831 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd828025040 con 0x565344dfadc0 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 10 client.15684 fetch_fsmap learned FSMap version 29 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565344e803d0 con 0x565344dfadc0 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd821ffb640 1 -- 172.21.15.103:0/2474443831 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8300f1050 con 0x565344dfadc0 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd821ffb640 1 -- 172.21.15.103:0/2474443831 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7fd828032020 con 0x565344dfadc0 2024-09-19T07:38:58.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 10 client.15684 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 20 client.15684 populate_metadata read hostname 'smithi103' 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 4 client.15684 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.220+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565344d67d60 con 0x565344e15060 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd83e567640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd83e567640 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e828a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f66f40f22c1fb3 server_cookie=21df95b9b43fda2a in_seq=0 out_seq=0 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd821ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd821ffb640 1 -- 172.21.15.103:0/2474443831 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565344d67d60 con 0x565344e15060 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd821ffb640 10 client.15684 handle_command_reply: tid=0 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 client.15684 shutdown 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 msgr2=0x565344e0bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e0bc00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd828004810 tx=0x7fd828018850 comp rx=0 tx=0).stop 2024-09-19T07:38:58.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 shutdown_connections 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565344e15060 0x565344e828a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565344e05680 0x565344e7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565344dfadc0 0x565344e0bc00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd821ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 --2- 172.21.15.103:0/2474443831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565344dfa4a0 0x565344e0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 >> 172.21.15.103:0/2474443831 conn(0x565344df6a30 msgr2=0x565344e03b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 shutdown_connections 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.221+0000 7fd842ba4740 1 -- 172.21.15.103:0/2474443831 wait complete. 2024-09-19T07:38:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd842ba4740 20 client.15684 trim_cache size 0 max 16384 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: "3473486d-6045-48ca-8ce0-1c28614627a4": { 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "3473486d-6045-48ca-8ce0-1c28614627a4", 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:38:58.243 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:38:58.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd8240a3440 msgr2=0x7fd8240a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd8240a3440 0x7fd8240a5900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd828014670 tx=0x7fd82800a940 comp rx=0 tx=0).stop 2024-09-19T07:38:58.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 msgr2=0x7fd83819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:38:58.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83819cca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd830031a60 tx=0x7fd830031a90 comp rx=0 tx=0).stop 2024-09-19T07:38:58.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 shutdown_connections 2024-09-19T07:38:58.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd8240a3440 0x7fd8240a5900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd838106f20 0x7fd83819cca0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8381027a0 0x7fd838198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 --2- 172.21.15.103:0/1215453636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd838101df0 0x7fd8381983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:38:58.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 >> 172.21.15.103:0/1215453636 conn(0x7fd8380fd8b0 msgr2=0x7fd8380feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:38:58.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 shutdown_connections 2024-09-19T07:38:58.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:38:58.222+0000 7fd8407cb640 1 -- 172.21.15.103:0/1215453636 wait complete. 2024-09-19T07:38:58.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:58 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2852229892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:58.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:58 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2852229892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:58.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:58 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2852229892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:38:59.039 INFO:tasks.cephfs.filesystem:scrub status for tag:3473486d-6045-48ca-8ce0-1c28614627a4 - {'path': '/', 'tag': '3473486d-6045-48ca-8ce0-1c28614627a4', 'options': 'recursive,force'} 2024-09-19T07:38:59.601 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 2c26da9c --release --await' 2024-09-19T07:38:59.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 2c26da9c --release --await 2024-09-19T07:38:59.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:38:59 smithi134 ceph-mon[35414]: pgmap v33: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.3 MiB/s wr, 395 op/s 2024-09-19T07:38:59.851 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:38:59.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:38:59 smithi174 ceph-mon[35370]: pgmap v33: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.3 MiB/s wr, 395 op/s 2024-09-19T07:38:59.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:38:59 smithi103 ceph-mon[31201]: pgmap v33: 97 pgs: 97 active+clean; 935 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.3 MiB/s wr, 395 op/s 2024-09-19T07:39:00.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 -- 172.21.15.103:0/3999147297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430810e4d0 msgr2=0x7f43081108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 --2- 172.21.15.103:0/3999147297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430810e4d0 0x7f43081108e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4300009be0 tx=0x7f430002a6c0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 -- 172.21.15.103:0/3999147297 shutdown_connections 2024-09-19T07:39:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 --2- 172.21.15.103:0/3999147297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f430810e4d0 0x7f43081108e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 --2- 172.21.15.103:0/3999147297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4308101360 0x7f430810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 --2- 172.21.15.103:0/3999147297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43081009b0 0x7f4308100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 -- 172.21.15.103:0/3999147297 >> 172.21.15.103:0/3999147297 conn(0x7f43080fc7d0 msgr2=0x7f43080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 -- 172.21.15.103:0/3999147297 shutdown_connections 2024-09-19T07:39:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.392+0000 7f430d4a6640 1 -- 172.21.15.103:0/3999147297 wait complete. 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 Processor -- start 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 -- start start 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43081009b0 0x7f430819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4308101360 0x7f430819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430810e4d0 0x7f430819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43081141b0 con 0x7f43081009b0 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4308114030 con 0x7f4308101360 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f430d4a6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4308114330 con 0x7f430810e4d0 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43081009b0 0x7f430819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43081009b0 0x7f430819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34500/0 (socket says 172.21.15.103:34500) 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f4306ffd640 1 -- 172.21.15.103:0/4086141736 learned_addr learned my addr 172.21.15.103:0/4086141736 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f43067fc640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4308101360 0x7f430819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.393+0000 7f43077fe640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430810e4d0 0x7f430819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f4306ffd640 1 -- 172.21.15.103:0/4086141736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430810e4d0 msgr2=0x7f430819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f4306ffd640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430810e4d0 0x7f430819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f4306ffd640 1 -- 172.21.15.103:0/4086141736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4308101360 msgr2=0x7f430819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f4306ffd640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4308101360 0x7f430819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f4306ffd640 1 -- 172.21.15.103:0/4086141736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430819f640 con 0x7f43081009b0 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f43067fc640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4308101360 0x7f430819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:39:00.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f4306ffd640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43081009b0 0x7f430819a690 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e9c0 tx=0x7f42f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f42effff640 1 -- 172.21.15.103:0/4086141736 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f800cde0 con 0x7f43081009b0 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430819f930 con 0x7f43081009b0 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43081a7210 con 0x7f43081009b0 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f42effff640 1 -- 172.21.15.103:0/4086141736 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f80035f0 con 0x7f43081009b0 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.394+0000 7f42effff640 1 -- 172.21.15.103:0/4086141736 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8018630 con 0x7f43081009b0 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f42effff640 1 -- 172.21.15.103:0/4086141736 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138532+0+0 (secure 0 0 0) 0x7f42f80187d0 con 0x7f43081009b0 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 Processor -- start 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 -- start start 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78f9f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78fac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562c78f04620 con 0x562c78f9f980 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c78eabe50 con 0x562c78faa240 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f430f87f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562c78f005e0 con 0x562c78f9f060 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34506/0 (socket says 172.21.15.103:34506) 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f4306ffd640 1 -- 172.21.15.103:0/283438834 learned_addr learned my addr 172.21.15.103:0/283438834 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f43077fe640 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78f9f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.397+0000 7f43067fc640 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78fac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f4306ffd640 1 -- 172.21.15.103:0/283438834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 msgr2=0x562c78f9f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f4306ffd640 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78f9f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f4306ffd640 1 -- 172.21.15.103:0/283438834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 msgr2=0x562c78fac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f4306ffd640 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78fac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f4306ffd640 1 -- 172.21.15.103:0/283438834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c78e9cd40 con 0x562c78f9f980 2024-09-19T07:39:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f42effff640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42f00a34c0 0x7f42f00a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f43077fe640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42f00a34c0 0x7f42f00a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f4306ffd640 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa9d00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f42f80cd690 tx=0x7f42f80d3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664de01a6c320858 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f42ed7fa640 1 -- 172.21.15.103:0/283438834 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8097d40 con 0x562c78f9f980 2024-09-19T07:39:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f42ed7fa640 1 -- 172.21.15.103:0/283438834 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f809c8d0 con 0x562c78f9f980 2024-09-19T07:39:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f42ed7fa640 1 -- 172.21.15.103:0/283438834 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f809bb30 con 0x562c78f9f980 2024-09-19T07:39:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.398+0000 7f42effff640 1 -- 172.21.15.103:0/4086141736 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f42f80d0050 con 0x7f43081009b0 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.399+0000 7f430f87f740 1 -- 172.21.15.103:0/283438834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 msgr2=0x562c78fa9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.399+0000 7f430f87f740 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa9d00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f42f80cd690 tx=0x7f42f80d3f90 comp rx=0 tx=0).stop 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f43077fe640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42f00a34c0 0x7f42f00a5980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f43000092f0 tx=0x7f4300009280 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f430f87f740 1 -- 172.21.15.103:0/283438834 shutdown_connections 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f430f87f740 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78fac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f430f87f740 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa9d00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f430f87f740 1 --2- 172.21.15.103:0/283438834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78f9f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f430f87f740 1 -- 172.21.15.103:0/283438834 >> 172.21.15.103:0/283438834 conn(0x562c78f9b5f0 msgr2=0x562c78f9b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f430f87f740 1 -- 172.21.15.103:0/283438834 shutdown_connections 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.400+0000 7f430f87f740 1 -- 172.21.15.103:0/283438834 wait complete. 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 -- start start 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 Processor -- start 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fa7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78faa240 0x562c78fa7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562c78face10 con 0x562c78f9f980 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c78e8cf10 con 0x562c78f9f060 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562c78facf90 con 0x562c78faa240 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f430f87f740 10 client.0 _ll_register_callbacks cb 0x562c78eb0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f43067fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f43067fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34514/0 (socket says 172.21.15.103:34514) 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.401+0000 7f43067fc640 1 -- 172.21.15.103:0/3718473634 learned_addr learned my addr 172.21.15.103:0/3718473634 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f4306ffd640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fa7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43077fe640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78faa240 0x562c78fa7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43067fc640 1 -- 172.21.15.103:0/3718473634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78faa240 msgr2=0x562c78fa7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43067fc640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78faa240 0x562c78fa7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43067fc640 1 -- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 msgr2=0x562c78fa7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43067fc640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fa7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43067fc640 1 -- 172.21.15.103:0/3718473634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c78c13580 con 0x562c78f9f980 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43077fe640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78faa240 0x562c78fa7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f43067fc640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa7690 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f42fc012a10 tx=0x7f42fc012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1e0ec8c0444465 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f42eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f42eaff5640 1 -- 172.21.15.103:0/3718473634 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00fa40 con 0x562c78f9f980 2024-09-19T07:39:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f42eaff5640 1 -- 172.21.15.103:0/3718473634 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f42fc00fbe0 con 0x562c78f9f980 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f809c8d0 con 0x562c78f9f980 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f42eaff5640 1 -- 172.21.15.103:0/3718473634 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42fc00fde0 con 0x562c78f9f980 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f430f87f740 10 client.15702 fetch_fsmap learned FSMap version 29 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.402+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c78cdd960 con 0x562c78f9f980 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f42eaff5640 1 -- 172.21.15.103:0/3718473634 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42f809c8d0 con 0x562c78f9f980 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f42eaff5640 1 -- 172.21.15.103:0/3718473634 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f42fc010430 con 0x562c78f9f980 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f430f87f740 10 client.15702 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f430f87f740 10 client.15702 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f430f87f740 20 client.15702 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f430f87f740 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c78fb9f50 0x562c78fbc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f430f87f740 4 client.15702 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c78f11b90 con 0x562c78fb9f50 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f4306ffd640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c78fb9f50 0x562c78fbc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.403+0000 7f4306ffd640 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c78fb9f50 0x562c78fbc3a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c44c2b2bbb5ba6 server_cookie=c499f95b9f1f411a in_seq=0 out_seq=0 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f42eaff5640 10 client.15702 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f42eaff5640 1 -- 172.21.15.103:0/3718473634 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c78f11b90 con 0x562c78fb9f50 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f42eaff5640 10 client.15702 handle_command_reply: tid=0 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 client.15702 shutdown 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 msgr2=0x562c78fa7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa7690 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f42fc012a10 tx=0x7f42fc012ed0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 shutdown_connections 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c78fb9f50 0x562c78fbc3a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78faa240 0x562c78fa7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78fa7690 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f42eaff5640 0 client.15702 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 --2- 172.21.15.103:0/3718473634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fa7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.404+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 >> 172.21.15.103:0/3718473634 conn(0x562c78f9b5f0 msgr2=0x562c78fa12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.405+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 shutdown_connections 2024-09-19T07:39:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.405+0000 7f430f87f740 1 -- 172.21.15.103:0/3718473634 wait complete. 2024-09-19T07:39:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.405+0000 7f430f87f740 20 client.15702 trim_cache size 0 max 16384 2024-09-19T07:39:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 Processor -- start 2024-09-19T07:39:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 -- start start 2024-09-19T07:39:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fc1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f980 0x562c78fc21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78faa240 0x562c78fa43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42f400e700 con 0x562c78faa240 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40102d0 con 0x562c78f9f060 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f430f87f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42f400edd0 con 0x562c78f9f980 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f43077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78faa240 0x562c78fa43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f43067fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f980 0x562c78fc21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f43077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78faa240 0x562c78fa43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34528/0 (socket says 172.21.15.103:34528) 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f43077fe640 1 -- 172.21.15.103:0/2678907443 learned_addr learned my addr 172.21.15.103:0/2678907443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f43067fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f980 0x562c78fc21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37548/0 (socket says 172.21.15.103:37548) 2024-09-19T07:39:00.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.423+0000 7f4306ffd640 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fc1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f43067fc640 1 -- 172.21.15.103:0/2678907443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 msgr2=0x562c78fc1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f43067fc640 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fc1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f43067fc640 1 -- 172.21.15.103:0/2678907443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78faa240 msgr2=0x562c78fa43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f43067fc640 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78faa240 0x562c78fa43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f43067fc640 1 -- 172.21.15.103:0/2678907443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c78f9f7b0 con 0x562c78f9f980 2024-09-19T07:39:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f43077fe640 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78faa240 0x562c78fa43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:39:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f43067fc640 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f980 0x562c78fc21f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0129e0 tx=0x7f42fc0106f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28413a3383a3d9fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f42ecff9640 1 -- 172.21.15.103:0/2678907443 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc029030 con 0x562c78f9f980 2024-09-19T07:39:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f42ecff9640 1 -- 172.21.15.103:0/2678907443 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42fc0055c0 con 0x562c78f9f980 2024-09-19T07:39:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.424+0000 7f42ecff9640 1 -- 172.21.15.103:0/2678907443 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0035f0 con 0x562c78f9f980 2024-09-19T07:39:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 -- 172.21.15.103:0/2678907443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f980 msgr2=0x562c78fc21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f980 0x562c78fc21f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0129e0 tx=0x7f42fc0106f0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 -- 172.21.15.103:0/2678907443 shutdown_connections 2024-09-19T07:39:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78faa240 0x562c78fa43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f980 0x562c78fc21f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 --2- 172.21.15.103:0/2678907443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78f9f060 0x562c78fc1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 -- 172.21.15.103:0/2678907443 >> 172.21.15.103:0/2678907443 conn(0x562c78f9b5f0 msgr2=0x562c78fa5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 -- 172.21.15.103:0/2678907443 shutdown_connections 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 -- 172.21.15.103:0/2678907443 wait complete. 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 -- start start 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.425+0000 7f430f87f740 1 Processor -- start 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f430f87f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78fb2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f430f87f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78faeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f430f87f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78faefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f430f87f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42f400fb70 con 0x562c78f9f980 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f430f87f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42f400f4a0 con 0x562c78faa240 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f430f87f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42f400c6e0 con 0x562c78f9f060 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f430f87f740 10 client.0 _ll_register_callbacks cb 0x7f42fc012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78fb2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78fb2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37552/0 (socket says 172.21.15.103:37552) 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 -- 172.21.15.103:0/1141223806 learned_addr learned my addr 172.21.15.103:0/1141223806 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f43067fc640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78faeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f43077fe640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78faefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 -- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 msgr2=0x562c78faefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78faefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 -- 172.21.15.103:0/1141223806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 msgr2=0x562c78faeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78faeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 -- 172.21.15.103:0/1141223806 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c78faf520 con 0x562c78f9f060 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f43077fe640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78faefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f43067fc640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78faeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f4306ffd640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78fb2710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f42f809bed0 tx=0x7f42f800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d6e931bd7613b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.426+0000 7f42eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f42eaff5640 1 -- 172.21.15.103:0/1141223806 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f809bd10 con 0x562c78f9f060 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f42eaff5640 1 -- 172.21.15.103:0/1141223806 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f42f80bc0c0 con 0x562c78f9f060 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42fc0055c0 con 0x562c78f9f060 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f42eaff5640 1 -- 172.21.15.103:0/1141223806 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42f809bfa0 con 0x562c78f9f060 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 10 client.25264 fetch_fsmap learned FSMap version 29 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562c790259c0 con 0x562c78f9f060 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f42eaff5640 1 -- 172.21.15.103:0/1141223806 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42fc0055c0 con 0x562c78f9f060 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f42eaff5640 1 -- 172.21.15.103:0/1141223806 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8374+0+0 (secure 0 0 0) 0x7f42fc009ce0 con 0x562c78f9f060 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 10 client.25264 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 10 client.25264 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 20 client.25264 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:00.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c79025d00 0x562c790280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:00.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 4 client.25264 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2c26da9c", "release": true, "await": true}] 2024-09-19T07:39:00.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.427+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2c26da9c", "release": true, "await": true}) -- 0x562c78f0c0a0 con 0x562c79025d00 2024-09-19T07:39:00.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.428+0000 7f43067fc640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c79025d00 0x562c790280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:00.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.428+0000 7f42eaff5640 10 client.25264 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:00.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.428+0000 7f43067fc640 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c79025d00 0x562c790280f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c57cbc01ea6d30f server_cookie=629a5171784a4cb2 in_seq=0 out_seq=0 2024-09-19T07:39:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 25.0 s, 24.8 tps, lat 7725.876 ms stddev 5522.878 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.486+0000 7f42eaff5640 1 -- 172.21.15.103:0/1141223806 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562c78f0c0a0 con 0x562c79025d00 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.486+0000 7f42eaff5640 10 client.25264 handle_command_reply: tid=0 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.486+0000 7f430f87f740 1 client.25264 shutdown 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 msgr2=0x562c78fb2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78fb2710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f42f809bed0 tx=0x7f42f800c860 comp rx=0 tx=0).stop 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 shutdown_connections 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562c79025d00 0x562c790280f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c78faa240 0x562c78faefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c78f9f980 0x562c78faeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 --2- 172.21.15.103:0/1141223806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c78f9f060 0x562c78fb2710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f42eaff5640 0 client.25264 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:00.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 >> 172.21.15.103:0/1141223806 conn(0x562c78f9b5f0 msgr2=0x562c78fa70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 shutdown_connections 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.487+0000 7f430f87f740 1 -- 172.21.15.103:0/1141223806 wait complete. 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.488+0000 7f430f87f740 20 client.25264 trim_cache size 0 max 16384 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 29, 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 4, 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:39:00.490 INFO:teuthology.orchestra.run.smithi103.stdout: "2c26da9c": { 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 4, 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 131.2, 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:39:00.492 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:00.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:00.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:00.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:00.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:00.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:00.492 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:39:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.488+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42f00a34c0 msgr2=0x7f42f00a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.488+0000 7f430d4a6640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42f00a34c0 0x7f42f00a5980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f43000092f0 tx=0x7f4300009280 comp rx=0 tx=0).stop 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.488+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43081009b0 msgr2=0x7f430819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43081009b0 0x7f430819a690 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f42f800e9c0 tx=0x7f42f800ee80 comp rx=0 tx=0).stop 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 shutdown_connections 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42f00a34c0 0x7f42f00a5980 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f430810e4d0 0x7f430819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4308101360 0x7f430819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 --2- 172.21.15.103:0/4086141736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43081009b0 0x7f430819a690 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 >> 172.21.15.103:0/4086141736 conn(0x7f43080fc7d0 msgr2=0x7f43080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 shutdown_connections 2024-09-19T07:39:00.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:00.489+0000 7f430d4a6640 1 -- 172.21.15.103:0/4086141736 wait complete. 2024-09-19T07:39:00.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:39:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:39:00.363+0000 7f0335715640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:39:00.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:39:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:39:00.363+0000 7f0335715640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:39:00.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:39:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:39:00.363+0000 7f0335715640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:39:00.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:39:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:39:00.363+0000 7f0335715640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:39:00.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:39:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:39:00.363+0000 7f0335715640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T07:39:01.186 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2c26da9c', total seconds elapsed: 16.4 2024-09-19T07:39:01.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:01 smithi134 ceph-mon[35414]: pgmap v34: 97 pgs: 97 active+clean; 935 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-09-19T07:39:01.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:01 smithi174 ceph-mon[35370]: pgmap v34: 97 pgs: 97 active+clean; 935 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-09-19T07:39:01.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:01 smithi103 ceph-mon[31201]: pgmap v34: 97 pgs: 97 active+clean; 935 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-09-19T07:39:03.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:03 smithi134 ceph-mon[35414]: pgmap v35: 97 pgs: 97 active+clean; 935 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 134 KiB/s wr, 68 op/s 2024-09-19T07:39:03.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:03 smithi134 ceph-mon[35414]: mgrmap e25: x(active, since 64s), standbys: y, z 2024-09-19T07:39:03.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:03 smithi174 ceph-mon[35370]: pgmap v35: 97 pgs: 97 active+clean; 935 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 134 KiB/s wr, 68 op/s 2024-09-19T07:39:03.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:03 smithi174 ceph-mon[35370]: mgrmap e25: x(active, since 64s), standbys: y, z 2024-09-19T07:39:03.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:03 smithi103 ceph-mon[31201]: pgmap v35: 97 pgs: 97 active+clean; 935 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 134 KiB/s wr, 68 op/s 2024-09-19T07:39:03.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:03 smithi103 ceph-mon[31201]: mgrmap e25: x(active, since 64s), standbys: y, z 2024-09-19T07:39:04.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:04 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:04.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:04 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:04.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:04 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:04.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:04 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:04.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:04 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:04.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:04 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 30.0 s, 675.4 tps, lat 209.886 ms stddev 856.749 2024-09-19T07:39:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:05 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:05 smithi134 ceph-mon[35414]: pgmap v36: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-09-19T07:39:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:05 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:05 smithi174 ceph-mon[35370]: pgmap v36: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-09-19T07:39:05.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:05 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:05.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:05 smithi103 ceph-mon[31201]: pgmap v36: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-09-19T07:39:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:07 smithi134 ceph-mon[35414]: pgmap v37: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.4 MiB/s wr, 374 op/s 2024-09-19T07:39:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:39:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:39:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:39:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:07 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:39:07.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:07 smithi174 ceph-mon[35370]: pgmap v37: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.4 MiB/s wr, 374 op/s 2024-09-19T07:39:07.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:39:07.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:39:07.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:39:07.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:07 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:39:07.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:07 smithi103 ceph-mon[31201]: pgmap v37: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.4 MiB/s wr, 374 op/s 2024-09-19T07:39:07.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:39:07.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:39:07.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:39:07.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:07 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:39:09.772 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:09 smithi103 ceph-mon[31201]: pgmap v38: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.4 MiB/s wr, 374 op/s 2024-09-19T07:39:09.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:09 smithi134 ceph-mon[35414]: pgmap v38: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.4 MiB/s wr, 374 op/s 2024-09-19T07:39:09.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:09 smithi174 ceph-mon[35370]: pgmap v38: 97 pgs: 97 active+clean; 943 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 2.4 MiB/s wr, 374 op/s 2024-09-19T07:39:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 35.0 s, 3418.0 tps, lat 29.236 ms stddev 32.807 2024-09-19T07:39:10.785 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:10 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:39:10.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:10 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:39:10.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:10 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:39:11.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:11 smithi103 ceph-mon[31201]: pgmap v39: 97 pgs: 97 active+clean; 998 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 46 MiB/s wr, 1.31k op/s 2024-09-19T07:39:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:11 smithi134 ceph-mon[35414]: pgmap v39: 97 pgs: 97 active+clean; 998 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 46 MiB/s wr, 1.31k op/s 2024-09-19T07:39:11.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:11 smithi174 ceph-mon[35370]: pgmap v39: 97 pgs: 97 active+clean; 998 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 46 MiB/s wr, 1.31k op/s 2024-09-19T07:39:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:13 smithi134 ceph-mon[35414]: pgmap v40: 97 pgs: 97 active+clean; 998 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 1.30k op/s 2024-09-19T07:39:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:13 smithi174 ceph-mon[35370]: pgmap v40: 97 pgs: 97 active+clean; 998 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 1.30k op/s 2024-09-19T07:39:13.972 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:13 smithi103 ceph-mon[31201]: pgmap v40: 97 pgs: 97 active+clean; 998 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 46 MiB/s wr, 1.30k op/s 2024-09-19T07:39:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 40.0 s, 3277.8 tps, lat 30.476 ms stddev 40.924 2024-09-19T07:39:15.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:15 smithi134 ceph-mon[35414]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 63 MiB/s wr, 2.64k op/s 2024-09-19T07:39:15.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:15 smithi174 ceph-mon[35370]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 63 MiB/s wr, 2.64k op/s 2024-09-19T07:39:15.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:15 smithi103 ceph-mon[31201]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 63 MiB/s wr, 2.64k op/s 2024-09-19T07:39:17.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:17 smithi134 ceph-mon[35414]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 MiB/s wr, 2.27k op/s 2024-09-19T07:39:17.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:17 smithi174 ceph-mon[35370]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 MiB/s wr, 2.27k op/s 2024-09-19T07:39:17.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:17 smithi103 ceph-mon[31201]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 MiB/s wr, 2.27k op/s 2024-09-19T07:39:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:19 smithi134 ceph-mon[35414]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 MiB/s wr, 2.27k op/s 2024-09-19T07:39:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:19 smithi174 ceph-mon[35370]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 MiB/s wr, 2.27k op/s 2024-09-19T07:39:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:19 smithi103 ceph-mon[31201]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 60 MiB/s wr, 2.27k op/s 2024-09-19T07:39:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 45.0 s, 4253.8 tps, lat 23.537 ms stddev 24.789 2024-09-19T07:39:21.811 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:21 smithi103 ceph-mon[31201]: pgmap v44: 97 pgs: 97 active+clean; 1.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 68 MiB/s wr, 2.79k op/s 2024-09-19T07:39:21.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:21 smithi134 ceph-mon[35414]: pgmap v44: 97 pgs: 97 active+clean; 1.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 68 MiB/s wr, 2.79k op/s 2024-09-19T07:39:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:21 smithi174 ceph-mon[35370]: pgmap v44: 97 pgs: 97 active+clean; 1.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 68 MiB/s wr, 2.79k op/s 2024-09-19T07:39:21.987 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:39:21.987 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T07:39:21.988 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:39:22.365 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:22.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.921+0000 7fa2d5440640 1 -- 172.21.15.103:0/104775199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 msgr2=0x7fa2d0114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:22.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.921+0000 7fa2d5440640 1 --2- 172.21.15.103:0/104775199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 0x7fa2d0114570 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009900 tx=0x7fa2c002a480 comp rx=0 tx=0).stop 2024-09-19T07:39:22.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.922+0000 7fa2d5440640 1 -- 172.21.15.103:0/104775199 shutdown_connections 2024-09-19T07:39:22.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.922+0000 7fa2d5440640 1 --2- 172.21.15.103:0/104775199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2d0114ab0 0x7fa2d0116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:22.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.922+0000 7fa2d5440640 1 --2- 172.21.15.103:0/104775199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 0x7fa2d0114570 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:22.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.922+0000 7fa2d5440640 1 --2- 172.21.15.103:0/104775199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d01091f0 0x7fa2d01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:22.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.922+0000 7fa2d5440640 1 -- 172.21.15.103:0/104775199 >> 172.21.15.103:0/104775199 conn(0x7fa2d006e140 msgr2=0x7fa2d006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.922+0000 7fa2d5440640 1 -- 172.21.15.103:0/104775199 shutdown_connections 2024-09-19T07:39:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.923+0000 7fa2d5440640 1 -- 172.21.15.103:0/104775199 wait complete. 2024-09-19T07:39:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 Processor -- start 2024-09-19T07:39:22.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 -- start start 2024-09-19T07:39:22.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2d01091f0 0x7fa2d01a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:22.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 0x7fa2d01a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:22.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d0114ab0 0x7fa2d01a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:22.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d011a7b0 con 0x7fa2d0109b10 2024-09-19T07:39:22.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d011a630 con 0x7fa2d01091f0 2024-09-19T07:39:22.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2d5440640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d011a930 con 0x7fa2d0114ab0 2024-09-19T07:39:22.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2ce7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 0x7fa2d01a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:22.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2cf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d0114ab0 0x7fa2d01a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:22.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2ce7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 0x7fa2d01a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45678/0 (socket says 172.21.15.103:45678) 2024-09-19T07:39:22.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2cf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d0114ab0 0x7fa2d01a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56188/0 (socket says 172.21.15.103:56188) 2024-09-19T07:39:22.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.924+0000 7fa2ce7fc640 1 -- 172.21.15.103:0/2524527097 learned_addr learned my addr 172.21.15.103:0/2524527097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:22.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2ceffd640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2d01091f0 0x7fa2d01a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:22.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2cf7fe640 1 -- 172.21.15.103:0/2524527097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2d01091f0 msgr2=0x7fa2d01a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:22.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2cf7fe640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2d01091f0 0x7fa2d01a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:22.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2cf7fe640 1 -- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 msgr2=0x7fa2d01a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:22.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2cf7fe640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 0x7fa2d01a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:22.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2cf7fe640 1 -- 172.21.15.103:0/2524527097 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d01a7ec0 con 0x7fa2d0114ab0 2024-09-19T07:39:22.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2ceffd640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2d01091f0 0x7fa2d01a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:22.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2cf7fe640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d0114ab0 0x7fa2d01a77e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00743f0 tx=0x7fa2c400cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:22.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2b7fff640 1 -- 172.21.15.103:0/2524527097 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4007bb0 con 0x7fa2d0114ab0 2024-09-19T07:39:22.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2b7fff640 1 -- 172.21.15.103:0/2524527097 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c4007d50 con 0x7fa2d0114ab0 2024-09-19T07:39:22.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2b7fff640 1 -- 172.21.15.103:0/2524527097 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40186c0 con 0x7fa2d0114ab0 2024-09-19T07:39:22.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d01a81b0 con 0x7fa2d0114ab0 2024-09-19T07:39:22.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.925+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d010c510 con 0x7fa2d0114ab0 2024-09-19T07:39:22.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.928+0000 7fa2b7fff640 1 -- 172.21.15.103:0/2524527097 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fa2c40213f0 con 0x7fa2d0114ab0 2024-09-19T07:39:22.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.928+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d0073a40 con 0x7fa2d0114ab0 2024-09-19T07:39:22.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.928+0000 7fa2b7fff640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2b80a34d0 0x7fa2b80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:22.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.929+0000 7fa2b7fff640 1 -- 172.21.15.103:0/2524527097 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fa2c40cffe0 con 0x7fa2d0114ab0 2024-09-19T07:39:22.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.929+0000 7fa2ceffd640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2b80a34d0 0x7fa2b80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:22.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.929+0000 7fa2ceffd640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2b80a34d0 0x7fa2b80a5990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005fd0 tx=0x7fa2c8009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:22.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:22.934+0000 7fa2b7fff640 1 -- 172.21.15.103:0/2524527097 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2c4014030 con 0x7fa2d0114ab0 2024-09-19T07:39:23.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.269+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2d010d9b0 con 0x7fa2d0114ab0 2024-09-19T07:39:23.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.270+0000 7fa2b7fff640 1 -- 172.21.15.103:0/2524527097 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11998 (secure 0 0 0) 0x7fa2c409c780 con 0x7fa2d0114ab0 2024-09-19T07:39:23.270 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:39:23.270 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":30,"btime":"2024-09-19T07:39:09:495445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:39:08.515410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:39:23.271 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 30 2024-09-19T07:39:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2b80a34d0 msgr2=0x7fa2b80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2b80a34d0 0x7fa2b80a5990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005fd0 tx=0x7fa2c8009500 comp rx=0 tx=0).stop 2024-09-19T07:39:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d0114ab0 msgr2=0x7fa2d01a77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d0114ab0 0x7fa2d01a77e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00743f0 tx=0x7fa2c400cdd0 comp rx=0 tx=0).stop 2024-09-19T07:39:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 shutdown_connections 2024-09-19T07:39:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2b80a34d0 0x7fa2b80a5990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d0114ab0 0x7fa2d01a77e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2d0109b10 0x7fa2d01a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 --2- 172.21.15.103:0/2524527097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2d01091f0 0x7fa2d01a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 >> 172.21.15.103:0/2524527097 conn(0x7fa2d006e140 msgr2=0x7fa2d010a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.273+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 shutdown_connections 2024-09-19T07:39:23.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:23.274+0000 7fa2d5440640 1 -- 172.21.15.103:0/2524527097 wait complete. 2024-09-19T07:39:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:23 smithi134 ceph-mon[35414]: pgmap v45: 97 pgs: 97 active+clean; 1.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.85k op/s 2024-09-19T07:39:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:23 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2524527097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:23 smithi174 ceph-mon[35370]: pgmap v45: 97 pgs: 97 active+clean; 1.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.85k op/s 2024-09-19T07:39:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:23 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2524527097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:23.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:23 smithi103 ceph-mon[31201]: pgmap v45: 97 pgs: 97 active+clean; 1.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.85k op/s 2024-09-19T07:39:23.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:23 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2524527097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:24.059 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T07:39:24.060 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T07:39:24.460 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:25.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.032+0000 7fe5449c0640 1 -- 172.21.15.103:0/2587065806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 msgr2=0x7fe5401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.032+0000 7fe5449c0640 1 --2- 172.21.15.103:0/2587065806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401029b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe534009860 tx=0x7fe53402a3f0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.035+0000 7fe5449c0640 1 -- 172.21.15.103:0/2587065806 shutdown_connections 2024-09-19T07:39:25.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.035+0000 7fe5449c0640 1 --2- 172.21.15.103:0/2587065806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe540102ef0 0x7fe54010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.035+0000 7fe5449c0640 1 --2- 172.21.15.103:0/2587065806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401029b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.035+0000 7fe5449c0640 1 --2- 172.21.15.103:0/2587065806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe540100e20 0x7fe540101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.035+0000 7fe5449c0640 1 -- 172.21.15.103:0/2587065806 >> 172.21.15.103:0/2587065806 conn(0x7fe5400fc7d0 msgr2=0x7fe5400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:25.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.036+0000 7fe5449c0640 1 -- 172.21.15.103:0/2587065806 shutdown_connections 2024-09-19T07:39:25.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.036+0000 7fe5449c0640 1 -- 172.21.15.103:0/2587065806 wait complete. 2024-09-19T07:39:25.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.037+0000 7fe5449c0640 1 Processor -- start 2024-09-19T07:39:25.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.037+0000 7fe5449c0640 1 -- start start 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.037+0000 7fe5449c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe540100e20 0x7fe540197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.037+0000 7fe5449c0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.037+0000 7fe5449c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe540102ef0 0x7fe54019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe5449c0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe540110ff0 con 0x7fe540102570 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe5449c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe540110e70 con 0x7fe540100e20 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe5449c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe540111170 con 0x7fe540102ef0 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45684/0 (socket says 172.21.15.103:45684) 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 -- 172.21.15.103:0/1750455723 learned_addr learned my addr 172.21.15.103:0/1750455723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53e59c640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe540100e20 0x7fe540197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe540102ef0 0x7fe54019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 -- 172.21.15.103:0/1750455723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe540102ef0 msgr2=0x7fe54019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe540102ef0 0x7fe54019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 -- 172.21.15.103:0/1750455723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe540100e20 msgr2=0x7fe540197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe540100e20 0x7fe540197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.038+0000 7fe53dd9b640 1 -- 172.21.15.103:0/1750455723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54019c420 con 0x7fe540102570 2024-09-19T07:39:25.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.040+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401979b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe534009460 tx=0x7fe5340309c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:25.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.040+0000 7fe52f7fe640 1 -- 172.21.15.103:0/1750455723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534040070 con 0x7fe540102570 2024-09-19T07:39:25.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.040+0000 7fe52f7fe640 1 -- 172.21.15.103:0/1750455723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe534005620 con 0x7fe540102570 2024-09-19T07:39:25.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.040+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54019c6b0 con 0x7fe540102570 2024-09-19T07:39:25.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.040+0000 7fe52f7fe640 1 -- 172.21.15.103:0/1750455723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534004920 con 0x7fe540102570 2024-09-19T07:39:25.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.041+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe540069f70 con 0x7fe540102570 2024-09-19T07:39:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.045+0000 7fe546d99740 1 Processor -- start 2024-09-19T07:39:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.045+0000 7fe52f7fe640 1 -- 172.21.15.103:0/1750455723 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7fe534043070 con 0x7fe540102570 2024-09-19T07:39:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.045+0000 7fe546d99740 1 -- start start 2024-09-19T07:39:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe546d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dc9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe546d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dc43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe546d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe546d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5d1f160 con 0x55c4a5dc9a90 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe546d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5cc6720 con 0x55c4a5dca3b0 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe546d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5d1b120 con 0x55c4a5dc4a50 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe53e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dc43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe53e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dc43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43568/0 (socket says 172.21.15.103:43568) 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe53e59c640 1 -- 172.21.15.103:0/2157319615 learned_addr learned my addr 172.21.15.103:0/2157319615 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe52f7fe640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5280a3550 0x7fe5280a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.046+0000 7fe53e59c640 1 -- 172.21.15.103:0/2157319615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 msgr2=0x55c4a5dc4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe52f7fe640 1 -- 172.21.15.103:0/1750455723 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe5340f0b90 con 0x7fe540102570 2024-09-19T07:39:25.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dc9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53e59c640 1 -- 172.21.15.103:0/2157319615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc9a90 msgr2=0x55c4a5dc9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dc9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53e59c640 1 -- 172.21.15.103:0/2157319615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a5a90ba0 con 0x55c4a5dca3b0 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dc9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5280a3550 0x7fe5280a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dc43e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe530009a70 tx=0x7fe530009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74710b290164f489 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe52cff9640 1 -- 172.21.15.103:0/2157319615 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530017070 con 0x55c4a5dca3b0 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe52cff9640 1 -- 172.21.15.103:0/2157319615 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe530013470 con 0x55c4a5dca3b0 2024-09-19T07:39:25.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.047+0000 7fe52cff9640 1 -- 172.21.15.103:0/2157319615 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530012630 con 0x55c4a5dca3b0 2024-09-19T07:39:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 -- 172.21.15.103:0/2157319615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dca3b0 msgr2=0x55c4a5dc43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5280a3550 0x7fe5280a5a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe538005fd0 tx=0x7fe538010040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dc43e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe530009a70 tx=0x7fe530009f30 comp rx=0 tx=0).stop 2024-09-19T07:39:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 -- 172.21.15.103:0/2157319615 shutdown_connections 2024-09-19T07:39:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dc43e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 --2- 172.21.15.103:0/2157319615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dc9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 -- 172.21.15.103:0/2157319615 >> 172.21.15.103:0/2157319615 conn(0x55c4a5db6890 msgr2=0x55c4a5db6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 -- 172.21.15.103:0/2157319615 shutdown_connections 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.048+0000 7fe546d99740 1 -- 172.21.15.103:0/2157319615 wait complete. 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 -- start start 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 Processor -- start 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dcd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dcdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5dd0dd0 con 0x55c4a5dca3b0 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5ca7900 con 0x55c4a5dc4a50 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe546d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5dd0f50 con 0x55c4a5dc9a90 2024-09-19T07:39:25.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45694/0 (socket says 172.21.15.103:45694) 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.049+0000 7fe53ed9d640 1 -- 172.21.15.103:0/2558223242 learned_addr learned my addr 172.21.15.103:0/2558223242 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dcd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe546d99740 10 client.0 _ll_register_callbacks cb 0x55c4a5ca5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dcdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53e59c640 1 -- 172.21.15.103:0/2558223242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc9a90 msgr2=0x55c4a5dcdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dcdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53e59c640 1 -- 172.21.15.103:0/2558223242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 msgr2=0x55c4a5dce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53e59c640 1 -- 172.21.15.103:0/2558223242 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a5ce7c50 con 0x55c4a5dc4a50 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dcdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:25.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe530013470 con 0x55c4a5dc4a50 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dcd7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55c4a5db7fa0 tx=0x7fe5300097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a57bbbc026fe234 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.050+0000 7fe525ffb640 10 client.25277 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe525ffb640 1 -- 172.21.15.103:0/2558223242 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530017050 con 0x55c4a5dc4a50 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe525ffb640 1 -- 172.21.15.103:0/2558223242 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe53001c450 con 0x55c4a5dc4a50 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe525ffb640 1 -- 172.21.15.103:0/2558223242 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe53001c690 con 0x55c4a5dc4a50 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe546d99740 10 client.25277 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c4a5dbd190 con 0x55c4a5dc4a50 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe525ffb640 1 -- 172.21.15.103:0/2558223242 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe53001c830 con 0x55c4a5dc4a50 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe546d99740 10 client.25277 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe546d99740 10 client.25277 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.051+0000 7fe546d99740 20 client.25277 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.052+0000 7fe546d99740 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5dbe4e0 0x55c4a5dc0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.052+0000 7fe546d99740 4 client.25277 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:25.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.052+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4a5d2c890 con 0x55c4a5dbe4e0 2024-09-19T07:39:25.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.052+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5dbe4e0 0x55c4a5dc0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.052+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5dbe4e0 0x55c4a5dc0980 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a56ff2321ded28 server_cookie=c7f6362c4ae78a51 in_seq=0 out_seq=0 2024-09-19T07:39:25.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.052+0000 7fe525ffb640 10 client.25277 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:25.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.053+0000 7fe525ffb640 1 -- 172.21.15.103:0/2558223242 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4a5d2c890 con 0x55c4a5dbe4e0 2024-09-19T07:39:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.053+0000 7fe525ffb640 10 client.25277 handle_command_reply: tid=0 2024-09-19T07:39:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.053+0000 7fe546d99740 1 client.25277 shutdown 2024-09-19T07:39:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.055+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc4a50 msgr2=0x55c4a5dcd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.055+0000 7fe546d99740 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dcd7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55c4a5db7fa0 tx=0x7fe5300097f0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.058+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 shutdown_connections 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.058+0000 7fe546d99740 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5dbe4e0 0x55c4a5dc0980 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.058+0000 7fe546d99740 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.058+0000 7fe546d99740 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dcdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.058+0000 7fe546d99740 1 --2- 172.21.15.103:0/2558223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dcd7b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.058+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 >> 172.21.15.103:0/2558223242 conn(0x55c4a5db6890 msgr2=0x55c4a5db5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.059+0000 7fe525ffb640 0 client.25277 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:25.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.059+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 shutdown_connections 2024-09-19T07:39:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.059+0000 7fe546d99740 1 -- 172.21.15.103:0/2558223242 wait complete. 2024-09-19T07:39:25.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.059+0000 7fe546d99740 20 client.25277 trim_cache size 0 max 16384 2024-09-19T07:39:25.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.093+0000 7fe546d99740 1 Processor -- start 2024-09-19T07:39:25.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.093+0000 7fe546d99740 1 -- start start 2024-09-19T07:39:25.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.093+0000 7fe546d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.093+0000 7fe546d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.093+0000 7fe546d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dcacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.094+0000 7fe546d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401ab6e0 con 0x55c4a5dca3b0 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.094+0000 7fe546d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401ad2e0 con 0x55c4a5dc9a90 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.094+0000 7fe546d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401abdb0 con 0x55c4a5dc4a50 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.094+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dcacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.094+0000 7fe53ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dcacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45698/0 (socket says 172.21.15.103:45698) 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.094+0000 7fe53ed9d640 1 -- 172.21.15.103:0/1040671710 learned_addr learned my addr 172.21.15.103:0/1040671710 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.094+0000 7fe53e59c640 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.095+0000 7fe53ed9d640 1 -- 172.21.15.103:0/1040671710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 msgr2=0x55c4a5dc4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.095+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.095+0000 7fe53ed9d640 1 -- 172.21.15.103:0/1040671710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 msgr2=0x55c4a5dca790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:39:25.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.095+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dca790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.095+0000 7fe53ed9d640 1 -- 172.21.15.103:0/1040671710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a5dc9020 con 0x55c4a5dca3b0 2024-09-19T07:39:25.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.096+0000 7fe53e59c640 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:39:25.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.097+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dcacd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe53800d8d0 tx=0x7fe538015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a89f207a2ca82b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:25.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.097+0000 7fe527fff640 1 -- 172.21.15.103:0/1040671710 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53802a070 con 0x55c4a5dca3b0 2024-09-19T07:39:25.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.097+0000 7fe527fff640 1 -- 172.21.15.103:0/1040671710 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53801f070 con 0x55c4a5dca3b0 2024-09-19T07:39:25.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.098+0000 7fe527fff640 1 -- 172.21.15.103:0/1040671710 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380047b0 con 0x55c4a5dca3b0 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 -- 172.21.15.103:0/1040671710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 msgr2=0x55c4a5dcacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dcacd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe53800d8d0 tx=0x7fe538015000 comp rx=0 tx=0).stop 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 -- 172.21.15.103:0/1040671710 shutdown_connections 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dcacd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5dca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 --2- 172.21.15.103:0/1040671710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dc4a50 0x55c4a5dc4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 -- 172.21.15.103:0/1040671710 >> 172.21.15.103:0/1040671710 conn(0x55c4a5db6890 msgr2=0x55c4a5ddf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 -- 172.21.15.103:0/1040671710 shutdown_connections 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 -- 172.21.15.103:0/1040671710 wait complete. 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 -- start start 2024-09-19T07:39:25.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.099+0000 7fe546d99740 1 Processor -- start 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe546d99740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc4a50 0x55c4a5e39510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe546d99740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5e39a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe53e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc4a50 0x55c4a5e39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe53e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc4a50 0x55c4a5e39510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45716/0 (socket says 172.21.15.103:45716) 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe546d99740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dd5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe546d99740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401acb50 con 0x55c4a5dc4a50 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe546d99740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401ac480 con 0x55c4a5dc9a90 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe546d99740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401a96c0 con 0x55c4a5dca3b0 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe546d99740 10 client.0 _ll_register_callbacks cb 0x7fe5300098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:25.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe53e59c640 1 -- 172.21.15.103:0/2577354588 learned_addr learned my addr 172.21.15.103:0/2577354588 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5e39a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.100+0000 7fe53ed9d640 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dd5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.101+0000 7fe53dd9b640 1 -- 172.21.15.103:0/2577354588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dca3b0 msgr2=0x55c4a5dd5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.101+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dd5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.101+0000 7fe53dd9b640 1 -- 172.21.15.103:0/2577354588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc4a50 msgr2=0x55c4a5e39510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.101+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc4a50 0x55c4a5e39510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.101+0000 7fe53dd9b640 1 -- 172.21.15.103:0/2577354588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a5dd5e90 con 0x55c4a5dc9a90 2024-09-19T07:39:25.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.101+0000 7fe53dd9b640 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5e39a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5340ee170 tx=0x7fe53404bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14d7281c1fe1094 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:25.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.101+0000 7fe525ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:39:25.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.102+0000 7fe525ffb640 1 -- 172.21.15.103:0/2577354588 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53404a040 con 0x55c4a5dc9a90 2024-09-19T07:39:25.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.102+0000 7fe525ffb640 1 -- 172.21.15.103:0/2577354588 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe53402fdc0 con 0x55c4a5dc9a90 2024-09-19T07:39:25.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.102+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe53801f070 con 0x55c4a5dc9a90 2024-09-19T07:39:25.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.102+0000 7fe525ffb640 1 -- 172.21.15.103:0/2577354588 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5340bd490 con 0x55c4a5dc9a90 2024-09-19T07:39:25.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.102+0000 7fe546d99740 10 client.25280 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:25.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.102+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4a5e38cb0 con 0x55c4a5dc9a90 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe525ffb640 1 -- 172.21.15.103:0/2577354588 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53801f070 con 0x55c4a5dc9a90 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe525ffb640 1 -- 172.21.15.103:0/2577354588 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe5340b9400 con 0x55c4a5dc9a90 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe546d99740 10 client.25280 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe546d99740 10 client.25280 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe546d99740 20 client.25280 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe546d99740 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5e51500 0x55c4a5e53920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe546d99740 4 client.25280 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T07:39:25.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x55c4a5d26be0 con 0x55c4a5e51500 2024-09-19T07:39:25.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.103+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5e51500 0x55c4a5e53920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:25.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.104+0000 7fe525ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:25.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.104+0000 7fe53e59c640 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5e51500 0x55c4a5e53920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b702a15b427bb4 server_cookie=99ae0395f825348a in_seq=0 out_seq=0 2024-09-19T07:39:25.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe525ffb640 1 -- 172.21.15.103:0/2577354588 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55c4a5d26be0 con 0x55c4a5e51500 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe525ffb640 10 client.25280 handle_command_reply: tid=0 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 client.25280 shutdown 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 msgr2=0x55c4a5e39a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5e39a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5340ee170 tx=0x7fe53404bf00 comp rx=0 tx=0).stop 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 shutdown_connections 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c4a5e51500 0x55c4a5e53920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4a5dca3b0 0x55c4a5dd5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c4a5dc9a90 0x55c4a5e39a50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 --2- 172.21.15.103:0/2577354588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4a5dc4a50 0x55c4a5e39510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe525ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.251+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 >> 172.21.15.103:0/2577354588 conn(0x55c4a5db6890 msgr2=0x55c4a5db5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 shutdown_connections 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe546d99740 1 -- 172.21.15.103:0/2577354588 wait complete. 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe546d99740 20 client.25280 trim_cache size 0 max 16384 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 30, 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:39:25.253 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 6, 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "e32035a": { 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 6, 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:39:25.254 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:25.255 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5280a3550 msgr2=0x7fe5280a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5280a3550 0x7fe5280a5a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe538005fd0 tx=0x7fe538010040 comp rx=0 tx=0).stop 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 msgr2=0x7fe5401979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401979b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe534009460 tx=0x7fe5340309c0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 shutdown_connections 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5280a3550 0x7fe5280a5a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe540102ef0 0x7fe54019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe540102570 0x7fe5401979b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 --2- 172.21.15.103:0/1750455723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe540100e20 0x7fe540197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:25.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 >> 172.21.15.103:0/1750455723 conn(0x7fe5400fc7d0 msgr2=0x7fe5400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:25.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 shutdown_connections 2024-09-19T07:39:25.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:25.252+0000 7fe5449c0640 1 -- 172.21.15.103:0/1750455723 wait complete. 2024-09-19T07:39:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 50.0 s, 3303.8 tps, lat 28.277 ms stddev 28.979 2024-09-19T07:39:25.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:25 smithi134 ceph-mon[35414]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 28 MiB/s wr, 2.18k op/s 2024-09-19T07:39:25.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:25 smithi174 ceph-mon[35370]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 28 MiB/s wr, 2.18k op/s 2024-09-19T07:39:25.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:25 smithi103 ceph-mon[31201]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 28 MiB/s wr, 2.18k op/s 2024-09-19T07:39:26.952 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e32035a', quiesce took 5.0 seconds. Will release after: 15.350000000000001 2024-09-19T07:39:27.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:27 smithi134 ceph-mon[35414]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-19T07:39:27.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:27 smithi174 ceph-mon[35370]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-19T07:39:27.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:27 smithi103 ceph-mon[31201]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-19T07:39:29.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:39:29.302 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:29.697 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:29 smithi103 ceph-mon[31201]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 -- 172.21.15.103:0/3986999084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba010c2f0 msgr2=0x7ffba010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 --2- 172.21.15.103:0/3986999084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba010c2f0 0x7ffba010e790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009ca0 tx=0x7ffb9802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 -- 172.21.15.103:0/3986999084 shutdown_connections 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 --2- 172.21.15.103:0/3986999084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba010c2f0 0x7ffba010e790 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 --2- 172.21.15.103:0/3986999084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba01037a0 0x7ffba010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 --2- 172.21.15.103:0/3986999084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffba0102df0 0x7ffba01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 -- 172.21.15.103:0/3986999084 >> 172.21.15.103:0/3986999084 conn(0x7ffba00fc7d0 msgr2=0x7ffba00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 -- 172.21.15.103:0/3986999084 shutdown_connections 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.779+0000 7ffba5aee640 1 -- 172.21.15.103:0/3986999084 wait complete. 2024-09-19T07:39:29.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 Processor -- start 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 -- start start 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0102df0 0x7ffba01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba01037a0 0x7ffba0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffba010c2f0 0x7ffba019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0112110 con 0x7ffba01037a0 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0111f90 con 0x7ffba0102df0 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffba5aee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0112290 con 0x7ffba010c2f0 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffb96ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba01037a0 0x7ffba0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffb96ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba01037a0 0x7ffba0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37538/0 (socket says 172.21.15.103:37538) 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffb96ffd640 1 -- 172.21.15.103:0/312381145 learned_addr learned my addr 172.21.15.103:0/312381145 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffba010c2f0 0x7ffba019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.780+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0102df0 0x7ffba01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb96ffd640 1 -- 172.21.15.103:0/312381145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffba010c2f0 msgr2=0x7ffba019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffba010c2f0 0x7ffba019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb96ffd640 1 -- 172.21.15.103:0/312381145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0102df0 msgr2=0x7ffba01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0102df0 0x7ffba01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb96ffd640 1 -- 172.21.15.103:0/312381145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba019d560 con 0x7ffba01037a0 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0102df0 0x7ffba01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba01037a0 0x7ffba0198af0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c970 tx=0x7ffb8800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb9d7fa640 1 -- 172.21.15.103:0/312381145 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88007bb0 con 0x7ffba01037a0 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb9d7fa640 1 -- 172.21.15.103:0/312381145 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb88007d50 con 0x7ffba01037a0 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba019d850 con 0x7ffba01037a0 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffb9d7fa640 1 -- 172.21.15.103:0/312381145 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800f450 con 0x7ffba01037a0 2024-09-19T07:39:29.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.781+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba01a5130 con 0x7ffba01037a0 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.783+0000 7ffb9d7fa640 1 -- 172.21.15.103:0/312381145 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138589+0+0 (secure 0 0 0) 0x7ffb88004f60 con 0x7ffba01037a0 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 Processor -- start 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 -- start start 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9d7fa640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb8c0a34d0 0x7ffb8c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a3810b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810bc80 0x555a3811a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3811aa90 0x555a3811ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555a380712e0 con 0x555a3810b360 2024-09-19T07:39:29.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555a38018ce0 con 0x555a3810bc80 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555a3806d2a0 con 0x555a3811aa90 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb8c0a34d0 0x7ffb8c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9d7fa640 1 -- 172.21.15.103:0/312381145 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ffb880cfd20 con 0x7ffba01037a0 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a3810b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a3810b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37546/0 (socket says 172.21.15.103:37546) 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/151376938 learned_addr learned my addr 172.21.15.103:0/151376938 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810bc80 0x555a3811a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3811aa90 0x555a3811ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.784+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/151376938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3811aa90 msgr2=0x555a3811ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3811aa90 0x555a3811ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/151376938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810bc80 msgr2=0x555a3811a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810bc80 0x555a3811a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/151376938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a37ffd440 con 0x555a3810b360 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3811aa90 0x555a3811ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb8c0a34d0 0x7ffb8c0a5990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb98004af0 tx=0x7ffb98009930 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a3810b740 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000bd40 tx=0x7ffb9000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583208b618ed07ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb95ffb640 1 -- 172.21.15.103:0/151376938 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90015070 con 0x555a3810b360 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb95ffb640 1 -- 172.21.15.103:0/151376938 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9000be30 con 0x555a3810b360 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.785+0000 7ffb95ffb640 1 -- 172.21.15.103:0/151376938 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011a10 con 0x555a3810b360 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- 172.21.15.103:0/151376938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 msgr2=0x555a3810b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a3810b740 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000bd40 tx=0x7ffb9000f730 comp rx=0 tx=0).stop 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- 172.21.15.103:0/151376938 shutdown_connections 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3811aa90 0x555a3811ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810bc80 0x555a3811a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/151376938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a3810b740 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- 172.21.15.103:0/151376938 >> 172.21.15.103:0/151376938 conn(0x555a38107a30 msgr2=0x555a38107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- 172.21.15.103:0/151376938 shutdown_connections 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- 172.21.15.103:0/151376938 wait complete. 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- start start 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 Processor -- start 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810b360 0x555a3811a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3811aa90 0x555a38117c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555a3811d660 con 0x555a3811aa90 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555a3802c050 con 0x555a3810b360 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.786+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555a3811d7e0 con 0x555a3810bc80 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffba7ec7740 10 client.0 _ll_register_callbacks cb 0x555a3806c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3811aa90 0x555a38117c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3811aa90 0x555a38117c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37552/0 (socket says 172.21.15.103:37552) 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 -- 172.21.15.103:0/450842016 learned_addr learned my addr 172.21.15.103:0/450842016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810b360 0x555a3811a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 -- 172.21.15.103:0/450842016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 msgr2=0x555a381176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381176b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 -- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810b360 msgr2=0x555a3811a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810b360 0x555a3811a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 -- 172.21.15.103:0/450842016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a37de23c0 con 0x555a3811aa90 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810b360 0x555a3811a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3811aa90 0x555a38117c10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009a60 tx=0x7ffb9802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea33c9e8b34e1913 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb7effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:29.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb7effd640 1 -- 172.21.15.103:0/450842016 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9804f070 con 0x555a3811aa90 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffb7effd640 1 -- 172.21.15.103:0/450842016 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ffb98004590 con 0x555a3811aa90 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.787+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9000be30 con 0x555a3811aa90 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffb7effd640 1 -- 172.21.15.103:0/450842016 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb9805b050 con 0x555a3811aa90 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 10 client.15741 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a38128fe0 con 0x555a3811aa90 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffb7effd640 1 -- 172.21.15.103:0/450842016 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9000be30 con 0x555a3811aa90 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffb7effd640 1 -- 172.21.15.103:0/450842016 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ffb98030650 con 0x555a3811aa90 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 10 client.15741 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 20 client.15741 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a3812a470 0x555a3812c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 4 client.15741 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a3807e850 con 0x555a3812a470 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.788+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a3812a470 0x555a3812c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.789+0000 7ffb7effd640 10 client.15741 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.789+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a3812a470 0x555a3812c8f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de71191cac8e8e0b server_cookie=e3010f94a94b3969 in_seq=0 out_seq=0 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.789+0000 7ffb7effd640 1 -- 172.21.15.103:0/450842016 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a3807e850 con 0x555a3812a470 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.789+0000 7ffb7effd640 10 client.15741 handle_command_reply: tid=0 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.789+0000 7ffba7ec7740 1 client.15741 shutdown 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.789+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3811aa90 msgr2=0x555a38117c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.789+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3811aa90 0x555a38117c10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009a60 tx=0x7ffb9802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 shutdown_connections 2024-09-19T07:39:29.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a3812a470 0x555a3812c8f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3811aa90 0x555a38117c10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffb7effd640 0 client.15741 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/450842016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3810b360 0x555a3811a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 >> 172.21.15.103:0/450842016 conn(0x555a38107a30 msgr2=0x555a3810d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 shutdown_connections 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 1 -- 172.21.15.103:0/450842016 wait complete. 2024-09-19T07:39:29.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.790+0000 7ffba7ec7740 20 client.15741 trim_cache size 0 max 16384 2024-09-19T07:39:29.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 Processor -- start 2024-09-19T07:39:29.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 -- start start 2024-09-19T07:39:29.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38126900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a38132700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01ab6a0 con 0x555a3810b360 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01ad2a0 con 0x555a3811aa90 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01abd70 con 0x555a3810bc80 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38126900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38126900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37558/0 (socket says 172.21.15.103:37558) 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/98742640 learned_addr learned my addr 172.21.15.103:0/98742640 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a38132700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/98742640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 msgr2=0x555a381321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/98742640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 msgr2=0x555a38132700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a38132700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/98742640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3811d7d0 con 0x555a3810b360 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38126900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000bd40 tx=0x7ffb9000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97433d8ca1f116d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.807+0000 7ffb957fa640 1 -- 172.21.15.103:0/98742640 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90015040 con 0x555a3810b360 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffb957fa640 1 -- 172.21.15.103:0/98742640 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb90011780 con 0x555a3810b360 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffb957fa640 1 -- 172.21.15.103:0/98742640 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb900048e0 con 0x555a3810b360 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 -- 172.21.15.103:0/98742640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 msgr2=0x555a38126900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38126900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000bd40 tx=0x7ffb9000b660 comp rx=0 tx=0).stop 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 -- 172.21.15.103:0/98742640 shutdown_connections 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a38132700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a381321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/98742640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38126900 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 -- 172.21.15.103:0/98742640 >> 172.21.15.103:0/98742640 conn(0x555a38107a30 msgr2=0x555a38112d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 -- 172.21.15.103:0/98742640 shutdown_connections 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.808+0000 7ffba7ec7740 1 -- 172.21.15.103:0/98742640 wait complete. 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 -- start start 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 Processor -- start 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38122a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a38122fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a3811fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01acb10 con 0x555a3810b360 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01ac440 con 0x555a3811aa90 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01a96a0 con 0x555a3810bc80 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffba7ec7740 10 client.0 _ll_register_callbacks cb 0x7ffb98009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38122a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38122a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37562/0 (socket says 172.21.15.103:37562) 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/3769982608 learned_addr learned my addr 172.21.15.103:0/3769982608 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a38122fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9ffff640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a3811fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/3769982608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 msgr2=0x555a38122fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a38122fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 msgr2=0x555a3811fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a3811fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb9f7fe640 1 -- 172.21.15.103:0/3769982608 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a38123670 con 0x555a3810b360 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.809+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a38122fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffb9f7fe640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38122a70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffb9001c010 tx=0x7ffb9000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60fc58212a45bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffb7effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffb7effd640 1 -- 172.21.15.103:0/3769982608 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011bf0 con 0x555a3810b360 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffb7effd640 1 -- 172.21.15.103:0/3769982608 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ffb90011d90 con 0x555a3810b360 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb90011780 con 0x555a3810b360 2024-09-19T07:39:29.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffb7effd640 1 -- 172.21.15.103:0/3769982608 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb90005040 con 0x555a3810b360 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 10 client.15747 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a38195770 con 0x555a3810b360 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffb7effd640 1 -- 172.21.15.103:0/3769982608 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb90011780 con 0x555a3810b360 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffb7effd640 1 -- 172.21.15.103:0/3769982608 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ffb9001a460 con 0x555a3810b360 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 10 client.15747 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 20 client.15747 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a38195d20 0x555a38198140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:29.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 4 client.15747 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:39:29.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.810+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a38078d60 con 0x555a38195d20 2024-09-19T07:39:29.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a38195d20 0x555a38198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:29.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffb7effd640 10 client.15747 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:29.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffb96ffd640 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a38195d20 0x555a38198140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d539cb84b5dd4ea7 server_cookie=a61bbe53b66c7636 in_seq=0 out_seq=0 2024-09-19T07:39:29.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffb7effd640 1 -- 172.21.15.103:0/3769982608 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a38078d60 con 0x555a38195d20 2024-09-19T07:39:29.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffb7effd640 10 client.15747 handle_command_reply: tid=0 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffba7ec7740 1 client.15747 shutdown 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 msgr2=0x555a38122a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38122a70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ffb9001c010 tx=0x7ffb9000beb0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 shutdown_connections 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.811+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555a38195d20 0x555a38198140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a3811aa90 0x555a3811fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555a3810bc80 0x555a38122fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba7ec7740 1 --2- 172.21.15.103:0/3769982608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555a3810b360 0x555a38122a70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 >> 172.21.15.103:0/3769982608 conn(0x555a38107a30 msgr2=0x555a38118f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:29.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffb7effd640 0 client.15747 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:29.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 shutdown_connections 2024-09-19T07:39:29.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba7ec7740 1 -- 172.21.15.103:0/3769982608 wait complete. 2024-09-19T07:39:29.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba7ec7740 20 client.15747 trim_cache size 0 max 16384 2024-09-19T07:39:29.824 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:39:29.824 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:39:29.824 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:39:29.824 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:39:29.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb8c0a34d0 msgr2=0x7ffb8c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb8c0a34d0 0x7ffb8c0a5990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb98004af0 tx=0x7ffb98009930 comp rx=0 tx=0).stop 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba01037a0 msgr2=0x7ffba0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba01037a0 0x7ffba0198af0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c970 tx=0x7ffb8800ce30 comp rx=0 tx=0).stop 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 shutdown_connections 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffb8c0a34d0 0x7ffb8c0a5990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffba010c2f0 0x7ffba019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffba01037a0 0x7ffba0198af0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.812+0000 7ffba5aee640 1 --2- 172.21.15.103:0/312381145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0102df0 0x7ffba01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.813+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 >> 172.21.15.103:0/312381145 conn(0x7ffba00fc7d0 msgr2=0x7ffba00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.813+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 shutdown_connections 2024-09-19T07:39:29.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:29.813+0000 7ffba5aee640 1 -- 172.21.15.103:0/312381145 wait complete. 2024-09-19T07:39:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:29 smithi134 ceph-mon[35414]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-19T07:39:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:29 smithi174 ceph-mon[35370]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-09-19T07:39:30.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 55.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:39:30.490 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:39:30.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:39:30.743 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:31.707 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:31 smithi103 ceph-mon[31201]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 900 op/s 2024-09-19T07:39:31.707 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:31 smithi103 ceph-mon[31201]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-19T07:39:31.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 -- 172.21.15.103:0/2302135991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834106ba0 msgr2=0x7f5834106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 --2- 172.21.15.103:0/2302135991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834106ba0 0x7f5834106f80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5824009900 tx=0x7f582402a410 comp rx=0 tx=0).stop 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 -- 172.21.15.103:0/2302135991 shutdown_connections 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 --2- 172.21.15.103:0/2302135991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5834100a70 0x7f583410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 --2- 172.21.15.103:0/2302135991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58341000d0 0x7f5834100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 --2- 172.21.15.103:0/2302135991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834106ba0 0x7f5834106f80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 -- 172.21.15.103:0/2302135991 >> 172.21.15.103:0/2302135991 conn(0x7f5834077210 msgr2=0x7f58340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 -- 172.21.15.103:0/2302135991 shutdown_connections 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.707+0000 7f583a3be640 1 -- 172.21.15.103:0/2302135991 wait complete. 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 Processor -- start 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 -- start start 2024-09-19T07:39:31.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58341000d0 0x7f58341984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834100a70 0x7f5834198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834106ba0 0x7f583419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112090 con 0x7f5834100a70 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5834111f10 con 0x7f58341000d0 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583a3be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5834112210 con 0x7f5834106ba0 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f58337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834100a70 0x7f5834198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f58337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834100a70 0x7f5834198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37582/0 (socket says 172.21.15.103:37582) 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f58337fe640 1 -- 172.21.15.103:0/2039081139 learned_addr learned my addr 172.21.15.103:0/2039081139 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f5833fff640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58341000d0 0x7f58341984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.708+0000 7f583895b640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834106ba0 0x7f583419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58337fe640 1 -- 172.21.15.103:0/2039081139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834106ba0 msgr2=0x7f583419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:31.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58337fe640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834106ba0 0x7f583419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58337fe640 1 -- 172.21.15.103:0/2039081139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58341000d0 msgr2=0x7f58341984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58337fe640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58341000d0 0x7f58341984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58337fe640 1 -- 172.21.15.103:0/2039081139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583419d480 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58337fe640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834100a70 0x7f5834198a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f581c00d770 tx=0x7f581c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58317fa640 1 -- 172.21.15.103:0/2039081139 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c017070 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58317fa640 1 -- 172.21.15.103:0/2039081139 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f581c005500 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583419d770 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f58317fa640 1 -- 172.21.15.103:0/2039081139 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c004e00 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.709+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a5050 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.710+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5834100530 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.711+0000 7f58317fa640 1 -- 172.21.15.103:0/2039081139 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f581c004fa0 con 0x7f5834100a70 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.712+0000 7f58317fa640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58200a33b0 0x7f58200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.712+0000 7f58317fa640 1 -- 172.21.15.103:0/2039081139 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f581c0cfb10 con 0x7f5834100a70 2024-09-19T07:39:31.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.712+0000 7f5833fff640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58200a33b0 0x7f58200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:31.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.712+0000 7f5833fff640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58200a33b0 0x7f58200a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5824009230 tx=0x7f58240058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:31.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.715+0000 7f58317fa640 1 -- 172.21.15.103:0/2039081139 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f581c09beb0 con 0x7f5834100a70 2024-09-19T07:39:31.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:31 smithi134 ceph-mon[35414]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 900 op/s 2024-09-19T07:39:31.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:31 smithi134 ceph-mon[35414]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-19T07:39:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.914+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f583410a090 con 0x7f5834100a70 2024-09-19T07:39:31.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.915+0000 7f58317fa640 1 -- 172.21.15.103:0/2039081139 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11998 (secure 0 0 0) 0x7f581c0a01b0 con 0x7f5834100a70 2024-09-19T07:39:31.915 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:39:31.915 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":30,"btime":"2024-09-19T07:39:09:495445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:39:08.515410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:39:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 30 2024-09-19T07:39:31.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:31 smithi174 ceph-mon[35370]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 900 op/s 2024-09-19T07:39:31.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:31 smithi174 ceph-mon[35370]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-19T07:39:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58200a33b0 msgr2=0x7f58200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58200a33b0 0x7f58200a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5824009230 tx=0x7f58240058d0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834100a70 msgr2=0x7f5834198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834100a70 0x7f5834198a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f581c00d770 tx=0x7f581c00dc30 comp rx=0 tx=0).stop 2024-09-19T07:39:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 shutdown_connections 2024-09-19T07:39:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58200a33b0 0x7f58200a5870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834106ba0 0x7f583419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5834100a70 0x7f5834198a10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 --2- 172.21.15.103:0/2039081139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58341000d0 0x7f58341984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 >> 172.21.15.103:0/2039081139 conn(0x7f5834077210 msgr2=0x7f5834104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 shutdown_connections 2024-09-19T07:39:31.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:31.917+0000 7f583a3be640 1 -- 172.21.15.103:0/2039081139 wait complete. 2024-09-19T07:39:32.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:39:32.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:32 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2039081139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:32.848 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:32.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:32 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2039081139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:32.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:32 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2039081139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 -- 172.21.15.103:0/1293054220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 msgr2=0x7f79281021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 --2- 172.21.15.103:0/1293054220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79281021b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7924009900 tx=0x7f792402a410 comp rx=0 tx=0).stop 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 -- 172.21.15.103:0/1293054220 shutdown_connections 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 --2- 172.21.15.103:0/1293054220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7928106f00 0x7f792810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 --2- 172.21.15.103:0/1293054220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7928102780 0x7f79281067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 --2- 172.21.15.103:0/1293054220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79281021b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 -- 172.21.15.103:0/1293054220 >> 172.21.15.103:0/1293054220 conn(0x7f79280fd890 msgr2=0x7f79280ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 -- 172.21.15.103:0/1293054220 shutdown_connections 2024-09-19T07:39:33.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 -- 172.21.15.103:0/1293054220 wait complete. 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 Processor -- start 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.317+0000 7f7930730640 1 -- start start 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7928102780 0x7f7928074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7928106f00 0x7f7928074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7928111f20 con 0x7f7928101dd0 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7928111da0 con 0x7f7928106f00 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79281120a0 con 0x7f7928102780 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37598/0 (socket says 172.21.15.103:37598) 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 -- 172.21.15.103:0/444719886 learned_addr learned my addr 172.21.15.103:0/444719886 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:33.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792eccd640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7928106f00 0x7f7928074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792dccb640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7928102780 0x7f7928074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 -- 172.21.15.103:0/444719886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7928102780 msgr2=0x7f7928074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7928102780 0x7f7928074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 -- 172.21.15.103:0/444719886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7928106f00 msgr2=0x7f7928074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7928106f00 0x7f7928074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 -- 172.21.15.103:0/444719886 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79281a6970 con 0x7f7928101dd0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f792e4cc640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79280744f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7924005ab0 tx=0x7f79240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f791f7fe640 1 -- 172.21.15.103:0/444719886 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792403f030 con 0x7f7928101dd0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f791f7fe640 1 -- 172.21.15.103:0/444719886 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7924003ce0 con 0x7f7928101dd0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f791f7fe640 1 -- 172.21.15.103:0/444719886 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924035070 con 0x7f7928101dd0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79281a6c00 con 0x7f7928101dd0 2024-09-19T07:39:33.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.318+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79281a70e0 con 0x7f7928101dd0 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.321+0000 7f791f7fe640 1 -- 172.21.15.103:0/444719886 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7924042030 con 0x7f7928101dd0 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.321+0000 7f7932b09740 1 Processor -- start 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.321+0000 7f7932b09740 1 -- start start 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f7932b09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f7932b09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f7932b09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c300760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f7932b09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf6c2592f0 con 0x55cf6c2f4b40 2024-09-19T07:39:33.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f7932b09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf6c200cf0 con 0x55cf6c2f35d0 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f7932b09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cf6c2552b0 con 0x55cf6c2fe370 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f791f7fe640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f79140a33b0 0x7f79140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37614/0 (socket says 172.21.15.103:37614) 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 -- 172.21.15.103:0/4040037393 learned_addr learned my addr 172.21.15.103:0/4040037393 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f79140a33b0 0x7f79140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792e4cc640 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792dccb640 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c300760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f791f7fe640 1 -- 172.21.15.103:0/444719886 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f79240f06d0 con 0x7f7928101dd0 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 -- 172.21.15.103:0/4040037393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 msgr2=0x55cf6c300760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c300760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 -- 172.21.15.103:0/4040037393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 msgr2=0x55cf6c2f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 -- 172.21.15.103:0/4040037393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6c1e53a0 con 0x55cf6c2f4b40 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792dccb640 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c300760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.322+0000 7f792eccd640 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2f3090 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7920013e70 tx=0x7f79200107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ec7d54687ecaf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f792eccd640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f79140a33b0 0x7f79140a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7920010ca0 tx=0x7f7920015000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f791cff9640 1 -- 172.21.15.103:0/4040037393 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792000ae10 con 0x55cf6c2f4b40 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f791cff9640 1 -- 172.21.15.103:0/4040037393 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920003e50 con 0x55cf6c2f4b40 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f791cff9640 1 -- 172.21.15.103:0/4040037393 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920004770 con 0x55cf6c2f4b40 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f7932b09740 1 -- 172.21.15.103:0/4040037393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 msgr2=0x55cf6c2f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f7932b09740 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2f3090 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7920013e70 tx=0x7f79200107a0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f7932b09740 1 -- 172.21.15.103:0/4040037393 shutdown_connections 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f7932b09740 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c300760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f7932b09740 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f7932b09740 1 --2- 172.21.15.103:0/4040037393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2f3090 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.323+0000 7f7932b09740 1 -- 172.21.15.103:0/4040037393 >> 172.21.15.103:0/4040037393 conn(0x55cf6c2bc310 msgr2=0x55cf6c2efc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 -- 172.21.15.103:0/4040037393 shutdown_connections 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 -- 172.21.15.103:0/4040037393 wait complete. 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 -- start start 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 Processor -- start 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c2fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf6c301010 con 0x55cf6c2f4b40 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf6c214060 con 0x55cf6c2f35d0 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cf6c301190 con 0x55cf6c2fe370 2024-09-19T07:39:33.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f7932b09740 10 client.0 _ll_register_callbacks cb 0x55cf6bf678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f792dccb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f792dccb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37630/0 (socket says 172.21.15.103:37630) 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f792dccb640 1 -- 172.21.15.103:0/3711946577 learned_addr learned my addr 172.21.15.103:0/3711946577 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.324+0000 7f792e4cc640 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f792eccd640 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c2fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f792dccb640 1 -- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 msgr2=0x55cf6c2fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f792dccb640 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c2fb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f792dccb640 1 -- 172.21.15.103:0/3711946577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 msgr2=0x55cf6c2fdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f792dccb640 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2fdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f792dccb640 1 -- 172.21.15.103:0/3711946577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6bfca3c0 con 0x55cf6c2f4b40 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f792dccb640 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fb020 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f791800e6b0 tx=0x7f791800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4122817f7ea9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:33.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7911ffb640 1 -- 172.21.15.103:0/3711946577 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918009a80 con 0x55cf6c2f4b40 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7911ffb640 1 -- 172.21.15.103:0/3711946577 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7918009c20 con 0x55cf6c2f4b40 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7920003e50 con 0x55cf6c2f4b40 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7911ffb640 1 -- 172.21.15.103:0/3711946577 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7918009e20 con 0x55cf6c2f4b40 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7932b09740 10 client.15765 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf6c30ca80 con 0x55cf6c2f4b40 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7911ffb640 1 -- 172.21.15.103:0/3711946577 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920003e50 con 0x55cf6c2f4b40 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.325+0000 7f7911ffb640 1 -- 172.21.15.103:0/3711946577 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7918004cc0 con 0x55cf6c2f4b40 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f7932b09740 10 client.15765 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f7932b09740 10 client.15765 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f7932b09740 20 client.15765 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f7932b09740 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c30dfe0 0x55cf6c310440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f7932b09740 4 client.15765 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf6c266860 con 0x55cf6c30dfe0 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f792e4cc640 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c30dfe0 0x55cf6c310440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f792e4cc640 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c30dfe0 0x55cf6c310440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d32de5c187e0c41 server_cookie=801c265b6fc6bb4 in_seq=0 out_seq=0 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.326+0000 7f7911ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7911ffb640 1 -- 172.21.15.103:0/3711946577 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf6c266860 con 0x55cf6c30dfe0 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7911ffb640 10 client.15765 handle_command_reply: tid=0 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 client.15765 shutdown 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 msgr2=0x55cf6c2fb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fb020 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f791800e6b0 tx=0x7f791800eb70 comp rx=0 tx=0).stop 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 shutdown_connections 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c30dfe0 0x55cf6c310440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2fe370 0x55cf6c2fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fb020 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 --2- 172.21.15.103:0/3711946577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f35d0 0x55cf6c2fdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 >> 172.21.15.103:0/3711946577 conn(0x55cf6c2bc310 msgr2=0x55cf6c2f5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7911ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:39:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 shutdown_connections 2024-09-19T07:39:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 1 -- 172.21.15.103:0/3711946577 wait complete. 2024-09-19T07:39:33.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.327+0000 7f7932b09740 20 client.15765 trim_cache size 0 max 16384 2024-09-19T07:39:33.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 Processor -- start 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 -- start start 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c30a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2fe370 0x55cf6c2f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79281afa30 con 0x55cf6c2fe370 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79281b1630 con 0x55cf6c2f4b40 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f7932b09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79281b0100 con 0x55cf6c2f35d0 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f792eccd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2fe370 0x55cf6c2f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f792eccd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2fe370 0x55cf6c2f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37644/0 (socket says 172.21.15.103:37644) 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f792eccd640 1 -- 172.21.15.103:0/395810025 learned_addr learned my addr 172.21.15.103:0/395810025 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f792e4cc640 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c30a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.340+0000 7f792dccb640 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792eccd640 1 -- 172.21.15.103:0/395810025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 msgr2=0x55cf6c30a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792eccd640 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c30a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792eccd640 1 -- 172.21.15.103:0/395810025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f4b40 msgr2=0x55cf6c2fa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792eccd640 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792eccd640 1 -- 172.21.15.103:0/395810025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6c2ef510 con 0x55cf6c2fe370 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792e4cc640 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c30a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792dccb640 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f792eccd640 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2fe370 0x55cf6c2f98f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f792000b640 tx=0x7f7920013cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19283ef0de0d78b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f7913fff640 1 -- 172.21.15.103:0/395810025 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792000ac10 con 0x55cf6c2fe370 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f7913fff640 1 -- 172.21.15.103:0/395810025 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920003e50 con 0x55cf6c2fe370 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f7913fff640 1 -- 172.21.15.103:0/395810025 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200043e0 con 0x55cf6c2fe370 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f7932b09740 1 -- 172.21.15.103:0/395810025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2fe370 msgr2=0x55cf6c2f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.341+0000 7f7932b09740 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2fe370 0x55cf6c2f98f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f792000b640 tx=0x7f7920013cf0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- 172.21.15.103:0/395810025 shutdown_connections 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2fe370 0x55cf6c2f98f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2f4b40 0x55cf6c2fa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 --2- 172.21.15.103:0/395810025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c30a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- 172.21.15.103:0/395810025 >> 172.21.15.103:0/395810025 conn(0x55cf6c2bc310 msgr2=0x55cf6c2f62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- 172.21.15.103:0/395810025 shutdown_connections 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- 172.21.15.103:0/395810025 wait complete. 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- start start 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 Processor -- start 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c3019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c301ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2fe370 0x55cf6c376cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79281b0ea0 con 0x55cf6c2f4b40 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79281b07d0 con 0x55cf6c2fe370 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79281ada10 con 0x55cf6c2f35d0 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f7932b09740 10 client.0 _ll_register_callbacks cb 0x7f791800ce30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f792dccb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c301ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f792dccb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c301ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37658/0 (socket says 172.21.15.103:37658) 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f792dccb640 1 -- 172.21.15.103:0/1433877539 learned_addr learned my addr 172.21.15.103:0/1433877539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.342+0000 7f792e4cc640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c3019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792eccd640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2fe370 0x55cf6c376cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792dccb640 1 -- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 msgr2=0x55cf6c3019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792dccb640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c3019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792dccb640 1 -- 172.21.15.103:0/1433877539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2fe370 msgr2=0x55cf6c376cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792dccb640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2fe370 0x55cf6c376cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792dccb640 1 -- 172.21.15.103:0/1433877539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6c377240 con 0x55cf6c2f4b40 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792eccd640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2fe370 0x55cf6c376cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f792dccb640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c301ee0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f79180206c0 tx=0x7f7918020a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79cd78f75a81b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7911ffb640 1 -- 172.21.15.103:0/1433877539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918009880 con 0x55cf6c2f4b40 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7911ffb640 1 -- 172.21.15.103:0/1433877539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7918009a20 con 0x55cf6c2f4b40 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7920003e50 con 0x55cf6c2f4b40 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7911ffb640 1 -- 172.21.15.103:0/1433877539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7918009c20 con 0x55cf6c2f4b40 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7932b09740 10 client.15771 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf6c3784b0 con 0x55cf6c2f4b40 2024-09-19T07:39:33.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.343+0000 7f7911ffb640 1 -- 172.21.15.103:0/1433877539 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920003e50 con 0x55cf6c2f4b40 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7911ffb640 1 -- 172.21.15.103:0/1433877539 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7918004be0 con 0x55cf6c2f4b40 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 10 client.15771 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 10 client.15771 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 20 client.15771 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c378c80 0x55cf6c37b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 4 client.15771 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf6c260d70 con 0x55cf6c378c80 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f792e4cc640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c378c80 0x55cf6c37b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7911ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f792e4cc640 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c378c80 0x55cf6c37b0a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd2c76e98fb73ef1 server_cookie=75111f931a1d19d1 in_seq=0 out_seq=0 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7911ffb640 1 -- 172.21.15.103:0/1433877539 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf6c260d70 con 0x55cf6c378c80 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7911ffb640 10 client.15771 handle_command_reply: tid=0 2024-09-19T07:39:33.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 1 client.15771 shutdown 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 msgr2=0x55cf6c301ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.344+0000 7f7932b09740 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c301ee0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f79180206c0 tx=0x7f7918020a50 comp rx=0 tx=0).stop 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 shutdown_connections 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55cf6c378c80 0x55cf6c37b0a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf6c2fe370 0x55cf6c376cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6c2f4b40 0x55cf6c301ee0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7911ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 --2- 172.21.15.103:0/1433877539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf6c2f35d0 0x55cf6c3019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 >> 172.21.15.103:0/1433877539 conn(0x55cf6c2bc310 msgr2=0x55cf6c2f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 shutdown_connections 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 1 -- 172.21.15.103:0/1433877539 wait complete. 2024-09-19T07:39:33.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7932b09740 20 client.15771 trim_cache size 0 max 16384 2024-09-19T07:39:33.352 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:39:33.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f79140a33b0 msgr2=0x7f79140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f79140a33b0 0x7f79140a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7920010ca0 tx=0x7f7920015000 comp rx=0 tx=0).stop 2024-09-19T07:39:33.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 msgr2=0x7f79280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:33.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79280744f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7924005ab0 tx=0x7f79240042f0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 shutdown_connections 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f79140a33b0 0x7f79140a5870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7928106f00 0x7f7928074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7928102780 0x7f7928074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 --2- 172.21.15.103:0/444719886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7928101dd0 0x7f79280744f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 >> 172.21.15.103:0/444719886 conn(0x7f79280fd890 msgr2=0x7f79280fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 shutdown_connections 2024-09-19T07:39:33.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:33.345+0000 7f7930730640 1 -- 172.21.15.103:0/444719886 wait complete. 2024-09-19T07:39:33.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:33 smithi134 ceph-mon[35414]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 381 op/s 2024-09-19T07:39:33.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:33 smithi174 ceph-mon[35370]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 381 op/s 2024-09-19T07:39:33.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:33 smithi103 ceph-mon[31201]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 381 op/s 2024-09-19T07:39:34.060 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:39:34.308 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 -- 172.21.15.103:0/182920376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc106f10 msgr2=0x7f10fc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 --2- 172.21.15.103:0/182920376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc106f10 0x7f10fc10ea90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009ca0 tx=0x7f10f402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 -- 172.21.15.103:0/182920376 shutdown_connections 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 --2- 172.21.15.103:0/182920376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc106f10 0x7f10fc10ea90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 --2- 172.21.15.103:0/182920376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10fc102790 0x7f10fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 --2- 172.21.15.103:0/182920376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc101de0 0x7f10fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 -- 172.21.15.103:0/182920376 >> 172.21.15.103:0/182920376 conn(0x7f10fc0fd8c0 msgr2=0x7f10fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 -- 172.21.15.103:0/182920376 shutdown_connections 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 -- 172.21.15.103:0/182920376 wait complete. 2024-09-19T07:39:34.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 Processor -- start 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.869+0000 7f1103098640 1 -- start start 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1103098640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc101de0 0x7f10fc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1103098640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10fc102790 0x7f10fc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1103098640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc106f10 0x7f10fc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1103098640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc111f70 con 0x7f10fc101de0 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1103098640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc111df0 con 0x7f10fc106f10 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1103098640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1120f0 con 0x7f10fc102790 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc101de0 0x7f10fc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc101de0 0x7f10fc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37686/0 (socket says 172.21.15.103:37686) 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 -- 172.21.15.103:0/284124940 learned_addr learned my addr 172.21.15.103:0/284124940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1101635640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc106f10 0x7f10fc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f10fbfff640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10fc102790 0x7f10fc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 -- 172.21.15.103:0/284124940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10fc102790 msgr2=0x7f10fc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10fc102790 0x7f10fc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 -- 172.21.15.103:0/284124940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc106f10 msgr2=0x7f10fc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc106f10 0x7f10fc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 -- 172.21.15.103:0/284124940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc19d3a0 con 0x7f10fc101de0 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f10fbfff640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10fc102790 0x7f10fc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1100e34640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc101de0 0x7f10fc1983f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f10f000d950 tx=0x7f10f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f10f9ffb640 1 -- 172.21.15.103:0/284124940 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0017070 con 0x7f10fc101de0 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f10f9ffb640 1 -- 172.21.15.103:0/284124940 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f0009c30 con 0x7f10fc101de0 2024-09-19T07:39:34.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc19d690 con 0x7f10fc101de0 2024-09-19T07:39:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.870+0000 7f10f9ffb640 1 -- 172.21.15.103:0/284124940 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f000f6b0 con 0x7f10fc101de0 2024-09-19T07:39:34.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.871+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc1a4f70 con 0x7f10fc101de0 2024-09-19T07:39:34.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.873+0000 7f10f9ffb640 1 -- 172.21.15.103:0/284124940 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f10f00109d0 con 0x7f10fc101de0 2024-09-19T07:39:34.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.873+0000 7f1105471740 1 Processor -- start 2024-09-19T07:39:34.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1105471740 1 -- start start 2024-09-19T07:39:34.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f10f9ffb640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10ec0a33b0 0x7f10ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1101635640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10ec0a33b0 0x7f10ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1105471740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cd7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1105471740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f10f9ffb640 1 -- 172.21.15.103:0/284124940 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f10f00cfa00 con 0x7f10fc101de0 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1105471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259ce4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1105471740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565259c3d2f0 con 0x565259cd8b40 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1105471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565259be4cf0 con 0x565259ce2370 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1105471740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565259c392b0 con 0x565259cd75d0 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cd7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cd7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37698/0 (socket says 172.21.15.103:37698) 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 -- 172.21.15.103:0/539995808 learned_addr learned my addr 172.21.15.103:0/539995808 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f10fbfff640 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1101635640 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259ce4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 -- 172.21.15.103:0/539995808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 msgr2=0x565259cd7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 -- 172.21.15.103:0/539995808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 msgr2=0x565259ce4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259ce4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1101635640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10ec0a33b0 0x7f10ec0a5870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10f4004af0 tx=0x7f10f4004a80 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.874+0000 7f1100e34640 1 -- 172.21.15.103:0/539995808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565259bc93a0 con 0x565259cd8b40 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.875+0000 7f1100e34640 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cd7090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f10f00a43d0 tx=0x7f10f0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b306c3d7b62ee944 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.875+0000 7f10e6ffd640 1 -- 172.21.15.103:0/539995808 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0020040 con 0x565259cd8b40 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.875+0000 7f10e6ffd640 1 -- 172.21.15.103:0/539995808 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f00a0040 con 0x565259cd8b40 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.875+0000 7f10e6ffd640 1 -- 172.21.15.103:0/539995808 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f009b320 con 0x565259cd8b40 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.875+0000 7f1105471740 1 -- 172.21.15.103:0/539995808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 msgr2=0x565259cd7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.875+0000 7f1105471740 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cd7090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f10f00a43d0 tx=0x7f10f0005f90 comp rx=0 tx=0).stop 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- 172.21.15.103:0/539995808 shutdown_connections 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259ce4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 --2- 172.21.15.103:0/539995808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cd7090 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- 172.21.15.103:0/539995808 >> 172.21.15.103:0/539995808 conn(0x565259ca0310 msgr2=0x565259cd3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- 172.21.15.103:0/539995808 shutdown_connections 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- 172.21.15.103:0/539995808 wait complete. 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- start start 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 Processor -- start 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd75d0 0x565259ce1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd8b40 0x565259cdf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259cdf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565259ce4fb0 con 0x565259ce2370 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565259bf8060 con 0x565259cd75d0 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565259ce5130 con 0x565259cd8b40 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.876+0000 7f1105471740 10 client.0 _ll_register_callbacks cb 0x56525994b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259cdf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259cdf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37714/0 (socket says 172.21.15.103:37714) 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 -- 172.21.15.103:0/1572391790 learned_addr learned my addr 172.21.15.103:0/1572391790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1100e34640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd75d0 0x565259ce1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f10fbfff640 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd8b40 0x565259cdf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 -- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd8b40 msgr2=0x565259cdf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd8b40 0x565259cdf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 -- 172.21.15.103:0/1572391790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd75d0 msgr2=0x565259ce1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd75d0 0x565259ce1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 -- 172.21.15.103:0/1572391790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652599ae3c0 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1101635640 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259cdf7f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009ca0 tx=0x7f10f402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5ef812ec693316 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f10dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f10dbfff640 1 -- 172.21.15.103:0/1572391790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40319a0 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f10dbfff640 1 -- 172.21.15.103:0/1572391790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f10f4004510 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10f00a0040 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f10dbfff640 1 -- 172.21.15.103:0/1572391790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10f4004710 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1105471740 10 client.15783 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.877+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565259cf0aa0 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f10dbfff640 1 -- 172.21.15.103:0/1572391790 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f00a0040 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f10dbfff640 1 -- 172.21.15.103:0/1572391790 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f10f40307f0 con 0x565259ce2370 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1105471740 10 client.15783 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:34.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1105471740 10 client.15783 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:39:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1105471740 20 client.15783 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1105471740 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259cf1eb0 0x565259cf4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1105471740 4 client.15783 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565259c4a860 con 0x565259cf1eb0 2024-09-19T07:39:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1100e34640 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259cf1eb0 0x565259cf4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f1100e34640 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259cf1eb0 0x565259cf4310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e6d6d076a6d0d97 server_cookie=8d4e16ddce4892a9 in_seq=0 out_seq=0 2024-09-19T07:39:34.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.878+0000 7f10dbfff640 10 client.15783 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:39:34.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f10dbfff640 1 -- 172.21.15.103:0/1572391790 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565259c4a860 con 0x565259cf1eb0 2024-09-19T07:39:34.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f10dbfff640 10 client.15783 handle_command_reply: tid=0 2024-09-19T07:39:34.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 client.15783 shutdown 2024-09-19T07:39:34.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 msgr2=0x565259cdf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259cdf7f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009ca0 tx=0x7f10f402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 shutdown_connections 2024-09-19T07:39:34.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259cf1eb0 0x565259cf4310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259cdf7f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd8b40 0x565259cdf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 --2- 172.21.15.103:0/1572391790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd75d0 0x565259ce1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 >> 172.21.15.103:0/1572391790 conn(0x565259ca0310 msgr2=0x565259cd9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.879+0000 7f10dbfff640 0 client.15783 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.880+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 shutdown_connections 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.880+0000 7f1105471740 1 -- 172.21.15.103:0/1572391790 wait complete. 2024-09-19T07:39:34.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.880+0000 7f1105471740 20 client.15783 trim_cache size 0 max 16384 2024-09-19T07:39:34.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 Processor -- start 2024-09-19T07:39:34.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 -- start start 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd8b40 0x565259cea7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259ceace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1ad8d0 con 0x565259ce2370 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1af4d0 con 0x565259cd8b40 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1105471740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1adfa0 con 0x565259cd75d0 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1101635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259ceace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1101635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259ceace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37720/0 (socket says 172.21.15.103:37720) 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.892+0000 7f1101635640 1 -- 172.21.15.103:0/1119484332 learned_addr learned my addr 172.21.15.103:0/1119484332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f10fbfff640 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd8b40 0x565259cea7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1100e34640 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1101635640 1 -- 172.21.15.103:0/1119484332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 msgr2=0x565259cd3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1101635640 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1101635640 1 -- 172.21.15.103:0/1119484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd8b40 msgr2=0x565259cea7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1101635640 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd8b40 0x565259cea7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1101635640 1 -- 172.21.15.103:0/1119484332 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565259cdd5b0 con 0x565259ce2370 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1100e34640 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f10fbfff640 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd8b40 0x565259cea7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f1101635640 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259ceace0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f10f4030c10 tx=0x7f10f40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcb1ed8b983ca24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f10e67fc640 1 -- 172.21.15.103:0/1119484332 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40319a0 con 0x565259ce2370 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f10e67fc640 1 -- 172.21.15.103:0/1119484332 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f405f020 con 0x565259ce2370 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.893+0000 7f10e67fc640 1 -- 172.21.15.103:0/1119484332 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f402f9d0 con 0x565259ce2370 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- 172.21.15.103:0/1119484332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 msgr2=0x565259ceace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259ceace0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f10f4030c10 tx=0x7f10f40095e0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- 172.21.15.103:0/1119484332 shutdown_connections 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259ce2370 0x565259ceace0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259cd8b40 0x565259cea7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 --2- 172.21.15.103:0/1119484332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cd3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- 172.21.15.103:0/1119484332 >> 172.21.15.103:0/1119484332 conn(0x565259ca0310 msgr2=0x565259cd3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- 172.21.15.103:0/1119484332 shutdown_connections 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- 172.21.15.103:0/1119484332 wait complete. 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- start start 2024-09-19T07:39:34.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 Processor -- start 2024-09-19T07:39:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cf9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cf9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259d57eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1aed40 con 0x565259cd8b40 2024-09-19T07:39:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1ae670 con 0x565259ce2370 2024-09-19T07:39:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.894+0000 7f1105471740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1ab8b0 con 0x565259cd75d0 2024-09-19T07:39:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f1105471740 10 client.0 _ll_register_callbacks cb 0x7f10f4003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cf9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cf9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37726/0 (socket says 172.21.15.103:37726) 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 -- 172.21.15.103:0/1761609172 learned_addr learned my addr 172.21.15.103:0/1761609172 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f1100e34640 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cf9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f1101635640 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259d57eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 -- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 msgr2=0x565259cf9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cf9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 -- 172.21.15.103:0/1761609172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 msgr2=0x565259d57eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259d57eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 -- 172.21.15.103:0/1761609172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565259d58550 con 0x565259cd8b40 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10fbfff640 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cf9ca0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f10e8005fd0 tx=0x7f10e8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb33952c02427686 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:34.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10dbfff640 1 -- 172.21.15.103:0/1761609172 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8010a80 con 0x565259cd8b40 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10dbfff640 1 -- 172.21.15.103:0/1761609172 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f10e8010c20 con 0x565259cd8b40 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10f405f020 con 0x565259cd8b40 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f10dbfff640 1 -- 172.21.15.103:0/1761609172 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10e8010e20 con 0x565259cd8b40 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f1105471740 10 client.15789 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.895+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565259d60ab0 con 0x565259cd8b40 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f10dbfff640 1 -- 172.21.15.103:0/1761609172 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f405f020 con 0x565259cd8b40 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f10dbfff640 1 -- 172.21.15.103:0/1761609172 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f10e801b070 con 0x565259cd8b40 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1105471740 10 client.15789 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1105471740 10 client.15789 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1105471740 20 client.15789 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1105471740 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259d612e0 0x565259d63700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1105471740 4 client.15789 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:39:34.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565259c44d70 con 0x565259d612e0 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1100e34640 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259d612e0 0x565259d63700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f10dbfff640 10 client.15789 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.896+0000 7f1100e34640 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259d612e0 0x565259d63700 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ab3d7f87f8156d8 server_cookie=1488bcc3e81f335f in_seq=0 out_seq=0 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f10dbfff640 1 -- 172.21.15.103:0/1761609172 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565259c44d70 con 0x565259d612e0 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f10dbfff640 10 client.15789 handle_command_reply: tid=0 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 client.15789 shutdown 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 msgr2=0x565259cf9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cf9ca0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f10e8005fd0 tx=0x7f10e8011b50 comp rx=0 tx=0).stop 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 shutdown_connections 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x565259d612e0 0x565259d63700 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f10dbfff640 0 client.15789 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565259ce2370 0x565259d57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565259cd8b40 0x565259cf9ca0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 --2- 172.21.15.103:0/1761609172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565259cd75d0 0x565259cf9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 >> 172.21.15.103:0/1761609172 conn(0x565259ca0310 msgr2=0x565259ce08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:34.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 shutdown_connections 2024-09-19T07:39:34.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 1 -- 172.21.15.103:0/1761609172 wait complete. 2024-09-19T07:39:34.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.897+0000 7f1105471740 20 client.15789 trim_cache size 0 max 16384 2024-09-19T07:39:34.904 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10ec0a33b0 msgr2=0x7f10ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10ec0a33b0 0x7f10ec0a5870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10f4004af0 tx=0x7f10f4004a80 comp rx=0 tx=0).stop 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc101de0 msgr2=0x7f10fc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc101de0 0x7f10fc1983f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f10f000d950 tx=0x7f10f000de10 comp rx=0 tx=0).stop 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 shutdown_connections 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10ec0a33b0 0x7f10ec0a5870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10fc106f10 0x7f10fc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10fc102790 0x7f10fc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 --2- 172.21.15.103:0/284124940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10fc101de0 0x7f10fc1983f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 >> 172.21.15.103:0/284124940 conn(0x7f10fc0fd8c0 msgr2=0x7f10fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:34.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 shutdown_connections 2024-09-19T07:39:34.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:34.898+0000 7f1103098640 1 -- 172.21.15.103:0/284124940 wait complete. 2024-09-19T07:39:35.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 60.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:39:35.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:39:35.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:35 smithi134 ceph-mon[35414]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 431 op/s 2024-09-19T07:39:35.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:35 smithi174 ceph-mon[35370]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 431 op/s 2024-09-19T07:39:35.937 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:35.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:35 smithi103 ceph-mon[31201]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 431 op/s 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.371+0000 7f42ee44a640 1 -- 172.21.15.103:0/1177487128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8105220 msgr2=0x7f42e8105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.371+0000 7f42ee44a640 1 --2- 172.21.15.103:0/1177487128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8105220 0x7f42e8105600 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009900 tx=0x7f42d802a410 comp rx=0 tx=0).stop 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 -- 172.21.15.103:0/1177487128 shutdown_connections 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 --2- 172.21.15.103:0/1177487128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e8101020 0x7f42e810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 --2- 172.21.15.103:0/1177487128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e8100680 0x7f42e8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 --2- 172.21.15.103:0/1177487128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8105220 0x7f42e8105600 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 -- 172.21.15.103:0/1177487128 >> 172.21.15.103:0/1177487128 conn(0x7f42e8077210 msgr2=0x7f42e8077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 -- 172.21.15.103:0/1177487128 shutdown_connections 2024-09-19T07:39:36.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 -- 172.21.15.103:0/1177487128 wait complete. 2024-09-19T07:39:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 Processor -- start 2024-09-19T07:39:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 -- start start 2024-09-19T07:39:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.372+0000 7f42ee44a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8100680 0x7f42e8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ee44a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e8101020 0x7f42e81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ee44a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e8105220 0x7f42e819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ee44a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8110f90 con 0x7f42e8100680 2024-09-19T07:39:36.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ee44a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8110e10 con 0x7f42e8101020 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ee44a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8111110 con 0x7f42e8105220 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8100680 0x7f42e8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8100680 0x7f42e8198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37750/0 (socket says 172.21.15.103:37750) 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 -- 172.21.15.103:0/952853693 learned_addr learned my addr 172.21.15.103:0/952853693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e77fe640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e8101020 0x7f42e81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e8105220 0x7f42e819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 -- 172.21.15.103:0/952853693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e8105220 msgr2=0x7f42e819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e8105220 0x7f42e819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 -- 172.21.15.103:0/952853693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e8101020 msgr2=0x7f42e81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e8101020 0x7f42e81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 -- 172.21.15.103:0/952853693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e819d430 con 0x7f42e8100680 2024-09-19T07:39:36.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e77fe640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e8101020 0x7f42e81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:36.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e8105220 0x7f42e819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:36.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e7fff640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8100680 0x7f42e8198480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006010 tx=0x7f42d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:36.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e57fa640 1 -- 172.21.15.103:0/952853693 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d803f070 con 0x7f42e8100680 2024-09-19T07:39:36.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e819d6c0 con 0x7f42e8100680 2024-09-19T07:39:36.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e57fa640 1 -- 172.21.15.103:0/952853693 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d8031c30 con 0x7f42e8100680 2024-09-19T07:39:36.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.373+0000 7f42e57fa640 1 -- 172.21.15.103:0/952853693 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8035030 con 0x7f42e8100680 2024-09-19T07:39:36.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.374+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81a5000 con 0x7f42e8100680 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.376+0000 7f42e57fa640 1 -- 172.21.15.103:0/952853693 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f42d8042070 con 0x7f42e8100680 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.376+0000 7f42f0823740 1 Processor -- start 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.376+0000 7f42f0823740 1 -- start start 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42f0823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42f0823740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42f0823740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42f0823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562b2282d2f0 con 0x562b228c8b40 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42f0823740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562b227d4cf0 con 0x562b228c75d0 2024-09-19T07:39:36.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42f0823740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b228292b0 con 0x562b228d2370 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42e57fa640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42d40a33b0 0x7f42d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37760/0 (socket says 172.21.15.103:37760) 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 -- 172.21.15.103:0/1868896770 learned_addr learned my addr 172.21.15.103:0/1868896770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42e77fe640 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42e7fff640 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42d40a33b0 0x7f42d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42e57fa640 1 -- 172.21.15.103:0/952853693 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f42d80f0680 con 0x7f42e8100680 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 -- 172.21.15.103:0/1868896770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 msgr2=0x562b228d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 -- 172.21.15.103:0/1868896770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 msgr2=0x562b228c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 -- 172.21.15.103:0/1868896770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b227b93a0 con 0x562b228c8b40 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.377+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228c7090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f42e00149a0 tx=0x7f42e0014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cfa39626bcc858 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42de7fc640 1 -- 172.21.15.103:0/1868896770 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0021070 con 0x562b228c8b40 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42de7fc640 1 -- 172.21.15.103:0/1868896770 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e0013460 con 0x562b228c8b40 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42de7fc640 1 -- 172.21.15.103:0/1868896770 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e000e630 con 0x562b228c8b40 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 -- 172.21.15.103:0/1868896770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 msgr2=0x562b228c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228c7090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f42e00149a0 tx=0x7f42e0014e60 comp rx=0 tx=0).stop 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42d40a33b0 0x7f42d40a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42e001e000 tx=0x7f42e0019000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 -- 172.21.15.103:0/1868896770 shutdown_connections 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 --2- 172.21.15.103:0/1868896770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228c7090 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 -- 172.21.15.103:0/1868896770 >> 172.21.15.103:0/1868896770 conn(0x562b22890310 msgr2=0x562b228c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:36.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 -- 172.21.15.103:0/1868896770 shutdown_connections 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.378+0000 7f42f0823740 1 -- 172.21.15.103:0/1868896770 wait complete. 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 -- start start 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 Processor -- start 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562b228d94d0 con 0x562b228c75d0 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562b227e8060 con 0x562b228c8b40 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b228d9650 con 0x562b228d2370 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42f0823740 10 client.0 _ll_register_callbacks cb 0x562b2253b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37766/0 (socket says 172.21.15.103:37766) 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42e7fff640 1 -- 172.21.15.103:0/2162247713 learned_addr learned my addr 172.21.15.103:0/2162247713 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.379+0000 7f42e77fe640 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42e7fff640 1 -- 172.21.15.103:0/2162247713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 msgr2=0x562b228cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42e7fff640 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42e7fff640 1 -- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 msgr2=0x562b228cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42e7fff640 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42e7fff640 1 -- 172.21.15.103:0/2162247713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b2259e3c0 con 0x562b228c75d0 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42e7fff640 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228d1d30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f42d80bc610 tx=0x7f42d802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9549745b3dc8d9a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42c77fe640 1 -- 172.21.15.103:0/2162247713 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8049040 con 0x562b228c75d0 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42c77fe640 1 -- 172.21.15.103:0/2162247713 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f42d80ee000 con 0x562b228c75d0 2024-09-19T07:39:36.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42e0013460 con 0x562b228c75d0 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42c77fe640 1 -- 172.21.15.103:0/2162247713 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42d80ee200 con 0x562b228c75d0 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42f0823740 10 client.15801 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b228e4f30 con 0x562b228c75d0 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42c77fe640 1 -- 172.21.15.103:0/2162247713 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e0013460 con 0x562b228c75d0 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.380+0000 7f42c77fe640 1 -- 172.21.15.103:0/2162247713 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f42d803b460 con 0x562b228c75d0 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.381+0000 7f42f0823740 10 client.15801 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.381+0000 7f42f0823740 10 client.15801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.381+0000 7f42f0823740 20 client.15801 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.381+0000 7f42f0823740 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b228e6280 0x562b228e8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.381+0000 7f42f0823740 4 client.15801 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.381+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b2283a860 con 0x562b228e6280 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.381+0000 7f42e77fe640 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b228e6280 0x562b228e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42e77fe640 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b228e6280 0x562b228e8690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfb0d4e3861159ca server_cookie=25d98a5cc2affe5c in_seq=0 out_seq=0 2024-09-19T07:39:36.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42c77fe640 10 client.15801 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42c77fe640 1 -- 172.21.15.103:0/2162247713 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b2283a860 con 0x562b228e6280 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42c77fe640 10 client.15801 handle_command_reply: tid=0 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42f0823740 1 client.15801 shutdown 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 msgr2=0x562b228d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42f0823740 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228d1d30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f42d80bc610 tx=0x7f42d802a410 comp rx=0 tx=0).stop 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 shutdown_connections 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.382+0000 7f42f0823740 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b228e6280 0x562b228e8690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42f0823740 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42f0823740 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42f0823740 1 --2- 172.21.15.103:0/2162247713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228d1d30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42c77fe640 0 client.15801 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:39:36.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 >> 172.21.15.103:0/2162247713 conn(0x562b22890310 msgr2=0x562b228c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:36.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 shutdown_connections 2024-09-19T07:39:36.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42f0823740 1 -- 172.21.15.103:0/2162247713 wait complete. 2024-09-19T07:39:36.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.383+0000 7f42f0823740 20 client.15801 trim_cache size 0 max 16384 2024-09-19T07:39:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.395+0000 7f42f0823740 1 Processor -- start 2024-09-19T07:39:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.395+0000 7f42f0823740 1 -- start start 2024-09-19T07:39:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42f0823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42f0823740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42f0823740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42f0823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ab630 con 0x562b228c75d0 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42f0823740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ad230 con 0x562b228c8b40 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42f0823740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81abd00 con 0x562b228d2370 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37772/0 (socket says 172.21.15.103:37772) 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 -- 172.21.15.103:0/1989472394 learned_addr learned my addr 172.21.15.103:0/1989472394 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e77fe640 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 -- 172.21.15.103:0/1989472394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 msgr2=0x562b228edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228edf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 -- 172.21.15.103:0/1989472394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 msgr2=0x562b228ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 -- 172.21.15.103:0/1989472394 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b228c4960 con 0x562b228c75d0 2024-09-19T07:39:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e77fe640 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42e7fff640 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228ca120 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f42d8005940 tx=0x7f42d8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4f11e31903ab8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42ddffb640 1 -- 172.21.15.103:0/1989472394 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80b5030 con 0x562b228c75d0 2024-09-19T07:39:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42ddffb640 1 -- 172.21.15.103:0/1989472394 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d80c0cc0 con 0x562b228c75d0 2024-09-19T07:39:36.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.396+0000 7f42ddffb640 1 -- 172.21.15.103:0/1989472394 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80c5140 con 0x562b228c75d0 2024-09-19T07:39:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 -- 172.21.15.103:0/1989472394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 msgr2=0x562b228ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228ca120 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f42d8005940 tx=0x7f42d8005c00 comp rx=0 tx=0).stop 2024-09-19T07:39:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 -- 172.21.15.103:0/1989472394 shutdown_connections 2024-09-19T07:39:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b228edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c8b40 0x562b228ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 --2- 172.21.15.103:0/1989472394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c75d0 0x562b228ca120 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 -- 172.21.15.103:0/1989472394 >> 172.21.15.103:0/1989472394 conn(0x562b22890310 msgr2=0x562b228c2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 -- 172.21.15.103:0/1989472394 shutdown_connections 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 -- 172.21.15.103:0/1989472394 wait complete. 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.397+0000 7f42f0823740 1 -- start start 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 1 Processor -- start 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228dad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b2294ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81acaa0 con 0x562b228c8b40 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81ac3d0 con 0x562b228c75d0 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81a9610 con 0x562b228d2370 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42f0823740 10 client.0 _ll_register_callbacks cb 0x7f42d80b8490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228db260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37774/0 (socket says 172.21.15.103:37774) 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 -- 172.21.15.103:0/1573809502 learned_addr learned my addr 172.21.15.103:0/1573809502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42ec9e7640 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b2294ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e7fff640 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 -- 172.21.15.103:0/1573809502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 msgr2=0x562b2294ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b2294ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 -- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 msgr2=0x562b228dad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228dad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.398+0000 7f42e77fe640 1 -- 172.21.15.103:0/1573809502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b22950470 con 0x562b228c8b40 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42e77fe640 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228db260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005fd0 tx=0x7f42d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76163367febad56f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42c77fe640 1 -- 172.21.15.103:0/1573809502 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0023070 con 0x562b228c8b40 2024-09-19T07:39:36.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42c77fe640 1 -- 172.21.15.103:0/1573809502 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f42d001f5c0 con 0x562b228c8b40 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d80c0cc0 con 0x562b228c8b40 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42c77fe640 1 -- 172.21.15.103:0/1573809502 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42d001f7c0 con 0x562b228c8b40 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 10 client.15807 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b229517a0 con 0x562b228c8b40 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42c77fe640 1 -- 172.21.15.103:0/1573809502 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d80c0cc0 con 0x562b228c8b40 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42c77fe640 1 -- 172.21.15.103:0/1573809502 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f42d0027400 con 0x562b228c8b40 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 10 client.15807 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 10 client.15807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 20 client.15807 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b22951ed0 0x562b229542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 4 client.15807 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.399+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b22834d70 con 0x562b22951ed0 2024-09-19T07:39:36.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42e7fff640 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b22951ed0 0x562b229542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42c77fe640 10 client.15807 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42e7fff640 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b22951ed0 0x562b229542f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6eab34cd99bb89c server_cookie=d3cd1321aa1515ed in_seq=0 out_seq=0 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42c77fe640 1 -- 172.21.15.103:0/1573809502 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b22834d70 con 0x562b22951ed0 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42c77fe640 10 client.15807 handle_command_reply: tid=0 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 client.15807 shutdown 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 msgr2=0x562b228db260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228db260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005fd0 tx=0x7f42d000b840 comp rx=0 tx=0).stop 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 shutdown_connections 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562b22951ed0 0x562b229542f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b228d2370 0x562b2294ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b228c8b40 0x562b228db260 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 --2- 172.21.15.103:0/1573809502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b228c75d0 0x562b228dad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 >> 172.21.15.103:0/1573809502 conn(0x562b22890310 msgr2=0x562b228d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.400+0000 7f42c77fe640 0 client.15807 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:39:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 shutdown_connections 2024-09-19T07:39:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42f0823740 1 -- 172.21.15.103:0/1573809502 wait complete. 2024-09-19T07:39:36.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42f0823740 20 client.15807 trim_cache size 0 max 16384 2024-09-19T07:39:36.409 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42d40a33b0 msgr2=0x7f42d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42d40a33b0 0x7f42d40a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42e001e000 tx=0x7f42e0019000 comp rx=0 tx=0).stop 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8100680 msgr2=0x7f42e8198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8100680 0x7f42e8198480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006010 tx=0x7f42d8003c10 comp rx=0 tx=0).stop 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 shutdown_connections 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42d40a33b0 0x7f42d40a5870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e8105220 0x7f42e819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e8101020 0x7f42e81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 --2- 172.21.15.103:0/952853693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e8100680 0x7f42e8198480 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:36.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 >> 172.21.15.103:0/952853693 conn(0x7f42e8077210 msgr2=0x7f42e810dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 shutdown_connections 2024-09-19T07:39:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:36.401+0000 7f42ee44a640 1 -- 172.21.15.103:0/952853693 wait complete. 2024-09-19T07:39:37.134 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:39:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:37.794 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:37 smithi103 ceph-mon[31201]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-19T07:39:37.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:37 smithi134 ceph-mon[35414]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1468881368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54106f10 msgr2=0x7f4a5410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1468881368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54106f10 0x7f4a5410ea90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009ca0 tx=0x7f4a4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1468881368 shutdown_connections 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1468881368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54106f10 0x7f4a5410ea90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1468881368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a54102790 0x7f4a541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1468881368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a54101de0 0x7f4a541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1468881368 >> 172.21.15.103:0/1468881368 conn(0x7f4a540fd8c0 msgr2=0x7f4a540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1468881368 shutdown_connections 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.872+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1468881368 wait complete. 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 Processor -- start 2024-09-19T07:39:37.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 -- start start 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54101de0 0x7f4a54198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a54102790 0x7f4a54198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a54106f10 0x7f4a5419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54111f70 con 0x7f4a54101de0 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54111df0 con 0x7f4a54106f10 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a5b1a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541120f0 con 0x7f4a54102790 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a58f41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54101de0 0x7f4a54198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a58f41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54101de0 0x7f4a54198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37786/0 (socket says 172.21.15.103:37786) 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a58f41640 1 -- 172.21.15.103:0/1614313293 learned_addr learned my addr 172.21.15.103:0/1614313293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.873+0000 7f4a53fff640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a54102790 0x7f4a54198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a59742640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a54106f10 0x7f4a5419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a58f41640 1 -- 172.21.15.103:0/1614313293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a54102790 msgr2=0x7f4a54198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a58f41640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a54102790 0x7f4a54198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a58f41640 1 -- 172.21.15.103:0/1614313293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a54106f10 msgr2=0x7f4a5419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a58f41640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a54106f10 0x7f4a5419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a58f41640 1 -- 172.21.15.103:0/1614313293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5419d3c0 con 0x7f4a54101de0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a58f41640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54101de0 0x7f4a54198410 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800d950 tx=0x7f4a4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a51ffb640 1 -- 172.21.15.103:0/1614313293 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48017070 con 0x7f4a54101de0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a51ffb640 1 -- 172.21.15.103:0/1614313293 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48009c30 con 0x7f4a54101de0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5419d6b0 con 0x7f4a54101de0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a51ffb640 1 -- 172.21.15.103:0/1614313293 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800f6b0 con 0x7f4a54101de0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.874+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a541a4f90 con 0x7f4a54101de0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.876+0000 7f4a51ffb640 1 -- 172.21.15.103:0/1614313293 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4a480109d0 con 0x7f4a54101de0 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 Processor -- start 2024-09-19T07:39:37.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 -- start start 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a51ffb640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4a440a33b0 0x7f4a440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e55d0 0x55f9202e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f92024b2f0 con 0x55f9202f0370 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f9201f2cf0 con 0x55f9202e6b40 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9202472b0 con 0x55f9202e55d0 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a59742640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4a440a33b0 0x7f4a440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a51ffb640 1 -- 172.21.15.103:0/1614313293 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4a480cfa10 con 0x7f4a54101de0 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e55d0 0x55f9202e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a59742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e55d0 0x55f9202e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:39302/0 (socket says 172.21.15.103:39302) 2024-09-19T07:39:37.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a53fff640 1 -- 172.21.15.103:0/1374078870 learned_addr learned my addr 172.21.15.103:0/1374078870 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.877+0000 7f4a58f41640 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a53fff640 1 -- 172.21.15.103:0/1374078870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 msgr2=0x55f9202e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a53fff640 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a53fff640 1 -- 172.21.15.103:0/1374078870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 msgr2=0x55f9202f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a53fff640 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a53fff640 1 -- 172.21.15.103:0/1374078870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9201d73a0 con 0x55f9202e55d0 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a59742640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4a440a33b0 0x7f4a440a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009a60 tx=0x7f4a4c02f330 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a53fff640 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e55d0 0x55f9202e5a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000bc00 tx=0x7f4a4000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d2e2db210ca490 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a3effd640 1 -- 172.21.15.103:0/1374078870 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40015070 con 0x55f9202e55d0 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a3effd640 1 -- 172.21.15.103:0/1374078870 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a40005500 con 0x55f9202e55d0 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.878+0000 7f4a3effd640 1 -- 172.21.15.103:0/1374078870 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011440 con 0x55f9202e55d0 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 -- 172.21.15.103:0/1374078870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e55d0 msgr2=0x55f9202e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e55d0 0x55f9202e5a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000bc00 tx=0x7f4a4000f730 comp rx=0 tx=0).stop 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 -- 172.21.15.103:0/1374078870 shutdown_connections 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e55d0 0x55f9202e5a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/1374078870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 -- 172.21.15.103:0/1374078870 >> 172.21.15.103:0/1374078870 conn(0x55f9202ae310 msgr2=0x55f9202e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 -- 172.21.15.103:0/1374078870 shutdown_connections 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 -- 172.21.15.103:0/1374078870 wait complete. 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 -- start start 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.879+0000 7f4a5d57e740 1 Processor -- start 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e55d0 0x55f9202efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e6b40 0x55f9202ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202ed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9202f3070 con 0x55f9202f0370 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f920206060 con 0x55f9202e55d0 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9202f31f0 con 0x55f9202e6b40 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 10 client.0 _ll_register_callbacks cb 0x55f91ff598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202ed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37798/0 (socket says 172.21.15.103:37798) 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 -- 172.21.15.103:0/3766912333 learned_addr learned my addr 172.21.15.103:0/3766912333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a58f41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e55d0 0x55f9202efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a53fff640 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e6b40 0x55f9202ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 -- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e6b40 msgr2=0x55f9202ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e6b40 0x55f9202ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 -- 172.21.15.103:0/3766912333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e55d0 msgr2=0x55f9202efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e55d0 0x55f9202efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 -- 172.21.15.103:0/3766912333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f91ffbc3c0 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a59742640 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202ed570 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009ca0 tx=0x7f4a4c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261de615d1622d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a33fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a33fff640 1 -- 172.21.15.103:0/3766912333 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c04f030 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a33fff640 1 -- 172.21.15.103:0/3766912333 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4a4c046040 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.880+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a40005500 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a33fff640 1 -- 172.21.15.103:0/3766912333 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a4c05b050 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 10 client.15819 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9202feaf0 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a33fff640 1 -- 172.21.15.103:0/3766912333 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a40005500 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a33fff640 1 -- 172.21.15.103:0/3766912333 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4a4c04a070 con 0x55f9202f0370 2024-09-19T07:39:37.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 10 client.15819 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 10 client.15819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 20 client.15819 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f9202fffa0 0x55f920302440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 4 client.15819 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f920258860 con 0x55f9202fffa0 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.881+0000 7f4a58f41640 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f9202fffa0 0x55f920302440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a33fff640 10 client.15819 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a58f41640 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f9202fffa0 0x55f920302440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7ecf860ed9c38e4 server_cookie=71932978a2c46313 in_seq=0 out_seq=0 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a33fff640 1 -- 172.21.15.103:0/3766912333 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f920258860 con 0x55f9202fffa0 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a33fff640 10 client.15819 handle_command_reply: tid=0 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a5d57e740 1 client.15819 shutdown 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 msgr2=0x55f9202ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202ed570 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009ca0 tx=0x7f4a4c047000 comp rx=0 tx=0).stop 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 shutdown_connections 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f9202fffa0 0x55f920302440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.882+0000 7f4a33fff640 0 client.15819 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.883+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202f0370 0x55f9202ed570 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.883+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202e6b40 0x55f9202ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.883+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3766912333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e55d0 0x55f9202efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.883+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 >> 172.21.15.103:0/3766912333 conn(0x55f9202ae310 msgr2=0x55f9202e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.883+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 shutdown_connections 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.883+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3766912333 wait complete. 2024-09-19T07:39:37.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.883+0000 7f4a5d57e740 20 client.15819 trim_cache size 0 max 16384 2024-09-19T07:39:37.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.895+0000 7f4a5d57e740 1 Processor -- start 2024-09-19T07:39:37.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a5d57e740 1 -- start start 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ad850 con 0x55f9202e55d0 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541af450 con 0x55f9202e6b40 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541adf20 con 0x55f9202f0370 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37808/0 (socket says 172.21.15.103:37808) 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 -- 172.21.15.103:0/2173396740 learned_addr learned my addr 172.21.15.103:0/2173396740 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a59742640 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 -- 172.21.15.103:0/2173396740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 msgr2=0x55f9202e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a53fff640 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 -- 172.21.15.103:0/2173396740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 msgr2=0x55f9202e8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 -- 172.21.15.103:0/2173396740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9202ebf80 con 0x55f9202e55d0 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a59742640 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a53fff640 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a58f41640 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202eef40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4a480cd1d0 tx=0x7f4a48086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434bc0b6b78463e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.896+0000 7f4a3e7fc640 1 -- 172.21.15.103:0/2173396740 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48020040 con 0x55f9202e55d0 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a3e7fc640 1 -- 172.21.15.103:0/2173396740 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a480a4970 con 0x55f9202e55d0 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a3e7fc640 1 -- 172.21.15.103:0/2173396740 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4806bd10 con 0x55f9202e55d0 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 -- 172.21.15.103:0/2173396740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 msgr2=0x55f9202eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202eef40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4a480cd1d0 tx=0x7f4a48086040 comp rx=0 tx=0).stop 2024-09-19T07:39:37.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 -- 172.21.15.103:0/2173396740 shutdown_connections 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202e8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/2173396740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202eef40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 -- 172.21.15.103:0/2173396740 >> 172.21.15.103:0/2173396740 conn(0x55f9202ae310 msgr2=0x55f9202ee830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 -- 172.21.15.103:0/2173396740 shutdown_connections 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.897+0000 7f4a5d57e740 1 -- 172.21.15.103:0/2173396740 wait complete. 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 -- start start 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 Processor -- start 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202f4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541aecc0 con 0x55f9202e55d0 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ae5f0 con 0x55f9202e6b40 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ab830 con 0x55f9202f0370 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a5d57e740 10 client.0 _ll_register_callbacks cb 0x7f4a4c031bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a58f41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a58f41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37824/0 (socket says 172.21.15.103:37824) 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a58f41640 1 -- 172.21.15.103:0/3546043175 learned_addr learned my addr 172.21.15.103:0/3546043175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a53fff640 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.898+0000 7f4a59742640 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a58f41640 1 -- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 msgr2=0x55f9202f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a58f41640 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a58f41640 1 -- 172.21.15.103:0/3546043175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 msgr2=0x55f9202f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a58f41640 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a58f41640 1 -- 172.21.15.103:0/3546043175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9202f0090 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a58f41640 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202f4360 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a480a4440 tx=0x7f4a48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965ad9daf3c07013 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a33fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a33fff640 1 -- 172.21.15.103:0/3546043175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4806a070 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a33fff640 1 -- 172.21.15.103:0/3546043175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4a480cd3f0 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a480a4970 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a33fff640 1 -- 172.21.15.103:0/3546043175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a48020050 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a5d57e740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f92036aea0 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a33fff640 1 -- 172.21.15.103:0/3546043175 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a480a4970 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a33fff640 1 -- 172.21.15.103:0/3546043175 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4a4809b2e0 con 0x55f9202e55d0 2024-09-19T07:39:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a5d57e740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a5d57e740 10 client.15825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a5d57e740 20 client.15825 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.899+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f92036b510 0x55f92036d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 4 client.15825 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f920252d70 con 0x55f92036b510 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a53fff640 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f92036b510 0x55f92036d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a53fff640 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f92036b510 0x55f92036d900 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=879a1d1a8f734fcc server_cookie=c25b261d52dd333b in_seq=0 out_seq=0 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a33fff640 10 client.15825 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a33fff640 1 -- 172.21.15.103:0/3546043175 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f920252d70 con 0x55f92036b510 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a33fff640 10 client.15825 handle_command_reply: tid=0 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 client.15825 shutdown 2024-09-19T07:39:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 msgr2=0x55f9202f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202f4360 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a480a4440 tx=0x7f4a48005880 comp rx=0 tx=0).stop 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 shutdown_connections 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f92036b510 0x55f92036d900 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a33fff640 0 client.15825 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9202f0370 0x55f9202f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9202e6b40 0x55f9202f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 --2- 172.21.15.103:0/3546043175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9202e55d0 0x55f9202f4360 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.900+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 >> 172.21.15.103:0/3546043175 conn(0x55f9202ae310 msgr2=0x55f9202e0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.901+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 shutdown_connections 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.901+0000 7f4a5d57e740 1 -- 172.21.15.103:0/3546043175 wait complete. 2024-09-19T07:39:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.901+0000 7f4a5d57e740 20 client.15825 trim_cache size 0 max 16384 2024-09-19T07:39:37.917 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:39:37.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:37 smithi174 ceph-mon[35370]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4a440a33b0 msgr2=0x7f4a440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4a440a33b0 0x7f4a440a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009a60 tx=0x7f4a4c02f330 comp rx=0 tx=0).stop 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54101de0 msgr2=0x7f4a54198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54101de0 0x7f4a54198410 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800d950 tx=0x7f4a4800de10 comp rx=0 tx=0).stop 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 shutdown_connections 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4a440a33b0 0x7f4a440a5870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a54106f10 0x7f4a5419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a54102790 0x7f4a54198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 --2- 172.21.15.103:0/1614313293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a54101de0 0x7f4a54198410 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:37.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 >> 172.21.15.103:0/1614313293 conn(0x7f4a540fd8c0 msgr2=0x7f4a540feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:37.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 shutdown_connections 2024-09-19T07:39:37.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:37.902+0000 7f4a5b1a5640 1 -- 172.21.15.103:0/1614313293 wait complete. 2024-09-19T07:39:38.670 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:39:38.914 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:39.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 -- 172.21.15.103:0/1112021744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100e30 msgr2=0x7f1e6c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 --2- 172.21.15.103:0/1112021744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100e30 0x7f1e6c104e80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009900 tx=0x7f1e5402a410 comp rx=0 tx=0).stop 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 -- 172.21.15.103:0/1112021744 shutdown_connections 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 --2- 172.21.15.103:0/1112021744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c1055b0 0x7f1e6c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 --2- 172.21.15.103:0/1112021744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100e30 0x7f1e6c104e80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 --2- 172.21.15.103:0/1112021744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c100480 0x7f1e6c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 -- 172.21.15.103:0/1112021744 >> 172.21.15.103:0/1112021744 conn(0x7f1e6c077210 msgr2=0x7f1e6c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 -- 172.21.15.103:0/1112021744 shutdown_connections 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.360+0000 7f1e71713640 1 -- 172.21.15.103:0/1112021744 wait complete. 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 Processor -- start 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 -- start start 2024-09-19T07:39:39.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100480 0x7f1e6c10e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c100e30 0x7f1e6c10ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c1055b0 0x7f1e6c10f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c111f90 con 0x7f1e6c100480 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c111e10 con 0x7f1e6c1055b0 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.361+0000 7f1e71713640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c112110 con 0x7f1e6c100e30 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100480 0x7f1e6c10e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100480 0x7f1e6c10e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54822/0 (socket says 172.21.15.103:54822) 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 -- 172.21.15.103:0/778580630 learned_addr learned my addr 172.21.15.103:0/778580630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c100e30 0x7f1e6c10ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c1055b0 0x7f1e6c10f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 -- 172.21.15.103:0/778580630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c100e30 msgr2=0x7f1e6c10ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c100e30 0x7f1e6c10ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 -- 172.21.15.103:0/778580630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c1055b0 msgr2=0x7f1e6c10f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c1055b0 0x7f1e6c10f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 -- 172.21.15.103:0/778580630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c1a6910 con 0x7f1e6c100480 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c1055b0 0x7f1e6c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e6affd640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100480 0x7f1e6c10e850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ea80 tx=0x7f1e6400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e63fff640 1 -- 172.21.15.103:0/778580630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400ce40 con 0x7f1e6c100480 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c1a6c00 con 0x7f1e6c100480 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e63fff640 1 -- 172.21.15.103:0/778580630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e64005020 con 0x7f1e6c100480 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e63fff640 1 -- 172.21.15.103:0/778580630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64011760 con 0x7f1e6c100480 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.362+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c1a70c0 con 0x7f1e6c100480 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.365+0000 7f1e63fff640 1 -- 172.21.15.103:0/778580630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1e640051d0 con 0x7f1e6c100480 2024-09-19T07:39:39.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.365+0000 7f1e63fff640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1e580a33b0 0x7f1e580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.365+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1e580a33b0 0x7f1e580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.365+0000 7f1e63fff640 1 -- 172.21.15.103:0/778580630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f1e640d0330 con 0x7f1e6c100480 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.365+0000 7f1e73aec740 1 Processor -- start 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e73aec740 1 -- start start 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1e580a33b0 0x7f1e580a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009230 tx=0x7f1e540058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeaca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e73aec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56135ae122f0 con 0x56135aeac5d0 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e73aec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56135adb9cf0 con 0x56135aeadb40 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e73aec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56135ae0e2b0 con 0x56135aeb7370 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeaca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeaca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54824/0 (socket says 172.21.15.103:54824) 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/1463658994 learned_addr learned my addr 172.21.15.103:0/1463658994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6affd640 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/1463658994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 msgr2=0x56135aeb9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/1463658994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 msgr2=0x56135aeac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.366+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/1463658994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135ad9e3a0 con 0x56135aeac5d0 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.367+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeaca10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009900 tx=0x7f1e5403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1be15ad060d84b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.367+0000 7f1e617fa640 1 -- 172.21.15.103:0/1463658994 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54035040 con 0x56135aeac5d0 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.367+0000 7f1e617fa640 1 -- 172.21.15.103:0/1463658994 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5404e070 con 0x56135aeac5d0 2024-09-19T07:39:39.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.367+0000 7f1e617fa640 1 -- 172.21.15.103:0/1463658994 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540496e0 con 0x56135aeac5d0 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 -- 172.21.15.103:0/1463658994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 msgr2=0x56135aeaca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeaca10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009900 tx=0x7f1e5403d000 comp rx=0 tx=0).stop 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 -- 172.21.15.103:0/1463658994 shutdown_connections 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeaca10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 --2- 172.21.15.103:0/1463658994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 -- 172.21.15.103:0/1463658994 >> 172.21.15.103:0/1463658994 conn(0x56135ae75310 msgr2=0x56135aea8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 -- 172.21.15.103:0/1463658994 shutdown_connections 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 -- 172.21.15.103:0/1463658994 wait complete. 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 -- start start 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.368+0000 7f1e73aec740 1 Processor -- start 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeb6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeb3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56135aeba010 con 0x56135aeac5d0 2024-09-19T07:39:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56135adcd060 con 0x56135aeadb40 2024-09-19T07:39:39.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56135aeba190 con 0x56135aeb7370 2024-09-19T07:39:39.429 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 10 client.0 _ll_register_callbacks cb 0x56135ab208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeb6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeb6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54830/0 (socket says 172.21.15.103:54830) 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 -- 172.21.15.103:0/930654269 learned_addr learned my addr 172.21.15.103:0/930654269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeb3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 -- 172.21.15.103:0/930654269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 msgr2=0x56135aeb4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 -- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 msgr2=0x56135aeb3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeb3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 -- 172.21.15.103:0/930654269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135ab833c0 con 0x56135aeac5d0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeb3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e6affd640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeb6d10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400c700 tx=0x7f1e6409c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d554971ff3de6e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e527fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e527fc640 1 -- 172.21.15.103:0/930654269 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640b4070 con 0x56135aeac5d0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e527fc640 1 -- 172.21.15.103:0/930654269 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f1e64098c90 con 0x56135aeac5d0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e5404e070 con 0x56135aeac5d0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e527fc640 1 -- 172.21.15.103:0/930654269 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e6401f040 con 0x56135aeac5d0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 10 client.15837 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.369+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56135aec5af0 con 0x56135aeac5d0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e527fc640 1 -- 172.21.15.103:0/930654269 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5404e070 con 0x56135aeac5d0 2024-09-19T07:39:39.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e527fc640 1 -- 172.21.15.103:0/930654269 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1e6407e050 con 0x56135aeac5d0 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e73aec740 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e73aec740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e73aec740 20 client.15837 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e73aec740 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135aec6ff0 0x56135aec9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e73aec740 4 client.15837 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56135ae1f860 con 0x56135aec6ff0 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.370+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135aec6ff0 0x56135aec9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135aec6ff0 0x56135aec9470 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d78c80fb34c1fa3 server_cookie=4f76cce443317fa9 in_seq=0 out_seq=0 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e527fc640 10 client.15837 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:39.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e527fc640 1 -- 172.21.15.103:0/930654269 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56135ae1f860 con 0x56135aec6ff0 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e527fc640 10 client.15837 handle_command_reply: tid=0 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 client.15837 shutdown 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 msgr2=0x56135aeb6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeb6d10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400c700 tx=0x7f1e6409c6e0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 shutdown_connections 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135aec6ff0 0x56135aec9470 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeb7370 0x56135aeb4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeb3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 --2- 172.21.15.103:0/930654269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeac5d0 0x56135aeb6d10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e527fc640 0 client.15837 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:39.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.371+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 >> 172.21.15.103:0/930654269 conn(0x56135ae75310 msgr2=0x56135aeae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.372+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 shutdown_connections 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.372+0000 7f1e73aec740 1 -- 172.21.15.103:0/930654269 wait complete. 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.372+0000 7f1e73aec740 20 client.15837 trim_cache size 0 max 16384 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.384+0000 7f1e73aec740 1 Processor -- start 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.384+0000 7f1e73aec740 1 -- start start 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aec32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeaf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeb7370 0x56135aeaf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e73aec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1af9a0 con 0x56135aeb7370 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e73aec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1b15a0 con 0x56135aeadb40 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e73aec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1b0070 con 0x56135aeac5d0 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeb7370 0x56135aeaf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeb7370 0x56135aeaf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54832/0 (socket says 172.21.15.103:54832) 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 -- 172.21.15.103:0/4250901417 learned_addr learned my addr 172.21.15.103:0/4250901417 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6affd640 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aec32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 -- 172.21.15.103:0/4250901417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 msgr2=0x56135aec32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aec32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeaf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 -- 172.21.15.103:0/4250901417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 msgr2=0x56135aeaf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeaf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 -- 172.21.15.103:0/4250901417 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135aeb7080 con 0x56135aeb7370 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.385+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeb7370 0x56135aeaf660 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009cb0 tx=0x7f1e5c0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742c792c56df0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e60ff9640 1 -- 172.21.15.103:0/4250901417 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c01b440 con 0x56135aeb7370 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e60ff9640 1 -- 172.21.15.103:0/4250901417 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5c01a8c0 con 0x56135aeb7370 2024-09-19T07:39:39.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e60ff9640 1 -- 172.21.15.103:0/4250901417 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c010ad0 con 0x56135aeb7370 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 -- 172.21.15.103:0/4250901417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeb7370 msgr2=0x56135aeaf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeb7370 0x56135aeaf660 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009cb0 tx=0x7f1e5c0127c0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 -- 172.21.15.103:0/4250901417 shutdown_connections 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeb7370 0x56135aeaf660 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeadb40 0x56135aeaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 --2- 172.21.15.103:0/4250901417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aec32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 -- 172.21.15.103:0/4250901417 >> 172.21.15.103:0/4250901417 conn(0x56135ae75310 msgr2=0x56135aeb30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 -- 172.21.15.103:0/4250901417 shutdown_connections 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.386+0000 7f1e73aec740 1 -- 172.21.15.103:0/4250901417 wait complete. 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 -- start start 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 Processor -- start 2024-09-19T07:39:39.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aebf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeadb40 0x56135aebfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeb7370 0x56135aeb09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1b0e10 con 0x56135aeadb40 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1b0740 con 0x56135aeb7370 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1ad980 con 0x56135aeac5d0 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e73aec740 10 client.0 _ll_register_callbacks cb 0x7f1e6409bf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeadb40 0x56135aebfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeadb40 0x56135aebfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54844/0 (socket says 172.21.15.103:54844) 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/3269266917 learned_addr learned my addr 172.21.15.103:0/3269266917 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6affd640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aebf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeb7370 0x56135aeb09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/3269266917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 msgr2=0x56135aebf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aebf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeb7370 msgr2=0x56135aeb09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeb7370 0x56135aeb09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 -- 172.21.15.103:0/3269266917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135aeb48d0 con 0x56135aeadb40 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6b7fe640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeb7370 0x56135aeb09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e6a7fc640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeadb40 0x56135aebfc40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009690 tx=0x7f1e54009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44287c126b325fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.387+0000 7f1e527fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e527fc640 1 -- 172.21.15.103:0/3269266917 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5405f060 con 0x56135aeadb40 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e527fc640 1 -- 172.21.15.103:0/3269266917 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f1e540035f0 con 0x56135aeadb40 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e5c01a8c0 con 0x56135aeadb40 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e527fc640 1 -- 172.21.15.103:0/3269266917 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e54040050 con 0x56135aeadb40 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 10 client.15843 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:39.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56135af31730 con 0x56135aeadb40 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e527fc640 1 -- 172.21.15.103:0/3269266917 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5c01a8c0 con 0x56135aeadb40 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e527fc640 1 -- 172.21.15.103:0/3269266917 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1e54049940 con 0x56135aeadb40 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 10 client.15843 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 20 client.15843 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135af31b30 0x56135af33f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 4 client.15843 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.388+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56135ae19d70 con 0x56135af31b30 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e6affd640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135af31b30 0x56135af33f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e527fc640 10 client.15843 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e6affd640 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135af31b30 0x56135af33f20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d4f6dedfe37e6c server_cookie=63414ccb9b2e0361 in_seq=0 out_seq=0 2024-09-19T07:39:39.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e527fc640 1 -- 172.21.15.103:0/3269266917 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56135ae19d70 con 0x56135af31b30 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e527fc640 10 client.15843 handle_command_reply: tid=0 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 client.15843 shutdown 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeadb40 msgr2=0x56135aebfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeadb40 0x56135aebfc40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009690 tx=0x7f1e54009620 comp rx=0 tx=0).stop 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 shutdown_connections 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56135af31b30 0x56135af33f20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56135aeb7370 0x56135aeb09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e527fc640 0 client.15843 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56135aeadb40 0x56135aebfc40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 --2- 172.21.15.103:0/3269266917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56135aeac5d0 0x56135aebf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 >> 172.21.15.103:0/3269266917 conn(0x56135ae75310 msgr2=0x56135aea7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:39.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 shutdown_connections 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.389+0000 7f1e73aec740 1 -- 172.21.15.103:0/3269266917 wait complete. 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e73aec740 20 client.15843 trim_cache size 0 max 16384 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1e580a33b0 msgr2=0x7f1e580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1e580a33b0 0x7f1e580a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009230 tx=0x7f1e540058d0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100480 msgr2=0x7f1e6c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100480 0x7f1e6c10e850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ea80 tx=0x7f1e6400ef40 comp rx=0 tx=0).stop 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 shutdown_connections 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f1e580a33b0 0x7f1e580a5870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e6c1055b0 0x7f1e6c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c100e30 0x7f1e6c10ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 --2- 172.21.15.103:0/778580630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c100480 0x7f1e6c10e850 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 >> 172.21.15.103:0/778580630 conn(0x7f1e6c077210 msgr2=0x7f1e6c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:39.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 shutdown_connections 2024-09-19T07:39:39.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:39.390+0000 7f1e71713640 1 -- 172.21.15.103:0/778580630 wait complete. 2024-09-19T07:39:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:39 smithi103 ceph-mon[31201]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-19T07:39:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:39 smithi134 ceph-mon[35414]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-19T07:39:39.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:39 smithi174 ceph-mon[35370]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-09-19T07:39:40.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 65.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:39:41.150 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:39:41.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:39:41.409 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:41.808 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:41 smithi103 ceph-mon[31201]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-09-19T07:39:41.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:41 smithi134 ceph-mon[35414]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-09-19T07:39:41.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.893+0000 7fdade54a640 1 -- 172.21.15.103:0/859273260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 msgr2=0x7fdad810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.893+0000 7fdade54a640 1 --2- 172.21.15.103:0/859273260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad810dee0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009900 tx=0x7fdacc02a480 comp rx=0 tx=0).stop 2024-09-19T07:39:41.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.893+0000 7fdade54a640 1 -- 172.21.15.103:0/859273260 shutdown_connections 2024-09-19T07:39:41.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.893+0000 7fdade54a640 1 --2- 172.21.15.103:0/859273260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad810e420 0x7fdad8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.893+0000 7fdade54a640 1 --2- 172.21.15.103:0/859273260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad810dee0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.893+0000 7fdade54a640 1 --2- 172.21.15.103:0/859273260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdad80745c0 0x7fdad80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.893+0000 7fdade54a640 1 -- 172.21.15.103:0/859273260 >> 172.21.15.103:0/859273260 conn(0x7fdad80fd8c0 msgr2=0x7fdad80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.894+0000 7fdade54a640 1 -- 172.21.15.103:0/859273260 shutdown_connections 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.894+0000 7fdade54a640 1 -- 172.21.15.103:0/859273260 wait complete. 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.894+0000 7fdade54a640 1 Processor -- start 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.894+0000 7fdade54a640 1 -- start start 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.894+0000 7fdade54a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdad80745c0 0x7fdad819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdade54a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdade54a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad810e420 0x7fdad819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdade54a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8114160 con 0x7fdad8074ee0 2024-09-19T07:39:41.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdade54a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8113fe0 con 0x7fdad80745c0 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdade54a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81142e0 con 0x7fdad810e420 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54860/0 (socket says 172.21.15.103:54860) 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 -- 172.21.15.103:0/3646105005 learned_addr learned my addr 172.21.15.103:0/3646105005 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad810e420 0x7fdad819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad7fff640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdad80745c0 0x7fdad819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 -- 172.21.15.103:0/3646105005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad810e420 msgr2=0x7fdad819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad810e420 0x7fdad819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 -- 172.21.15.103:0/3646105005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdad80745c0 msgr2=0x7fdad819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdad80745c0 0x7fdad819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 -- 172.21.15.103:0/3646105005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad819f620 con 0x7fdad8074ee0 2024-09-19T07:39:41.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad810e420 0x7fdad819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad819abb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdacc006010 tx=0x7fdacc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad57fa640 1 -- 172.21.15.103:0/3646105005 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc03f030 con 0x7fdad8074ee0 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad819f8b0 con 0x7fdad8074ee0 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad57fa640 1 -- 172.21.15.103:0/3646105005 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc003ce0 con 0x7fdad8074ee0 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdad57fa640 1 -- 172.21.15.103:0/3646105005 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc035070 con 0x7fdad8074ee0 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.895+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad81a71f0 con 0x7fdad8074ee0 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.898+0000 7fdad57fa640 1 -- 172.21.15.103:0/3646105005 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdacc042030 con 0x7fdad8074ee0 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.898+0000 7fdae0923740 1 Processor -- start 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.898+0000 7fdae0923740 1 -- start start 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.898+0000 7fdae0923740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdae0923740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d41800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdae0923740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdae0923740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618f6c9c280 con 0x5618f6d36b60 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdae0923740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618f6c43b10 con 0x5618f6d41d40 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdae0923740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5618f6c98240 con 0x5618f6d37480 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdad57fa640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdac00a33b0 0x7fdac00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d36f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54870/0 (socket says 172.21.15.103:54870) 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 -- 172.21.15.103:0/3868845035 learned_addr learned my addr 172.21.15.103:0/3868845035 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdac00a33b0 0x7fdac00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdad7fff640 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 -- 172.21.15.103:0/3868845035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 msgr2=0x5618f6d41800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d41800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdad57fa640 1 -- 172.21.15.103:0/3646105005 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdacc0f0450 con 0x7fdad8074ee0 2024-09-19T07:39:41.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 -- 172.21.15.103:0/3868845035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 msgr2=0x5618f6d44130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d44130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 -- 172.21.15.103:0/3868845035 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f6c501f0 con 0x5618f6d36b60 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdad7fff640 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d41800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d36f40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdad0013a70 tx=0x7fdad0013f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346b58c79576793c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdabe7fc640 1 -- 172.21.15.103:0/3868845035 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0013480 con 0x5618f6d36b60 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdabe7fc640 1 -- 172.21.15.103:0/3868845035 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0011d40 con 0x5618f6d36b60 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.899+0000 7fdabe7fc640 1 -- 172.21.15.103:0/3868845035 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad001db10 con 0x5618f6d36b60 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 -- 172.21.15.103:0/3868845035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 msgr2=0x5618f6d36f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdac00a33b0 0x7fdac00a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdad0020000 tx=0x7fdad001f000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d36f40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdad0013a70 tx=0x7fdad0013f30 comp rx=0 tx=0).stop 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 -- 172.21.15.103:0/3868845035 shutdown_connections 2024-09-19T07:39:41.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d41800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 --2- 172.21.15.103:0/3868845035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d36f40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 -- 172.21.15.103:0/3868845035 >> 172.21.15.103:0/3868845035 conn(0x5618f6d330f0 msgr2=0x5618f6d334d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 -- 172.21.15.103:0/3868845035 shutdown_connections 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 -- 172.21.15.103:0/3868845035 wait complete. 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.900+0000 7fdae0923740 1 -- start start 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 1 Processor -- start 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d3eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d3f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618f6d44910 con 0x5618f6d36b60 2024-09-19T07:39:41.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5618f6c56f10 con 0x5618f6d41d40 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5618f6d44a90 con 0x5618f6d37480 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d3eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdae0923740 10 client.0 _ll_register_callbacks cb 0x5618f6c1fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d3eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54876/0 (socket says 172.21.15.103:54876) 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 -- 172.21.15.103:0/1894447929 learned_addr learned my addr 172.21.15.103:0/1894447929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad77fe640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d3f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdadcae7640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 -- 172.21.15.103:0/1894447929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 msgr2=0x5618f6d3f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d3f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 -- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 msgr2=0x5618f6d3f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d3f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdad7fff640 1 -- 172.21.15.103:0/1894447929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f6a0d6a0 con 0x5618f6d36b60 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.901+0000 7fdadcae7640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d3f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdad7fff640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d3eb80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdac80106a0 tx=0x7fdac8010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104867ac41fb8fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdab77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdab77fe640 1 -- 172.21.15.103:0/1894447929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8010ea0 con 0x5618f6d36b60 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdab77fe640 1 -- 172.21.15.103:0/1894447929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdac8004230 con 0x5618f6d36b60 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad0011d40 con 0x5618f6d36b60 2024-09-19T07:39:41.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdab77fe640 1 -- 172.21.15.103:0/1894447929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdac8004430 con 0x5618f6d36b60 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 10 client.15855 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618f6d50450 con 0x5618f6d36b60 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdab77fe640 1 -- 172.21.15.103:0/1894447929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0011d40 con 0x5618f6d36b60 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdab77fe640 1 -- 172.21.15.103:0/1894447929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdac800f040 con 0x5618f6d36b60 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 10 client.15855 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 20 client.15855 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6d53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 4 client.15855 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.902+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f6ca97f0 con 0x5618f6d51910 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.903+0000 7fdad77fe640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6d53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.903+0000 7fdab77fe640 10 client.15855 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.903+0000 7fdad77fe640 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6d53db0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30dc03f3359f32a8 server_cookie=b435e997ae20d255 in_seq=0 out_seq=0 2024-09-19T07:39:41.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdab77fe640 1 -- 172.21.15.103:0/1894447929 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618f6ca97f0 con 0x5618f6d51910 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdab77fe640 10 client.15855 handle_command_reply: tid=0 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 client.15855 shutdown 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 msgr2=0x5618f6d3eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d3eb80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdac80106a0 tx=0x7fdac8010b60 comp rx=0 tx=0).stop 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 shutdown_connections 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6d53db0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d41d40 0x5618f6d3f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d37480 0x5618f6d3f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdab77fe640 0 client.15855 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 --2- 172.21.15.103:0/1894447929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d36b60 0x5618f6d3eb80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 >> 172.21.15.103:0/1894447929 conn(0x5618f6d330f0 msgr2=0x5618f6d32270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:41.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 shutdown_connections 2024-09-19T07:39:41.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 1 -- 172.21.15.103:0/1894447929 wait complete. 2024-09-19T07:39:41.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.904+0000 7fdae0923740 20 client.15855 trim_cache size 0 max 16384 2024-09-19T07:39:41.920 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:41 smithi174 ceph-mon[35370]: pgmap v54: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 Processor -- start 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 -- start start 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d3da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d59680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81afa80 con 0x5618f6d37480 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b1680 con 0x5618f6d36b60 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.921+0000 7fdae0923740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b0150 con 0x5618f6d41d40 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d3da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d3da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54880/0 (socket says 172.21.15.103:54880) 2024-09-19T07:39:41.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 -- 172.21.15.103:0/3919511228 learned_addr learned my addr 172.21.15.103:0/3919511228 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad7fff640 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdadcae7640 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d59680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 -- 172.21.15.103:0/3919511228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 msgr2=0x5618f6d59680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d59680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 -- 172.21.15.103:0/3919511228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 msgr2=0x5618f6d395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 -- 172.21.15.103:0/3919511228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f6d32f20 con 0x5618f6d37480 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdad77fe640 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d3da30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0b8c00 tx=0x7fdacc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ab9d4fbd1bc51c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdabdffb640 1 -- 172.21.15.103:0/3919511228 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0d9070 con 0x5618f6d37480 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdabdffb640 1 -- 172.21.15.103:0/3919511228 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc03b580 con 0x5618f6d37480 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.922+0000 7fdabdffb640 1 -- 172.21.15.103:0/3919511228 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc049040 con 0x5618f6d37480 2024-09-19T07:39:41.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 -- 172.21.15.103:0/3919511228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 msgr2=0x5618f6d3da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d3da30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0b8c00 tx=0x7fdacc02a480 comp rx=0 tx=0).stop 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 -- 172.21.15.103:0/3919511228 shutdown_connections 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d3da30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 --2- 172.21.15.103:0/3919511228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 -- 172.21.15.103:0/3919511228 >> 172.21.15.103:0/3919511228 conn(0x5618f6d330f0 msgr2=0x5618f6d3d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 -- 172.21.15.103:0/3919511228 shutdown_connections 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 -- 172.21.15.103:0/3919511228 wait complete. 2024-09-19T07:39:41.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.923+0000 7fdae0923740 1 -- start start 2024-09-19T07:39:41.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 1 Processor -- start 2024-09-19T07:39:41.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d45c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d46180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b0ef0 con 0x5618f6d37480 2024-09-19T07:39:41.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81b0820 con 0x5618f6d36b60 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81ada60 con 0x5618f6d41d40 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdae0923740 10 client.0 _ll_register_callbacks cb 0x7fdac800b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d46180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d46180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54896/0 (socket says 172.21.15.103:54896) 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 -- 172.21.15.103:0/408677286 learned_addr learned my addr 172.21.15.103:0/408677286 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdadcae7640 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad7fff640 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d45c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 -- 172.21.15.103:0/408677286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 msgr2=0x5618f6d466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 -- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 msgr2=0x5618f6d45c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d45c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 -- 172.21.15.103:0/408677286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f6dbb850 con 0x5618f6d37480 2024-09-19T07:39:41.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdad77fe640 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d46180 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0c4ee0 tx=0x7fdacc0c0c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358b5e37cb809e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.924+0000 7fdab77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdab77fe640 1 -- 172.21.15.103:0/408677286 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc076060 con 0x5618f6d37480 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdab77fe640 1 -- 172.21.15.103:0/408677286 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdacc0eddd0 con 0x5618f6d37480 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618f6ca3d00 con 0x5618f6d37480 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdab77fe640 1 -- 172.21.15.103:0/408677286 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdacc049070 con 0x5618f6d37480 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 10 client.15861 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618f6dbcb70 con 0x5618f6d37480 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdab77fe640 1 -- 172.21.15.103:0/408677286 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5618f6ca3d00 con 0x5618f6d37480 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdab77fe640 1 -- 172.21.15.103:0/408677286 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdacc0bbbf0 con 0x5618f6d37480 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 10 client.15861 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:41.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 20 client.15861 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6dbf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 4 client.15861 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618f6c20110 con 0x5618f6d51910 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.925+0000 7fdad7fff640 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6dbf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.926+0000 7fdab77fe640 10 client.15861 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.926+0000 7fdad7fff640 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6dbf3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bab1afb0e45a22e server_cookie=7224ec8e63bb0a05 in_seq=0 out_seq=0 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdab77fe640 1 -- 172.21.15.103:0/408677286 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618f6c20110 con 0x5618f6d51910 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdab77fe640 10 client.15861 handle_command_reply: tid=0 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 client.15861 shutdown 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 msgr2=0x5618f6d46180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d46180 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0c4ee0 tx=0x7fdacc0c0c40 comp rx=0 tx=0).stop 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 shutdown_connections 2024-09-19T07:39:41.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5618f6d51910 0x5618f6dbf3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618f6d41d40 0x5618f6d466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618f6d37480 0x5618f6d46180 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdab77fe640 0 client.15861 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 --2- 172.21.15.103:0/408677286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5618f6d36b60 0x5618f6d45c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 >> 172.21.15.103:0/408677286 conn(0x5618f6d330f0 msgr2=0x5618f6d32270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 shutdown_connections 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 1 -- 172.21.15.103:0/408677286 wait complete. 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.927+0000 7fdae0923740 20 client.15861 trim_cache size 0 max 16384 2024-09-19T07:39:41.933 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:39:41.934 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:39:41.934 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "70b70ced-e91d-40b2-a5c8-d30cd89fb96e", 2024-09-19T07:39:41.934 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:39:41.934 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdac00a33b0 msgr2=0x7fdac00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdac00a33b0 0x7fdac00a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdad0020000 tx=0x7fdad001f000 comp rx=0 tx=0).stop 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 msgr2=0x7fdad819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad819abb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdacc006010 tx=0x7fdacc0042f0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 shutdown_connections 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdac00a33b0 0x7fdac00a5870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad810e420 0x7fdad819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdad8074ee0 0x7fdad819abb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 --2- 172.21.15.103:0/3646105005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdad80745c0 0x7fdad819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:41.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 >> 172.21.15.103:0/3646105005 conn(0x7fdad80fd8c0 msgr2=0x7fdad80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:41.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 shutdown_connections 2024-09-19T07:39:41.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:41.928+0000 7fdade54a640 1 -- 172.21.15.103:0/3646105005 wait complete. 2024-09-19T07:39:42.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:39:42.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:39:42.743 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:42.871 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:39:42.871 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.871 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:39:42.871 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.872 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:42 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:39:42.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:39:42.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:39:42.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:42.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:42 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:39:43.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:42 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 -- 172.21.15.103:0/1087006569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c101de0 msgr2=0x7fb70c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 --2- 172.21.15.103:0/1087006569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c101de0 0x7fb70c1021c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb708009900 tx=0x7fb70802a410 comp rx=0 tx=0).stop 2024-09-19T07:39:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 -- 172.21.15.103:0/1087006569 shutdown_connections 2024-09-19T07:39:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 --2- 172.21.15.103:0/1087006569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb70c106f10 0x7fb70c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 --2- 172.21.15.103:0/1087006569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c102790 0x7fb70c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 --2- 172.21.15.103:0/1087006569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c101de0 0x7fb70c1021c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 -- 172.21.15.103:0/1087006569 >> 172.21.15.103:0/1087006569 conn(0x7fb70c0fd8c0 msgr2=0x7fb70c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 -- 172.21.15.103:0/1087006569 shutdown_connections 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.241+0000 7fb713d17640 1 -- 172.21.15.103:0/1087006569 wait complete. 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 Processor -- start 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 -- start start 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb70c101de0 0x7fb70c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c102790 0x7fb70c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c106f10 0x7fb70c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c111f90 con 0x7fb70c106f10 2024-09-19T07:39:43.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c111e10 con 0x7fb70c101de0 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb713d17640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c112110 con 0x7fb70c102790 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb7122b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c106f10 0x7fb70c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb7122b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c106f10 0x7fb70c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54910/0 (socket says 172.21.15.103:54910) 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb7122b4640 1 -- 172.21.15.103:0/566357340 learned_addr learned my addr 172.21.15.103:0/566357340 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb711ab3640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb70c101de0 0x7fb70c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb7112b2640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c102790 0x7fb70c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.242+0000 7fb7122b4640 1 -- 172.21.15.103:0/566357340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c102790 msgr2=0x7fb70c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb7122b4640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c102790 0x7fb70c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb7122b4640 1 -- 172.21.15.103:0/566357340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb70c101de0 msgr2=0x7fb70c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb7122b4640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb70c101de0 0x7fb70c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb7122b4640 1 -- 172.21.15.103:0/566357340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c19d440 con 0x7fb70c106f10 2024-09-19T07:39:43.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb7112b2640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c102790 0x7fb70c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb7122b4640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c106f10 0x7fb70c19cd60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb70400eaa0 tx=0x7fb70400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb702ffd640 1 -- 172.21.15.103:0/566357340 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70400cdb0 con 0x7fb70c106f10 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb702ffd640 1 -- 172.21.15.103:0/566357340 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb704005020 con 0x7fb70c106f10 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb702ffd640 1 -- 172.21.15.103:0/566357340 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7040116b0 con 0x7fb70c106f10 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c19d730 con 0x7fb70c106f10 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.243+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c1a5010 con 0x7fb70c106f10 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.244+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c1021c0 con 0x7fb70c106f10 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.245+0000 7fb702ffd640 1 -- 172.21.15.103:0/566357340 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb704005560 con 0x7fb70c106f10 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.246+0000 7fb702ffd640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb6f80a33b0 0x7fb6f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.246+0000 7fb711ab3640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb6f80a33b0 0x7fb6f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.246+0000 7fb702ffd640 1 -- 172.21.15.103:0/566357340 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb7040cff90 con 0x7fb70c106f10 2024-09-19T07:39:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.246+0000 7fb711ab3640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb6f80a33b0 0x7fb6f80a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb708009230 tx=0x7fb7080058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.250+0000 7fb702ffd640 1 -- 172.21.15.103:0/566357340 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb704016030 con 0x7fb70c106f10 2024-09-19T07:39:43.338 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:43.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.458+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb70c02d7c0 con 0x7fb70c106f10 2024-09-19T07:39:43.461 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:39:43.461 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":30,"btime":"2024-09-19T07:39:09:495445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:39:08.515410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:39:43.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.459+0000 7fb702ffd640 1 -- 172.21.15.103:0/566357340 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11998 (secure 0 0 0) 0x7fb70409c980 con 0x7fb70c106f10 2024-09-19T07:39:43.461 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 30 2024-09-19T07:39:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb6f80a33b0 msgr2=0x7fb6f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb6f80a33b0 0x7fb6f80a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb708009230 tx=0x7fb7080058d0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c106f10 msgr2=0x7fb70c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c106f10 0x7fb70c19cd60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb70400eaa0 tx=0x7fb70400ef60 comp rx=0 tx=0).stop 2024-09-19T07:39:43.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 shutdown_connections 2024-09-19T07:39:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb6f80a33b0 0x7fb6f80a5870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb70c106f10 0x7fb70c19cd60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c102790 0x7fb70c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 --2- 172.21.15.103:0/566357340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb70c101de0 0x7fb70c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:43.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.461+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 >> 172.21.15.103:0/566357340 conn(0x7fb70c0fd8c0 msgr2=0x7fb70c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.462+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 shutdown_connections 2024-09-19T07:39:43.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:43.462+0000 7fb713d17640 1 -- 172.21.15.103:0/566357340 wait complete. 2024-09-19T07:39:43.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:43 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:39:43.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:43 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:43 smithi103 ceph-mon[31201]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-09-19T07:39:43.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:43 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/566357340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:43 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:39:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:43 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:43 smithi174 ceph-mon[35370]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-09-19T07:39:43.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:43 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/566357340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:44.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 -- 172.21.15.103:0/3380728834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 msgr2=0x7f2a9c10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/3380728834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c10eaf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009cd0 tx=0x7f2a9402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 -- 172.21.15.103:0/3380728834 shutdown_connections 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/3380728834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c10eaf0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/3380728834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9c100ca0 0x7f2a9c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/3380728834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c1002f0 0x7f2a9c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 -- 172.21.15.103:0/3380728834 >> 172.21.15.103:0/3380728834 conn(0x7f2a9c077210 msgr2=0x7f2a9c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 -- 172.21.15.103:0/3380728834 shutdown_connections 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 -- 172.21.15.103:0/3380728834 wait complete. 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.007+0000 7f2aa1df9640 1 Processor -- start 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2aa1df9640 1 -- start start 2024-09-19T07:39:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2aa1df9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9c1002f0 0x7f2a9c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2aa1df9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c100ca0 0x7f2a9c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2aa1df9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2aa1df9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c111fa0 con 0x7f2a9c106f50 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2aa1df9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c111e20 con 0x7f2a9c1002f0 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2aa1df9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c112120 con 0x7f2a9c100ca0 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54934/0 (socket says 172.21.15.103:54934) 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 -- 172.21.15.103:0/1720875586 learned_addr learned my addr 172.21.15.103:0/1720875586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:44.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c100ca0 0x7f2a9c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9c1002f0 0x7f2a9c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 -- 172.21.15.103:0/1720875586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c100ca0 msgr2=0x7f2a9c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c100ca0 0x7f2a9c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 -- 172.21.15.103:0/1720875586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9c1002f0 msgr2=0x7f2a9c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9c1002f0 0x7f2a9c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 -- 172.21.15.103:0/1720875586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c19d410 con 0x7f2a9c106f50 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9c1002f0 0x7f2a9c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.008+0000 7f2a9bfff640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c19cd30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2a94005cc0 tx=0x7f2a94003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.009+0000 7f2a98ff9640 1 -- 172.21.15.103:0/1720875586 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94004040 con 0x7f2a9c106f50 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.009+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c19d6a0 con 0x7f2a9c106f50 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.009+0000 7f2a98ff9640 1 -- 172.21.15.103:0/1720875586 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a940046c0 con 0x7f2a9c106f50 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.009+0000 7f2a98ff9640 1 -- 172.21.15.103:0/1720875586 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94046480 con 0x7f2a9c106f50 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.009+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c1a4fe0 con 0x7f2a9c106f50 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.011+0000 7f2a98ff9640 1 -- 172.21.15.103:0/1720875586 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2a940305a0 con 0x7f2a9c106f50 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.011+0000 7f2aa41d2740 1 Processor -- start 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.011+0000 7f2aa41d2740 1 -- start start 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986b360 0x564dc986b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc986bc80 0x564dc987a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc987aa90 0x564dc987ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564dc97d12e0 con 0x564dc986b360 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564dc9778ce0 con 0x564dc987aa90 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564dc97cd2a0 con 0x564dc986bc80 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986b360 0x564dc986b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a98ff9640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a840a33b0 0x7f2a840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986b360 0x564dc986b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54950/0 (socket says 172.21.15.103:54950) 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 -- 172.21.15.103:0/1258361423 learned_addr learned my addr 172.21.15.103:0/1258361423 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9bfff640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a840a33b0 0x7f2a840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc986bc80 0x564dc987a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc987aa90 0x564dc987ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 -- 172.21.15.103:0/1258361423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc986bc80 msgr2=0x564dc987a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc986bc80 0x564dc987a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 -- 172.21.15.103:0/1258361423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc987aa90 msgr2=0x564dc987ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc987aa90 0x564dc987ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a98ff9640 1 -- 172.21.15.103:0/1720875586 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2a9402e030 con 0x7f2a9c106f50 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 -- 172.21.15.103:0/1258361423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc975d440 con 0x564dc986b360 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc986bc80 0x564dc987a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986b360 0x564dc986b740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000bf70 tx=0x7f2a9000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474e05290932c897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a9bfff640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a840a33b0 0x7f2a840a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a940b9c80 tx=0x7f2a94088040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a81ffb640 1 -- 172.21.15.103:0/1258361423 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90004030 con 0x564dc986b360 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.012+0000 7f2a81ffb640 1 -- 172.21.15.103:0/1258361423 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a90005500 con 0x564dc986b360 2024-09-19T07:39:44.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2a81ffb640 1 -- 172.21.15.103:0/1258361423 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90004bc0 con 0x564dc986b360 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1258361423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986b360 msgr2=0x564dc986b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986b360 0x564dc986b740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000bf70 tx=0x7f2a9000b780 comp rx=0 tx=0).stop 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1258361423 shutdown_connections 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc987aa90 0x564dc987ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc986bc80 0x564dc987a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1258361423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986b360 0x564dc986b740 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1258361423 >> 172.21.15.103:0/1258361423 conn(0x564dc9867a30 msgr2=0x564dc9867e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1258361423 shutdown_connections 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1258361423 wait complete. 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.013+0000 7f2aa41d2740 1 -- start start 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 1 Processor -- start 2024-09-19T07:39:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9877730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9877d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564dc987d660 con 0x564dc986bc80 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564dc978c050 con 0x564dc986b360 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564dc987d7e0 con 0x564dc987aa90 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2aa41d2740 10 client.0 _ll_register_callbacks cb 0x564dc97cc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9877730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9877730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54960/0 (socket says 172.21.15.103:54960) 2024-09-19T07:39:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/2045488598 learned_addr learned my addr 172.21.15.103:0/2045488598 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9affd640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9877d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9affd640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/2045488598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 msgr2=0x564dc9877d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9877d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 msgr2=0x564dc987a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/2045488598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc95423c0 con 0x564dc986bc80 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.014+0000 7f2a9affd640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9877d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9877730 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bfa0 tx=0x7f2a8c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dab16aa000fe800 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:44.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2a7affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2a7affd640 1 -- 172.21.15.103:0/2045488598 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004230 con 0x564dc986bc80 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2a7affd640 1 -- 172.21.15.103:0/2045488598 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2a8c0043d0 con 0x564dc986bc80 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a90005500 con 0x564dc986bc80 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2a7affd640 1 -- 172.21.15.103:0/2045488598 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a8c012720 con 0x564dc986bc80 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 10 client.15879 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564dc9889150 con 0x564dc986bc80 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2a7affd640 1 -- 172.21.15.103:0/2045488598 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a90005500 con 0x564dc986bc80 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2a7affd640 1 -- 172.21.15.103:0/2045488598 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2a8c01c420 con 0x564dc986bc80 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 10 client.15879 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 20 client.15879 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:44.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc988a660 0x564dc988ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 4 client.15879 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.015+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dc97de850 con 0x564dc988a660 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.016+0000 7f2a9affd640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc988a660 0x564dc988ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.016+0000 7f2a9affd640 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc988a660 0x564dc988ca50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64391b25b167b863 server_cookie=e2246f3c8377d717 in_seq=0 out_seq=0 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.016+0000 7f2a7affd640 10 client.15879 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2a7affd640 1 -- 172.21.15.103:0/2045488598 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564dc97de850 con 0x564dc988a660 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2a7affd640 10 client.15879 handle_command_reply: tid=0 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 client.15879 shutdown 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 msgr2=0x564dc9877730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9877730 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00bfa0 tx=0x7f2a8c011870 comp rx=0 tx=0).stop 2024-09-19T07:39:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 shutdown_connections 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc988a660 0x564dc988ca50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9877d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9877730 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2a7affd640 0 client.15879 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/2045488598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 >> 172.21.15.103:0/2045488598 conn(0x564dc9867a30 msgr2=0x564dc986d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 shutdown_connections 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 1 -- 172.21.15.103:0/2045488598 wait complete. 2024-09-19T07:39:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.017+0000 7f2aa41d2740 20 client.15879 trim_cache size 0 max 16384 2024-09-19T07:39:44.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 Processor -- start 2024-09-19T07:39:44.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 -- start start 2024-09-19T07:39:44.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc9886900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9892340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9892880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1ab640 con 0x564dc986bc80 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1ad240 con 0x564dc986b360 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1abd10 con 0x564dc987aa90 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.034+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9892340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9892340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54970/0 (socket says 172.21.15.103:54970) 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/3970309750 learned_addr learned my addr 172.21.15.103:0/3970309750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9affd640 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc9886900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9affd640 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9892880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/3970309750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 msgr2=0x564dc9892880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9892880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/3970309750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 msgr2=0x564dc9886900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc9886900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/3970309750 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc98780b0 con 0x564dc986bc80 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9892340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c006010 tx=0x7f2a8c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae11f35b80f7b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a817fa640 1 -- 172.21.15.103:0/3970309750 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c032070 con 0x564dc986bc80 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a817fa640 1 -- 172.21.15.103:0/3970309750 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c01c620 con 0x564dc986bc80 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.035+0000 7f2a817fa640 1 -- 172.21.15.103:0/3970309750 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c029040 con 0x564dc986bc80 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 -- 172.21.15.103:0/3970309750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 msgr2=0x564dc9892340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9892340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c006010 tx=0x7f2a8c01ccc0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 -- 172.21.15.103:0/3970309750 shutdown_connections 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987aa90 0x564dc9892880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc986bc80 0x564dc9892340 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/3970309750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc9886900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 -- 172.21.15.103:0/3970309750 >> 172.21.15.103:0/3970309750 conn(0x564dc9867a30 msgr2=0x564dc9872d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 -- 172.21.15.103:0/3970309750 shutdown_connections 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.036+0000 7f2aa41d2740 1 -- 172.21.15.103:0/3970309750 wait complete. 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 -- start start 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 Processor -- start 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc987aa90 0x564dc987efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987f500 0x564dc98f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1acab0 con 0x564dc987aa90 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1ac3e0 con 0x564dc986b360 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1a9620 con 0x564dc987f500 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.037+0000 7f2aa41d2740 10 client.0 _ll_register_callbacks cb 0x7f2a8c00b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc987aa90 0x564dc987efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc987aa90 0x564dc987efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54976/0 (socket says 172.21.15.103:54976) 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/1135486266 learned_addr learned my addr 172.21.15.103:0/1135486266 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987f500 0x564dc98f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/1135486266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987f500 msgr2=0x564dc98f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987f500 0x564dc98f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 msgr2=0x564dc987ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 -- 172.21.15.103:0/1135486266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc98f49b0 con 0x564dc987aa90 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a9b7fe640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc987aa90 0x564dc987efc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c01c8c0 tx=0x7f2a8c00b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ccbfa911459c30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a7affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a7affd640 1 -- 172.21.15.103:0/1135486266 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c029050 con 0x564dc987aa90 2024-09-19T07:39:44.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a7affd640 1 -- 172.21.15.103:0/1135486266 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2a8c005110 con 0x564dc987aa90 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8c01c620 con 0x564dc987aa90 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2a7affd640 1 -- 172.21.15.103:0/1135486266 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a8c032070 con 0x564dc987aa90 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2aa41d2740 10 client.15885 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.038+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564dc98f5c80 con 0x564dc987aa90 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2a7affd640 1 -- 172.21.15.103:0/1135486266 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c01c620 con 0x564dc987aa90 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2a7affd640 1 -- 172.21.15.103:0/1135486266 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2a8c02c070 con 0x564dc987aa90 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2aa41d2740 10 client.15885 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2aa41d2740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2aa41d2740 20 client.15885 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc98f6350 0x564dc98f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:44.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2aa41d2740 4 client.15885 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564dc97d8d60 con 0x564dc98f6350 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.039+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc98f6350 0x564dc98f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2a9affd640 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc98f6350 0x564dc98f8770 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f98a417fe1c544 server_cookie=9eb07bd72e65811d in_seq=0 out_seq=0 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2a7affd640 10 client.15885 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2a7affd640 1 -- 172.21.15.103:0/1135486266 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564dc97d8d60 con 0x564dc98f6350 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2a7affd640 10 client.15885 handle_command_reply: tid=0 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 client.15885 shutdown 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc987aa90 msgr2=0x564dc987efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc987aa90 0x564dc987efc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c01c8c0 tx=0x7f2a8c00b600 comp rx=0 tx=0).stop 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 shutdown_connections 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564dc98f6350 0x564dc98f8770 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564dc987f500 0x564dc98f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dc987aa90 0x564dc987efc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 --2- 172.21.15.103:0/1135486266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564dc986b360 0x564dc987ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 >> 172.21.15.103:0/1135486266 conn(0x564dc9867a30 msgr2=0x564dc9878f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2a7affd640 0 client.15885 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 shutdown_connections 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 1 -- 172.21.15.103:0/1135486266 wait complete. 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.040+0000 7f2aa41d2740 20 client.15885 trim_cache size 0 max 16384 2024-09-19T07:39:44.053 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: "70b70ced-e91d-40b2-a5c8-d30cd89fb96e": { 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "70b70ced-e91d-40b2-a5c8-d30cd89fb96e", 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:44.054 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a840a33b0 msgr2=0x7f2a840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a840a33b0 0x7f2a840a5870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a940b9c80 tx=0x7f2a94088040 comp rx=0 tx=0).stop 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 msgr2=0x7f2a9c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c19cd30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2a94005cc0 tx=0x7f2a94003970 comp rx=0 tx=0).stop 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 shutdown_connections 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a840a33b0 0x7f2a840a5870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a9c106f50 0x7f2a9c19cd30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c100ca0 0x7f2a9c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 --2- 172.21.15.103:0/1720875586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a9c1002f0 0x7f2a9c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 >> 172.21.15.103:0/1720875586 conn(0x7f2a9c077210 msgr2=0x7f2a9c105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:44.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 shutdown_connections 2024-09-19T07:39:44.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:44.041+0000 7f2aa1df9640 1 -- 172.21.15.103:0/1720875586 wait complete. 2024-09-19T07:39:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:43 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:39:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:43 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:39:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:43 smithi134 ceph-mon[35414]: pgmap v55: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-09-19T07:39:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:43 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/566357340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:44.552 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id e32035a --release --await' 2024-09-19T07:39:44.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id e32035a --release --await 2024-09-19T07:39:44.879 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:45.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.364+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/2029243645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d81008d0 msgr2=0x7fe3d8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.364+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/2029243645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d81008d0 0x7fe3d8100cb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009900 tx=0x7fe3d402a410 comp rx=0 tx=0).stop 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.364+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/2029243645 shutdown_connections 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.364+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/2029243645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3d810e480 0x7fe3d8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.364+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/2029243645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3d8101280 0x7fe3d810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.364+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/2029243645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d81008d0 0x7fe3d8100cb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.364+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/2029243645 >> 172.21.15.103:0/2029243645 conn(0x7fe3d80fc7a0 msgr2=0x7fe3d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.365+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/2029243645 shutdown_connections 2024-09-19T07:39:45.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.365+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/2029243645 wait complete. 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.365+0000 7fe3dfd3c640 1 Processor -- start 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.365+0000 7fe3dfd3c640 1 -- start start 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3d81008d0 0x7fe3d81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d8101280 0x7fe3d81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3d810e480 0x7fe3d819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8114260 con 0x7fe3d8101280 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81140e0 con 0x7fe3d81008d0 2024-09-19T07:39:45.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81143e0 con 0x7fe3d810e480 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d8101280 0x7fe3d81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d8101280 0x7fe3d81a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54998/0 (socket says 172.21.15.103:54998) 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/4087115309 learned_addr learned my addr 172.21.15.103:0/4087115309 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3d810e480 0x7fe3d819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3d81008d0 0x7fe3d81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/4087115309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3d810e480 msgr2=0x7fe3d819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3d810e480 0x7fe3d819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/4087115309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3d81008d0 msgr2=0x7fe3d81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3d81008d0 0x7fe3d81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/4087115309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d819af40 con 0x7fe3d8101280 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d8101280 0x7fe3d81a0b40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b4a0 tx=0x7fe3c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:45.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3ceffd640 1 -- 172.21.15.103:0/4087115309 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8014070 con 0x7fe3d8101280 2024-09-19T07:39:45.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3ceffd640 1 -- 172.21.15.103:0/4087115309 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c8010af0 con 0x7fe3d8101280 2024-09-19T07:39:45.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d819b230 con 0x7fe3d8101280 2024-09-19T07:39:45.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3ceffd640 1 -- 172.21.15.103:0/4087115309 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8017d10 con 0x7fe3d8101280 2024-09-19T07:39:45.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.366+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81a7580 con 0x7fe3d8101280 2024-09-19T07:39:45.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.369+0000 7fe3e2115740 1 Processor -- start 2024-09-19T07:39:45.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.369+0000 7fe3ceffd640 1 -- 172.21.15.103:0/4087115309 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe3c8017070 con 0x7fe3d8101280 2024-09-19T07:39:45.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3e2115740 1 -- start start 2024-09-19T07:39:45.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce060 0x55b56ecce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecd8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecdb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3e2115740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b56ec33620 con 0x55b56ecce980 2024-09-19T07:39:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3e2115740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b56ebdae50 con 0x55b56ecd9240 2024-09-19T07:39:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3e2115740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b56ec2f5e0 con 0x55b56ecce060 2024-09-19T07:39:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecd8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecd8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55012/0 (socket says 172.21.15.103:55012) 2024-09-19T07:39:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 -- 172.21.15.103:0/3270876915 learned_addr learned my addr 172.21.15.103:0/3270876915 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce060 0x55b56ecce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecdb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 -- 172.21.15.103:0/3270876915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce060 msgr2=0x55b56ecce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce060 0x55b56ecce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 -- 172.21.15.103:0/3270876915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 msgr2=0x55b56ecdb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecdb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3ddad8640 1 -- 172.21.15.103:0/3270876915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56ebcbd40 con 0x55b56ecce980 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.370+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecdb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.371+0000 7fe3ceffd640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3c40a3430 0x7fe3c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.371+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecd8d00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4033000 tx=0x7fe3d4031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f01521d436d6e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.371+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3c40a3430 0x7fe3c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.371+0000 7fe3ceffd640 1 -- 172.21.15.103:0/4087115309 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe3c80cefa0 con 0x7fe3d8101280 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.371+0000 7fe3c3fff640 1 -- 172.21.15.103:0/3270876915 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d403f070 con 0x55b56ecce980 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.371+0000 7fe3c3fff640 1 -- 172.21.15.103:0/3270876915 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d40035f0 con 0x55b56ecce980 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.371+0000 7fe3c3fff640 1 -- 172.21.15.103:0/3270876915 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4035070 con 0x55b56ecce980 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 -- 172.21.15.103:0/3270876915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 msgr2=0x55b56ecd8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3c40a3430 0x7fe3c40a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe3d806a0f0 tx=0x7fe3d000e040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecd8d00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4033000 tx=0x7fe3d4031660 comp rx=0 tx=0).stop 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 -- 172.21.15.103:0/3270876915 shutdown_connections 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecdb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecd8d00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3270876915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce060 0x55b56ecce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 -- 172.21.15.103:0/3270876915 >> 172.21.15.103:0/3270876915 conn(0x55b56ecca5f0 msgr2=0x55b56ecca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 -- 172.21.15.103:0/3270876915 shutdown_connections 2024-09-19T07:39:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 -- 172.21.15.103:0/3270876915 wait complete. 2024-09-19T07:39:45.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 -- start start 2024-09-19T07:39:45.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.372+0000 7fe3e2115740 1 Processor -- start 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ecd60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce980 0x55b56ecd6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecd6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3e2115740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b56ecdbe10 con 0x55b56ecce060 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3e2115740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b56ebbbf10 con 0x55b56ecd9240 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3e2115740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b56ecdbf90 con 0x55b56ecce980 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3e2115740 10 client.0 _ll_register_callbacks cb 0x55b56ebdfdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ecd60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ecd60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55014/0 (socket says 172.21.15.103:55014) 2024-09-19T07:39:45.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 -- 172.21.15.103:0/223338168 learned_addr learned my addr 172.21.15.103:0/223338168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 -- 172.21.15.103:0/223338168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce980 msgr2=0x55b56ecd6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce980 0x55b56ecd6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 -- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 msgr2=0x55b56ecd6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecd6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecd6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 -- 172.21.15.103:0/223338168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56e942580 con 0x55b56ecce060 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ecd60d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005ec0 tx=0x7fe3d40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aadfea5dfbcd51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.373+0000 7fe3c17fa640 1 -- 172.21.15.103:0/223338168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4035070 con 0x55b56ecce060 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3c17fa640 1 -- 172.21.15.103:0/223338168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe3d40041f0 con 0x55b56ecce060 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d40035f0 con 0x55b56ecce060 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3c17fa640 1 -- 172.21.15.103:0/223338168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3d4054050 con 0x55b56ecce060 2024-09-19T07:39:45.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 10 client.15897 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b56ea0c960 con 0x55b56ecce060 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3c17fa640 1 -- 172.21.15.103:0/223338168 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d40035f0 con 0x55b56ecce060 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3c17fa640 1 -- 172.21.15.103:0/223338168 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe3d4004aa0 con 0x55b56ecce060 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 10 client.15897 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 10 client.15897 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 20 client.15897 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ece8c00 0x55b56eceaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 4 client.15897 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b56ec40b90 con 0x55b56ece8c00 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.374+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ece8c00 0x55b56eceaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.375+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ece8c00 0x55b56eceaff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6941cfe51a69fb68 server_cookie=f845341badaf34e5 in_seq=0 out_seq=0 2024-09-19T07:39:45.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.375+0000 7fe3c17fa640 10 client.15897 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.375+0000 7fe3c17fa640 1 -- 172.21.15.103:0/223338168 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b56ec40b90 con 0x55b56ece8c00 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.375+0000 7fe3c17fa640 10 client.15897 handle_command_reply: tid=0 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.375+0000 7fe3e2115740 1 client.15897 shutdown 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 msgr2=0x55b56ecd60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ecd60d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005ec0 tx=0x7fe3d40039d0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 shutdown_connections 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ece8c00 0x55b56eceaff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecd9240 0x55b56ecd6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecce980 0x55b56ecd6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3c17fa640 0 client.15897 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:45.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 --2- 172.21.15.103:0/223338168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ecd60d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 >> 172.21.15.103:0/223338168 conn(0x55b56ecca5f0 msgr2=0x55b56ecd02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:45.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 shutdown_connections 2024-09-19T07:39:45.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 1 -- 172.21.15.103:0/223338168 wait complete. 2024-09-19T07:39:45.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.376+0000 7fe3e2115740 20 client.15897 trim_cache size 0 max 16384 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.392+0000 7fe3e2115740 1 Processor -- start 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.392+0000 7fe3e2115740 1 -- start start 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce060 0x55b56ecc9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecf05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecf0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3e2115740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81afc80 con 0x55b56ecce980 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3e2115740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b1880 con 0x55b56ecce060 2024-09-19T07:39:45.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3e2115740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b0350 con 0x55b56ecd9240 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecf05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecf05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55026/0 (socket says 172.21.15.103:55026) 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/3669019751 learned_addr learned my addr 172.21.15.103:0/3669019751 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce060 0x55b56ecc9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecf0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/3669019751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 msgr2=0x55b56ecf0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecf0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/3669019751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce060 msgr2=0x55b56ecc9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce060 0x55b56ecc9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 -- 172.21.15.103:0/3669019751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56ecca420 con 0x55b56ecce980 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce060 0x55b56ecc9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:45.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecf0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecf05c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8017fd0 tx=0x7fe3c80d14b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee985468eb96e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3c37fe640 1 -- 172.21.15.103:0/3669019751 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c801e040 con 0x55b56ecce980 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3c37fe640 1 -- 172.21.15.103:0/3669019751 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c80cce50 con 0x55b56ecce980 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.393+0000 7fe3c37fe640 1 -- 172.21.15.103:0/3669019751 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80cc980 con 0x55b56ecce980 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 -- 172.21.15.103:0/3669019751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 msgr2=0x55b56ecf05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecf05c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8017fd0 tx=0x7fe3c80d14b0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 -- 172.21.15.103:0/3669019751 shutdown_connections 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecf0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce980 0x55b56ecf05c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 --2- 172.21.15.103:0/3669019751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce060 0x55b56ecc9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 -- 172.21.15.103:0/3669019751 >> 172.21.15.103:0/3669019751 conn(0x55b56ecca5f0 msgr2=0x55b56ecc8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:45.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 -- 172.21.15.103:0/3669019751 shutdown_connections 2024-09-19T07:39:45.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 -- 172.21.15.103:0/3669019751 wait complete. 2024-09-19T07:39:45.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 -- start start 2024-09-19T07:39:45.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.394+0000 7fe3e2115740 1 Processor -- start 2024-09-19T07:39:45.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ece16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce980 0x55b56ecdda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3e2115740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3e2115740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b10f0 con 0x55b56ecce060 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3e2115740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b0a20 con 0x55b56ecce980 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3e2115740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81adc60 con 0x55b56ecd9240 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3e2115740 10 client.0 _ll_register_callbacks cb 0x7fe3d4003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ece16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ece16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55034/0 (socket says 172.21.15.103:55034) 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 -- 172.21.15.103:0/1776998963 learned_addr learned my addr 172.21.15.103:0/1776998963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce980 0x55b56ecdda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3de2d9640 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 -- 172.21.15.103:0/1776998963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 msgr2=0x55b56ecddf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecddf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 -- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce980 msgr2=0x55b56ecdda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:45.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce980 0x55b56ecdda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 -- 172.21.15.103:0/1776998963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56ecd0ed0 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.395+0000 7fe3ddad8640 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ece16e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4031550 tx=0x7fe3d4003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cc7b781c4f2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3c17fa640 1 -- 172.21.15.103:0/1776998963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d405d060 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3c17fa640 1 -- 172.21.15.103:0/1776998963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe3d4003a40 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c80cce50 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3c17fa640 1 -- 172.21.15.103:0/1776998963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3d4035040 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 10 client.15903 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b56ed540f0 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3c17fa640 1 -- 172.21.15.103:0/1776998963 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c80cce50 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3c17fa640 1 -- 172.21.15.103:0/1776998963 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe3d403b8d0 con 0x55b56ecce060 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 10 client.15903 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:45.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 10 client.15903 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:45.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 20 client.15903 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:45.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ed54870 0x55b56ed56c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:45.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 4 client.15903 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e32035a", "release": true, "await": true}] 2024-09-19T07:39:45.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.396+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e32035a", "release": true, "await": true}) -- 0x55b56ec3b0a0 con 0x55b56ed54870 2024-09-19T07:39:45.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.397+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ed54870 0x55b56ed56c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:45.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.397+0000 7fe3c17fa640 10 client.15903 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:45.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.397+0000 7fe3dd2d7640 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ed54870 0x55b56ed56c90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9626267eecf7c01 server_cookie=b0171d9dbad8e9d7 in_seq=0 out_seq=0 2024-09-19T07:39:45.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3c17fa640 1 -- 172.21.15.103:0/1776998963 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55b56ec3b0a0 con 0x55b56ed54870 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3c17fa640 10 client.15903 handle_command_reply: tid=0 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 client.15903 shutdown 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 msgr2=0x55b56ece16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ece16e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4031550 tx=0x7fe3d4003f40 comp rx=0 tx=0).stop 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 shutdown_connections 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b56ed54870 0x55b56ed56c90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56ecd9240 0x55b56ecddf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b56ecce980 0x55b56ecdda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 --2- 172.21.15.103:0/1776998963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b56ecce060 0x55b56ece16e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 >> 172.21.15.103:0/1776998963 conn(0x55b56ecca5f0 msgr2=0x55b56ecd6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.429+0000 7fe3c17fa640 0 client.15903 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.430+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 shutdown_connections 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.430+0000 7fe3e2115740 1 -- 172.21.15.103:0/1776998963 wait complete. 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.430+0000 7fe3e2115740 20 client.15903 trim_cache size 0 max 16384 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 30, 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 8, 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:39:45.432 INFO:teuthology.orchestra.run.smithi103.stdout: "e32035a": { 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 8, 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:45.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:39:45.434 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:45.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:45.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:45.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:45.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:45.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:45.434 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:39:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3c40a3430 msgr2=0x7fe3c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3c40a3430 0x7fe3c40a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe3d806a0f0 tx=0x7fe3d000e040 comp rx=0 tx=0).stop 2024-09-19T07:39:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d8101280 msgr2=0x7fe3d81a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d8101280 0x7fe3d81a0b40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b4a0 tx=0x7fe3c800b960 comp rx=0 tx=0).stop 2024-09-19T07:39:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 shutdown_connections 2024-09-19T07:39:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3c40a3430 0x7fe3c40a58f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3d810e480 0x7fe3d819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3d8101280 0x7fe3d81a0b40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 --2- 172.21.15.103:0/4087115309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3d81008d0 0x7fe3d81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:45.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 >> 172.21.15.103:0/4087115309 conn(0x7fe3d80fc7a0 msgr2=0x7fe3d80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:45.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 shutdown_connections 2024-09-19T07:39:45.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:45.431+0000 7fe3dfd3c640 1 -- 172.21.15.103:0/4087115309 wait complete. 2024-09-19T07:39:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 70.0 s, 20.0 tps, lat 14654.114 ms stddev 9130.928 2024-09-19T07:39:45.853 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:45 smithi103 ceph-mon[31201]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-19T07:39:45.891 INFO:tasks.cephfs.filesystem:scrub status for tag:70b70ced-e91d-40b2-a5c8-d30cd89fb96e - {'path': '/', 'tag': '70b70ced-e91d-40b2-a5c8-d30cd89fb96e', 'options': 'recursive,force'} 2024-09-19T07:39:45.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:45 smithi174 ceph-mon[35370]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-19T07:39:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:45 smithi134 ceph-mon[35414]: pgmap v56: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-09-19T07:39:46.520 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e32035a', total seconds elapsed: 24.5 2024-09-19T07:39:46.521 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T07:39:46.521 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:39:46.877 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 -- 172.21.15.103:0/4108427836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244106a70 msgr2=0x7f2244106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 --2- 172.21.15.103:0/4108427836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244106a70 0x7f2244106e50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f223c009cd0 tx=0x7f223c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 -- 172.21.15.103:0/4108427836 shutdown_connections 2024-09-19T07:39:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 --2- 172.21.15.103:0/4108427836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244101410 0x7f224410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 --2- 172.21.15.103:0/4108427836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2244100a70 0x7f2244100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 --2- 172.21.15.103:0/4108427836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244106a70 0x7f2244106e50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 -- 172.21.15.103:0/4108427836 >> 172.21.15.103:0/4108427836 conn(0x7f22440fc790 msgr2=0x7f22440febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 -- 172.21.15.103:0/4108427836 shutdown_connections 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.460+0000 7f224b6e6640 1 -- 172.21.15.103:0/4108427836 wait complete. 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 Processor -- start 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 -- start start 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244100a70 0x7f2244198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2244101410 0x7f2244198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244106a70 0x7f224419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2244110e70 con 0x7f2244100a70 2024-09-19T07:39:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2244110cf0 con 0x7f2244101410 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f224b6e6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2244110ff0 con 0x7f2244106a70 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244100a70 0x7f2244198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249c83640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244106a70 0x7f224419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244100a70 0x7f2244198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55044/0 (socket says 172.21.15.103:55044) 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2248c81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2244101410 0x7f2244198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249482640 1 -- 172.21.15.103:0/910886518 learned_addr learned my addr 172.21.15.103:0/910886518 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249c83640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244106a70 0x7f224419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:53654/0 (socket says 172.21.15.103:53654) 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249c83640 1 -- 172.21.15.103:0/910886518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2244101410 msgr2=0x7f2244198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249c83640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2244101410 0x7f2244198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249c83640 1 -- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244100a70 msgr2=0x7f2244198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249c83640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244100a70 0x7f2244198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249c83640 1 -- 172.21.15.103:0/910886518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224419d400 con 0x7f2244106a70 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2249482640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244100a70 0x7f2244198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.461+0000 7f2248c81640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2244101410 0x7f2244198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.462+0000 7f2249c83640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244106a70 0x7f224419cd20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f223400e970 tx=0x7f223400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.462+0000 7f223a7fc640 1 -- 172.21.15.103:0/910886518 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223400ccc0 con 0x7f2244106a70 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.462+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224419d6f0 con 0x7f2244106a70 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.462+0000 7f223a7fc640 1 -- 172.21.15.103:0/910886518 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2234005020 con 0x7f2244106a70 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.462+0000 7f223a7fc640 1 -- 172.21.15.103:0/910886518 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234004a00 con 0x7f2244106a70 2024-09-19T07:39:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.462+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22441a4fd0 con 0x7f2244106a70 2024-09-19T07:39:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.464+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2244106e50 con 0x7f2244106a70 2024-09-19T07:39:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.465+0000 7f223a7fc640 1 -- 172.21.15.103:0/910886518 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2234004ba0 con 0x7f2244106a70 2024-09-19T07:39:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.467+0000 7f223a7fc640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f22300a3430 0x7f22300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.467+0000 7f223a7fc640 1 -- 172.21.15.103:0/910886518 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f22340cfc60 con 0x7f2244106a70 2024-09-19T07:39:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.468+0000 7f2249482640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f22300a3430 0x7f22300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.468+0000 7f2249482640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f22300a3430 0x7f22300a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f223c009430 tx=0x7f223c009360 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.470+0000 7f223a7fc640 1 -- 172.21.15.103:0/910886518 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f223409c5f0 con 0x7f2244106a70 2024-09-19T07:39:47.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:47 smithi103 ceph-mon[31201]: pgmap v57: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 859 KiB/s wr, 69 op/s 2024-09-19T07:39:47.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.805+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22441a5310 con 0x7f2244106a70 2024-09-19T07:39:47.807 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:39:47.807 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":30,"btime":"2024-09-19T07:39:09:495445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:39:08.515410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:39:47.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.806+0000 7f223a7fc640 1 -- 172.21.15.103:0/910886518 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11998 (secure 0 0 0) 0x7f223409c5f0 con 0x7f2244106a70 2024-09-19T07:39:47.808 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 30 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f22300a3430 msgr2=0x7f22300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f22300a3430 0x7f22300a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f223c009430 tx=0x7f223c009360 comp rx=0 tx=0).stop 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244106a70 msgr2=0x7f224419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244106a70 0x7f224419cd20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f223400e970 tx=0x7f223400ee30 comp rx=0 tx=0).stop 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 shutdown_connections 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f22300a3430 0x7f22300a58f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2244106a70 0x7f224419cd20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2244101410 0x7f2244198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 --2- 172.21.15.103:0/910886518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2244100a70 0x7f2244198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 >> 172.21.15.103:0/910886518 conn(0x7f22440fc790 msgr2=0x7f22440fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 shutdown_connections 2024-09-19T07:39:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:47.809+0000 7f224b6e6640 1 -- 172.21.15.103:0/910886518 wait complete. 2024-09-19T07:39:47.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:47 smithi174 ceph-mon[35370]: pgmap v57: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 859 KiB/s wr, 69 op/s 2024-09-19T07:39:48.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:47 smithi134 ceph-mon[35414]: pgmap v57: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 859 KiB/s wr, 69 op/s 2024-09-19T07:39:48.534 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T07:39:48.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T07:39:48.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/910886518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:48.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:48 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:48.917 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:39:48.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/910886518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:48.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:48 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/910886518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:39:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:48 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 -- 172.21.15.103:0/3008723571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 msgr2=0x7f3b90109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 --2- 172.21.15.103:0/3008723571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b90109830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b8002b500 comp rx=0 tx=0).stop 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 -- 172.21.15.103:0/3008723571 shutdown_connections 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 --2- 172.21.15.103:0/3008723571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b9010ea80 0x7f3b901154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 --2- 172.21.15.103:0/3008723571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b90109e00 0x7f3b9010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 --2- 172.21.15.103:0/3008723571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b90109830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 -- 172.21.15.103:0/3008723571 >> 172.21.15.103:0/3008723571 conn(0x7f3b9006e2d0 msgr2=0x7f3b9006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:49.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 -- 172.21.15.103:0/3008723571 shutdown_connections 2024-09-19T07:39:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.446+0000 7f3b970aa640 1 -- 172.21.15.103:0/3008723571 wait complete. 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.447+0000 7f3b970aa640 1 Processor -- start 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.447+0000 7f3b970aa640 1 -- start start 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b970aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b901a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b970aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b90109e00 0x7f3b901a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b970aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b9010ea80 0x7f3b901abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b94e46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b901a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b970aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b901185a0 con 0x7f3b90109e00 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b970aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90118420 con 0x7f3b90109450 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b970aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90118720 con 0x7f3b9010ea80 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b94e46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b901a71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34792/0 (socket says 172.21.15.103:34792) 2024-09-19T07:39:49.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b94e46640 1 -- 172.21.15.103:0/1514389818 learned_addr learned my addr 172.21.15.103:0/1514389818 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.448+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b90109e00 0x7f3b901a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.449+0000 7f3b8ffff640 1 -- 172.21.15.103:0/1514389818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b9010ea80 msgr2=0x7f3b901abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.449+0000 7f3b95647640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b9010ea80 0x7f3b901abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.449+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b9010ea80 0x7f3b901abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.449+0000 7f3b8ffff640 1 -- 172.21.15.103:0/1514389818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 msgr2=0x7f3b901a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.449+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b901a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.449+0000 7f3b8ffff640 1 -- 172.21.15.103:0/1514389818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b901ac1a0 con 0x7f3b90109e00 2024-09-19T07:39:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.450+0000 7f3b95647640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b9010ea80 0x7f3b901abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:39:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.450+0000 7f3b94e46640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b901a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:39:49.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.451+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b90109e00 0x7f3b901a7730 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400e9c0 tx=0x7f3b8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.452+0000 7f3b8dffb640 1 -- 172.21.15.103:0/1514389818 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400ccc0 con 0x7f3b90109e00 2024-09-19T07:39:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.452+0000 7f3b8dffb640 1 -- 172.21.15.103:0/1514389818 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b84010910 con 0x7f3b90109e00 2024-09-19T07:39:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.452+0000 7f3b8dffb640 1 -- 172.21.15.103:0/1514389818 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b840188c0 con 0x7f3b90109e00 2024-09-19T07:39:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.452+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b901ac490 con 0x7f3b90109e00 2024-09-19T07:39:49.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.452+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b901b3d70 con 0x7f3b90109e00 2024-09-19T07:39:49.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.456+0000 7f3b8dffb640 1 -- 172.21.15.103:0/1514389818 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3b84018a60 con 0x7f3b90109e00 2024-09-19T07:39:49.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.459+0000 7f3b8dffb640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b780a3580 0x7f3b780a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.459+0000 7f3b8dffb640 1 -- 172.21.15.103:0/1514389818 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3b840d1080 con 0x7f3b90109e00 2024-09-19T07:39:49.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.462+0000 7f3b99483740 1 Processor -- start 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.464+0000 7f3b94e46640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b780a3580 0x7f3b780a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.465+0000 7f3b99483740 1 -- start start 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.465+0000 7f3b99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf1850 0x5563a7cf1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.465+0000 7f3b99483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf2170 0x5563a7cfc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.465+0000 7f3b99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7cfee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.465+0000 7f3b99483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563a7c56160 con 0x5563a7cf1850 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.465+0000 7f3b99483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563a7bfd720 con 0x5563a7cfca30 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.465+0000 7f3b99483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563a7c52120 con 0x5563a7cf2170 2024-09-19T07:39:49.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7cfee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7cfee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34806/0 (socket says 172.21.15.103:34806) 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 -- 172.21.15.103:0/198925612 learned_addr learned my addr 172.21.15.103:0/198925612 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 -- 172.21.15.103:0/198925612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf2170 msgr2=0x5563a7cfc4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 --2- 172.21.15.103:0/198925612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf2170 0x5563a7cfc4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 -- 172.21.15.103:0/198925612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf1850 msgr2=0x5563a7cf1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 --2- 172.21.15.103:0/198925612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf1850 0x5563a7cf1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.466+0000 7f3b95647640 1 -- 172.21.15.103:0/198925612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a79c7ba0 con 0x5563a7cfca30 2024-09-19T07:39:49.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.467+0000 7f3b95647640 1 --2- 172.21.15.103:0/198925612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7cfee20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b88008de0 tx=0x7f3b8800d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3fe028262b8fba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:49.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.468+0000 7f3b76ffd640 1 -- 172.21.15.103:0/198925612 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800f830 con 0x5563a7cfca30 2024-09-19T07:39:49.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.468+0000 7f3b76ffd640 1 -- 172.21.15.103:0/198925612 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8800e3b0 con 0x5563a7cfca30 2024-09-19T07:39:49.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.469+0000 7f3b76ffd640 1 -- 172.21.15.103:0/198925612 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800b600 con 0x5563a7cfca30 2024-09-19T07:39:49.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.470+0000 7f3b99483740 1 -- 172.21.15.103:0/198925612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 msgr2=0x5563a7cfee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.470+0000 7f3b99483740 1 --2- 172.21.15.103:0/198925612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7cfee20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b88008de0 tx=0x7f3b8800d710 comp rx=0 tx=0).stop 2024-09-19T07:39:49.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b99483740 1 -- 172.21.15.103:0/198925612 shutdown_connections 2024-09-19T07:39:49.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b99483740 1 --2- 172.21.15.103:0/198925612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7cfee20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b99483740 1 --2- 172.21.15.103:0/198925612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf2170 0x5563a7cfc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b99483740 1 --2- 172.21.15.103:0/198925612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf1850 0x5563a7cf1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b99483740 1 -- 172.21.15.103:0/198925612 >> 172.21.15.103:0/198925612 conn(0x5563a7cedde0 msgr2=0x5563a7cee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:49.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b94e46640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b780a3580 0x7f3b780a5a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b800057c0 tx=0x7f3b80032480 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:49.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b99483740 1 -- 172.21.15.103:0/198925612 shutdown_connections 2024-09-19T07:39:49.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.471+0000 7f3b99483740 1 -- 172.21.15.103:0/198925612 wait complete. 2024-09-19T07:39:49.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.472+0000 7f3b99483740 1 -- start start 2024-09-19T07:39:49.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.472+0000 7f3b99483740 1 Processor -- start 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf1850 0x5563a7cf9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b94e46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf1850 0x5563a7cf9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b94e46640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf1850 0x5563a7cf9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34818/0 (socket says 172.21.15.103:34818) 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b94e46640 1 -- 172.21.15.103:0/1223270388 learned_addr learned my addr 172.21.15.103:0/1223270388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b99483740 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cfca30 0x5563a7cfa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563a7cff650 con 0x5563a7cf2170 2024-09-19T07:39:49.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563a7bde900 con 0x5563a7cf1850 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563a7cff7d0 con 0x5563a7cfca30 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.473+0000 7f3b95647640 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cfca30 0x5563a7cfa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b99483740 10 client.0 _ll_register_callbacks cb 0x5563a7bdc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b95647640 1 -- 172.21.15.103:0/1223270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf1850 msgr2=0x5563a7cf9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b95647640 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf1850 0x5563a7cf9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b95647640 1 -- 172.21.15.103:0/1223270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 msgr2=0x5563a7cf9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b95647640 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b95647640 1 -- 172.21.15.103:0/1223270388 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a7c1ec50 con 0x5563a7cfca30 2024-09-19T07:39:49.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8800e3b0 con 0x5563a7cfca30 2024-09-19T07:39:49.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.474+0000 7f3b95647640 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cfca30 0x5563a7cfa3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b88000f80 tx=0x7f3b8800f860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57fb222d71168a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:49.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b6ffff640 10 client.25411 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b6ffff640 1 -- 172.21.15.103:0/1223270388 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880089d0 con 0x5563a7cfca30 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b6ffff640 1 -- 172.21.15.103:0/1223270388 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3b8800fb60 con 0x5563a7cfca30 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b6ffff640 1 -- 172.21.15.103:0/1223270388 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b8800fde0 con 0x5563a7cfca30 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 10 client.25411 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5563a7d0b350 con 0x5563a7cfca30 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b6ffff640 1 -- 172.21.15.103:0/1223270388 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3b8802b410 con 0x5563a7cfca30 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 10 client.25411 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 10 client.25411 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 20 client.25411 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d0c6a0 0x5563a7d0ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 4 client.25411 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:39:49.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.475+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563a7c63890 con 0x5563a7d0c6a0 2024-09-19T07:39:49.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.476+0000 7f3b94e46640 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d0c6a0 0x5563a7d0ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.476+0000 7f3b94e46640 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d0c6a0 0x5563a7d0ea90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b878fdbc3cd39867 server_cookie=61a5ab8b80ada53b in_seq=0 out_seq=0 2024-09-19T07:39:49.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.476+0000 7f3b6ffff640 10 client.25411 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:49.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.477+0000 7f3b6ffff640 1 -- 172.21.15.103:0/1223270388 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563a7c63890 con 0x5563a7d0c6a0 2024-09-19T07:39:49.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.477+0000 7f3b6ffff640 10 client.25411 handle_command_reply: tid=0 2024-09-19T07:39:49.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.477+0000 7f3b99483740 1 client.25411 shutdown 2024-09-19T07:39:49.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cfca30 msgr2=0x5563a7cfa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cfca30 0x5563a7cfa3e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b88000f80 tx=0x7f3b8800f860 comp rx=0 tx=0).stop 2024-09-19T07:39:49.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 shutdown_connections 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d0c6a0 0x5563a7d0ea90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cfca30 0x5563a7cfa3e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 --2- 172.21.15.103:0/1223270388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf1850 0x5563a7cf9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 >> 172.21.15.103:0/1223270388 conn(0x5563a7cedde0 msgr2=0x5563a7cf3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b6ffff640 0 client.25411 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 shutdown_connections 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 1 -- 172.21.15.103:0/1223270388 wait complete. 2024-09-19T07:39:49.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.478+0000 7f3b99483740 20 client.25411 trim_cache size 0 max 16384 2024-09-19T07:39:49.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 Processor -- start 2024-09-19T07:39:49.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 -- start start 2024-09-19T07:39:49.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7cf88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7d141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c00e830 con 0x5563a7cf2170 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c010430 con 0x5563a7cfca30 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b99483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c00ef00 con 0x5563a7cf1850 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54236/0 (socket says 172.21.15.103:54236) 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b8ffff640 1 -- 172.21.15.103:0/2314791524 learned_addr learned my addr 172.21.15.103:0/2314791524 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b95647640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7d141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.506+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7cf88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.507+0000 7f3b94e46640 1 -- 172.21.15.103:0/2314791524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 msgr2=0x5563a7d141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:49.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.507+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7d141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.507+0000 7f3b94e46640 1 -- 172.21.15.103:0/2314791524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 msgr2=0x5563a7cf42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:49.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.507+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.507+0000 7f3b94e46640 1 -- 172.21.15.103:0/2314791524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a7cf7eb0 con 0x5563a7cf1850 2024-09-19T07:39:49.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.508+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7cf88a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5563a7d0f120 tx=0x7f3b8003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8383dfc8801051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:49.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.508+0000 7f3b767fc640 1 -- 172.21.15.103:0/2314791524 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80047040 con 0x5563a7cf1850 2024-09-19T07:39:49.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.508+0000 7f3b767fc640 1 -- 172.21.15.103:0/2314791524 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b80051070 con 0x5563a7cf1850 2024-09-19T07:39:49.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.508+0000 7f3b767fc640 1 -- 172.21.15.103:0/2314791524 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80036040 con 0x5563a7cf1850 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.509+0000 7f3b99483740 1 -- 172.21.15.103:0/2314791524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 msgr2=0x5563a7cf88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.509+0000 7f3b99483740 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7cf88a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5563a7d0f120 tx=0x7f3b8003e000 comp rx=0 tx=0).stop 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.509+0000 7f3b99483740 1 -- 172.21.15.103:0/2314791524 shutdown_connections 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.509+0000 7f3b99483740 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cfca30 0x5563a7d141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.509+0000 7f3b99483740 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cf2170 0x5563a7cf42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.509+0000 7f3b99483740 1 --2- 172.21.15.103:0/2314791524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7cf88a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.509+0000 7f3b99483740 1 -- 172.21.15.103:0/2314791524 >> 172.21.15.103:0/2314791524 conn(0x5563a7cedde0 msgr2=0x5563a7cfb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.510+0000 7f3b99483740 1 -- 172.21.15.103:0/2314791524 shutdown_connections 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.510+0000 7f3b99483740 1 -- 172.21.15.103:0/2314791524 wait complete. 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.510+0000 7f3b99483740 1 -- start start 2024-09-19T07:39:49.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.510+0000 7f3b99483740 1 Processor -- start 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b99483740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7d050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b99483740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf2170 0x5563a7cf6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b99483740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cfca30 0x5563a7cf6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b99483740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c00fca0 con 0x5563a7cfca30 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b99483740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c00f5d0 con 0x5563a7cf2170 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b99483740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c00c810 con 0x5563a7cf1850 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b99483740 10 client.0 _ll_register_callbacks cb 0x7f3b88008c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf2170 0x5563a7cf6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf2170 0x5563a7cf6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:34828/0 (socket says 172.21.15.103:34828) 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 -- 172.21.15.103:0/2359896914 learned_addr learned my addr 172.21.15.103:0/2359896914 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7d050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b95647640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cfca30 0x5563a7cf6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 -- 172.21.15.103:0/2359896914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 msgr2=0x5563a7d050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7d050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 -- 172.21.15.103:0/2359896914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cfca30 msgr2=0x5563a7cf6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cfca30 0x5563a7cf6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b8ffff640 1 -- 172.21.15.103:0/2359896914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a7cf7100 con 0x5563a7cf2170 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.511+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7d050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.512+0000 7f3b8ffff640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf2170 0x5563a7cf6520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000c00 tx=0x7f3b840bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d9fd0ae5811888 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:39:49.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.512+0000 7f3b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:39:49.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.513+0000 7f3b6ffff640 1 -- 172.21.15.103:0/2359896914 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b840bb060 con 0x5563a7cf2170 2024-09-19T07:39:49.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.513+0000 7f3b6ffff640 1 -- 172.21.15.103:0/2359896914 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3b840b6270 con 0x5563a7cf2170 2024-09-19T07:39:49.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.513+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b80051070 con 0x5563a7cf2170 2024-09-19T07:39:49.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.513+0000 7f3b6ffff640 1 -- 172.21.15.103:0/2359896914 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b840b5030 con 0x5563a7cf2170 2024-09-19T07:39:49.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.513+0000 7f3b99483740 10 client.25436 fetch_fsmap learned FSMap version 30 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.513+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563a7d76ff0 con 0x5563a7cf2170 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b6ffff640 1 -- 172.21.15.103:0/2359896914 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b80051070 con 0x5563a7cf2170 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b6ffff640 1 -- 172.21.15.103:0/2359896914 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3b8409c760 con 0x5563a7cf2170 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b99483740 10 client.25436 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b99483740 10 client.25436 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b99483740 20 client.25436 populate_metadata read hostname 'smithi103' 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b99483740 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d778a0 0x5563a7d79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b99483740 4 client.25436 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x5563a7c5dbe0 con 0x5563a7d778a0 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.514+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d778a0 0x5563a7d79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.515+0000 7f3b6ffff640 10 client.25436 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:49.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.515+0000 7f3b94e46640 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d778a0 0x5563a7d79cc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4cdeb2ba30e6f server_cookie=fc32c056df2e29 in_seq=0 out_seq=0 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.656+0000 7f3b6ffff640 1 -- 172.21.15.103:0/2359896914 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5563a7c5dbe0 con 0x5563a7d778a0 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.656+0000 7f3b6ffff640 10 client.25436 handle_command_reply: tid=0 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.656+0000 7f3b99483740 1 client.25436 shutdown 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf2170 msgr2=0x5563a7cf6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf2170 0x5563a7cf6520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000c00 tx=0x7f3b840bd040 comp rx=0 tx=0).stop 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 shutdown_connections 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563a7d778a0 0x5563a7d79cc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563a7cfca30 0x5563a7cf6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563a7cf2170 0x5563a7cf6520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 --2- 172.21.15.103:0/2359896914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7cf1850 0x5563a7d050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b6ffff640 0 client.25436 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 >> 172.21.15.103:0/2359896914 conn(0x5563a7cedde0 msgr2=0x5563a7cf9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 shutdown_connections 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 1 -- 172.21.15.103:0/2359896914 wait complete. 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b99483740 20 client.25436 trim_cache size 0 max 16384 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:39:49.659 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 30, 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 10, 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "ae2c5d5c": { 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 10, 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:39:49.660 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:49.661 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:49.662 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:39:49.662 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:39:49.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b780a3580 msgr2=0x7f3b780a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b970aa640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b780a3580 0x7f3b780a5a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b800057c0 tx=0x7f3b80032480 comp rx=0 tx=0).stop 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b90109e00 msgr2=0x7f3b901a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b970aa640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b90109e00 0x7f3b901a7730 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400e9c0 tx=0x7f3b8400ee80 comp rx=0 tx=0).stop 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 shutdown_connections 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.657+0000 7f3b970aa640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b780a3580 0x7f3b780a5a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.658+0000 7f3b970aa640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b9010ea80 0x7f3b901abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.658+0000 7f3b970aa640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b90109e00 0x7f3b901a7730 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.658+0000 7f3b970aa640 1 --2- 172.21.15.103:0/1514389818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b90109450 0x7f3b901a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.658+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 >> 172.21.15.103:0/1514389818 conn(0x7f3b9006e2d0 msgr2=0x7f3b9010c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.658+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 shutdown_connections 2024-09-19T07:39:49.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:39:49.658+0000 7f3b970aa640 1 -- 172.21.15.103:0/1514389818 wait complete. 2024-09-19T07:39:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:49 smithi174 ceph-mon[35370]: pgmap v58: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 859 KiB/s wr, 69 op/s 2024-09-19T07:39:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:49 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:49 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:49 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:49 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:39:49.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:49 smithi103 ceph-mon[31201]: pgmap v58: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 859 KiB/s wr, 69 op/s 2024-09-19T07:39:49.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:49 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:49.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:49 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:49.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:49 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:49.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:49 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:39:50.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:49 smithi134 ceph-mon[35414]: pgmap v58: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 859 KiB/s wr, 69 op/s 2024-09-19T07:39:50.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:49 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:50.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:49 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:50.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:49 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:50.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:49 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:39:50.382 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ae2c5d5c', quiesce took 3.9 seconds. Will release after: 16.450000000000003 2024-09-19T07:39:50.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 75.0 s, 2816.5 tps, lat 69.220 ms stddev 907.333 2024-09-19T07:39:51.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:51 smithi174 ceph-mon[35370]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.6 MiB/s wr, 331 op/s 2024-09-19T07:39:51.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:51 smithi103 ceph-mon[31201]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.6 MiB/s wr, 331 op/s 2024-09-19T07:39:52.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:51 smithi134 ceph-mon[35414]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.6 MiB/s wr, 331 op/s 2024-09-19T07:39:53.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:53 smithi174 ceph-mon[35370]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-09-19T07:39:53.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:53 smithi103 ceph-mon[31201]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-09-19T07:39:54.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:53 smithi134 ceph-mon[35414]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-09-19T07:39:55.473 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 80.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:39:55.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:55 smithi134 ceph-mon[35414]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-09-19T07:39:55.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:55 smithi174 ceph-mon[35370]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-09-19T07:39:55.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:55 smithi103 ceph-mon[31201]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.0 MiB/s wr, 288 op/s 2024-09-19T07:39:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:57 smithi174 ceph-mon[35370]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-09-19T07:39:57.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:57 smithi103 ceph-mon[31201]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-09-19T07:39:58.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:57 smithi134 ceph-mon[35414]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-09-19T07:39:59.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:39:59 smithi174 ceph-mon[35370]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-09-19T07:39:59.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:39:59 smithi103 ceph-mon[31201]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-09-19T07:40:00.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:39:59 smithi134 ceph-mon[35414]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 3.9 MiB/s wr, 281 op/s 2024-09-19T07:40:00.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 85.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:40:00.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:00 smithi174 ceph-mon[35370]: overall HEALTH_OK 2024-09-19T07:40:00.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:00 smithi103 ceph-mon[31201]: overall HEALTH_OK 2024-09-19T07:40:01.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:00 smithi134 ceph-mon[35414]: overall HEALTH_OK 2024-09-19T07:40:01.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:01 smithi174 ceph-mon[35370]: pgmap v64: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.4 MiB/s wr, 592 op/s 2024-09-19T07:40:01.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:01 smithi103 ceph-mon[31201]: pgmap v64: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.4 MiB/s wr, 592 op/s 2024-09-19T07:40:02.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:01 smithi134 ceph-mon[35414]: pgmap v64: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.4 MiB/s wr, 592 op/s 2024-09-19T07:40:03.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:03 smithi174 ceph-mon[35370]: pgmap v65: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 329 op/s 2024-09-19T07:40:03.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:03 smithi103 ceph-mon[31201]: pgmap v65: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 329 op/s 2024-09-19T07:40:04.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:03 smithi134 ceph-mon[35414]: pgmap v65: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 329 op/s 2024-09-19T07:40:05.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 90.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:40:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:05 smithi174 ceph-mon[35370]: pgmap v66: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 387 op/s 2024-09-19T07:40:05.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:05 smithi103 ceph-mon[31201]: pgmap v66: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 387 op/s 2024-09-19T07:40:06.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:05 smithi134 ceph-mon[35414]: pgmap v66: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.3 MiB/s wr, 387 op/s 2024-09-19T07:40:06.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:40:07.097 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:07.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 -- 172.21.15.103:0/1422316651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102570 msgr2=0x7fa5e41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:07.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/1422316651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102570 0x7fa5e41029b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009900 tx=0x7fa5d802a480 comp rx=0 tx=0).stop 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 -- 172.21.15.103:0/1422316651 shutdown_connections 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/1422316651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e4102ef0 0x7fa5e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/1422316651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102570 0x7fa5e41029b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/1422316651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e4100e20 0x7fa5e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 -- 172.21.15.103:0/1422316651 >> 172.21.15.103:0/1422316651 conn(0x7fa5e40fc7d0 msgr2=0x7fa5e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 -- 172.21.15.103:0/1422316651 shutdown_connections 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.551+0000 7fa5ea19e640 1 -- 172.21.15.103:0/1422316651 wait complete. 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 Processor -- start 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 -- start start 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e4100e20 0x7fa5e41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:07.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e4102570 0x7fa5e41998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102ef0 0x7fa5e41a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e4110f50 con 0x7fa5e4102ef0 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e4110dd0 con 0x7fa5e4102570 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.552+0000 7fa5ea19e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e41110d0 con 0x7fa5e4100e20 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102ef0 0x7fa5e41a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102ef0 0x7fa5e41a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58810/0 (socket says 172.21.15.103:58810) 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 -- 172.21.15.103:0/3310920988 learned_addr learned my addr 172.21.15.103:0/3310920988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e2ffd640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e4102570 0x7fa5e41998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e37fe640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e4100e20 0x7fa5e41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:07.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 -- 172.21.15.103:0/3310920988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e4100e20 msgr2=0x7fa5e41993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e4100e20 0x7fa5e41993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 -- 172.21.15.103:0/3310920988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e4102570 msgr2=0x7fa5e41998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e4102570 0x7fa5e41998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 -- 172.21.15.103:0/3310920988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e41a4960 con 0x7fa5e4102ef0 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e2ffd640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e4102570 0x7fa5e41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e3fff640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102ef0 0x7fa5e41a4420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00eb10 tx=0x7fa5dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e0ff9640 1 -- 172.21.15.103:0/3310920988 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc004b20 con 0x7fa5e4102ef0 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e0ff9640 1 -- 172.21.15.103:0/3310920988 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5dc004cc0 con 0x7fa5e4102ef0 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e41a4b30 con 0x7fa5e4102ef0 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5e0ff9640 1 -- 172.21.15.103:0/3310920988 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc010760 con 0x7fa5e4102ef0 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.553+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e41a4ff0 con 0x7fa5e4102ef0 2024-09-19T07:40:07.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.554+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e41021b0 con 0x7fa5e4102ef0 2024-09-19T07:40:07.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.555+0000 7fa5e0ff9640 1 -- 172.21.15.103:0/3310920988 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa5dc005000 con 0x7fa5e4102ef0 2024-09-19T07:40:07.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.556+0000 7fa5e0ff9640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa5cc0a33b0 0x7fa5cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:07.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.556+0000 7fa5e37fe640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa5cc0a33b0 0x7fa5cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:07.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.556+0000 7fa5e0ff9640 1 -- 172.21.15.103:0/3310920988 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fa5dc0d0420 con 0x7fa5e4102ef0 2024-09-19T07:40:07.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.557+0000 7fa5e37fe640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa5cc0a33b0 0x7fa5cc0a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4005d00 tx=0x7fa5d400b410 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:07.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.560+0000 7fa5e0ff9640 1 -- 172.21.15.103:0/3310920988 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa5dc054210 con 0x7fa5e4102ef0 2024-09-19T07:40:07.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:07 smithi103 ceph-mon[31201]: pgmap v67: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T07:40:07.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:07 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:40:07.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.773+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e402d7c0 con 0x7fa5e4102ef0 2024-09-19T07:40:07.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.776+0000 7fa5e0ff9640 1 -- 172.21.15.103:0/3310920988 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11998 (secure 0 0 0) 0x7fa5dc09ce10 con 0x7fa5e4102ef0 2024-09-19T07:40:07.777 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:40:07.777 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":30,"btime":"2024-09-19T07:39:09:495445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:39:08.515410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:40:07.778 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 30 2024-09-19T07:40:07.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa5cc0a33b0 msgr2=0x7fa5cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:07.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa5cc0a33b0 0x7fa5cc0a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4005d00 tx=0x7fa5d400b410 comp rx=0 tx=0).stop 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102ef0 msgr2=0x7fa5e41a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102ef0 0x7fa5e41a4420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00eb10 tx=0x7fa5dc00c2d0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 shutdown_connections 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa5cc0a33b0 0x7fa5cc0a5870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5e4102ef0 0x7fa5e41a4420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5e4102570 0x7fa5e41998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 --2- 172.21.15.103:0/3310920988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5e4100e20 0x7fa5e41993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.778+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 >> 172.21.15.103:0/3310920988 conn(0x7fa5e40fc7d0 msgr2=0x7fa5e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.779+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 shutdown_connections 2024-09-19T07:40:07.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:07.779+0000 7fa5ea19e640 1 -- 172.21.15.103:0/3310920988 wait complete. 2024-09-19T07:40:07.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:07 smithi174 ceph-mon[35370]: pgmap v67: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T07:40:07.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:07 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:40:08.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:07 smithi134 ceph-mon[35414]: pgmap v67: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T07:40:08.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:07 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:40:08.451 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id ae2c5d5c --release --await' 2024-09-19T07:40:08.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id ae2c5d5c --release --await 2024-09-19T07:40:08.696 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:09.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.071 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3310920988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:40:09.072 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.072 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.072 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:40:09.072 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:40:09.072 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:08 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3310920988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:40:09.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:40:09.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:40:09.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:08 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.169 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3310920988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:40:09.208 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.209 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.209 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:40:09.209 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:40:09.209 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:08 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:40:09.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 -- 172.21.15.103:0/3236946704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe1810c170 msgr2=0x7fbe1810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/3236946704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe1810c170 0x7fbe1810e610 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009ca0 tx=0x7fbe1002a950 comp rx=0 tx=0).stop 2024-09-19T07:40:09.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 -- 172.21.15.103:0/3236946704 shutdown_connections 2024-09-19T07:40:09.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/3236946704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe1810c170 0x7fbe1810e610 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/3236946704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe18103660 0x7fbe1810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/3236946704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe18102d40 0x7fbe18103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 -- 172.21.15.103:0/3236946704 >> 172.21.15.103:0/3236946704 conn(0x7fbe180fc7a0 msgr2=0x7fbe180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 -- 172.21.15.103:0/3236946704 shutdown_connections 2024-09-19T07:40:09.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.156+0000 7fbe1dd22640 1 -- 172.21.15.103:0/3236946704 wait complete. 2024-09-19T07:40:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 Processor -- start 2024-09-19T07:40:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 -- start start 2024-09-19T07:40:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe18102d40 0x7fbe181009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe18103660 0x7fbe18100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1810c170 0x7fbe18101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18111fb0 con 0x7fbe18103660 2024-09-19T07:40:09.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18111e30 con 0x7fbe1810c170 2024-09-19T07:40:09.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe1dd22640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18112130 con 0x7fbe18102d40 2024-09-19T07:40:09.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe18103660 0x7fbe18100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe18103660 0x7fbe18100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60492/0 (socket says 172.21.15.103:60492) 2024-09-19T07:40:09.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.157+0000 7fbe16ffd640 1 -- 172.21.15.103:0/390089476 learned_addr learned my addr 172.21.15.103:0/390089476 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:09.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe177fe640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe18102d40 0x7fbe181009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe17fff640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1810c170 0x7fbe18101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe16ffd640 1 -- 172.21.15.103:0/390089476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe18102d40 msgr2=0x7fbe181009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe18102d40 0x7fbe181009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe16ffd640 1 -- 172.21.15.103:0/390089476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1810c170 msgr2=0x7fbe18101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1810c170 0x7fbe18101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe16ffd640 1 -- 172.21.15.103:0/390089476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe181a6930 con 0x7fbe18103660 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe18103660 0x7fbe18100ee0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c940 tx=0x7fbe0c00cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe14ff9640 1 -- 172.21.15.103:0/390089476 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c004220 con 0x7fbe18103660 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe14ff9640 1 -- 172.21.15.103:0/390089476 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c0043c0 con 0x7fbe18103660 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe181a6c20 con 0x7fbe18103660 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe14ff9640 1 -- 172.21.15.103:0/390089476 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c005650 con 0x7fbe18103660 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.158+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe181a7160 con 0x7fbe18103660 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.161+0000 7fbe14ff9640 1 -- 172.21.15.103:0/390089476 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbe0c0079b0 con 0x7fbe18103660 2024-09-19T07:40:09.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.161+0000 7fbe200fb740 1 Processor -- start 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.161+0000 7fbe200fb740 1 -- start start 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c39329c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c980 0x55c3932a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe200fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c393201620 con 0x55c39329c060 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe200fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c3931a8e50 con 0x55c39329c980 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe200fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3931fd5e0 con 0x55c3932a7240 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c39329c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c39329c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60502/0 (socket says 172.21.15.103:60502) 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe17fff640 1 -- 172.21.15.103:0/2604950879 learned_addr learned my addr 172.21.15.103:0/2604950879 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe16ffd640 1 -- 172.21.15.103:0/2604950879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c980 msgr2=0x55c3932a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c980 0x55c3932a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe16ffd640 1 -- 172.21.15.103:0/2604950879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 msgr2=0x55c39329c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c39329c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.162+0000 7fbe16ffd640 1 -- 172.21.15.103:0/2604950879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c393199d40 con 0x55c3932a7240 2024-09-19T07:40:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.163+0000 7fbe14ff9640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe000a33b0 0x7fbe000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.163+0000 7fbe17fff640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe000a33b0 0x7fbe000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.163+0000 7fbe14ff9640 1 -- 172.21.15.103:0/390089476 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fbe0c0cf880 con 0x7fbe18103660 2024-09-19T07:40:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.163+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932a9630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0ccfe0 tx=0x7fbe0c0d7690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f7f7171d1e58982 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.163+0000 7fbe17fff640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe000a33b0 0x7fbe000a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005fe0 tx=0x7fbe10005c50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbdfdffb640 1 -- 172.21.15.103:0/2604950879 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c01d070 con 0x55c3932a7240 2024-09-19T07:40:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbdfdffb640 1 -- 172.21.15.103:0/2604950879 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c09c270 con 0x55c3932a7240 2024-09-19T07:40:09.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbdfdffb640 1 -- 172.21.15.103:0/2604950879 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c09b940 con 0x55c3932a7240 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbe200fb740 1 -- 172.21.15.103:0/2604950879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 msgr2=0x55c3932a9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbe200fb740 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932a9630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0ccfe0 tx=0x7fbe0c0d7690 comp rx=0 tx=0).stop 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbe200fb740 1 -- 172.21.15.103:0/2604950879 shutdown_connections 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbe200fb740 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932a9630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbe200fb740 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c980 0x55c3932a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbe200fb740 1 --2- 172.21.15.103:0/2604950879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c39329c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.164+0000 7fbe200fb740 1 -- 172.21.15.103:0/2604950879 >> 172.21.15.103:0/2604950879 conn(0x55c3932985f0 msgr2=0x55c3932989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:09.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 -- 172.21.15.103:0/2604950879 shutdown_connections 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 -- 172.21.15.103:0/2604950879 wait complete. 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 -- start start 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 Processor -- start 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c060 0x55c3932a4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932a4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c3932a9e10 con 0x55c39329c980 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c393189f10 con 0x55c3932a7240 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3932a9f90 con 0x55c39329c060 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe200fb740 10 client.0 _ll_register_callbacks cb 0x55c3931addb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:09.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.165+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60516/0 (socket says 172.21.15.103:60516) 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3592690315 learned_addr learned my addr 172.21.15.103:0/3592690315 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe17fff640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe177fe640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c060 0x55c3932a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3592690315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c060 msgr2=0x55c3932a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c060 0x55c3932a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 msgr2=0x55c3932a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3592690315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c392f10580 con 0x55c39329c980 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe177fe640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c060 0x55c3932a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932a4650 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c09bb00 tx=0x7fbe0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fce68a2b136beee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:09.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbdf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/3592690315 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c01d040 con 0x55c39329c980 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0c09c270 con 0x55c39329c980 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/3592690315 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fbe0c0981e0 con 0x55c39329c980 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/3592690315 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe0c09b4d0 con 0x55c39329c980 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe200fb740 10 client.15942 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.166+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c392fda960 con 0x55c39329c980 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/3592690315 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c09c270 con 0x55c39329c980 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/3592690315 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fbe0c007ca0 con 0x55c39329c980 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbe200fb740 10 client.15942 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbe200fb740 10 client.15942 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbe200fb740 20 client.15942 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:09.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c3932b6c20 0x55c3932b9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbe200fb740 4 client.15942 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c39320eb90 con 0x55c3932b6c20 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.167+0000 7fbe177fe640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c3932b6c20 0x55c3932b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.168+0000 7fbdf6ffd640 10 client.15942 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.168+0000 7fbe177fe640 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c3932b6c20 0x55c3932b9010 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367a08fb31e00661 server_cookie=3a6b21f5b55e11 in_seq=0 out_seq=0 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.168+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/3592690315 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c39320eb90 con 0x55c3932b6c20 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.168+0000 7fbdf6ffd640 10 client.15942 handle_command_reply: tid=0 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.168+0000 7fbe200fb740 1 client.15942 shutdown 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 msgr2=0x55c3932a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932a4650 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c09bb00 tx=0x7fbe0c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 shutdown_connections 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c3932b6c20 0x55c3932b9010 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932a4650 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3592690315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c060 0x55c3932a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 >> 172.21.15.103:0/3592690315 conn(0x55c3932985f0 msgr2=0x55c39329e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbdf6ffd640 0 client.15942 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 shutdown_connections 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 1 -- 172.21.15.103:0/3592690315 wait complete. 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.169+0000 7fbe200fb740 20 client.15942 trim_cache size 0 max 16384 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.185+0000 7fbe200fb740 1 Processor -- start 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.185+0000 7fbe200fb740 1 -- start start 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c060 0x55c3932b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932bbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe200fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181afb20 con 0x55c39329c980 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe200fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181b1720 con 0x55c39329c060 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe200fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181b01f0 con 0x55c3932a7240 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932bbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932bbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60530/0 (socket says 172.21.15.103:60530) 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3652637496 learned_addr learned my addr 172.21.15.103:0/3652637496 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe17fff640 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe177fe640 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c060 0x55c3932b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3652637496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 msgr2=0x55c3932bc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932bc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3652637496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c060 msgr2=0x55c3932b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c060 0x55c3932b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 -- 172.21.15.103:0/3652637496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3932a30e0 con 0x55c39329c980 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe177fe640 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c060 0x55c3932b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932bbc50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c097fd0 tx=0x7fbe0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463233e5fb130116 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbdfd7fa640 1 -- 172.21.15.103:0/3652637496 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c09b6b0 con 0x55c39329c980 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbdfd7fa640 1 -- 172.21.15.103:0/3652637496 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c09c270 con 0x55c39329c980 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.186+0000 7fbdfd7fa640 1 -- 172.21.15.103:0/3652637496 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c09b940 con 0x55c39329c980 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 -- 172.21.15.103:0/3652637496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 msgr2=0x55c3932bbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932bbc50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c097fd0 tx=0x7fbe0c004730 comp rx=0 tx=0).stop 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 -- 172.21.15.103:0/3652637496 shutdown_connections 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3932a7240 0x55c3932bc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c980 0x55c3932bbc50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 --2- 172.21.15.103:0/3652637496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c39329c060 0x55c3932b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 -- 172.21.15.103:0/3652637496 >> 172.21.15.103:0/3652637496 conn(0x55c3932985f0 msgr2=0x55c3932b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 -- 172.21.15.103:0/3652637496 shutdown_connections 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 -- 172.21.15.103:0/3652637496 wait complete. 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.187+0000 7fbe200fb740 1 -- start start 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 1 Processor -- start 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c3932b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c980 0x55c3932ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932ba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181b0f90 con 0x55c39329c060 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181b08c0 con 0x55c3932a7240 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181adb00 con 0x55c39329c980 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe200fb740 10 client.0 _ll_register_callbacks cb 0x7fbe0c00cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:09.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c3932b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c3932b9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60538/0 (socket says 172.21.15.103:60538) 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 -- 172.21.15.103:0/1487475082 learned_addr learned my addr 172.21.15.103:0/1487475082 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c980 0x55c3932ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe17fff640 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932ba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 -- 172.21.15.103:0/1487475082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c980 msgr2=0x55c3932ba0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c980 0x55c3932ba0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 -- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 msgr2=0x55c3932ba5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932ba5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 -- 172.21.15.103:0/1487475082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c393320660 con 0x55c39329c060 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbe177fe640 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c3932b9b60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009be0 tx=0x7fbe080157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad6a70d9989d7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.188+0000 7fbdf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/1487475082 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe080059c0 con 0x55c39329c060 2024-09-19T07:40:09.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/1487475082 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fbe08005b60 con 0x55c39329c060 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0c09c270 con 0x55c39329c060 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/1487475082 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe08005d60 con 0x55c39329c060 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 10 client.15948 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3933219c0 con 0x55c39329c060 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/1487475082 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c09c270 con 0x55c39329c060 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/1487475082 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fbe08022020 con 0x55c39329c060 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 10 client.15948 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 20 client.15948 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c393322170 0x55c393324590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 4 client.15948 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ae2c5d5c", "release": true, "await": true}] 2024-09-19T07:40:09.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ae2c5d5c", "release": true, "await": true}) -- 0x55c3932090a0 con 0x55c393322170 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.189+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c393322170 0x55c393324590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.190+0000 7fbe16ffd640 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c393322170 0x55c393324590 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d1fe5a97dc9237 server_cookie=86f9cbfb5798bd8c in_seq=0 out_seq=0 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.190+0000 7fbdf6ffd640 10 client.15948 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbdf6ffd640 1 -- 172.21.15.103:0/1487475082 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c3932090a0 con 0x55c393322170 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbdf6ffd640 10 client.15948 handle_command_reply: tid=0 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 client.15948 shutdown 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 msgr2=0x55c3932b9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c3932b9b60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009be0 tx=0x7fbe080157f0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 shutdown_connections 2024-09-19T07:40:09.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c393322170 0x55c393324590 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3932a7240 0x55c3932ba5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbdf6ffd640 0 client.15948 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c39329c980 0x55c3932ba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 --2- 172.21.15.103:0/1487475082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c39329c060 0x55c3932b9b60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 >> 172.21.15.103:0/1487475082 conn(0x55c3932985f0 msgr2=0x55c393297750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 shutdown_connections 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 1 -- 172.21.15.103:0/1487475082 wait complete. 2024-09-19T07:40:09.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.216+0000 7fbe200fb740 20 client.15948 trim_cache size 0 max 16384 2024-09-19T07:40:09.247 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:40:09.247 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 30, 2024-09-19T07:40:09.247 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 12, 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "ae2c5d5c": { 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 12, 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.3, 2024-09-19T07:40:09.248 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:40:09.249 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:40:09.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe000a33b0 msgr2=0x7fbe000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe000a33b0 0x7fbe000a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005fe0 tx=0x7fbe10005c50 comp rx=0 tx=0).stop 2024-09-19T07:40:09.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe18103660 msgr2=0x7fbe18100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:09.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe18103660 0x7fbe18100ee0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c940 tx=0x7fbe0c00cd40 comp rx=0 tx=0).stop 2024-09-19T07:40:09.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 shutdown_connections 2024-09-19T07:40:09.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe000a33b0 0x7fbe000a5870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe1810c170 0x7fbe18101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbe18103660 0x7fbe18100ee0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 --2- 172.21.15.103:0/390089476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe18102d40 0x7fbe181009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:09.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.217+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 >> 172.21.15.103:0/390089476 conn(0x7fbe180fc7a0 msgr2=0x7fbe180fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:09.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.218+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 shutdown_connections 2024-09-19T07:40:09.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:09.218+0000 7fbe1dd22640 1 -- 172.21.15.103:0/390089476 wait complete. 2024-09-19T07:40:09.960 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ae2c5d5c', total seconds elapsed: 23.4 2024-09-19T07:40:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 95.0 s, 303.7 tps, lat 1379.536 ms stddev 4877.951 2024-09-19T07:40:10.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:10 smithi103 ceph-mon[31201]: pgmap v68: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T07:40:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:10 smithi134 ceph-mon[35414]: pgmap v68: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T07:40:10.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:10 smithi174 ceph-mon[35370]: pgmap v68: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.2 MiB/s wr, 369 op/s 2024-09-19T07:40:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:12 smithi103 ceph-mon[31201]: pgmap v69: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 381 op/s 2024-09-19T07:40:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:12 smithi134 ceph-mon[35414]: pgmap v69: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 381 op/s 2024-09-19T07:40:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:12 smithi174 ceph-mon[35370]: pgmap v69: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 381 op/s 2024-09-19T07:40:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:14 smithi103 ceph-mon[31201]: pgmap v70: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 767 KiB/s wr, 70 op/s 2024-09-19T07:40:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:14 smithi134 ceph-mon[35414]: pgmap v70: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 767 KiB/s wr, 70 op/s 2024-09-19T07:40:14.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:14 smithi174 ceph-mon[35370]: pgmap v70: 97 pgs: 97 active+clean; 1.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 767 KiB/s wr, 70 op/s 2024-09-19T07:40:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 100.0 s, 3510.0 tps, lat 28.499 ms stddev 28.682 2024-09-19T07:40:15.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:40:16.244 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:16.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:16 smithi103 ceph-mon[31201]: pgmap v71: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-19T07:40:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:16 smithi134 ceph-mon[35414]: pgmap v71: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-19T07:40:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:16 smithi174 ceph-mon[35370]: pgmap v71: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-19T07:40:16.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.759+0000 7ff846a10640 1 -- 172.21.15.103:0/2165016183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840109130 msgr2=0x7ff840110cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.759+0000 7ff846a10640 1 --2- 172.21.15.103:0/2165016183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840109130 0x7ff840110cb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff838009900 tx=0x7ff83802a410 comp rx=0 tx=0).stop 2024-09-19T07:40:16.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.759+0000 7ff846a10640 1 -- 172.21.15.103:0/2165016183 shutdown_connections 2024-09-19T07:40:16.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.759+0000 7ff846a10640 1 --2- 172.21.15.103:0/2165016183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff840109130 0x7ff840110cb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.759+0000 7ff846a10640 1 --2- 172.21.15.103:0/2165016183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8401049b0 0x7ff840108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.760+0000 7ff846a10640 1 --2- 172.21.15.103:0/2165016183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff840104000 0x7ff8401043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.760+0000 7ff846a10640 1 -- 172.21.15.103:0/2165016183 >> 172.21.15.103:0/2165016183 conn(0x7ff840077a90 msgr2=0x7ff840079eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:16.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.760+0000 7ff846a10640 1 -- 172.21.15.103:0/2165016183 shutdown_connections 2024-09-19T07:40:16.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.760+0000 7ff846a10640 1 -- 172.21.15.103:0/2165016183 wait complete. 2024-09-19T07:40:16.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.761+0000 7ff846a10640 1 Processor -- start 2024-09-19T07:40:16.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.762+0000 7ff846a10640 1 -- start start 2024-09-19T07:40:16.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.762+0000 7ff846a10640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840104000 0x7ff84019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff846a10640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8401049b0 0x7ff84019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840104000 0x7ff84019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840104000 0x7ff84019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54354/0 (socket says 172.21.15.103:54354) 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8401049b0 0x7ff84019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff83ffff640 1 -- 172.21.15.103:0/2103554921 learned_addr learned my addr 172.21.15.103:0/2103554921 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff846a10640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff840109130 0x7ff84019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff840114160 con 0x7ff8401049b0 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff840113fe0 con 0x7ff840109130 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.763+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401142e0 con 0x7ff840104000 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff844fad640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff840109130 0x7ff84019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83f7fe640 1 -- 172.21.15.103:0/2103554921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840104000 msgr2=0x7ff84019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840104000 0x7ff84019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83f7fe640 1 -- 172.21.15.103:0/2103554921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff840109130 msgr2=0x7ff84019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff840109130 0x7ff84019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83f7fe640 1 -- 172.21.15.103:0/2103554921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84019f5f0 con 0x7ff8401049b0 2024-09-19T07:40:16.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83ffff640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840104000 0x7ff84019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:16.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8401049b0 0x7ff84019ab80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e9f0 tx=0x7ff82800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:16.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83d7fa640 1 -- 172.21.15.103:0/2103554921 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82800ccc0 con 0x7ff8401049b0 2024-09-19T07:40:16.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.764+0000 7ff83d7fa640 1 -- 172.21.15.103:0/2103554921 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff828005020 con 0x7ff8401049b0 2024-09-19T07:40:16.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.765+0000 7ff83d7fa640 1 -- 172.21.15.103:0/2103554921 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82800ccc0 con 0x7ff8401049b0 2024-09-19T07:40:16.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.765+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84019f8e0 con 0x7ff8401049b0 2024-09-19T07:40:16.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.766+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84007a670 con 0x7ff8401049b0 2024-09-19T07:40:16.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.768+0000 7ff83d7fa640 1 -- 172.21.15.103:0/2103554921 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff8280051d0 con 0x7ff8401049b0 2024-09-19T07:40:16.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.768+0000 7ff848de9740 1 Processor -- start 2024-09-19T07:40:16.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.768+0000 7ff848de9740 1 -- start start 2024-09-19T07:40:16.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83d7fa640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff82c0a35b0 0x7ff82c0a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83d7fa640 1 -- 172.21.15.103:0/2103554921 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff8280cfdb0 con 0x7ff8401049b0 2024-09-19T07:40:16.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff848de9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff844fad640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff82c0a35b0 0x7ff82c0a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff848de9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d73140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff848de9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d75a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff848de9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559fa5cce2e0 con 0x559fa5d73680 2024-09-19T07:40:16.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff848de9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559fa5c75ce0 con 0x559fa5d68dc0 2024-09-19T07:40:16.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff848de9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559fa5cca2a0 con 0x559fa5d684a0 2024-09-19T07:40:16.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff844fad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d75a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d68880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54358/0 (socket says 172.21.15.103:54358) 2024-09-19T07:40:16.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 -- 172.21.15.103:0/117148217 learned_addr learned my addr 172.21.15.103:0/117148217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:16.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d73140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 -- 172.21.15.103:0/117148217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 msgr2=0x559fa5d73140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d73140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 -- 172.21.15.103:0/117148217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 msgr2=0x559fa5d75a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d75a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff83ffff640 1 -- 172.21.15.103:0/117148217 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa5c5a440 con 0x559fa5d684a0 2024-09-19T07:40:16.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.769+0000 7ff844fad640 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d75a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:16.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.770+0000 7ff844fad640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff82c0a35b0 0x7ff82c0a5a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff838006010 tx=0x7ff838031820 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:16.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.770+0000 7ff83ffff640 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d68880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff830016c10 tx=0x7ff830009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a119bb74bc7af00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:16.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.771+0000 7ff8367fc640 1 -- 172.21.15.103:0/117148217 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83001f420 con 0x559fa5d684a0 2024-09-19T07:40:16.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.771+0000 7ff8367fc640 1 -- 172.21.15.103:0/117148217 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff830024070 con 0x559fa5d684a0 2024-09-19T07:40:16.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.771+0000 7ff8367fc640 1 -- 172.21.15.103:0/117148217 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83001ea80 con 0x559fa5d684a0 2024-09-19T07:40:16.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 -- 172.21.15.103:0/117148217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 msgr2=0x559fa5d68880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d68880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff830016c10 tx=0x7ff830009500 comp rx=0 tx=0).stop 2024-09-19T07:40:16.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 -- 172.21.15.103:0/117148217 shutdown_connections 2024-09-19T07:40:16.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d75a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d73140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 --2- 172.21.15.103:0/117148217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d68880 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 -- 172.21.15.103:0/117148217 >> 172.21.15.103:0/117148217 conn(0x559fa5d64a30 msgr2=0x559fa5d64e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:16.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 -- 172.21.15.103:0/117148217 shutdown_connections 2024-09-19T07:40:16.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.772+0000 7ff848de9740 1 -- 172.21.15.103:0/117148217 wait complete. 2024-09-19T07:40:16.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.773+0000 7ff848de9740 1 -- start start 2024-09-19T07:40:16.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.773+0000 7ff848de9740 1 Processor -- start 2024-09-19T07:40:16.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff848de9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d72f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff848de9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d70200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff848de9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d70740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff848de9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559fa5d762a0 con 0x559fa5d73680 2024-09-19T07:40:16.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff848de9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559fa5c89050 con 0x559fa5d68dc0 2024-09-19T07:40:16.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff848de9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559fa5d76420 con 0x559fa5d684a0 2024-09-19T07:40:16.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff848de9740 10 client.0 _ll_register_callbacks cb 0x559fa5cc9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:16.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d70200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d70200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38650/0 (socket says 172.21.15.103:38650) 2024-09-19T07:40:16.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 -- 172.21.15.103:0/1492083345 learned_addr learned my addr 172.21.15.103:0/1492083345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:16.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83ffff640 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d72f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 -- 172.21.15.103:0/1492083345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 msgr2=0x559fa5d72f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d72f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 -- 172.21.15.103:0/1492083345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 msgr2=0x559fa5d70740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d70740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83f7fe640 1 -- 172.21.15.103:0/1492083345 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa5a3f3c0 con 0x559fa5d68dc0 2024-09-19T07:40:16.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.774+0000 7ff83ffff640 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d72f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:40:16.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d70200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e9c0 tx=0x7ff8280a0450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638c25bcfdd44dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:16.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:40:16.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff81f7fe640 1 -- 172.21.15.103:0/1492083345 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828098710 con 0x559fa5d68dc0 2024-09-19T07:40:16.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff81f7fe640 1 -- 172.21.15.103:0/1492083345 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff8280a4880 con 0x559fa5d68dc0 2024-09-19T07:40:16.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff830024070 con 0x559fa5d68dc0 2024-09-19T07:40:16.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff81f7fe640 1 -- 172.21.15.103:0/1492083345 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8280988b0 con 0x559fa5d68dc0 2024-09-19T07:40:16.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff848de9740 10 client.25466 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:16.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.775+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fa5d81ca0 con 0x559fa5d68dc0 2024-09-19T07:40:16.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff81f7fe640 1 -- 172.21.15.103:0/1492083345 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff830024070 con 0x559fa5d68dc0 2024-09-19T07:40:16.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff81f7fe640 1 -- 172.21.15.103:0/1492083345 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff8280b7030 con 0x559fa5d68dc0 2024-09-19T07:40:16.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff848de9740 10 client.25466 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:16.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff848de9740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:16.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff848de9740 20 client.25466 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:16.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff848de9740 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5d830b0 0x559fa5d85520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff848de9740 4 client.25466 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:16.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.776+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa5cdb850 con 0x559fa5d830b0 2024-09-19T07:40:16.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.779+0000 7ff83ffff640 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5d830b0 0x559fa5d85520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.780+0000 7ff83ffff640 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5d830b0 0x559fa5d85520 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc05777b25c7147 server_cookie=e7b7711959ea3c0b in_seq=0 out_seq=0 2024-09-19T07:40:16.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.780+0000 7ff81f7fe640 10 client.25466 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:16.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.780+0000 7ff81f7fe640 1 -- 172.21.15.103:0/1492083345 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fa5cdb850 con 0x559fa5d830b0 2024-09-19T07:40:16.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.780+0000 7ff81f7fe640 10 client.25466 handle_command_reply: tid=0 2024-09-19T07:40:16.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.780+0000 7ff848de9740 1 client.25466 shutdown 2024-09-19T07:40:16.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.781+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 msgr2=0x559fa5d70200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.781+0000 7ff848de9740 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d70200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e9c0 tx=0x7ff8280a0450 comp rx=0 tx=0).stop 2024-09-19T07:40:16.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 shutdown_connections 2024-09-19T07:40:16.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5d830b0 0x559fa5d85520 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d70740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d68dc0 0x559fa5d70200 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff81f7fe640 0 client.25466 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:16.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 --2- 172.21.15.103:0/1492083345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d684a0 0x559fa5d72f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 >> 172.21.15.103:0/1492083345 conn(0x559fa5d64a30 msgr2=0x559fa5d63b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:16.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 shutdown_connections 2024-09-19T07:40:16.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.782+0000 7ff848de9740 1 -- 172.21.15.103:0/1492083345 wait complete. 2024-09-19T07:40:16.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.783+0000 7ff848de9740 20 client.25466 trim_cache size 0 max 16384 2024-09-19T07:40:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.807+0000 7ff848de9740 1 Processor -- start 2024-09-19T07:40:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.807+0000 7ff848de9740 1 -- start start 2024-09-19T07:40:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff848de9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d7f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff848de9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d68dc0 0x559fa5d8ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff848de9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d8b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff848de9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff840070260 con 0x559fa5d73680 2024-09-19T07:40:16.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff848de9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff840071e60 con 0x559fa5d684a0 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff848de9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff840070930 con 0x559fa5d68dc0 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d7f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d8b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d8b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60596/0 (socket says 172.21.15.103:60596) 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 -- 172.21.15.103:0/2328977594 learned_addr learned my addr 172.21.15.103:0/2328977594 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d68dc0 0x559fa5d8ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 -- 172.21.15.103:0/2328977594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d68dc0 msgr2=0x559fa5d8ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d68dc0 0x559fa5d8ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 -- 172.21.15.103:0/2328977594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 msgr2=0x559fa5d7f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d7f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 -- 172.21.15.103:0/2328977594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa5d65ba0 con 0x559fa5d73680 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d68dc0 0x559fa5d8ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:16.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff83ffff640 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d7f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:16.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.808+0000 7ff844fad640 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d8b370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff838009900 tx=0x7ff838044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f695362021d7d451 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff835ffb640 1 -- 172.21.15.103:0/2328977594 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838035040 con 0x559fa5d73680 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff835ffb640 1 -- 172.21.15.103:0/2328977594 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff838003ad0 con 0x559fa5d73680 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff835ffb640 1 -- 172.21.15.103:0/2328977594 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83804aa10 con 0x559fa5d73680 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 -- 172.21.15.103:0/2328977594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 msgr2=0x559fa5d8b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d8b370 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff838009900 tx=0x7ff838044000 comp rx=0 tx=0).stop 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 -- 172.21.15.103:0/2328977594 shutdown_connections 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d73680 0x559fa5d8b370 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d68dc0 0x559fa5d8ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 --2- 172.21.15.103:0/2328977594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d7f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 -- 172.21.15.103:0/2328977594 >> 172.21.15.103:0/2328977594 conn(0x559fa5d64a30 msgr2=0x559fa5d6ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.809+0000 7ff848de9740 1 -- 172.21.15.103:0/2328977594 shutdown_connections 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.810+0000 7ff848de9740 1 -- 172.21.15.103:0/2328977594 wait complete. 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.810+0000 7ff848de9740 1 -- start start 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.810+0000 7ff848de9740 1 Processor -- start 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff848de9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff848de9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d68dc0 0x559fa5d6d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff848de9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d73680 0x559fa5d6de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff848de9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400716d0 con 0x559fa5d68dc0 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff848de9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff840071000 con 0x559fa5d684a0 2024-09-19T07:40:16.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff848de9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff84006e240 con 0x559fa5d73680 2024-09-19T07:40:16.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff83ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d6d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38680/0 (socket says 172.21.15.103:38680) 2024-09-19T07:40:16.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff83ffff640 1 -- 172.21.15.103:0/2072973378 learned_addr learned my addr 172.21.15.103:0/2072973378 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:16.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.811+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d68dc0 0x559fa5d6d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.812+0000 7ff83f7fe640 1 -- 172.21.15.103:0/2072973378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d73680 msgr2=0x559fa5d6de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.812+0000 7ff848de9740 10 client.0 _ll_register_callbacks cb 0x7ff82809ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:16.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.812+0000 7ff844fad640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d73680 0x559fa5d6de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.813+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d73680 0x559fa5d6de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.813+0000 7ff83f7fe640 1 -- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 msgr2=0x559fa5d6d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:16.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.813+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d6d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.813+0000 7ff83f7fe640 1 -- 172.21.15.103:0/2072973378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa5decf60 con 0x559fa5d68dc0 2024-09-19T07:40:16.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.813+0000 7ff83ffff640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:16.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.813+0000 7ff844fad640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d73680 0x559fa5d6de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:40:16.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.813+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff838003ad0 con 0x559fa5d68dc0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff83f7fe640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d68dc0 0x559fa5d6d8c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff8280cdb10 tx=0x7ff82809b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb8d95d38ab77d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff81f7fe640 10 client.15963 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff81f7fe640 1 -- 172.21.15.103:0/2072973378 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828098710 con 0x559fa5d68dc0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff81f7fe640 1 -- 172.21.15.103:0/2072973378 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff828097df0 con 0x559fa5d68dc0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff81f7fe640 1 -- 172.21.15.103:0/2072973378 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff828097ff0 con 0x559fa5d68dc0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff848de9740 10 client.15963 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559fa5dee130 con 0x559fa5d68dc0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff81f7fe640 1 -- 172.21.15.103:0/2072973378 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff8280b7040 con 0x559fa5d68dc0 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff848de9740 10 client.15963 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff848de9740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.814+0000 7ff848de9740 20 client.15963 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff848de9740 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5dee4f0 0x559fa5df0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:16.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff848de9740 4 client.15963 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fa5d70ad0 con 0x559fa5dee4f0 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff83ffff640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5dee4f0 0x559fa5df0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff83ffff640 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5dee4f0 0x559fa5df0910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d7dba24dcbbbc3 server_cookie=2662f6dcbceedbe9 in_seq=0 out_seq=0 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff81f7fe640 10 client.15963 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff81f7fe640 1 -- 172.21.15.103:0/2072973378 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559fa5d70ad0 con 0x559fa5dee4f0 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff81f7fe640 10 client.15963 handle_command_reply: tid=0 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff848de9740 1 client.15963 shutdown 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d68dc0 msgr2=0x559fa5d6d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff848de9740 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d68dc0 0x559fa5d6d8c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff8280cdb10 tx=0x7ff82809b700 comp rx=0 tx=0).stop 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.815+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 shutdown_connections 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559fa5dee4f0 0x559fa5df0910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fa5d73680 0x559fa5d6de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559fa5d68dc0 0x559fa5d6d8c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 1 --2- 172.21.15.103:0/2072973378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa5d684a0 0x559fa5d6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff81f7fe640 0 client.15963 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 >> 172.21.15.103:0/2072973378 conn(0x559fa5d64a30 msgr2=0x559fa5d69ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 shutdown_connections 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 1 -- 172.21.15.103:0/2072973378 wait complete. 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.816+0000 7ff848de9740 20 client.15963 trim_cache size 0 max 16384 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:40:16.820 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:40:16.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff82c0a35b0 msgr2=0x7ff82c0a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff82c0a35b0 0x7ff82c0a5a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff838006010 tx=0x7ff838031820 comp rx=0 tx=0).stop 2024-09-19T07:40:16.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8401049b0 msgr2=0x7ff84019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:16.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8401049b0 0x7ff84019ab80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e9f0 tx=0x7ff82800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 shutdown_connections 2024-09-19T07:40:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff82c0a35b0 0x7ff82c0a5a70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff840109130 0x7ff84019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8401049b0 0x7ff84019ab80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 --2- 172.21.15.103:0/2103554921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff840104000 0x7ff84019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 >> 172.21.15.103:0/2103554921 conn(0x7ff840077a90 msgr2=0x7ff840078e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:16.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 shutdown_connections 2024-09-19T07:40:16.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:16.817+0000 7ff846a10640 1 -- 172.21.15.103:0/2103554921 wait complete. 2024-09-19T07:40:17.939 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:40:17.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:40:18.291 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:18 smithi103 ceph-mon[31201]: pgmap v72: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-19T07:40:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:18 smithi134 ceph-mon[35414]: pgmap v72: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-19T07:40:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:18 smithi174 ceph-mon[35370]: pgmap v72: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-19T07:40:19.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.898+0000 7f5b46466640 1 -- 172.21.15.103:0/989228038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40100a70 msgr2=0x7f5b40100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:19.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.898+0000 7f5b46466640 1 --2- 172.21.15.103:0/989228038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40100a70 0x7f5b40100ed0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009930 tx=0x7f5b3402a420 comp rx=0 tx=0).stop 2024-09-19T07:40:19.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 -- 172.21.15.103:0/989228038 shutdown_connections 2024-09-19T07:40:19.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 --2- 172.21.15.103:0/989228038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b40101410 0x7f5b4010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:19.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 --2- 172.21.15.103:0/989228038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40100a70 0x7f5b40100ed0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:19.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 --2- 172.21.15.103:0/989228038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b40106a70 0x7f5b40106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:19.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 -- 172.21.15.103:0/989228038 >> 172.21.15.103:0/989228038 conn(0x7f5b400fc790 msgr2=0x7f5b400febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:19.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 -- 172.21.15.103:0/989228038 shutdown_connections 2024-09-19T07:40:19.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 -- 172.21.15.103:0/989228038 wait complete. 2024-09-19T07:40:19.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 Processor -- start 2024-09-19T07:40:19.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.899+0000 7f5b46466640 1 -- start start 2024-09-19T07:40:19.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b46466640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b40100a70 0x7f5b40198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:19.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b46466640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b40101410 0x7f5b40198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:19.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b46466640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40106a70 0x7f5b4019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:19.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b46466640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40110e70 con 0x7f5b40106a70 2024-09-19T07:40:19.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b46466640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40110cf0 con 0x7f5b40100a70 2024-09-19T07:40:19.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b46466640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40110ff0 con 0x7f5b40101410 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b44a03640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40106a70 0x7f5b4019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b3f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b40101410 0x7f5b40198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b44a03640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40106a70 0x7f5b4019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55796/0 (socket says 172.21.15.103:55796) 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b44a03640 1 -- 172.21.15.103:0/1811871988 learned_addr learned my addr 172.21.15.103:0/1811871988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.900+0000 7f5b3ffff640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b40100a70 0x7f5b40198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.901+0000 7f5b44a03640 1 -- 172.21.15.103:0/1811871988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b40101410 msgr2=0x7f5b40198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.901+0000 7f5b44a03640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b40101410 0x7f5b40198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.901+0000 7f5b44a03640 1 -- 172.21.15.103:0/1811871988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b40100a70 msgr2=0x7f5b40198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.901+0000 7f5b44a03640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b40100a70 0x7f5b40198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.901+0000 7f5b44a03640 1 -- 172.21.15.103:0/1811871988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4019d3b0 con 0x7f5b40106a70 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.901+0000 7f5b44a03640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40106a70 0x7f5b4019ccd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800cce0 tx=0x7f5b3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:19.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.901+0000 7f5b3d7fa640 1 -- 172.21.15.103:0/1811871988 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800f450 con 0x7f5b40106a70 2024-09-19T07:40:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.902+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4019d6a0 con 0x7f5b40106a70 2024-09-19T07:40:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.902+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401a4f80 con 0x7f5b40106a70 2024-09-19T07:40:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.902+0000 7f5b3d7fa640 1 -- 172.21.15.103:0/1811871988 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b3800fad0 con 0x7f5b40106a70 2024-09-19T07:40:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.902+0000 7f5b3d7fa640 1 -- 172.21.15.103:0/1811871988 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38018c10 con 0x7f5b40106a70 2024-09-19T07:40:19.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.905+0000 7f5b3d7fa640 1 -- 172.21.15.103:0/1811871988 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5b38018400 con 0x7f5b40106a70 2024-09-19T07:40:19.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.905+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c0052c0 con 0x7f5b40106a70 2024-09-19T07:40:19.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.906+0000 7f5b3d7fa640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5b280a3430 0x7f5b280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:19.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.906+0000 7f5b3d7fa640 1 -- 172.21.15.103:0/1811871988 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5b380d0160 con 0x7f5b40106a70 2024-09-19T07:40:19.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.906+0000 7f5b3ffff640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5b280a3430 0x7f5b280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:19.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.907+0000 7f5b3ffff640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5b280a3430 0x7f5b280a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009b50 tx=0x7f5b30005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:19.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:19.912+0000 7f5b3d7fa640 1 -- 172.21.15.103:0/1811871988 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b38014030 con 0x7f5b40106a70 2024-09-19T07:40:20.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.225+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b2c005880 con 0x7f5b40106a70 2024-09-19T07:40:20.231 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:40:20.231 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":30,"btime":"2024-09-19T07:39:09:495445+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:39:08.515410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:40:20.231 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 30 2024-09-19T07:40:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.231+0000 7f5b3d7fa640 1 -- 172.21.15.103:0/1811871988 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11998 (secure 0 0 0) 0x7f5b3809caf0 con 0x7f5b40106a70 2024-09-19T07:40:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.236+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5b280a3430 msgr2=0x7f5b280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.236+0000 7f5b46466640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5b280a3430 0x7f5b280a58f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009b50 tx=0x7f5b30005f90 comp rx=0 tx=0).stop 2024-09-19T07:40:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.236+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40106a70 msgr2=0x7f5b4019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.236+0000 7f5b46466640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40106a70 0x7f5b4019ccd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800cce0 tx=0x7f5b3800a620 comp rx=0 tx=0).stop 2024-09-19T07:40:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 shutdown_connections 2024-09-19T07:40:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5b280a3430 0x7f5b280a58f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:20.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b40106a70 0x7f5b4019ccd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b40101410 0x7f5b40198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 --2- 172.21.15.103:0/1811871988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b40100a70 0x7f5b40198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 >> 172.21.15.103:0/1811871988 conn(0x7f5b400fc790 msgr2=0x7f5b4006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 shutdown_connections 2024-09-19T07:40:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:20.237+0000 7f5b46466640 1 -- 172.21.15.103:0/1811871988 wait complete. 2024-09-19T07:40:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 105.0 s, 3809.8 tps, lat 26.268 ms stddev 27.507 2024-09-19T07:40:20.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:20 smithi103 ceph-mon[31201]: pgmap v73: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-19T07:40:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:20 smithi134 ceph-mon[35414]: pgmap v73: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-19T07:40:20.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:20 smithi174 ceph-mon[35370]: pgmap v73: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 19 MiB/s wr, 1.19k op/s 2024-09-19T07:40:21.014 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:40:21.384 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:21.514 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:21 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1811871988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:40:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:21 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1811871988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:40:21.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:21 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1811871988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:40:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 -- 172.21.15.103:0/3841438891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388106a90 msgr2=0x7f6388106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 --2- 172.21.15.103:0/3841438891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388106a90 0x7f6388106e70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6378009930 tx=0x7f637802a420 comp rx=0 tx=0).stop 2024-09-19T07:40:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 -- 172.21.15.103:0/3841438891 shutdown_connections 2024-09-19T07:40:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 --2- 172.21.15.103:0/3841438891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6388101430 0x7f638810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 --2- 172.21.15.103:0/3841438891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 0x7f6388100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 --2- 172.21.15.103:0/3841438891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388106a90 0x7f6388106e70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 -- 172.21.15.103:0/3841438891 >> 172.21.15.103:0/3841438891 conn(0x7f63880fc7d0 msgr2=0x7f63880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 -- 172.21.15.103:0/3841438891 shutdown_connections 2024-09-19T07:40:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.191+0000 7f638e023640 1 -- 172.21.15.103:0/3841438891 wait complete. 2024-09-19T07:40:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 Processor -- start 2024-09-19T07:40:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 -- start start 2024-09-19T07:40:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 0x7f6388198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388101430 0x7f6388198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6388106a90 0x7f638819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6388111f90 con 0x7f6388101430 2024-09-19T07:40:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6388111e10 con 0x7f6388100a90 2024-09-19T07:40:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f638e023640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6388112110 con 0x7f6388106a90 2024-09-19T07:40:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f63877fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 0x7f6388198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f63877fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 0x7f6388198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43460/0 (socket says 172.21.15.103:43460) 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f63877fe640 1 -- 172.21.15.103:0/1320546932 learned_addr learned my addr 172.21.15.103:0/1320546932 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f637ffff640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388101430 0x7f6388198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.192+0000 7f6387fff640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6388106a90 0x7f638819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.193+0000 7f63877fe640 1 -- 172.21.15.103:0/1320546932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6388106a90 msgr2=0x7f638819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.193+0000 7f63877fe640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6388106a90 0x7f638819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.193+0000 7f63877fe640 1 -- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388101430 msgr2=0x7f6388198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.193+0000 7f63877fe640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388101430 0x7f6388198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.193+0000 7f63877fe640 1 -- 172.21.15.103:0/1320546932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638819d3c0 con 0x7f6388100a90 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.193+0000 7f63877fe640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 0x7f6388198410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6378009230 tx=0x7f63780045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.194+0000 7f63857fa640 1 -- 172.21.15.103:0/1320546932 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378040070 con 0x7f6388100a90 2024-09-19T07:40:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.194+0000 7f63857fa640 1 -- 172.21.15.103:0/1320546932 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6378031c80 con 0x7f6388100a90 2024-09-19T07:40:22.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.194+0000 7f63857fa640 1 -- 172.21.15.103:0/1320546932 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378035020 con 0x7f6388100a90 2024-09-19T07:40:22.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.194+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638819d650 con 0x7f6388100a90 2024-09-19T07:40:22.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.194+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638819dae0 con 0x7f6388100a90 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.196+0000 7f63857fa640 1 -- 172.21.15.103:0/1320546932 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6378043070 con 0x7f6388100a90 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 Processor -- start 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 -- start start 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63857fa640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63680a3500 0x7f63680a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594679342f0 con 0x5594679ce5d0 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5594678dbcf0 con 0x5594679cfb40 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63903fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5594679302b0 con 0x5594679d9370 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f6387fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63857fa640 1 -- 172.21.15.103:0/1320546932 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f63780f06c0 con 0x7f6388100a90 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f6387fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43466/0 (socket says 172.21.15.103:43466) 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f6387fff640 1 -- 172.21.15.103:0/2159771662 learned_addr learned my addr 172.21.15.103:0/2159771662 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f63877fe640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63680a3500 0x7f63680a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.197+0000 7f6387fff640 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.198+0000 7f637ffff640 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.198+0000 7f63877fe640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63680a3500 0x7f63680a59c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63780b9320 tx=0x7f637802a420 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.199+0000 7f637ffff640 1 -- 172.21.15.103:0/2159771662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 msgr2=0x5594679db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.199+0000 7f637ffff640 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679db760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.199+0000 7f637ffff640 1 -- 172.21.15.103:0/2159771662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 msgr2=0x5594679ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.199+0000 7f637ffff640 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.199+0000 7f637ffff640 1 -- 172.21.15.103:0/2159771662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594678c03a0 con 0x5594679ce5d0 2024-09-19T07:40:22.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.200+0000 7f637ffff640 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679cea10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6370009bf0 tx=0x7f6370011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de203c5834362f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:22.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.200+0000 7f637dffb640 1 -- 172.21.15.103:0/2159771662 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637001e070 con 0x5594679ce5d0 2024-09-19T07:40:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.201+0000 7f637dffb640 1 -- 172.21.15.103:0/2159771662 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6370009da0 con 0x5594679ce5d0 2024-09-19T07:40:22.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.201+0000 7f637dffb640 1 -- 172.21.15.103:0/2159771662 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370018730 con 0x5594679ce5d0 2024-09-19T07:40:22.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 -- 172.21.15.103:0/2159771662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 msgr2=0x5594679cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679cea10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6370009bf0 tx=0x7f6370011810 comp rx=0 tx=0).stop 2024-09-19T07:40:22.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 -- 172.21.15.103:0/2159771662 shutdown_connections 2024-09-19T07:40:22.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679cea10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 --2- 172.21.15.103:0/2159771662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 -- 172.21.15.103:0/2159771662 >> 172.21.15.103:0/2159771662 conn(0x559467997310 msgr2=0x5594679cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 -- 172.21.15.103:0/2159771662 shutdown_connections 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 -- 172.21.15.103:0/2159771662 wait complete. 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 -- start start 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.202+0000 7f63903fc740 1 Processor -- start 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f63903fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679dd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f63903fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f63903fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f63903fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594679dc110 con 0x5594679ce5d0 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f63903fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5594678ef060 con 0x5594679cfb40 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f63903fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5594679dc290 con 0x5594679d9370 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f63903fc740 10 client.0 _ll_register_callbacks cb 0x5594676428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f6387fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679dd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ddd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43476/0 (socket says 172.21.15.103:43476) 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 -- 172.21.15.103:0/2623266946 learned_addr learned my addr 172.21.15.103:0/2623266946 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f6387fff640 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 -- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 msgr2=0x5594679e1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679e1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 -- 172.21.15.103:0/2623266946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 msgr2=0x5594679dd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679dd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.203+0000 7f637ffff640 1 -- 172.21.15.103:0/2623266946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594676a53c0 con 0x5594679cfb40 2024-09-19T07:40:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.204+0000 7f6387fff640 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:22.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.204+0000 7f637ffff640 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ddd00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63700097e0 tx=0x7f6370019d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16aba67263ffe12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.204+0000 7f6376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.204+0000 7f6376ffd640 1 -- 172.21.15.103:0/2623266946 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370026020 con 0x5594679cfb40 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.204+0000 7f6376ffd640 1 -- 172.21.15.103:0/2623266946 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6370005020 con 0x5594679cfb40 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.204+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6370009da0 con 0x5594679cfb40 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.204+0000 7f6376ffd640 1 -- 172.21.15.103:0/2623266946 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f637001e040 con 0x5594679cfb40 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 10 client.25490 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5594679ebf70 con 0x5594679cfb40 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f6376ffd640 1 -- 172.21.15.103:0/2623266946 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6370009da0 con 0x5594679cfb40 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f6376ffd640 1 -- 172.21.15.103:0/2623266946 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6370021070 con 0x5594679cfb40 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 10 client.25490 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 10 client.25490 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:40:22.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 20 client.25490 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 4 client.25490 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559467941860 con 0x5594679ed2c0 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.205+0000 7f6387fff640 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.206+0000 7f6376ffd640 10 client.25490 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.206+0000 7f6387fff640 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679ef6b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1b1c82e56d66c67 server_cookie=b90f8bf5731fe07b in_seq=0 out_seq=0 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.206+0000 7f6376ffd640 1 -- 172.21.15.103:0/2623266946 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559467941860 con 0x5594679ed2c0 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.206+0000 7f6376ffd640 10 client.25490 handle_command_reply: tid=0 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 client.25490 shutdown 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 msgr2=0x5594679ddd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ddd00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63700097e0 tx=0x7f6370019d50 comp rx=0 tx=0).stop 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 shutdown_connections 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679ef6b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679cfb40 0x5594679ddd00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f6376ffd640 0 client.25490 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:40:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 --2- 172.21.15.103:0/2623266946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679dd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.207+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 >> 172.21.15.103:0/2623266946 conn(0x559467997310 msgr2=0x5594679d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.208+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 shutdown_connections 2024-09-19T07:40:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.208+0000 7f63903fc740 1 -- 172.21.15.103:0/2623266946 wait complete. 2024-09-19T07:40:22.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.208+0000 7f63903fc740 20 client.25490 trim_cache size 0 max 16384 2024-09-19T07:40:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 Processor -- start 2024-09-19T07:40:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 -- start start 2024-09-19T07:40:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679ce5d0 0x5594679e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679cfb40 0x5594679cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63881ab620 con 0x5594679cfb40 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63881ad220 con 0x5594679ce5d0 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f63903fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63881abcf0 con 0x5594679d9370 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f6387fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679ce5d0 0x5594679e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f6387fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679ce5d0 0x5594679e1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43486/0 (socket says 172.21.15.103:43486) 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.224+0000 7f6387fff640 1 -- 172.21.15.103:0/767588660 learned_addr learned my addr 172.21.15.103:0/767588660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f6387fff640 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f637ffff640 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679cfb40 0x5594679cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f6387fff640 1 -- 172.21.15.103:0/767588660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 msgr2=0x5594679f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f6387fff640 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f6387fff640 1 -- 172.21.15.103:0/767588660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679cfb40 msgr2=0x5594679cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f6387fff640 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679cfb40 0x5594679cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f6387fff640 1 -- 172.21.15.103:0/767588660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594679ca510 con 0x5594679ce5d0 2024-09-19T07:40:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f6387fff640 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679ce5d0 0x5594679e1e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6380004a80 tx=0x7f6380009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ced3655bf611cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f637d7fa640 1 -- 172.21.15.103:0/767588660 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380004b20 con 0x5594679ce5d0 2024-09-19T07:40:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f637d7fa640 1 -- 172.21.15.103:0/767588660 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6380026070 con 0x5594679ce5d0 2024-09-19T07:40:22.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.225+0000 7f637d7fa640 1 -- 172.21.15.103:0/767588660 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63800045b0 con 0x5594679ce5d0 2024-09-19T07:40:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.227+0000 7f63903fc740 1 -- 172.21.15.103:0/767588660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679ce5d0 msgr2=0x5594679e1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.227+0000 7f63903fc740 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679ce5d0 0x5594679e1e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6380004a80 tx=0x7f6380009a20 comp rx=0 tx=0).stop 2024-09-19T07:40:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.227+0000 7f63903fc740 1 -- 172.21.15.103:0/767588660 shutdown_connections 2024-09-19T07:40:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.227+0000 7f63903fc740 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679d9370 0x5594679f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.227+0000 7f63903fc740 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679cfb40 0x5594679cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.227+0000 7f63903fc740 1 --2- 172.21.15.103:0/767588660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679ce5d0 0x5594679e1e60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.227+0000 7f63903fc740 1 -- 172.21.15.103:0/767588660 >> 172.21.15.103:0/767588660 conn(0x559467997310 msgr2=0x5594679ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.228+0000 7f63903fc740 1 -- 172.21.15.103:0/767588660 shutdown_connections 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.228+0000 7f63903fc740 1 -- 172.21.15.103:0/767588660 wait complete. 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.228+0000 7f63903fc740 1 -- start start 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 1 Processor -- start 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679cfb40 0x5594679e4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679d9370 0x5594679e5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63881aca90 con 0x5594679ce5d0 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63881ac3c0 con 0x5594679d9370 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63881a9600 con 0x5594679cfb40 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.229+0000 7f63903fc740 10 client.0 _ll_register_callbacks cb 0x7f637001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:22.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679e9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55832/0 (socket says 172.21.15.103:55832) 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 -- 172.21.15.103:0/4192115538 learned_addr learned my addr 172.21.15.103:0/4192115538 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f637ffff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679cfb40 0x5594679e4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679d9370 0x5594679e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 -- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679cfb40 msgr2=0x5594679e4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679cfb40 0x5594679e4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 -- 172.21.15.103:0/4192115538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679d9370 msgr2=0x5594679e5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679d9370 0x5594679e5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 -- 172.21.15.103:0/4192115538 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594679e5900 con 0x5594679ce5d0 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f637ffff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679cfb40 0x5594679e4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6387fff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679e9890 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6380009540 tx=0x7f6380012970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b612ab722f23ee8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:22.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.230+0000 7f6376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:40:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.231+0000 7f6376ffd640 1 -- 172.21.15.103:0/4192115538 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380004dc0 con 0x5594679ce5d0 2024-09-19T07:40:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.231+0000 7f6376ffd640 1 -- 172.21.15.103:0/4192115538 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f638001b040 con 0x5594679ce5d0 2024-09-19T07:40:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.231+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6380026070 con 0x5594679ce5d0 2024-09-19T07:40:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.231+0000 7f6376ffd640 1 -- 172.21.15.103:0/4192115538 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6380040050 con 0x5594679ce5d0 2024-09-19T07:40:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.231+0000 7f63903fc740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.231+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5594679d1690 con 0x5594679ce5d0 2024-09-19T07:40:22.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f6376ffd640 1 -- 172.21.15.103:0/4192115538 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6380026070 con 0x5594679ce5d0 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f6376ffd640 1 -- 172.21.15.103:0/4192115538 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6380021040 con 0x5594679ce5d0 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f63903fc740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f63903fc740 10 client.15987 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f63903fc740 20 client.15987 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f63903fc740 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f63903fc740 4 client.15987 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55946793bd70 con 0x5594679ed2c0 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.232+0000 7f637ffff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.233+0000 7f6376ffd640 10 client.15987 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:40:22.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f637ffff640 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679d3d70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1ee3170d229e15e server_cookie=fb68ff92e725ed83 in_seq=0 out_seq=0 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f6376ffd640 1 -- 172.21.15.103:0/4192115538 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55946793bd70 con 0x5594679ed2c0 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f6376ffd640 10 client.15987 handle_command_reply: tid=0 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 client.15987 shutdown 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 msgr2=0x5594679e9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679e9890 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6380009540 tx=0x7f6380012970 comp rx=0 tx=0).stop 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 shutdown_connections 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5594679ed2c0 0x5594679d3d70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5594679d9370 0x5594679e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594679cfb40 0x5594679e4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 --2- 172.21.15.103:0/4192115538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594679ce5d0 0x5594679e9890 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 >> 172.21.15.103:0/4192115538 conn(0x559467997310 msgr2=0x5594679c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f6376ffd640 0 client.15987 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:40:22.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.234+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 shutdown_connections 2024-09-19T07:40:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f63903fc740 1 -- 172.21.15.103:0/4192115538 wait complete. 2024-09-19T07:40:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f63903fc740 20 client.15987 trim_cache size 0 max 16384 2024-09-19T07:40:22.238 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:40:22.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63680a3500 msgr2=0x7f63680a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63680a3500 0x7f63680a59c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63780b9320 tx=0x7f637802a420 comp rx=0 tx=0).stop 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 msgr2=0x7f6388198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 0x7f6388198410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6378009230 tx=0x7f63780045e0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 shutdown_connections 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63680a3500 0x7f63680a59c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6388106a90 0x7f638819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6388101430 0x7f6388198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 --2- 172.21.15.103:0/1320546932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6388100a90 0x7f6388198410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 >> 172.21.15.103:0/1320546932 conn(0x7f63880fc7d0 msgr2=0x7f63880fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 shutdown_connections 2024-09-19T07:40:22.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:22.235+0000 7f638e023640 1 -- 172.21.15.103:0/1320546932 wait complete. 2024-09-19T07:40:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:22 smithi103 ceph-mon[31201]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 1.51k op/s 2024-09-19T07:40:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:22 smithi134 ceph-mon[35414]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 1.51k op/s 2024-09-19T07:40:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:22 smithi174 ceph-mon[35370]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 1.51k op/s 2024-09-19T07:40:23.099 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:40:23.465 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:23.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.991+0000 7f17f2b13640 1 -- 172.21.15.103:0/236690697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec101dd0 msgr2=0x7f17ec1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:23.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.991+0000 7f17f2b13640 1 --2- 172.21.15.103:0/236690697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec101dd0 0x7f17ec1021b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e002a410 comp rx=0 tx=0).stop 2024-09-19T07:40:23.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.991+0000 7f17f2b13640 1 -- 172.21.15.103:0/236690697 shutdown_connections 2024-09-19T07:40:23.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.991+0000 7f17f2b13640 1 --2- 172.21.15.103:0/236690697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec106f00 0x7f17ec10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:23.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.991+0000 7f17f2b13640 1 --2- 172.21.15.103:0/236690697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17ec102780 0x7f17ec1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:23.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.991+0000 7f17f2b13640 1 --2- 172.21.15.103:0/236690697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec101dd0 0x7f17ec1021b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:23.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.991+0000 7f17f2b13640 1 -- 172.21.15.103:0/236690697 >> 172.21.15.103:0/236690697 conn(0x7f17ec0fd8b0 msgr2=0x7f17ec0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:23.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.992+0000 7f17f2b13640 1 -- 172.21.15.103:0/236690697 shutdown_connections 2024-09-19T07:40:23.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.992+0000 7f17f2b13640 1 -- 172.21.15.103:0/236690697 wait complete. 2024-09-19T07:40:23.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.992+0000 7f17f2b13640 1 Processor -- start 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f2b13640 1 -- start start 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f2b13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec101dd0 0x7f17ec1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f2b13640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17ec102780 0x7f17ec198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f2b13640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec106f00 0x7f17ec19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f2b13640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec111ff0 con 0x7f17ec101dd0 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f2b13640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec111e70 con 0x7f17ec106f00 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f2b13640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec112170 con 0x7f17ec102780 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f08af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec101dd0 0x7f17ec1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f10b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec106f00 0x7f17ec19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f08af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec101dd0 0x7f17ec1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55858/0 (socket says 172.21.15.103:55858) 2024-09-19T07:40:23.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f08af640 1 -- 172.21.15.103:0/210484809 learned_addr learned my addr 172.21.15.103:0/210484809 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:23.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17ebfff640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17ec102780 0x7f17ec198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:23.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f10b0640 1 -- 172.21.15.103:0/210484809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17ec102780 msgr2=0x7f17ec198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:23.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f10b0640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17ec102780 0x7f17ec198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:23.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f10b0640 1 -- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec101dd0 msgr2=0x7f17ec1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:23.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f10b0640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec101dd0 0x7f17ec1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:23.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f10b0640 1 -- 172.21.15.103:0/210484809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec19d470 con 0x7f17ec106f00 2024-09-19T07:40:23.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.993+0000 7f17f08af640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec101dd0 0x7f17ec1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:23.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.994+0000 7f17f10b0640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec106f00 0x7f17ec19cd90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17e400e940 tx=0x7f17e400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:23.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.994+0000 7f17e9ffb640 1 -- 172.21.15.103:0/210484809 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4004d20 con 0x7f17ec106f00 2024-09-19T07:40:23.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.994+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ec19d760 con 0x7f17ec106f00 2024-09-19T07:40:23.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.994+0000 7f17e9ffb640 1 -- 172.21.15.103:0/210484809 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4005020 con 0x7f17ec106f00 2024-09-19T07:40:23.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.994+0000 7f17e9ffb640 1 -- 172.21.15.103:0/210484809 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e401f060 con 0x7f17ec106f00 2024-09-19T07:40:23.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.994+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ec1a5040 con 0x7f17ec106f00 2024-09-19T07:40:23.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.998+0000 7f17e9ffb640 1 -- 172.21.15.103:0/210484809 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f17e40051d0 con 0x7f17ec106f00 2024-09-19T07:40:24.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 Processor -- start 2024-09-19T07:40:24.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17e9ffb640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17dc0a33b0 0x7f17dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 -- start start 2024-09-19T07:40:24.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17ebfff640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17dc0a33b0 0x7f17dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17e9ffb640 1 -- 172.21.15.103:0/210484809 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f17e40cfe00 con 0x7f17ec106f00 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa711090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa711a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa722b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b5fa6772f0 con 0x55b5fa712b40 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b5fa61ecf0 con 0x55b5fa7115d0 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f4eec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5fa6732b0 con 0x55b5fa720780 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f08af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa711090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f08af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa711090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55874/0 (socket says 172.21.15.103:55874) 2024-09-19T07:40:24.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f08af640 1 -- 172.21.15.103:0/809868165 learned_addr learned my addr 172.21.15.103:0/809868165 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:24.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17ebfff640 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa711a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:23.999+0000 7f17f10b0640 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa722b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.000+0000 7f17f08af640 1 -- 172.21.15.103:0/809868165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 msgr2=0x55b5fa722b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.000+0000 7f17f08af640 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa722b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.000+0000 7f17f08af640 1 -- 172.21.15.103:0/809868165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 msgr2=0x55b5fa711a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.000+0000 7f17f08af640 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa711a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.000+0000 7f17f08af640 1 -- 172.21.15.103:0/809868165 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fa6033a0 con 0x55b5fa712b40 2024-09-19T07:40:24.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.000+0000 7f17ebfff640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17dc0a33b0 0x7f17dc0a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0fdcd0 tx=0x7f17d8009c70 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:24.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.000+0000 7f17f08af640 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa711090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f17e00096c0 tx=0x7f17e0004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f63ee7135ea770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.001+0000 7f17d6ffd640 1 -- 172.21.15.103:0/809868165 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0040070 con 0x55b5fa712b40 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.001+0000 7f17d6ffd640 1 -- 172.21.15.103:0/809868165 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e00047a0 con 0x55b5fa712b40 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.001+0000 7f17d6ffd640 1 -- 172.21.15.103:0/809868165 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0035040 con 0x55b5fa712b40 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 -- 172.21.15.103:0/809868165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 msgr2=0x55b5fa711090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa711090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f17e00096c0 tx=0x7f17e0004520 comp rx=0 tx=0).stop 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 -- 172.21.15.103:0/809868165 shutdown_connections 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa722b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa711a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 --2- 172.21.15.103:0/809868165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa711090 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 -- 172.21.15.103:0/809868165 >> 172.21.15.103:0/809868165 conn(0x55b5fa6da310 msgr2=0x55b5fa70dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 -- 172.21.15.103:0/809868165 shutdown_connections 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.002+0000 7f17f4eec740 1 -- 172.21.15.103:0/809868165 wait complete. 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.003+0000 7f17f4eec740 1 -- start start 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.003+0000 7f17f4eec740 1 Processor -- start 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.003+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 0x55b5fa720180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.003+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa712b40 0x55b5fa71d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa71da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17f4eec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b5fa723420 con 0x55b5fa7115d0 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa712b40 0x55b5fa71d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa712b40 0x55b5fa71d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43536/0 (socket says 172.21.15.103:43536) 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 -- 172.21.15.103:0/4139180554 learned_addr learned my addr 172.21.15.103:0/4139180554 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b5fa632060 con 0x55b5fa712b40 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5fa7235a0 con 0x55b5fa720780 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17f10b0640 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa71da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17f4eec740 10 client.0 _ll_register_callbacks cb 0x55b5fa3858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 -- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 msgr2=0x55b5fa71da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa71da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 -- 172.21.15.103:0/4139180554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 msgr2=0x55b5fa720180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 0x55b5fa720180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.004+0000 7f17ebfff640 1 -- 172.21.15.103:0/4139180554 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fa3e83c0 con 0x55b5fa712b40 2024-09-19T07:40:24.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.005+0000 7f17ebfff640 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa712b40 0x55b5fa71d430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d8016680 tx=0x7f17d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bafa3a6a18483cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:24.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.005+0000 7f17cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:40:24.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.005+0000 7f17cbfff640 1 -- 172.21.15.103:0/4139180554 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8024020 con 0x55b5fa712b40 2024-09-19T07:40:24.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.005+0000 7f17cbfff640 1 -- 172.21.15.103:0/4139180554 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f17d80178b0 con 0x55b5fa712b40 2024-09-19T07:40:24.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.005+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e00047a0 con 0x55b5fa712b40 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17cbfff640 1 -- 172.21.15.103:0/4139180554 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17d8017ab0 con 0x55b5fa712b40 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 10 client.25508 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b5fa72ef20 con 0x55b5fa712b40 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17cbfff640 1 -- 172.21.15.103:0/4139180554 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d801b040 con 0x55b5fa712b40 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17cbfff640 1 -- 172.21.15.103:0/4139180554 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f17d8031020 con 0x55b5fa712b40 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 10 client.25508 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 10 client.25508 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:40:24.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 20 client.25508 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa732860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 4 client.25508 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.006+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f17e00047a0 con 0x55b5fa730430 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.007+0000 7f17f08af640 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa732860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.007+0000 7f17cbfff640 10 client.25508 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.007+0000 7f17f08af640 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa732860 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4f6dc1601fe9d5b server_cookie=f6b78d54be6528c7 in_seq=0 out_seq=0 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.008+0000 7f17cbfff640 1 -- 172.21.15.103:0/4139180554 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f17e00047a0 con 0x55b5fa730430 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.008+0000 7f17cbfff640 10 client.25508 handle_command_reply: tid=0 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.008+0000 7f17f4eec740 1 client.25508 shutdown 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.008+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa712b40 msgr2=0x55b5fa71d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:24.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.008+0000 7f17f4eec740 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa712b40 0x55b5fa71d430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d8016680 tx=0x7f17d80046c0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 shutdown_connections 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa732860 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17cbfff640 0 client.25508 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa71da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa712b40 0x55b5fa71d430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 --2- 172.21.15.103:0/4139180554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 0x55b5fa720180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 >> 172.21.15.103:0/4139180554 conn(0x55b5fa6da310 msgr2=0x55b5fa713960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 shutdown_connections 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 1 -- 172.21.15.103:0/4139180554 wait complete. 2024-09-19T07:40:24.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.009+0000 7f17f4eec740 20 client.25508 trim_cache size 0 max 16384 2024-09-19T07:40:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 Processor -- start 2024-09-19T07:40:24.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 -- start start 2024-09-19T07:40:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 0x55b5fa72c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa712b40 0x55b5fa70cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa720780 0x55b5fa738170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1ab790 con 0x55b5fa7115d0 2024-09-19T07:40:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1ad390 con 0x55b5fa720780 2024-09-19T07:40:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.027+0000 7f17f4eec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1abe60 con 0x55b5fa712b40 2024-09-19T07:40:24.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa720780 0x55b5fa738170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa720780 0x55b5fa738170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43540/0 (socket says 172.21.15.103:43540) 2024-09-19T07:40:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 -- 172.21.15.103:0/3425138069 learned_addr learned my addr 172.21.15.103:0/3425138069 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f08af640 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 0x55b5fa72c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17ebfff640 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa712b40 0x55b5fa70cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 -- 172.21.15.103:0/3425138069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa712b40 msgr2=0x55b5fa70cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa712b40 0x55b5fa70cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 -- 172.21.15.103:0/3425138069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 msgr2=0x55b5fa72c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 0x55b5fa72c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 -- 172.21.15.103:0/3425138069 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fa71dd90 con 0x55b5fa720780 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.028+0000 7f17f10b0640 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa720780 0x55b5fa738170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17e4000f80 tx=0x7f17e40bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe03ee55701c7be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.029+0000 7f17d67fc640 1 -- 172.21.15.103:0/3425138069 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e409c0f0 con 0x55b5fa720780 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.029+0000 7f17d67fc640 1 -- 172.21.15.103:0/3425138069 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e40a04a0 con 0x55b5fa720780 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.029+0000 7f17d67fc640 1 -- 172.21.15.103:0/3425138069 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40bc0b0 con 0x55b5fa720780 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 -- 172.21.15.103:0/3425138069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa720780 msgr2=0x55b5fa738170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa720780 0x55b5fa738170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17e4000f80 tx=0x7f17e40bc040 comp rx=0 tx=0).stop 2024-09-19T07:40:24.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 -- 172.21.15.103:0/3425138069 shutdown_connections 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa720780 0x55b5fa738170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa712b40 0x55b5fa70cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3425138069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa7115d0 0x55b5fa72c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 -- 172.21.15.103:0/3425138069 >> 172.21.15.103:0/3425138069 conn(0x55b5fa6da310 msgr2=0x55b5fa70c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 -- 172.21.15.103:0/3425138069 shutdown_connections 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 -- 172.21.15.103:0/3425138069 wait complete. 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 -- start start 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.030+0000 7f17f4eec740 1 Processor -- start 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa7954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa795a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f4eec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa717150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f4eec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1acc00 con 0x55b5fa712b40 2024-09-19T07:40:24.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f4eec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1ac530 con 0x55b5fa7115d0 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f4eec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1a9770 con 0x55b5fa720780 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f4eec740 10 client.0 _ll_register_callbacks cb 0x7f17d800df00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa795a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa795a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55898/0 (socket says 172.21.15.103:55898) 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17ebfff640 1 -- 172.21.15.103:0/3983880927 learned_addr learned my addr 172.21.15.103:0/3983880927 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f08af640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa7954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f10b0640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa717150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f08af640 1 -- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 msgr2=0x55b5fa717150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f08af640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa717150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f08af640 1 -- 172.21.15.103:0/3983880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 msgr2=0x55b5fa795a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f08af640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa795a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f08af640 1 -- 172.21.15.103:0/3983880927 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fa717690 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17ebfff640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.031+0000 7f17f10b0640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa717150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17f08af640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa7954d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17e0003950 tx=0x7f17e0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a804c41d02fab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17cbfff640 1 -- 172.21.15.103:0/3983880927 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0040040 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17cbfff640 1 -- 172.21.15.103:0/3983880927 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f17e004b070 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e40a04a0 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17cbfff640 1 -- 172.21.15.103:0/3983880927 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17e005a050 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17f4eec740 10 client.25514 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b5fa7279d0 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17cbfff640 1 -- 172.21.15.103:0/3983880927 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e40a04a0 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.032+0000 7f17cbfff640 1 -- 172.21.15.103:0/3983880927 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f17e0004890 con 0x55b5fa7115d0 2024-09-19T07:40:24.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17f4eec740 10 client.25514 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17f4eec740 10 client.25514 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:40:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17f4eec740 20 client.25514 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa727c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17f4eec740 4 client.25514 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:40:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5fa684860 con 0x55b5fa730430 2024-09-19T07:40:24.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17ebfff640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa727c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17ebfff640 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa727c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f596894585c60ca3 server_cookie=2aacd14fc510de8d in_seq=0 out_seq=0 2024-09-19T07:40:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.033+0000 7f17cbfff640 10 client.25514 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:40:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17cbfff640 1 -- 172.21.15.103:0/3983880927 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5fa684860 con 0x55b5fa730430 2024-09-19T07:40:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17cbfff640 10 client.25514 handle_command_reply: tid=0 2024-09-19T07:40:24.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 client.25514 shutdown 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 msgr2=0x55b5fa7954d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa7954d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17e0003950 tx=0x7f17e0009260 comp rx=0 tx=0).stop 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 shutdown_connections 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55b5fa730430 0x55b5fa727c60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fa720780 0x55b5fa717150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5fa712b40 0x55b5fa795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 --2- 172.21.15.103:0/3983880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5fa7115d0 0x55b5fa7954d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.034+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 >> 172.21.15.103:0/3983880927 conn(0x55b5fa6da310 msgr2=0x55b5fa71f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.035+0000 7f17cbfff640 0 client.25514 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.035+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 shutdown_connections 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.035+0000 7f17f4eec740 1 -- 172.21.15.103:0/3983880927 wait complete. 2024-09-19T07:40:24.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.035+0000 7f17f4eec740 20 client.25514 trim_cache size 0 max 16384 2024-09-19T07:40:24.041 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:40:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17dc0a33b0 msgr2=0x7f17dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:24.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17dc0a33b0 0x7f17dc0a5870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0fdcd0 tx=0x7f17d8009c70 comp rx=0 tx=0).stop 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec106f00 msgr2=0x7f17ec19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec106f00 0x7f17ec19cd90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17e400e940 tx=0x7f17e400ee00 comp rx=0 tx=0).stop 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 shutdown_connections 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17dc0a33b0 0x7f17dc0a5870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17ec106f00 0x7f17ec19cd90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17ec102780 0x7f17ec198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 --2- 172.21.15.103:0/210484809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17ec101dd0 0x7f17ec1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 >> 172.21.15.103:0/210484809 conn(0x7f17ec0fd8b0 msgr2=0x7f17ec0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.036+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 shutdown_connections 2024-09-19T07:40:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:24.037+0000 7f17f2b13640 1 -- 172.21.15.103:0/210484809 wait complete. 2024-09-19T07:40:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:24 smithi134 ceph-mon[35414]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2024-09-19T07:40:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:24 smithi174 ceph-mon[35370]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2024-09-19T07:40:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:24 smithi103 ceph-mon[31201]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2024-09-19T07:40:24.775 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:40:25.154 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 110.0 s, 3333.0 tps, lat 29.941 ms stddev 31.044 2024-09-19T07:40:25.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.699+0000 7f106b3e0640 1 -- 172.21.15.103:0/1802331901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064100e20 msgr2=0x7f1064101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.699+0000 7f106b3e0640 1 --2- 172.21.15.103:0/1802331901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064100e20 0x7f1064101200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1060009930 tx=0x7f106002a4f0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.700+0000 7f106b3e0640 1 -- 172.21.15.103:0/1802331901 shutdown_connections 2024-09-19T07:40:25.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.700+0000 7f106b3e0640 1 --2- 172.21.15.103:0/1802331901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1064102ef0 0x7f106410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.700+0000 7f106b3e0640 1 --2- 172.21.15.103:0/1802331901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064102570 0x7f10641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.700+0000 7f106b3e0640 1 --2- 172.21.15.103:0/1802331901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064100e20 0x7f1064101200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.700+0000 7f106b3e0640 1 -- 172.21.15.103:0/1802331901 >> 172.21.15.103:0/1802331901 conn(0x7f10640fc7d0 msgr2=0x7f10640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.700+0000 7f106b3e0640 1 -- 172.21.15.103:0/1802331901 shutdown_connections 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.700+0000 7f106b3e0640 1 -- 172.21.15.103:0/1802331901 wait complete. 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.701+0000 7f106b3e0640 1 Processor -- start 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.701+0000 7f106b3e0640 1 -- start start 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106b3e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1064100e20 0x7f1064197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106b3e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064102570 0x7f1064197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106b3e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064102ef0 0x7f106419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106b3e0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1064111040 con 0x7f1064102570 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106b3e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1064110ec0 con 0x7f1064100e20 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106b3e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10641111c0 con 0x7f1064102ef0 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106917c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1064100e20 0x7f1064197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064102ef0 0x7f106419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106917c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1064100e20 0x7f1064197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43562/0 (socket says 172.21.15.103:43562) 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064102ef0 0x7f106419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56152/0 (socket says 172.21.15.103:56152) 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 -- 172.21.15.103:0/2531920207 learned_addr learned my addr 172.21.15.103:0/2531920207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106897b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064102570 0x7f1064197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 -- 172.21.15.103:0/2531920207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1064100e20 msgr2=0x7f1064197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1064100e20 0x7f1064197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 -- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064102570 msgr2=0x7f1064197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064102570 0x7f1064197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.702+0000 7f106997d640 1 -- 172.21.15.103:0/2531920207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106419c5b0 con 0x7f1064102ef0 2024-09-19T07:40:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.703+0000 7f106997d640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064102ef0 0x7f106419bed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f105c00b990 tx=0x7f105c00bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.703+0000 7f105a7fc640 1 -- 172.21.15.103:0/2531920207 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c019070 con 0x7f1064102ef0 2024-09-19T07:40:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.703+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106419c8a0 con 0x7f1064102ef0 2024-09-19T07:40:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.703+0000 7f105a7fc640 1 -- 172.21.15.103:0/2531920207 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105c014440 con 0x7f1064102ef0 2024-09-19T07:40:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.703+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641a4180 con 0x7f1064102ef0 2024-09-19T07:40:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.703+0000 7f105a7fc640 1 -- 172.21.15.103:0/2531920207 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0075d0 con 0x7f1064102ef0 2024-09-19T07:40:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.706+0000 7f105a7fc640 1 -- 172.21.15.103:0/2531920207 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f105c01c070 con 0x7f1064102ef0 2024-09-19T07:40:25.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.707+0000 7f106d7b9740 1 Processor -- start 2024-09-19T07:40:25.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f105a7fc640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10500a3430 0x7f10500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106d7b9740 1 -- start start 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106897b640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10500a3430 0x7f10500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f105a7fc640 1 -- 172.21.15.103:0/2531920207 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f105c0d0140 con 0x7f1064102ef0 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f70b40 0x558377f78360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f85160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106d7b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558377ed52f0 con 0x558377f6f090 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106d7b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558377e7ccf0 con 0x558377f70b40 2024-09-19T07:40:25.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.708+0000 7f106d7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558377ed12b0 con 0x558377f6fa10 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f70b40 0x558377f78360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f70b40 0x558377f78360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43572/0 (socket says 172.21.15.103:43572) 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 -- 172.21.15.103:0/2801114159 learned_addr learned my addr 172.21.15.103:0/2801114159 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106897b640 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106897b640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10500a3430 0x7f10500a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10640fec30 tx=0x7f1054009110 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106997d640 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f85160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 -- 172.21.15.103:0/2801114159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 msgr2=0x558377f85160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f85160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 -- 172.21.15.103:0/2801114159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 msgr2=0x558377f6f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106917c640 1 -- 172.21.15.103:0/2801114159 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558377e613a0 con 0x558377f70b40 2024-09-19T07:40:25.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.709+0000 7f106897b640 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:25.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.710+0000 7f106917c640 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f70b40 0x558377f78360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f106002aa10 tx=0x7f10600316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8be7537bd0d096a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:25.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.710+0000 7f104f7fe640 1 -- 172.21.15.103:0/2801114159 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060040070 con 0x558377f70b40 2024-09-19T07:40:25.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.710+0000 7f104f7fe640 1 -- 172.21.15.103:0/2801114159 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060005020 con 0x558377f70b40 2024-09-19T07:40:25.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.710+0000 7f104f7fe640 1 -- 172.21.15.103:0/2801114159 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060035040 con 0x558377f70b40 2024-09-19T07:40:25.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.711+0000 7f106d7b9740 1 -- 172.21.15.103:0/2801114159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f70b40 msgr2=0x558377f78360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.711+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f70b40 0x558377f78360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f106002aa10 tx=0x7f10600316e0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.711+0000 7f106d7b9740 1 -- 172.21.15.103:0/2801114159 shutdown_connections 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.711+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f85160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.711+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.711+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2801114159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f70b40 0x558377f78360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.711+0000 7f106d7b9740 1 -- 172.21.15.103:0/2801114159 >> 172.21.15.103:0/2801114159 conn(0x558377f38310 msgr2=0x558377f6bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.712+0000 7f106d7b9740 1 -- 172.21.15.103:0/2801114159 shutdown_connections 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.712+0000 7f106d7b9740 1 -- 172.21.15.103:0/2801114159 wait complete. 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.712+0000 7f106d7b9740 1 -- start start 2024-09-19T07:40:25.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.712+0000 7f106d7b9740 1 Processor -- start 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.713+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f78160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.713+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f70b40 0x558377f75950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106d7b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558377f85880 con 0x558377f6f090 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106d7b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558377e90060 con 0x558377f6fa10 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106d7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558377f85a00 con 0x558377f70b40 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106917c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f78160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106997d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f70b40 0x558377f75950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106897b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106917c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f78160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55926/0 (socket says 172.21.15.103:55926) 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.714+0000 7f106917c640 1 -- 172.21.15.103:0/654136388 learned_addr learned my addr 172.21.15.103:0/654136388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106d7b9740 10 client.0 _ll_register_callbacks cb 0x558377be38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106897b640 1 -- 172.21.15.103:0/654136388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f70b40 msgr2=0x558377f75950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106897b640 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f70b40 0x558377f75950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106897b640 1 -- 172.21.15.103:0/654136388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 msgr2=0x558377f78160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106897b640 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f78160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106897b640 1 -- 172.21.15.103:0/654136388 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558377c463c0 con 0x558377f6fa10 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1060005020 con 0x558377f6fa10 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106897b640 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f75410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x558377f6cac0 tx=0x7f10540047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607bb541b18494a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f104cff9640 10 client.25523 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f104cff9640 1 -- 172.21.15.103:0/654136388 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105402f050 con 0x558377f6fa10 2024-09-19T07:40:25.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f104cff9640 1 -- 172.21.15.103:0/654136388 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f1054013720 con 0x558377f6fa10 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f104cff9640 1 -- 172.21.15.103:0/654136388 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1054025070 con 0x558377f6fa10 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106d7b9740 10 client.25523 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.715+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=30}) -- 0x558377f91190 con 0x558377f6fa10 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f104cff9640 1 -- 172.21.15.103:0/654136388 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1054013960 con 0x558377f6fa10 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f106d7b9740 10 client.25523 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f106d7b9740 10 client.25523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f106d7b9740 20 client.25523 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f106d7b9740 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377f92540 0x558377f949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f106d7b9740 4 client.25523 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558377ee2860 con 0x558377f92540 2024-09-19T07:40:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.716+0000 7f106917c640 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377f92540 0x558377f949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.718+0000 7f106917c640 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377f92540 0x558377f949b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1aa8e534221c7bad server_cookie=9242552fb8c69172 in_seq=0 out_seq=0 2024-09-19T07:40:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.718+0000 7f104cff9640 10 client.25523 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:40:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.718+0000 7f104cff9640 1 -- 172.21.15.103:0/654136388 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558377ee2860 con 0x558377f92540 2024-09-19T07:40:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.718+0000 7f104cff9640 10 client.25523 handle_command_reply: tid=0 2024-09-19T07:40:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 client.25523 shutdown 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 msgr2=0x558377f75410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f75410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x558377f6cac0 tx=0x7f10540047d0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 shutdown_connections 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377f92540 0x558377f949b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f70b40 0x558377f75950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f75410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 --2- 172.21.15.103:0/654136388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f78160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 >> 172.21.15.103:0/654136388 conn(0x558377f38310 msgr2=0x558377f719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.719+0000 7f104cff9640 0 client.25523 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.720+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 shutdown_connections 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.720+0000 7f106d7b9740 1 -- 172.21.15.103:0/654136388 wait complete. 2024-09-19T07:40:25.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.721+0000 7f106d7b9740 20 client.25523 trim_cache size 0 max 16384 2024-09-19T07:40:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.737+0000 7f106d7b9740 1 Processor -- start 2024-09-19T07:40:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.737+0000 7f106d7b9740 1 -- start start 2024-09-19T07:40:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f38310 0x558377f8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f9a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106d7b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f104800e800 con 0x558377f6f090 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106d7b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1048010400 con 0x558377f38310 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106d7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f104800eed0 con 0x558377f6fa10 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106997d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f9a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106897b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106897b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55942/0 (socket says 172.21.15.103:55942) 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106897b640 1 -- 172.21.15.103:0/3560918840 learned_addr learned my addr 172.21.15.103:0/3560918840 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.738+0000 7f106917c640 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f38310 0x558377f8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.739+0000 7f106897b640 1 -- 172.21.15.103:0/3560918840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 msgr2=0x558377f9a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.739+0000 7f106897b640 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f9a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.739+0000 7f106897b640 1 -- 172.21.15.103:0/3560918840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f38310 msgr2=0x558377f8eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.739+0000 7f106897b640 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f38310 0x558377f8eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.739+0000 7f106897b640 1 -- 172.21.15.103:0/3560918840 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558377f6c1a0 con 0x558377f6f090 2024-09-19T07:40:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.739+0000 7f106897b640 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6abf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f105400ffa0 tx=0x7f1054018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb9bf7764ae4ed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.740+0000 7f104effd640 1 -- 172.21.15.103:0/3560918840 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105402f040 con 0x558377f6f090 2024-09-19T07:40:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.740+0000 7f104effd640 1 -- 172.21.15.103:0/3560918840 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060005020 con 0x558377f6f090 2024-09-19T07:40:25.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.740+0000 7f104effd640 1 -- 172.21.15.103:0/3560918840 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105401b040 con 0x558377f6f090 2024-09-19T07:40:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.742+0000 7f106d7b9740 1 -- 172.21.15.103:0/3560918840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 msgr2=0x558377f6abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.742+0000 7f106d7b9740 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6abf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f105400ffa0 tx=0x7f1054018000 comp rx=0 tx=0).stop 2024-09-19T07:40:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.742+0000 7f106d7b9740 1 -- 172.21.15.103:0/3560918840 shutdown_connections 2024-09-19T07:40:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.743+0000 7f106d7b9740 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f6fa10 0x558377f9a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.743+0000 7f106d7b9740 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f6f090 0x558377f6abf0 secure :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f105400ffa0 tx=0x7f1054018000 comp rx=0 tx=0).stop 2024-09-19T07:40:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.743+0000 7f106d7b9740 1 --2- 172.21.15.103:0/3560918840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f38310 0x558377f8eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.743+0000 7f106d7b9740 1 -- 172.21.15.103:0/3560918840 >> 172.21.15.103:0/3560918840 conn(0x558377f75ab0 msgr2=0x558377f6a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:25.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.750+0000 7f106d7b9740 1 -- 172.21.15.103:0/3560918840 shutdown_connections 2024-09-19T07:40:25.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.750+0000 7f106d7b9740 1 -- 172.21.15.103:0/3560918840 wait complete. 2024-09-19T07:40:25.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.750+0000 7f106d7b9740 1 -- start start 2024-09-19T07:40:25.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.751+0000 7f106d7b9740 1 Processor -- start 2024-09-19T07:40:25.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.751+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f38310 0x558377f8b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.751+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.752+0000 7f106d7b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f70b40 0x558377f88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.752+0000 7f106d7b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f104800fc70 con 0x558377f70b40 2024-09-19T07:40:25.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.752+0000 7f106d7b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f104800f5a0 con 0x558377f6fa10 2024-09-19T07:40:25.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.752+0000 7f106d7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f104800c7e0 con 0x558377f38310 2024-09-19T07:40:25.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.752+0000 7f106d7b9740 10 client.0 _ll_register_callbacks cb 0x7f105400ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:25.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.753+0000 7f106897b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.753+0000 7f106917c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f38310 0x558377f8b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.753+0000 7f106897b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43602/0 (socket says 172.21.15.103:43602) 2024-09-19T07:40:25.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.753+0000 7f106897b640 1 -- 172.21.15.103:0/2307437131 learned_addr learned my addr 172.21.15.103:0/2307437131 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.755+0000 7f106917c640 1 -- 172.21.15.103:0/2307437131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 msgr2=0x558377f886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.755+0000 7f106917c640 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.755+0000 7f106917c640 1 -- 172.21.15.103:0/2307437131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f70b40 msgr2=0x558377f88bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.755+0000 7f106917c640 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f70b40 0x558377f88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.755+0000 7f106917c640 1 -- 172.21.15.103:0/2307437131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558377f893f0 con 0x558377f38310 2024-09-19T07:40:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.756+0000 7f106917c640 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f38310 0x558377f8b400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x558377f95220 tx=0x7f106002a4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18369306ed036f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:25.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.756+0000 7f104cff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:40:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.757+0000 7f104cff9640 1 -- 172.21.15.103:0/2307437131 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060040040 con 0x558377f38310 2024-09-19T07:40:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.757+0000 7f104cff9640 1 -- 172.21.15.103:0/2307437131 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f10600055e0 con 0x558377f38310 2024-09-19T07:40:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.757+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1060005020 con 0x558377f38310 2024-09-19T07:40:25.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.758+0000 7f104cff9640 1 -- 172.21.15.103:0/2307437131 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1060035040 con 0x558377f38310 2024-09-19T07:40:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.760+0000 7f106d7b9740 10 client.25501 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:25.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.760+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558377ffd230 con 0x558377f38310 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f104cff9640 1 -- 172.21.15.103:0/2307437131 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1060005020 con 0x558377f38310 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f104cff9640 1 -- 172.21.15.103:0/2307437131 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f106004e070 con 0x558377f38310 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f106d7b9740 10 client.25501 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f106d7b9740 10 client.25501 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f106d7b9740 20 client.25501 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377ffd4c0 0x558377fff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f106d7b9740 4 client.25501 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.761+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558377edcd70 con 0x558377ffd4c0 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.762+0000 7f106897b640 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377ffd4c0 0x558377fff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:25.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f104cff9640 10 client.25501 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f106897b640 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377ffd4c0 0x558377fff8b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4d1dd04e8c9f11 server_cookie=3b5ea0fd15bd01fa in_seq=0 out_seq=0 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f104cff9640 1 -- 172.21.15.103:0/2307437131 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558377edcd70 con 0x558377ffd4c0 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f104cff9640 10 client.25501 handle_command_reply: tid=0 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f106d7b9740 1 client.25501 shutdown 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f38310 msgr2=0x558377f8b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f38310 0x558377f8b400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x558377f95220 tx=0x7f106002a4f0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 shutdown_connections 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.763+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x558377ffd4c0 0x558377fff8b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.764+0000 7f104cff9640 0 client.25501 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.764+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558377f70b40 0x558377f88bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.764+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558377f6fa10 0x558377f886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.764+0000 7f106d7b9740 1 --2- 172.21.15.103:0/2307437131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558377f38310 0x558377f8b400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.764+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 >> 172.21.15.103:0/2307437131 conn(0x558377f75ab0 msgr2=0x558377f77490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:25.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.765+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 shutdown_connections 2024-09-19T07:40:25.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.766+0000 7f106d7b9740 1 -- 172.21.15.103:0/2307437131 wait complete. 2024-09-19T07:40:25.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.766+0000 7f106d7b9740 20 client.25501 trim_cache size 0 max 16384 2024-09-19T07:40:25.768 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:40:25.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10500a3430 msgr2=0x7f10500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10500a3430 0x7f10500a58f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10640fec30 tx=0x7f1054009110 comp rx=0 tx=0).stop 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064102ef0 msgr2=0x7f106419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064102ef0 0x7f106419bed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f105c00b990 tx=0x7f105c00bd90 comp rx=0 tx=0).stop 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 shutdown_connections 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f10500a3430 0x7f10500a58f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1064102ef0 0x7f106419bed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1064102570 0x7f1064197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 --2- 172.21.15.103:0/2531920207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1064100e20 0x7f1064197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 >> 172.21.15.103:0/2531920207 conn(0x7f10640fc7d0 msgr2=0x7f1064104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 shutdown_connections 2024-09-19T07:40:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:25.767+0000 7f106b3e0640 1 -- 172.21.15.103:0/2531920207 wait complete. 2024-09-19T07:40:26.524 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:26 smithi103 ceph-mon[31201]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 MiB/s wr, 1.81k op/s 2024-09-19T07:40:26.565 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:40:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:26 smithi134 ceph-mon[35414]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 MiB/s wr, 1.81k op/s 2024-09-19T07:40:26.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:26 smithi174 ceph-mon[35370]: pgmap v76: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 MiB/s wr, 1.81k op/s 2024-09-19T07:40:26.952 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:27.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/93400037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102ef0 msgr2=0x7f2eb810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/93400037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102ef0 0x7f2eb810da40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800ba20 tx=0x7f2ea802c170 comp rx=0 tx=0).stop 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/93400037 shutdown_connections 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/93400037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102ef0 0x7f2eb810da40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/93400037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2eb8102570 0x7f2eb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/93400037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eb8100e20 0x7f2eb8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/93400037 >> 172.21.15.103:0/93400037 conn(0x7f2eb80fc7d0 msgr2=0x7f2eb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/93400037 shutdown_connections 2024-09-19T07:40:27.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.757+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/93400037 wait complete. 2024-09-19T07:40:27.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.758+0000 7f2ebd9fd640 1 Processor -- start 2024-09-19T07:40:27.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.758+0000 7f2ebd9fd640 1 -- start start 2024-09-19T07:40:27.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2ebd9fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eb8100e20 0x7f2eb8196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2ebd9fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102570 0x7f2eb8196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2ebd9fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2eb8102ef0 0x7f2eb819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2ebd9fd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810fe80 con 0x7f2eb8102ef0 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2ebd9fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810fd00 con 0x7f2eb8102570 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2ebd9fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb8110000 con 0x7f2eb8100e20 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102570 0x7f2eb8196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2eb8102ef0 0x7f2eb819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102570 0x7f2eb8196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43618/0 (socket says 172.21.15.103:43618) 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 -- 172.21.15.103:0/1417040119 learned_addr learned my addr 172.21.15.103:0/1417040119 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eb8100e20 0x7f2eb8196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 -- 172.21.15.103:0/1417040119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eb8100e20 msgr2=0x7f2eb8196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eb8100e20 0x7f2eb8196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 -- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2eb8102ef0 msgr2=0x7f2eb819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2eb8102ef0 0x7f2eb819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 -- 172.21.15.103:0/1417040119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb819b2d0 con 0x7f2eb8102570 2024-09-19T07:40:27.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2eb8102ef0 0x7f2eb819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:27.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.759+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102570 0x7f2eb8196860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000eb90 tx=0x7f2eb000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:27.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.760+0000 7f2ebc9fb640 1 -- 172.21.15.103:0/1417040119 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0004d20 con 0x7f2eb8102570 2024-09-19T07:40:27.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.760+0000 7f2ebc9fb640 1 -- 172.21.15.103:0/1417040119 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb000ee90 con 0x7f2eb8102570 2024-09-19T07:40:27.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.760+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb819b5c0 con 0x7f2eb8102570 2024-09-19T07:40:27.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.760+0000 7f2ebc9fb640 1 -- 172.21.15.103:0/1417040119 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0010830 con 0x7f2eb8102570 2024-09-19T07:40:27.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.760+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb81a72b0 con 0x7f2eb8102570 2024-09-19T07:40:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebc9fb640 1 -- 172.21.15.103:0/1417040119 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2eb0005020 con 0x7f2eb8102570 2024-09-19T07:40:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 Processor -- start 2024-09-19T07:40:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 -- start start 2024-09-19T07:40:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebc9fb640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2ea00a3430 0x7f2ea00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8df0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8dfd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8d562f0 con 0x55dcf8df1b40 2024-09-19T07:40:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8cfdcf0 con 0x55dcf8df05d0 2024-09-19T07:40:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8d522b0 con 0x55dcf8dfb370 2024-09-19T07:40:27.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2ebc9fb640 1 -- 172.21.15.103:0/1417040119 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2eb00d02b0 con 0x7f2eb8102570 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.764+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2ea00a3430 0x7f2ea00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55974/0 (socket says 172.21.15.103:55974) 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/767815686 learned_addr learned my addr 172.21.15.103:0/767815686 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8df0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8dfd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/767815686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 msgr2=0x55dcf8dfd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8dfd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/767815686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 msgr2=0x55dcf8df0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8df0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/767815686 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf8ce23a0 con 0x55dcf8df1b40 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2ea00a3430 0x7f2ea00a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80057c0 tx=0x7f2ea80092c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df0090 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009cc0 tx=0x7f2eac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91e42fb5f09d075 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2e9dffb640 1 -- 172.21.15.103:0/767815686 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac005220 con 0x55dcf8df1b40 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.765+0000 7f2e9dffb640 1 -- 172.21.15.103:0/767815686 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac004c80 con 0x55dcf8df1b40 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2e9dffb640 1 -- 172.21.15.103:0/767815686 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac019810 con 0x55dcf8df1b40 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/767815686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 msgr2=0x55dcf8df0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df0090 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009cc0 tx=0x7f2eac011810 comp rx=0 tx=0).stop 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/767815686 shutdown_connections 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8dfd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8df0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/767815686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df0090 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.766+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/767815686 >> 172.21.15.103:0/767815686 conn(0x55dcf8db9310 msgr2=0x55dcf8decc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.767+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/767815686 shutdown_connections 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.767+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/767815686 wait complete. 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.767+0000 7f2ebfdd6740 1 -- start start 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.767+0000 7f2ebfdd6740 1 Processor -- start 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8dface0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55988/0 (socket says 172.21.15.103:55988) 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8dface0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8dface0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43638/0 (socket says 172.21.15.103:43638) 2024-09-19T07:40:27.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8dfe0c0 con 0x55dcf8df1b40 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8d11060 con 0x55dcf8df05d0 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcf8dfe240 con 0x55dcf8dfb370 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56240/0 (socket says 172.21.15.103:56240) 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 -- 172.21.15.103:0/728388537 learned_addr learned my addr 172.21.15.103:0/728388537 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 -- 172.21.15.103:0/728388537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 msgr2=0x55dcf8dface0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8dface0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 -- 172.21.15.103:0/728388537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 msgr2=0x55dcf8df7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 10 client.0 _ll_register_callbacks cb 0x55dcf8a648c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2eb77fe640 1 -- 172.21.15.103:0/728388537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf8ac73c0 con 0x55dcf8dfb370 2024-09-19T07:40:27.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.768+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eac004c80 con 0x55dcf8dfb370 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.769+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df84f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800ba20 tx=0x7f2ea80058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c468c7f5649f85c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.769+0000 7f2e96ffd640 10 client.25513 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.769+0000 7f2e96ffd640 1 -- 172.21.15.103:0/728388537 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8039050 con 0x55dcf8dfb370 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.769+0000 7f2e96ffd640 1 -- 172.21.15.103:0/728388537 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2ea8034dc0 con 0x55dcf8dfb370 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.769+0000 7f2e96ffd640 1 -- 172.21.15.103:0/728388537 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ea80035f0 con 0x55dcf8dfb370 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.769+0000 7f2ebfdd6740 10 client.25513 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.769+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55dcf8e09a80 con 0x55dcf8dfb370 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2e96ffd640 1 -- 172.21.15.103:0/728388537 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2ea8003790 con 0x55dcf8dfb370 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2ebfdd6740 10 client.25513 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2ebfdd6740 10 client.25513 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2ebfdd6740 20 client.25513 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e0b170 0x55dcf8e0d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2ebfdd6740 4 client.25513 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:27.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf8d63860 con 0x55dcf8e0b170 2024-09-19T07:40:27.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.770+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e0b170 0x55dcf8e0d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.771+0000 7f2e96ffd640 10 client.25513 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:40:27.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.771+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e0b170 0x55dcf8e0d5a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45c44d4d1e4e0d6b server_cookie=c1a890613c9fbcd in_seq=0 out_seq=0 2024-09-19T07:40:27.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.771+0000 7f2e96ffd640 1 -- 172.21.15.103:0/728388537 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcf8d63860 con 0x55dcf8e0b170 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.771+0000 7f2e96ffd640 10 client.25513 handle_command_reply: tid=0 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 client.25513 shutdown 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 msgr2=0x55dcf8df84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df84f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800ba20 tx=0x7f2ea80058b0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 shutdown_connections 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e0b170 0x55dcf8e0d5a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df84f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/728388537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df05d0 0x55dcf8dface0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 >> 172.21.15.103:0/728388537 conn(0x55dcf8db9310 msgr2=0x55dcf8df2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2e96ffd640 0 client.25513 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 shutdown_connections 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/728388537 wait complete. 2024-09-19T07:40:27.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.772+0000 7f2ebfdd6740 20 client.25513 trim_cache size 0 max 16384 2024-09-19T07:40:27.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 Processor -- start 2024-09-19T07:40:27.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 -- start start 2024-09-19T07:40:27.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8df05d0 0x55dcf8e07360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8dfb370 0x55dcf8df3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81ab670 con 0x55dcf8df1b40 2024-09-19T07:40:27.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81ad270 con 0x55dcf8dfb370 2024-09-19T07:40:27.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.792+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81abd40 con 0x55dcf8df05d0 2024-09-19T07:40:27.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8df05d0 0x55dcf8e07360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8df05d0 0x55dcf8e07360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56254/0 (socket says 172.21.15.103:56254) 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/3445807969 learned_addr learned my addr 172.21.15.103:0/3445807969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8dfb370 0x55dcf8df3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/3445807969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8dfb370 msgr2=0x55dcf8df3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8dfb370 0x55dcf8df3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/3445807969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 msgr2=0x55dcf8df9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/3445807969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf8deced0 con 0x55dcf8df05d0 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8df05d0 0x55dcf8e07360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009c90 tx=0x7f2eac005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee52590d07632b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2e9d7fa640 1 -- 172.21.15.103:0/3445807969 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac005200 con 0x55dcf8df05d0 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2e9d7fa640 1 -- 172.21.15.103:0/3445807969 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac02b420 con 0x55dcf8df05d0 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.793+0000 7f2e9d7fa640 1 -- 172.21.15.103:0/3445807969 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0196e0 con 0x55dcf8df05d0 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/3445807969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8df05d0 msgr2=0x55dcf8e07360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8df05d0 0x55dcf8e07360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009c90 tx=0x7f2eac005fd0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/3445807969 shutdown_connections 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8dfb370 0x55dcf8df3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df1b40 0x55dcf8df9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/3445807969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8df05d0 0x55dcf8e07360 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/3445807969 >> 172.21.15.103:0/3445807969 conn(0x55dcf8db9310 msgr2=0x55dcf8df97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/3445807969 shutdown_connections 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.794+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/3445807969 wait complete. 2024-09-19T07:40:27.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 -- start start 2024-09-19T07:40:27.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 Processor -- start 2024-09-19T07:40:27.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df05d0 0x55dcf8e03c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df1b40 0x55dcf8df4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81acae0 con 0x55dcf8df05d0 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81ac410 con 0x55dcf8df1b40 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb81a96a0 con 0x55dcf8dfb370 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df05d0 0x55dcf8e03c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2eb6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df05d0 0x55dcf8e03c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56010/0 (socket says 172.21.15.103:56010) 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/701796637 learned_addr learned my addr 172.21.15.103:0/701796637 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2ebfdd6740 10 client.0 _ll_register_callbacks cb 0x7f2eac005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df1b40 0x55dcf8df4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.795+0000 7f2eb77fe640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 msgr2=0x55dcf8df5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/701796637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df1b40 msgr2=0x55dcf8df4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df1b40 0x55dcf8df4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2eb6ffd640 1 -- 172.21.15.103:0/701796637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf8dfb050 con 0x55dcf8df05d0 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df1b40 0x55dcf8df4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2eb6ffd640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df05d0 0x55dcf8e03c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cd1e2271b7c75d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2e96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2e96ffd640 1 -- 172.21.15.103:0/701796637 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac039060 con 0x55dcf8df05d0 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2e96ffd640 1 -- 172.21.15.103:0/701796637 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2eac004660 con 0x55dcf8df05d0 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eac02b420 con 0x55dcf8df05d0 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2e96ffd640 1 -- 172.21.15.103:0/701796637 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2eac030050 con 0x55dcf8df05d0 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2ebfdd6740 10 client.16032 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:27.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.796+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dcf8e75cd0 con 0x55dcf8df05d0 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2e96ffd640 1 -- 172.21.15.103:0/701796637 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac02b420 con 0x55dcf8df05d0 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2e96ffd640 1 -- 172.21.15.103:0/701796637 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2eac034430 con 0x55dcf8df05d0 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2ebfdd6740 10 client.16032 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2ebfdd6740 10 client.16032 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2ebfdd6740 20 client.16032 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e761a0 0x55dcf8e78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2ebfdd6740 4 client.16032 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcf8d5dd70 con 0x55dcf8e761a0 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e761a0 0x55dcf8e78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2eb67fc640 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e761a0 0x55dcf8e78590 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=376dfcf74e3e8233 server_cookie=8359498b2873b574 in_seq=0 out_seq=0 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2e96ffd640 10 client.16032 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.797+0000 7f2e96ffd640 1 -- 172.21.15.103:0/701796637 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcf8d5dd70 con 0x55dcf8e761a0 2024-09-19T07:40:27.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2e96ffd640 10 client.16032 handle_command_reply: tid=0 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 client.16032 shutdown 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df05d0 msgr2=0x55dcf8e03c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df05d0 0x55dcf8e03c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac005e80 comp rx=0 tx=0).stop 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 shutdown_connections 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55dcf8e761a0 0x55dcf8e78590 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2e96ffd640 0 client.16032 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcf8dfb370 0x55dcf8df5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dcf8df1b40 0x55dcf8df4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 --2- 172.21.15.103:0/701796637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcf8df05d0 0x55dcf8e03c60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 >> 172.21.15.103:0/701796637 conn(0x55dcf8db9310 msgr2=0x55dcf8dfa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 shutdown_connections 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 1 -- 172.21.15.103:0/701796637 wait complete. 2024-09-19T07:40:27.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.798+0000 7f2ebfdd6740 20 client.16032 trim_cache size 0 max 16384 2024-09-19T07:40:27.806 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2ea00a3430 msgr2=0x7f2ea00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2ea00a3430 0x7f2ea00a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80057c0 tx=0x7f2ea80092c0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102570 msgr2=0x7f2eb8196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102570 0x7f2eb8196860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000eb90 tx=0x7f2eb000c6a0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 shutdown_connections 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2ea00a3430 0x7f2ea00a58f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2eb8102ef0 0x7f2eb819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eb8102570 0x7f2eb8196860 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 --2- 172.21.15.103:0/1417040119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eb8100e20 0x7f2eb8196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:27.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 >> 172.21.15.103:0/1417040119 conn(0x7f2eb80fc7d0 msgr2=0x7f2eb8104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:27.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 shutdown_connections 2024-09-19T07:40:27.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:27.799+0000 7f2ebd9fd640 1 -- 172.21.15.103:0/1417040119 wait complete. 2024-09-19T07:40:28.495 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:28 smithi103 ceph-mon[31201]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-09-19T07:40:28.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:40:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:28 smithi134 ceph-mon[35414]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-09-19T07:40:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:28 smithi174 ceph-mon[35370]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-09-19T07:40:28.906 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.417+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/3635069334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 msgr2=0x7f0ea4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.417+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/3635069334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 0x7f0ea4104e30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009900 tx=0x7f0e9802a410 comp rx=0 tx=0).stop 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.418+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/3635069334 shutdown_connections 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.418+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/3635069334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 0x7f0ea410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.418+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/3635069334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 0x7f0ea4104e30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.418+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/3635069334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ea4100430 0x7f0ea4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.418+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/3635069334 >> 172.21.15.103:0/3635069334 conn(0x7f0ea4077230 msgr2=0x7f0ea40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.418+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/3635069334 shutdown_connections 2024-09-19T07:40:29.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.418+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/3635069334 wait complete. 2024-09-19T07:40:29.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.419+0000 7f0ea8c7e640 1 Processor -- start 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.419+0000 7f0ea8c7e640 1 -- start start 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.419+0000 7f0ea8c7e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ea4100430 0x7f0ea41a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.419+0000 7f0ea8c7e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 0x7f0ea41a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.419+0000 7f0ea8c7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 0x7f0ea410c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.419+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 0x7f0ea41a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 0x7f0ea410c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 0x7f0ea410c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:37422/0 (socket says 172.21.15.103:37422) 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 0x7f0ea41a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45630/0 (socket says 172.21.15.103:45630) 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/539213786 learned_addr learned my addr 172.21.15.103:0/539213786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4111f40 con 0x7f0ea4100de0 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4111dc0 con 0x7f0ea4105560 2024-09-19T07:40:29.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41120c0 con 0x7f0ea4100430 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea259c640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ea4100430 0x7f0ea41a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/539213786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ea4100430 msgr2=0x7f0ea41a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ea4100430 0x7f0ea41a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 msgr2=0x7f0ea41a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 0x7f0ea41a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/539213786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea410ca40 con 0x7f0ea4105560 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.420+0000 7f0ea259c640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ea4100430 0x7f0ea41a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.421+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 0x7f0ea410c2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009fd0 tx=0x7f0e9c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.421+0000 7f0e937fe640 1 -- 172.21.15.103:0/539213786 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c017070 con 0x7f0ea4105560 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.421+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea410cd30 con 0x7f0ea4105560 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.421+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea406bc10 con 0x7f0ea4105560 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.421+0000 7f0e937fe640 1 -- 172.21.15.103:0/539213786 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c003e50 con 0x7f0ea4105560 2024-09-19T07:40:29.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.421+0000 7f0e937fe640 1 -- 172.21.15.103:0/539213786 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0106c0 con 0x7f0ea4105560 2024-09-19T07:40:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.424+0000 7f0eab057740 1 Processor -- start 2024-09-19T07:40:29.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.425+0000 7f0e937fe640 1 -- 172.21.15.103:0/539213786 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0e9c010860 con 0x7f0ea4105560 2024-09-19T07:40:29.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.425+0000 7f0eab057740 1 -- start start 2024-09-19T07:40:29.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.425+0000 7f0eab057740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72eb40 0x56130e72d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.425+0000 7f0eab057740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e72da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0eab057740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e73a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0eab057740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56130e6932f0 con 0x56130e72eb40 2024-09-19T07:40:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0eab057740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56130e63acf0 con 0x56130e738370 2024-09-19T07:40:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0eab057740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56130e68f2b0 con 0x56130e72d5d0 2024-09-19T07:40:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e73a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e73a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:37438/0 (socket says 172.21.15.103:37438) 2024-09-19T07:40:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/1096328408 learned_addr learned my addr 172.21.15.103:0/1096328408 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72eb40 0x56130e72d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.426+0000 7f0ea259c640 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e72da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.427+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/1096328408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 msgr2=0x56130e72da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.427+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e72da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.427+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/1096328408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72eb40 msgr2=0x56130e72d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.427+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72eb40 0x56130e72d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.427+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/1096328408 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130e61f3a0 con 0x56130e738370 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.427+0000 7f0e937fe640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0e8c0a34b0 0x7f0e8c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.428+0000 7f0e937fe640 1 -- 172.21.15.103:0/539213786 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0e9c0cf880 con 0x7f0ea4105560 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.428+0000 7f0ea259c640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0e8c0a34b0 0x7f0e8c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.428+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e73a760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e98000c00 tx=0x7f0e980314e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb2f2e69408d707 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.428+0000 7f0e90ff9640 1 -- 172.21.15.103:0/1096328408 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98035070 con 0x56130e738370 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.428+0000 7f0e90ff9640 1 -- 172.21.15.103:0/1096328408 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e98005020 con 0x56130e738370 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.428+0000 7f0e90ff9640 1 -- 172.21.15.103:0/1096328408 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98004a00 con 0x56130e738370 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0eab057740 1 -- 172.21.15.103:0/1096328408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 msgr2=0x56130e73a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0ea259c640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0e8c0a34b0 0x7f0e8c0a5970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56130e72aac0 tx=0x7f0e9400b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0eab057740 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e73a760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0e98000c00 tx=0x7f0e980314e0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0eab057740 1 -- 172.21.15.103:0/1096328408 shutdown_connections 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0eab057740 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e73a760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0eab057740 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e72da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0eab057740 1 --2- 172.21.15.103:0/1096328408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72eb40 0x56130e72d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.429+0000 7f0eab057740 1 -- 172.21.15.103:0/1096328408 >> 172.21.15.103:0/1096328408 conn(0x56130e6f6310 msgr2=0x56130e729c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.430+0000 7f0eab057740 1 -- 172.21.15.103:0/1096328408 shutdown_connections 2024-09-19T07:40:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.431+0000 7f0eab057740 1 -- 172.21.15.103:0/1096328408 wait complete. 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.431+0000 7f0eab057740 1 -- start start 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.431+0000 7f0eab057740 1 Processor -- start 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0eab057740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72d5d0 0x56130e737dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0eab057740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e735070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0eab057740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e735800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0eab057740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56130e73b010 con 0x56130e738370 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0eab057740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56130e64e060 con 0x56130e72d5d0 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0eab057740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56130e73b190 con 0x56130e72eb40 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0eab057740 10 client.0 _ll_register_callbacks cb 0x56130e3a18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e735070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e735070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36198/0 (socket says 172.21.15.103:36198) 2024-09-19T07:40:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/2211989971 learned_addr learned my addr 172.21.15.103:0/2211989971 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e735800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.432+0000 7f0ea259c640 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72d5d0 0x56130e737dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.433+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72d5d0 msgr2=0x56130e737dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.433+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72d5d0 0x56130e737dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.433+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/2211989971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 msgr2=0x56130e735800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.433+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e735800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.433+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/2211989971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130e4043c0 con 0x56130e72eb40 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.433+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e735070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e98001eb0 tx=0x7f0e980315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=403ff3d8cc458e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.433+0000 7f0e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.434+0000 7f0e89ffb640 1 -- 172.21.15.103:0/2211989971 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98035070 con 0x56130e72eb40 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.434+0000 7f0e89ffb640 1 -- 172.21.15.103:0/2211989971 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0e980055e0 con 0x56130e72eb40 2024-09-19T07:40:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.434+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e98005020 con 0x56130e72eb40 2024-09-19T07:40:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.434+0000 7f0e89ffb640 1 -- 172.21.15.103:0/2211989971 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e98058050 con 0x56130e72eb40 2024-09-19T07:40:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.434+0000 7f0eab057740 10 client.25531 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.434+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56130e73b010 con 0x56130e72eb40 2024-09-19T07:40:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.434+0000 7f0e89ffb640 1 -- 172.21.15.103:0/2211989971 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e98005020 con 0x56130e72eb40 2024-09-19T07:40:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0e89ffb640 1 -- 172.21.15.103:0/2211989971 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0e9803a040 con 0x56130e72eb40 2024-09-19T07:40:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0eab057740 10 client.25531 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0eab057740 10 client.25531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0eab057740 20 client.25531 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0eab057740 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e747d30 0x56130e74a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0eab057740 4 client.25531 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56130e6a0860 con 0x56130e747d30 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0ea259c640 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e747d30 0x56130e74a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0ea259c640 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e747d30 0x56130e74a120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=683a0666520dbea3 server_cookie=2d17e239c1ab82ac in_seq=0 out_seq=0 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.435+0000 7f0e89ffb640 10 client.25531 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.436+0000 7f0e89ffb640 1 -- 172.21.15.103:0/2211989971 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56130e6a0860 con 0x56130e747d30 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.436+0000 7f0e89ffb640 10 client.25531 handle_command_reply: tid=0 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.436+0000 7f0eab057740 1 client.25531 shutdown 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.436+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 msgr2=0x56130e735070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.436+0000 7f0eab057740 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e735070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e98001eb0 tx=0x7f0e980315b0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 shutdown_connections 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e747d30 0x56130e74a120 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e735800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e735070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0e89ffb640 0 client.25531 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 --2- 172.21.15.103:0/2211989971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72d5d0 0x56130e737dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 >> 172.21.15.103:0/2211989971 conn(0x56130e6f6310 msgr2=0x56130e72f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 shutdown_connections 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 1 -- 172.21.15.103:0/2211989971 wait complete. 2024-09-19T07:40:29.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.437+0000 7f0eab057740 20 client.25531 trim_cache size 0 max 16384 2024-09-19T07:40:29.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 Processor -- start 2024-09-19T07:40:29.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 -- start start 2024-09-19T07:40:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72d5d0 0x56130e72e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e734930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e734050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8400e810 con 0x56130e72d5d0 2024-09-19T07:40:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84010410 con 0x56130e738370 2024-09-19T07:40:29.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.456+0000 7f0eab057740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8400eee0 con 0x56130e72eb40 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e734050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e734050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:37456/0 (socket says 172.21.15.103:37456) 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/2616551120 learned_addr learned my addr 172.21.15.103:0/2616551120 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea259c640 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72d5d0 0x56130e72e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e734930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/2616551120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 msgr2=0x56130e734930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e734930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/2616551120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72d5d0 msgr2=0x56130e72e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72d5d0 0x56130e72e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 -- 172.21.15.103:0/2616551120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130e7335c0 con 0x56130e738370 2024-09-19T07:40:29.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e734930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:40:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.457+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e734050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c098020 tx=0x7f0e9c09b5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baaef6577ea4c879 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.458+0000 7f0e8bfff640 1 -- 172.21.15.103:0/2616551120 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c020040 con 0x56130e738370 2024-09-19T07:40:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.458+0000 7f0e8bfff640 1 -- 172.21.15.103:0/2616551120 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0a4780 con 0x56130e738370 2024-09-19T07:40:29.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.458+0000 7f0e8bfff640 1 -- 172.21.15.103:0/2616551120 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c010b50 con 0x56130e738370 2024-09-19T07:40:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 -- 172.21.15.103:0/2616551120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 msgr2=0x56130e734050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e734050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c098020 tx=0x7f0e9c09b5e0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 -- 172.21.15.103:0/2616551120 shutdown_connections 2024-09-19T07:40:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e738370 0x56130e734050 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72eb40 0x56130e734930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 --2- 172.21.15.103:0/2616551120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e72d5d0 0x56130e72e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 -- 172.21.15.103:0/2616551120 >> 172.21.15.103:0/2616551120 conn(0x56130e6f6310 msgr2=0x56130e733940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 -- 172.21.15.103:0/2616551120 shutdown_connections 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.459+0000 7f0eab057740 1 -- 172.21.15.103:0/2616551120 wait complete. 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 -- start start 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 Processor -- start 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e740860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72eb40 0x56130e740da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e731ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8400fc80 con 0x56130e738370 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8400f5b0 con 0x56130e72eb40 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8400c7f0 con 0x56130e72d5d0 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72eb40 0x56130e740da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0ea1d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72eb40 0x56130e740da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:37466/0 (socket says 172.21.15.103:37466) 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/3818521841 learned_addr learned my addr 172.21.15.103:0/3818521841 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0eab057740 10 client.0 _ll_register_callbacks cb 0x7f0e98032da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.460+0000 7f0ea2d9d640 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e731ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0ea259c640 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e740860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/3818521841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 msgr2=0x56130e740860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e740860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/3818521841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 msgr2=0x56130e731ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e731ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0ea1d9b640 1 -- 172.21.15.103:0/3818521841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130e7324d0 con 0x56130e72eb40 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0ea1d9b640 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72eb40 0x56130e740da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e9803da00 tx=0x7f0e9803dd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2cf01d5ddca7296 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0e89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0e89ffb640 1 -- 172.21.15.103:0/3818521841 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98035070 con 0x56130e72eb40 2024-09-19T07:40:29.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0e89ffb640 1 -- 172.21.15.103:0/3818521841 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0e980055e0 con 0x56130e72eb40 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c0a4780 con 0x56130e72eb40 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0e89ffb640 1 -- 172.21.15.103:0/3818521841 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e98041040 con 0x56130e72eb40 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0eab057740 10 client.25565 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.461+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56130e7b2540 con 0x56130e72eb40 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0e89ffb640 1 -- 172.21.15.103:0/3818521841 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0a4780 con 0x56130e72eb40 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0e89ffb640 1 -- 172.21.15.103:0/3818521841 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0e98004c50 con 0x56130e72eb40 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0eab057740 10 client.25565 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0eab057740 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0eab057740 20 client.25565 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0eab057740 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e7b2d80 0x56130e7b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0eab057740 4 client.25565 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56130e69ad70 con 0x56130e7b2d80 2024-09-19T07:40:29.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0ea259c640 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e7b2d80 0x56130e7b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0e89ffb640 10 client.25565 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0ea259c640 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e7b2d80 0x56130e7b51a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38751e0bc03b90ae server_cookie=cf223045de992b31 in_seq=0 out_seq=0 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0e89ffb640 1 -- 172.21.15.103:0/3818521841 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56130e69ad70 con 0x56130e7b2d80 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.462+0000 7f0e89ffb640 10 client.25565 handle_command_reply: tid=0 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 client.25565 shutdown 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72eb40 msgr2=0x56130e740da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72eb40 0x56130e740da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e9803da00 tx=0x7f0e9803dd30 comp rx=0 tx=0).stop 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 shutdown_connections 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56130e7b2d80 0x56130e7b51a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56130e738370 0x56130e731ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0e89ffb640 0 client.25565 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56130e72eb40 0x56130e740da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 --2- 172.21.15.103:0/3818521841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56130e72d5d0 0x56130e740860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.463+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 >> 172.21.15.103:0/3818521841 conn(0x56130e6f6310 msgr2=0x56130e734160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.464+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 shutdown_connections 2024-09-19T07:40:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.464+0000 7f0eab057740 1 -- 172.21.15.103:0/3818521841 wait complete. 2024-09-19T07:40:29.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.464+0000 7f0eab057740 20 client.25565 trim_cache size 0 max 16384 2024-09-19T07:40:29.470 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0e8c0a34b0 msgr2=0x7f0e8c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0e8c0a34b0 0x7f0e8c0a5970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56130e72aac0 tx=0x7f0e9400b390 comp rx=0 tx=0).stop 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 msgr2=0x7f0ea410c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 0x7f0ea410c2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009fd0 tx=0x7f0e9c00b260 comp rx=0 tx=0).stop 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 shutdown_connections 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0e8c0a34b0 0x7f0e8c0a5970 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ea4105560 0x7f0ea410c2c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ea4100de0 0x7f0ea41a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 --2- 172.21.15.103:0/539213786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ea4100430 0x7f0ea41a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 >> 172.21.15.103:0/539213786 conn(0x7f0ea4077230 msgr2=0x7f0ea4103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:29.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 shutdown_connections 2024-09-19T07:40:29.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:29.465+0000 7f0ea8c7e640 1 -- 172.21.15.103:0/539213786 wait complete. 2024-09-19T07:40:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 115.0 s, 3382.6 tps, lat 29.488 ms stddev 29.693 2024-09-19T07:40:30.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:30 smithi103 ceph-mon[31201]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-09-19T07:40:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:30 smithi134 ceph-mon[35414]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-09-19T07:40:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:30 smithi174 ceph-mon[35370]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 639 op/s 2024-09-19T07:40:31.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:40:31.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:40:31.519 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:32.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.072+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1004696956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781008c0 msgr2=0x7fdb78100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.072+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1004696956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781008c0 0x7fdb78100ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009890 tx=0x7fdb6802a380 comp rx=0 tx=0).stop 2024-09-19T07:40:32.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.072+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1004696956 shutdown_connections 2024-09-19T07:40:32.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.072+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1004696956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb78105ec0 0x7fdb78106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.072+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1004696956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb781011e0 0x7fdb78105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.072+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1004696956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781008c0 0x7fdb78100ca0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009890 tx=0x7fdb6802a380 comp rx=0 tx=0).stop 2024-09-19T07:40:32.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.072+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1004696956 >> 172.21.15.103:0/1004696956 conn(0x7fdb780fc7d0 msgr2=0x7fdb780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:32.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.073+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1004696956 shutdown_connections 2024-09-19T07:40:32.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.073+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1004696956 wait complete. 2024-09-19T07:40:32.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 Processor -- start 2024-09-19T07:40:32.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 -- start start 2024-09-19T07:40:32.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781011e0 0x7fdb7819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb78105ec0 0x7fdb7819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb781a3640 0x7fdb781a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78118610 con 0x7fdb78105ec0 2024-09-19T07:40:32.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78118490 con 0x7fdb781a3640 2024-09-19T07:40:32.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb7d5bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78118790 con 0x7fdb781011e0 2024-09-19T07:40:32.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb78105ec0 0x7fdb7819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb78105ec0 0x7fdb7819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45682/0 (socket says 172.21.15.103:45682) 2024-09-19T07:40:32.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb6ffff640 1 -- 172.21.15.103:0/1793343370 learned_addr learned my addr 172.21.15.103:0/1793343370 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:32.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781011e0 0x7fdb7819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.074+0000 7fdb777fe640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb781a3640 0x7fdb781a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.075+0000 7fdb777fe640 1 -- 172.21.15.103:0/1793343370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781011e0 msgr2=0x7fdb7819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.075+0000 7fdb777fe640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781011e0 0x7fdb7819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.075+0000 7fdb777fe640 1 -- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb78105ec0 msgr2=0x7fdb7819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.075+0000 7fdb777fe640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb78105ec0 0x7fdb7819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.075+0000 7fdb777fe640 1 -- 172.21.15.103:0/1793343370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb781a4100 con 0x7fdb781a3640 2024-09-19T07:40:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.075+0000 7fdb777fe640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb781a3640 0x7fdb781a3af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000ef00 tx=0x7fdb7000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.076+0000 7fdb74ff9640 1 -- 172.21.15.103:0/1793343370 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70004b20 con 0x7fdb781a3640 2024-09-19T07:40:32.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.076+0000 7fdb74ff9640 1 -- 172.21.15.103:0/1793343370 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb70003e50 con 0x7fdb781a3640 2024-09-19T07:40:32.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.076+0000 7fdb74ff9640 1 -- 172.21.15.103:0/1793343370 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000b570 con 0x7fdb781a3640 2024-09-19T07:40:32.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.076+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb781a43f0 con 0x7fdb781a3640 2024-09-19T07:40:32.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.076+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb781abb80 con 0x7fdb781a3640 2024-09-19T07:40:32.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.079+0000 7fdb74ff9640 1 -- 172.21.15.103:0/1793343370 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdb7001d020 con 0x7fdb781a3640 2024-09-19T07:40:32.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.081+0000 7fdb7f996740 1 Processor -- start 2024-09-19T07:40:32.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.081+0000 7fdb7f996740 1 -- start start 2024-09-19T07:40:32.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.081+0000 7fdb74ff9640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdb640a3500 0x7fdb640a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26594af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 0x55b265955800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b265958130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb7f996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2658b0280 con 0x55b26594b480 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb7f996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b265857b10 con 0x55b265955d40 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb7f996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b2658ac240 con 0x55b26594ab60 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdb640a3500 0x7fdb640a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.082+0000 7fdb74ff9640 1 -- 172.21.15.103:0/1793343370 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdb700cfab0 con 0x7fdb781a3640 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b265958130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b265958130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:37498/0 (socket says 172.21.15.103:37498) 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 -- 172.21.15.103:0/3136362404 learned_addr learned my addr 172.21.15.103:0/3136362404 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdb640a3500 0x7fdb640a59c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000b770 tx=0x7fdb6000b700 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 -- 172.21.15.103:0/3136362404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 msgr2=0x55b26594af40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:40:32.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 --2- 172.21.15.103:0/3136362404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26594af40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 -- 172.21.15.103:0/3136362404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 msgr2=0x55b265955800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 --2- 172.21.15.103:0/3136362404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 0x55b265955800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 -- 172.21.15.103:0/3136362404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2658641f0 con 0x55b265955d40 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.083+0000 7fdb777fe640 1 --2- 172.21.15.103:0/3136362404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b265958130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb70098200 tx=0x7fdb7009c5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25fc94e03b325303 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.084+0000 7fdb6d7fa640 1 -- 172.21.15.103:0/3136362404 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7009f070 con 0x55b265955d40 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.084+0000 7fdb6d7fa640 1 -- 172.21.15.103:0/3136362404 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb70004cc0 con 0x55b265955d40 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.084+0000 7fdb6d7fa640 1 -- 172.21.15.103:0/3136362404 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7001a040 con 0x55b265955d40 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.084+0000 7fdb7f996740 1 -- 172.21.15.103:0/3136362404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 msgr2=0x55b265958130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.084+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3136362404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b265958130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb70098200 tx=0x7fdb7009c5e0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 -- 172.21.15.103:0/3136362404 shutdown_connections 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3136362404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b265958130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3136362404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 0x55b265955800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3136362404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26594af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 -- 172.21.15.103:0/3136362404 >> 172.21.15.103:0/3136362404 conn(0x55b2659470f0 msgr2=0x55b2659474d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:32.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 -- 172.21.15.103:0/3136362404 shutdown_connections 2024-09-19T07:40:32.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 -- 172.21.15.103:0/3136362404 wait complete. 2024-09-19T07:40:32.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 -- start start 2024-09-19T07:40:32.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.085+0000 7fdb7f996740 1 Processor -- start 2024-09-19T07:40:32.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b265952d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594b480 0x55b265953290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b2659537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb7f996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b265969920 con 0x55b265955d40 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb7f996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b26586af10 con 0x55b26594b480 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb7f996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b265969aa0 con 0x55b26594ab60 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb7f996740 10 client.0 _ll_register_callbacks cb 0x55b265833a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb76ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b265952d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb76ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b265952d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36234/0 (socket says 172.21.15.103:36234) 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb76ffd640 1 -- 172.21.15.103:0/2100655626 learned_addr learned my addr 172.21.15.103:0/2100655626 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594b480 0x55b265953290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb777fe640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b2659537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb6ffff640 1 -- 172.21.15.103:0/2100655626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 msgr2=0x55b265952d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b265952d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb6ffff640 1 -- 172.21.15.103:0/2100655626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 msgr2=0x55b2659537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b2659537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb6ffff640 1 -- 172.21.15.103:0/2100655626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2656216a0 con 0x55b26594b480 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb777fe640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b2659537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b265952d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594b480 0x55b265953290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb781000c0 tx=0x7fdb6000b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=382df272c059ab2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.086+0000 7fdb5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.087+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/2100655626 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6001d810 con 0x55b26594b480 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.087+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb70004cc0 con 0x55b26594b480 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.087+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/2100655626 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdb6001d9b0 con 0x55b26594b480 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.087+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/2100655626 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb6001dbb0 con 0x55b26594b480 2024-09-19T07:40:32.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.087+0000 7fdb7f996740 10 client.25577 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.087+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b265975700 con 0x55b26594b480 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.087+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/2100655626 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb70004cc0 con 0x55b26594b480 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/2100655626 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdb60019030 con 0x55b26594b480 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb7f996740 10 client.25577 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb7f996740 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb7f996740 20 client.25577 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb7f996740 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b265976d00 0x55b265979160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb7f996740 4 client.25577 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2658bd7f0 con 0x55b265976d00 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.088+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b265976d00 0x55b265979160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.089+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b265976d00 0x55b265979160 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d077a17b7a4f8f6 server_cookie=ce8d325bf59906f4 in_seq=0 out_seq=0 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.089+0000 7fdb5e7fc640 10 client.25577 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:32.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/2100655626 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2658bd7f0 con 0x55b265976d00 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb5e7fc640 10 client.25577 handle_command_reply: tid=0 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 client.25577 shutdown 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594b480 msgr2=0x55b265953290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594b480 0x55b265953290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdb781000c0 tx=0x7fdb6000b480 comp rx=0 tx=0).stop 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 shutdown_connections 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b265976d00 0x55b265979160 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b2659537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594b480 0x55b265953290 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 --2- 172.21.15.103:0/2100655626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b265952d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 >> 172.21.15.103:0/2100655626 conn(0x55b2659470f0 msgr2=0x55b26594ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb5e7fc640 0 client.25577 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 shutdown_connections 2024-09-19T07:40:32.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 1 -- 172.21.15.103:0/2100655626 wait complete. 2024-09-19T07:40:32.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.090+0000 7fdb7f996740 20 client.25577 trim_cache size 0 max 16384 2024-09-19T07:40:32.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.113+0000 7fdb7f996740 1 Processor -- start 2024-09-19T07:40:32.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb7f996740 1 -- start start 2024-09-19T07:40:32.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594ab60 0x55b265953620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594b480 0x55b265954bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b26597ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb7f996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781abde0 con 0x55b265955d40 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb7f996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781ad9b0 con 0x55b26594ab60 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb7f996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781ac480 con 0x55b26594b480 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594b480 0x55b265954bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594b480 0x55b265954bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36248/0 (socket says 172.21.15.103:36248) 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb6ffff640 1 -- 172.21.15.103:0/3277199010 learned_addr learned my addr 172.21.15.103:0/3277199010 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb76ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594ab60 0x55b265953620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb777fe640 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b26597ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb76ffd640 1 -- 172.21.15.103:0/3277199010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594b480 msgr2=0x55b265954bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594b480 0x55b265954bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb76ffd640 1 -- 172.21.15.103:0/3277199010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 msgr2=0x55b26597ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:32.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b26597ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb76ffd640 1 -- 172.21.15.103:0/3277199010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b265951d00 con 0x55b26594ab60 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.114+0000 7fdb777fe640 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b26597ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.115+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594ab60 0x55b265953620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55b265979a10 tx=0x7fdb680058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cc265efaa7120a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.115+0000 7fdb6cff9640 1 -- 172.21.15.103:0/3277199010 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68037040 con 0x55b26594ab60 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.115+0000 7fdb6cff9640 1 -- 172.21.15.103:0/3277199010 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb68038040 con 0x55b26594ab60 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.115+0000 7fdb6cff9640 1 -- 172.21.15.103:0/3277199010 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb680055e0 con 0x55b26594ab60 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.115+0000 7fdb7f996740 1 -- 172.21.15.103:0/3277199010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594ab60 msgr2=0x55b265953620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.115+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594ab60 0x55b265953620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55b265979a10 tx=0x7fdb680058f0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 -- 172.21.15.103:0/3277199010 shutdown_connections 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b265955d40 0x55b26597ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594b480 0x55b265954bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3277199010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b26594ab60 0x55b265953620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 -- 172.21.15.103:0/3277199010 >> 172.21.15.103:0/3277199010 conn(0x55b2659470f0 msgr2=0x55b2659544a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 -- 172.21.15.103:0/3277199010 shutdown_connections 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 -- 172.21.15.103:0/3277199010 wait complete. 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 -- start start 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.116+0000 7fdb7f996740 1 Processor -- start 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26596aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 0x55b26596afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb7f996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b26596b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb7f996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781ad220 con 0x55b26594b480 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb7f996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781acb50 con 0x55b265955d40 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb7f996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781a00d0 con 0x55b26594ab60 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb76ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26596aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb76ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26596aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36258/0 (socket says 172.21.15.103:36258) 2024-09-19T07:40:32.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb76ffd640 1 -- 172.21.15.103:0/3348048693 learned_addr learned my addr 172.21.15.103:0/3348048693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:32.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb7f996740 10 client.0 _ll_register_callbacks cb 0x7fdb6000bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:32.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.117+0000 7fdb777fe640 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b26596b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.119+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 0x55b26596afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.121+0000 7fdb76ffd640 1 -- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 msgr2=0x55b26596b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:32.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.121+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b26596b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.121+0000 7fdb76ffd640 1 -- 172.21.15.103:0/3348048693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 msgr2=0x55b26596afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:32.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.121+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 0x55b26596afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.121+0000 7fdb76ffd640 1 -- 172.21.15.103:0/3348048693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2659e0a90 con 0x55b26594ab60 2024-09-19T07:40:32.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.124+0000 7fdb76ffd640 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26596aaa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb680099b0 tx=0x7fdb68005310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d03c8adf2c9bdc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:32.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.124+0000 7fdb5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:40:32.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.125+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/3348048693 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6804f030 con 0x55b26594ab60 2024-09-19T07:40:32.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.125+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/3348048693 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fdb680040c0 con 0x55b26594ab60 2024-09-19T07:40:32.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.126+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb68038040 con 0x55b26594ab60 2024-09-19T07:40:32.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.126+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/3348048693 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb68037040 con 0x55b26594ab60 2024-09-19T07:40:32.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.127+0000 7fdb7f996740 10 client.25549 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:32.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.127+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2659e1d90 con 0x55b26594ab60 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/3348048693 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb680042c0 con 0x55b26594ab60 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/3348048693 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdb6803d040 con 0x55b26594ab60 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb7f996740 10 client.25549 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb7f996740 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb7f996740 20 client.25549 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b2659e24f0 0x55b2659e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb7f996740 4 client.25549 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:40:32.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.128+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fdb68038040 con 0x55b2659e24f0 2024-09-19T07:40:32.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.129+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b2659e24f0 0x55b2659e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:32.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.131+0000 7fdb6ffff640 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b2659e24f0 0x55b2659e4910 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b04088fdf8e942d server_cookie=64b0e636a0f21ec8 in_seq=0 out_seq=0 2024-09-19T07:40:32.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.131+0000 7fdb5e7fc640 10 client.25549 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:32.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.132+0000 7fdb5e7fc640 1 -- 172.21.15.103:0/3348048693 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fdb68038040 con 0x55b2659e24f0 2024-09-19T07:40:32.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.132+0000 7fdb5e7fc640 10 client.25549 handle_command_reply: tid=0 2024-09-19T07:40:32.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.132+0000 7fdb7f996740 1 client.25549 shutdown 2024-09-19T07:40:32.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 msgr2=0x55b26596aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26596aaa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb680099b0 tx=0x7fdb68005310 comp rx=0 tx=0).stop 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 shutdown_connections 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b2659e24f0 0x55b2659e4910 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb5e7fc640 0 client.25549 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b265955d40 0x55b26596b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b26594b480 0x55b26596afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 --2- 172.21.15.103:0/3348048693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26594ab60 0x55b26596aaa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.133+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 >> 172.21.15.103:0/3348048693 conn(0x55b2659470f0 msgr2=0x55b265946270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.134+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 shutdown_connections 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.134+0000 7fdb7f996740 1 -- 172.21.15.103:0/3348048693 wait complete. 2024-09-19T07:40:32.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.134+0000 7fdb7f996740 20 client.25549 trim_cache size 0 max 16384 2024-09-19T07:40:32.136 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:40:32.136 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:40:32.136 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "b885cb25-988e-406d-816d-762133960997", 2024-09-19T07:40:32.136 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:40:32.136 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdb640a3500 msgr2=0x7fdb640a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdb640a3500 0x7fdb640a59c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000b770 tx=0x7fdb6000b700 comp rx=0 tx=0).stop 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb781a3640 msgr2=0x7fdb781a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb781a3640 0x7fdb781a3af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000ef00 tx=0x7fdb7000c740 comp rx=0 tx=0).stop 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 shutdown_connections 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdb640a3500 0x7fdb640a59c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb781a3640 0x7fdb781a3af0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb78105ec0 0x7fdb7819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 --2- 172.21.15.103:0/1793343370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb781011e0 0x7fdb7819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:32.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 >> 172.21.15.103:0/1793343370 conn(0x7fdb780fc7d0 msgr2=0x7fdb78103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:32.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 shutdown_connections 2024-09-19T07:40:32.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:32.135+0000 7fdb7d5bd640 1 -- 172.21.15.103:0/1793343370 wait complete. 2024-09-19T07:40:32.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:32 smithi134 ceph-mon[35414]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 947 op/s 2024-09-19T07:40:32.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:32 smithi174 ceph-mon[35370]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 947 op/s 2024-09-19T07:40:32.678 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:32 smithi103 ceph-mon[31201]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 947 op/s 2024-09-19T07:40:32.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:40:33.290 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:40:33.526 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:40:33.526 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.526 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.527 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:33 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:40:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:40:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:40:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:33 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:40:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:40:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:40:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:33 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:40:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 -- 172.21.15.103:0/1323783768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c102d30 msgr2=0x7f2a5c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 --2- 172.21.15.103:0/1323783768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c102d30 0x7f2a5c103110 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009930 tx=0x7f2a4c02a420 comp rx=0 tx=0).stop 2024-09-19T07:40:33.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 -- 172.21.15.103:0/1323783768 shutdown_connections 2024-09-19T07:40:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 --2- 172.21.15.103:0/1323783768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c1082a0 0x7f2a5c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 --2- 172.21.15.103:0/1323783768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c1036e0 0x7f2a5c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 --2- 172.21.15.103:0/1323783768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c102d30 0x7f2a5c103110 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 -- 172.21.15.103:0/1323783768 >> 172.21.15.103:0/1323783768 conn(0x7f2a5c0fc7d0 msgr2=0x7f2a5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 -- 172.21.15.103:0/1323783768 shutdown_connections 2024-09-19T07:40:33.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.773+0000 7f2a608f9640 1 -- 172.21.15.103:0/1323783768 wait complete. 2024-09-19T07:40:33.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.775+0000 7f2a608f9640 1 Processor -- start 2024-09-19T07:40:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a608f9640 1 -- start start 2024-09-19T07:40:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a608f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c102d30 0x7f2a5c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a608f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c1036e0 0x7f2a5c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a608f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c1082a0 0x7f2a5c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a608f9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1120e0 con 0x7f2a5c1036e0 2024-09-19T07:40:33.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a608f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c111f60 con 0x7f2a5c1082a0 2024-09-19T07:40:33.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a608f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c112260 con 0x7f2a5c102d30 2024-09-19T07:40:33.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c102d30 0x7f2a5c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c102d30 0x7f2a5c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36270/0 (socket says 172.21.15.103:36270) 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a5a59c640 1 -- 172.21.15.103:0/3727488777 learned_addr learned my addr 172.21.15.103:0/3727488777 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c1036e0 0x7f2a5c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.776+0000 7f2a5ad9d640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c1082a0 0x7f2a5c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.777+0000 7f2a59d9b640 1 -- 172.21.15.103:0/3727488777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c102d30 msgr2=0x7f2a5c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.777+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c102d30 0x7f2a5c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.777+0000 7f2a59d9b640 1 -- 172.21.15.103:0/3727488777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c1082a0 msgr2=0x7f2a5c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.777+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c1082a0 0x7f2a5c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.777+0000 7f2a59d9b640 1 -- 172.21.15.103:0/3727488777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c19d550 con 0x7f2a5c1036e0 2024-09-19T07:40:33.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.777+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c1036e0 0x7f2a5c198ae0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0697d0 tx=0x7f2a5000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:33.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.779+0000 7f2a4b7fe640 1 -- 172.21.15.103:0/3727488777 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5000cc40 con 0x7f2a5c1036e0 2024-09-19T07:40:33.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.779+0000 7f2a4b7fe640 1 -- 172.21.15.103:0/3727488777 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5000cde0 con 0x7f2a5c1036e0 2024-09-19T07:40:33.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.779+0000 7f2a4b7fe640 1 -- 172.21.15.103:0/3727488777 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a500188b0 con 0x7f2a5c1036e0 2024-09-19T07:40:33.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.779+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c19d840 con 0x7f2a5c1036e0 2024-09-19T07:40:33.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.779+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c06bc10 con 0x7f2a5c1036e0 2024-09-19T07:40:33.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.783+0000 7f2a62cd2740 1 Processor -- start 2024-09-19T07:40:33.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.784+0000 7f2a62cd2740 1 -- start start 2024-09-19T07:40:33.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a4b7fe640 1 -- 172.21.15.103:0/3727488777 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2a50018a50 con 0x7f2a5c1036e0 2024-09-19T07:40:33.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x55748798f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x55748798fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574878f52e0 con 0x5574879a2d20 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55748789cce0 con 0x557487990a20 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5574878f12a0 con 0x55748798f880 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.785+0000 7f2a59d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.786+0000 7f2a59d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45764/0 (socket says 172.21.15.103:45764) 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.786+0000 7f2a59d9b640 1 -- 172.21.15.103:0/3026324487 learned_addr learned my addr 172.21.15.103:0/3026324487 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.786+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x55748798fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.786+0000 7f2a5ad9d640 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x55748798f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.787+0000 7f2a59d9b640 1 -- 172.21.15.103:0/3026324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 msgr2=0x55748798fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:33.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.787+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x55748798fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.787+0000 7f2a59d9b640 1 -- 172.21.15.103:0/3026324487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 msgr2=0x55748798f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:33.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.787+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x55748798f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.787+0000 7f2a4b7fe640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a440a3540 0x7f2a440a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.788+0000 7f2a4b7fe640 1 -- 172.21.15.103:0/3727488777 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2a500d00b0 con 0x7f2a5c1036e0 2024-09-19T07:40:33.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.788+0000 7f2a59d9b640 1 -- 172.21.15.103:0/3026324487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557487881440 con 0x5574879a2d20 2024-09-19T07:40:33.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.788+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x55748798fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:40:33.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.788+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a5110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2a500cd780 tx=0x7f2a5000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cbc25ee75d33db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:33.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.788+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a440a3540 0x7f2a440a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.788+0000 7f2a48ff9640 1 -- 172.21.15.103:0/3026324487 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5009c170 con 0x5574879a2d20 2024-09-19T07:40:33.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.788+0000 7f2a48ff9640 1 -- 172.21.15.103:0/3026324487 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5009caa0 con 0x5574879a2d20 2024-09-19T07:40:33.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.789+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3026324487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 msgr2=0x5574879a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.789+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a5110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2a500cd780 tx=0x7f2a5000cf90 comp rx=0 tx=0).stop 2024-09-19T07:40:33.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.790+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3026324487 shutdown_connections 2024-09-19T07:40:33.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.790+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a5110 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.791+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x55748798fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.791+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3026324487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x55748798f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.791+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3026324487 >> 172.21.15.103:0/3026324487 conn(0x55748798ba30 msgr2=0x55748798be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:33.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.791+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a440a3540 0x7f2a440a5a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55748798c890 tx=0x7f2a4c02a920 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:33.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.793+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3026324487 shutdown_connections 2024-09-19T07:40:33.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.793+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3026324487 wait complete. 2024-09-19T07:40:33.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.794+0000 7f2a62cd2740 1 -- start start 2024-09-19T07:40:33.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.794+0000 7f2a62cd2740 1 Processor -- start 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.794+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55748798f880 0x557487995a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x557487992d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x557487993280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574879a5a30 con 0x5574879a2d20 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574878b0050 con 0x55748798f880 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5574879a5bb0 con 0x557487990a20 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a62cd2740 10 client.0 _ll_register_callbacks cb 0x5574878f0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x557487993280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55748798f880 0x557487995a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55748798f880 0x557487995a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:37548/0 (socket says 172.21.15.103:37548) 2024-09-19T07:40:33.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x557487993280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45766/0 (socket says 172.21.15.103:45766) 2024-09-19T07:40:33.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a5a59c640 1 -- 172.21.15.103:0/1269915772 learned_addr learned my addr 172.21.15.103:0/1269915772 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:33.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x557487992d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a59d9b640 1 -- 172.21.15.103:0/1269915772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55748798f880 msgr2=0x557487995a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:33.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55748798f880 0x557487995a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a59d9b640 1 -- 172.21.15.103:0/1269915772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 msgr2=0x557487993280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:33.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x557487993280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.795+0000 7f2a59d9b640 1 -- 172.21.15.103:0/1269915772 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574876663c0 con 0x557487990a20 2024-09-19T07:40:33.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.796+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x557487992d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a500cd780 tx=0x7f2a5009a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b968743d0baf9c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:33.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.796+0000 7f2a41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:40:33.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.796+0000 7f2a41ffb640 1 -- 172.21.15.103:0/1269915772 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5009bee0 con 0x557487990a20 2024-09-19T07:40:33.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.796+0000 7f2a41ffb640 1 -- 172.21.15.103:0/1269915772 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2a5009b9f0 con 0x557487990a20 2024-09-19T07:40:33.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.796+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5009caa0 con 0x557487990a20 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.797+0000 7f2a41ffb640 1 -- 172.21.15.103:0/1269915772 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a5009c170 con 0x557487990a20 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.797+0000 7f2a62cd2740 10 client.25558 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.797+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574879a5a30 con 0x557487990a20 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a41ffb640 1 -- 172.21.15.103:0/1269915772 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5009caa0 con 0x557487990a20 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a41ffb640 1 -- 172.21.15.103:0/1269915772 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2a500bc320 con 0x557487990a20 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a62cd2740 10 client.25558 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a62cd2740 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a62cd2740 20 client.25558 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5574879b25e0 0x5574879b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a62cd2740 4 client.25558 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:40:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557487902850 con 0x5574879b25e0 2024-09-19T07:40:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.798+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5574879b25e0 0x5574879b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.799+0000 7f2a41ffb640 10 client.25558 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.799+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5574879b25e0 0x5574879b49d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234dd8438a2bb586 server_cookie=f3759bd68ec19b5e in_seq=0 out_seq=0 2024-09-19T07:40:33.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.800+0000 7f2a41ffb640 1 -- 172.21.15.103:0/1269915772 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557487902850 con 0x5574879b25e0 2024-09-19T07:40:33.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.800+0000 7f2a41ffb640 10 client.25558 handle_command_reply: tid=0 2024-09-19T07:40:33.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.800+0000 7f2a62cd2740 1 client.25558 shutdown 2024-09-19T07:40:33.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 msgr2=0x557487992d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x557487992d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a500cd780 tx=0x7f2a5009a8b0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 shutdown_connections 2024-09-19T07:40:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5574879b25e0 0x5574879b49d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a41ffb640 0 client.25558 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x557487993280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x557487992d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1269915772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55748798f880 0x557487995a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 >> 172.21.15.103:0/1269915772 conn(0x55748798ba30 msgr2=0x557487991750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 shutdown_connections 2024-09-19T07:40:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.801+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1269915772 wait complete. 2024-09-19T07:40:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.802+0000 7f2a62cd2740 20 client.25558 trim_cache size 0 max 16384 2024-09-19T07:40:33.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.833+0000 7f2a62cd2740 1 Processor -- start 2024-09-19T07:40:33.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.837+0000 7f2a62cd2740 1 -- start start 2024-09-19T07:40:33.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.837+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55748798f880 0x5574879aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.837+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x55748798aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.837+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574879a2d20 0x5574879ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.837+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c072280 con 0x55748798f880 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.837+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c073e80 con 0x5574879a2d20 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.837+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c072950 con 0x557487990a20 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.838+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55748798f880 0x5574879aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.838+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55748798f880 0x5574879aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45782/0 (socket says 172.21.15.103:45782) 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.838+0000 7f2a5a59c640 1 -- 172.21.15.103:0/1136121258 learned_addr learned my addr 172.21.15.103:0/1136121258 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:33.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.838+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x55748798aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.839+0000 7f2a5a59c640 1 -- 172.21.15.103:0/1136121258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 msgr2=0x55748798aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:33.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.839+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x55748798aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.839+0000 7f2a5a59c640 1 -- 172.21.15.103:0/1136121258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574879a2d20 msgr2=0x5574879ba190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:40:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.839+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574879a2d20 0x5574879ba190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.839+0000 7f2a5a59c640 1 -- 172.21.15.103:0/1136121258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557487996260 con 0x55748798f880 2024-09-19T07:40:33.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.841+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55748798f880 0x5574879aed50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5414f5c907d643f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:33.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.844+0000 7f2a43fff640 1 -- 172.21.15.103:0/1136121258 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c035040 con 0x55748798f880 2024-09-19T07:40:33.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.844+0000 7f2a43fff640 1 -- 172.21.15.103:0/1136121258 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557487902850 con 0x55748798f880 2024-09-19T07:40:33.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.845+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1136121258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55748798f880 msgr2=0x5574879aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.845+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55748798f880 0x5574879aed50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c000c00 tx=0x7f2a4c03e000 comp rx=0 tx=0).stop 2024-09-19T07:40:33.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.846+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1136121258 shutdown_connections 2024-09-19T07:40:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.846+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574879a2d20 0x5574879ba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.846+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557487990a20 0x55748798aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.846+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/1136121258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55748798f880 0x5574879aed50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.846+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1136121258 >> 172.21.15.103:0/1136121258 conn(0x55748798ba30 msgr2=0x55748798a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:33.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.846+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1136121258 shutdown_connections 2024-09-19T07:40:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.846+0000 7f2a62cd2740 1 -- 172.21.15.103:0/1136121258 wait complete. 2024-09-19T07:40:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.847+0000 7f2a62cd2740 1 -- start start 2024-09-19T07:40:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.847+0000 7f2a62cd2740 1 Processor -- start 2024-09-19T07:40:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.848+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x5574879aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.848+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x5574879ab400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.848+0000 7f2a62cd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.849+0000 7f2a62cd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0736f0 con 0x5574879a2d20 2024-09-19T07:40:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.849+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x5574879aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.849+0000 7f2a5a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x5574879aaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36310/0 (socket says 172.21.15.103:36310) 2024-09-19T07:40:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.849+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.849+0000 7f2a5a59c640 1 -- 172.21.15.103:0/3538329896 learned_addr learned my addr 172.21.15.103:0/3538329896 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:40:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c073020 con 0x557487990a20 2024-09-19T07:40:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c070260 con 0x55748798f880 2024-09-19T07:40:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a62cd2740 10 client.0 _ll_register_callbacks cb 0x7f2a500d8300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:40:33.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x5574879ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a5a59c640 1 -- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 msgr2=0x5574879ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:33.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x5574879ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a5a59c640 1 -- 172.21.15.103:0/3538329896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 msgr2=0x5574879a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a5a59c640 1 -- 172.21.15.103:0/3538329896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574879a8480 con 0x55748798f880 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.850+0000 7f2a5a59c640 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x5574879aaec0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c02a990 tx=0x7f2a4c0051d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d9fb0c7e7b52b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.851+0000 7f2a41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.851+0000 7f2a41ffb640 1 -- 172.21.15.103:0/3538329896 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004d10 con 0x55748798f880 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.851+0000 7f2a41ffb640 1 -- 172.21.15.103:0/3538329896 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f2a4c0325c0 con 0x55748798f880 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.851+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5574878fcd60 con 0x55748798f880 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.851+0000 7f2a41ffb640 1 -- 172.21.15.103:0/3538329896 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2a4c0327c0 con 0x55748798f880 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.851+0000 7f2a62cd2740 10 client.25564 fetch_fsmap learned FSMap version 30 2024-09-19T07:40:33.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.851+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557487a1daf0 con 0x55748798f880 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a41ffb640 1 -- 172.21.15.103:0/3538329896 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5574878fcd60 con 0x55748798f880 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a41ffb640 1 -- 172.21.15.103:0/3538329896 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2a4c053490 con 0x55748798f880 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a62cd2740 10 client.25564 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a62cd2740 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a62cd2740 20 client.25564 populate_metadata read hostname 'smithi103' 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557487a1e250 0x557487a20670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a62cd2740 4 client.25564 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557487879210 con 0x557487a1e250 2024-09-19T07:40:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.852+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557487a1e250 0x557487a20670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:40:33.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.853+0000 7f2a41ffb640 10 client.25564 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:33.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.853+0000 7f2a59d9b640 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557487a1e250 0x557487a20670 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842cabae10c5fd6b server_cookie=5a27d49c7151b2d in_seq=0 out_seq=0 2024-09-19T07:40:33.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.854+0000 7f2a41ffb640 1 -- 172.21.15.103:0/3538329896 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557487879210 con 0x557487a1e250 2024-09-19T07:40:33.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.854+0000 7f2a41ffb640 10 client.25564 handle_command_reply: tid=0 2024-09-19T07:40:33.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.854+0000 7f2a62cd2740 1 client.25564 shutdown 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 msgr2=0x5574879aaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x5574879aaec0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c02a990 tx=0x7f2a4c0051d0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 shutdown_connections 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557487a1e250 0x557487a20670 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574879a2d20 0x5574879a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557487990a20 0x5574879ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 --2- 172.21.15.103:0/3538329896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55748798f880 0x5574879aaec0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.855+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 >> 172.21.15.103:0/3538329896 conn(0x55748798ba30 msgr2=0x55748798ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.856+0000 7f2a41ffb640 0 client.25564 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.856+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 shutdown_connections 2024-09-19T07:40:33.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.856+0000 7f2a62cd2740 1 -- 172.21.15.103:0/3538329896 wait complete. 2024-09-19T07:40:33.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.856+0000 7f2a62cd2740 20 client.25564 trim_cache size 0 max 16384 2024-09-19T07:40:33.859 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:40:33.859 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:40:33.859 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:40:33.859 INFO:teuthology.orchestra.run.smithi103.stdout: "b885cb25-988e-406d-816d-762133960997": { 2024-09-19T07:40:33.859 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:40:33.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "b885cb25-988e-406d-816d-762133960997", 2024-09-19T07:40:33.860 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:40:33.860 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:40:33.860 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:40:33.860 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:40:33.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a440a3540 msgr2=0x7f2a440a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a440a3540 0x7f2a440a5a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55748798c890 tx=0x7f2a4c02a920 comp rx=0 tx=0).stop 2024-09-19T07:40:33.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c1036e0 msgr2=0x7f2a5c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:40:33.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c1036e0 0x7f2a5c198ae0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0697d0 tx=0x7f2a5000ee50 comp rx=0 tx=0).stop 2024-09-19T07:40:33.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 shutdown_connections 2024-09-19T07:40:33.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f2a440a3540 0x7f2a440a5a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a5c1082a0 0x7f2a5c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.859+0000 7f2a608f9640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a5c1036e0 0x7f2a5c198ae0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.860+0000 7f2a608f9640 1 --2- 172.21.15.103:0/3727488777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c102d30 0x7f2a5c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:40:33.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.860+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 >> 172.21.15.103:0/3727488777 conn(0x7f2a5c0fc7d0 msgr2=0x7f2a5c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:40:33.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.863+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 shutdown_connections 2024-09-19T07:40:33.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:40:33.864+0000 7f2a608f9640 1 -- 172.21.15.103:0/3727488777 wait complete. 2024-09-19T07:40:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:34 smithi103 ceph-mon[31201]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.4 MiB/s wr, 622 op/s 2024-09-19T07:40:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:34 smithi134 ceph-mon[35414]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.4 MiB/s wr, 622 op/s 2024-09-19T07:40:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:34 smithi174 ceph-mon[35370]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.4 MiB/s wr, 622 op/s 2024-09-19T07:40:34.949 INFO:tasks.cephfs.filesystem:scrub status for tag:b885cb25-988e-406d-816d-762133960997 - {'path': '/', 'tag': 'b885cb25-988e-406d-816d-762133960997', 'options': 'recursive,force'} 2024-09-19T07:40:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 120.0 s, 3518.8 tps, lat 28.574 ms stddev 29.000 2024-09-19T07:40:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:36 smithi134 ceph-mon[35414]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 962 op/s 2024-09-19T07:40:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:36 smithi174 ceph-mon[35370]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 962 op/s 2024-09-19T07:40:36.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:36 smithi103 ceph-mon[31201]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 962 op/s 2024-09-19T07:40:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:38 smithi134 ceph-mon[35414]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-19T07:40:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:38 smithi174 ceph-mon[35370]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-19T07:40:38.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:38 smithi103 ceph-mon[31201]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-19T07:40:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:39 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:40:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:39 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:40:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:39 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:40:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:39 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:40:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:39 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:40:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:39 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:40:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:39 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:40:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:39 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:40:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:39 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:40:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 125.0 s, 3277.6 tps, lat 30.404 ms stddev 30.778 2024-09-19T07:40:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:40 smithi134 ceph-mon[35414]: pgmap v83: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-19T07:40:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:40 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:40:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:40 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:40:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:40 smithi174 ceph-mon[35370]: pgmap v83: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-19T07:40:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:40 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:40:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:40 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:40:40.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:40 smithi103 ceph-mon[31201]: pgmap v83: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 647 op/s 2024-09-19T07:40:40.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:40 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:40:40.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:40 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:40:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:42 smithi134 ceph-mon[35414]: pgmap v84: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 958 op/s 2024-09-19T07:40:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:42 smithi174 ceph-mon[35370]: pgmap v84: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 958 op/s 2024-09-19T07:40:42.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:42 smithi103 ceph-mon[31201]: pgmap v84: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 958 op/s 2024-09-19T07:40:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:44 smithi134 ceph-mon[35414]: pgmap v85: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.1 MiB/s wr, 650 op/s 2024-09-19T07:40:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:44 smithi174 ceph-mon[35370]: pgmap v85: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.1 MiB/s wr, 650 op/s 2024-09-19T07:40:44.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:44 smithi103 ceph-mon[31201]: pgmap v85: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.1 MiB/s wr, 650 op/s 2024-09-19T07:40:45.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 130.0 s, 3275.4 tps, lat 30.611 ms stddev 32.748 2024-09-19T07:40:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:46 smithi134 ceph-mon[35414]: pgmap v86: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-19T07:40:46.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:46 smithi174 ceph-mon[35370]: pgmap v86: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-19T07:40:46.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:46 smithi103 ceph-mon[31201]: pgmap v86: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-19T07:40:48.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:48 smithi134 ceph-mon[35414]: pgmap v87: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 654 op/s 2024-09-19T07:40:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:48 smithi174 ceph-mon[35370]: pgmap v87: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 654 op/s 2024-09-19T07:40:48.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:48 smithi103 ceph-mon[31201]: pgmap v87: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 654 op/s 2024-09-19T07:40:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 135.0 s, 3270.9 tps, lat 30.529 ms stddev 52.916 2024-09-19T07:40:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:50 smithi134 ceph-mon[35414]: pgmap v88: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 654 op/s 2024-09-19T07:40:50.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:50 smithi174 ceph-mon[35370]: pgmap v88: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 654 op/s 2024-09-19T07:40:50.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:50 smithi103 ceph-mon[31201]: pgmap v88: 97 pgs: 97 active+clean; 1.6 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 MiB/s wr, 654 op/s 2024-09-19T07:40:51.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:51 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:40:51.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:51 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:40:51.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:51 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:40:52.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:52 smithi134 ceph-mon[35414]: pgmap v89: 97 pgs: 97 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 1.74k op/s 2024-09-19T07:40:52.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:52 smithi174 ceph-mon[35370]: pgmap v89: 97 pgs: 97 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 1.74k op/s 2024-09-19T07:40:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:52 smithi103 ceph-mon[31201]: pgmap v89: 97 pgs: 97 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 1.74k op/s 2024-09-19T07:40:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:54 smithi134 ceph-mon[35414]: pgmap v90: 97 pgs: 97 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 35 MiB/s wr, 1.43k op/s 2024-09-19T07:40:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:54 smithi174 ceph-mon[35370]: pgmap v90: 97 pgs: 97 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 35 MiB/s wr, 1.43k op/s 2024-09-19T07:40:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:54 smithi103 ceph-mon[31201]: pgmap v90: 97 pgs: 97 active+clean; 1.9 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 35 MiB/s wr, 1.43k op/s 2024-09-19T07:40:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 140.0 s, 3289.7 tps, lat 30.386 ms stddev 31.623 2024-09-19T07:40:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:56 smithi134 ceph-mon[35414]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 38 MiB/s wr, 1.75k op/s 2024-09-19T07:40:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:56 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:40:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:56 smithi174 ceph-mon[35370]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 38 MiB/s wr, 1.75k op/s 2024-09-19T07:40:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:56 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:40:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:56 smithi103 ceph-mon[31201]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 38 MiB/s wr, 1.75k op/s 2024-09-19T07:40:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:56 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:40:58.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:40:58 smithi134 ceph-mon[35414]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 34 MiB/s wr, 1.41k op/s 2024-09-19T07:40:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:40:58 smithi174 ceph-mon[35370]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 34 MiB/s wr, 1.41k op/s 2024-09-19T07:40:58.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:40:58 smithi103 ceph-mon[31201]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 34 MiB/s wr, 1.41k op/s 2024-09-19T07:41:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 145.0 s, 3307.6 tps, lat 30.255 ms stddev 32.847 2024-09-19T07:41:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:00 smithi134 ceph-mon[35414]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 34 MiB/s wr, 1.41k op/s 2024-09-19T07:41:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:00 smithi174 ceph-mon[35370]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 34 MiB/s wr, 1.41k op/s 2024-09-19T07:41:00.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:00 smithi103 ceph-mon[31201]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 34 MiB/s wr, 1.41k op/s 2024-09-19T07:41:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:02 smithi134 ceph-mon[35414]: pgmap v94: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 41 MiB/s wr, 1.87k op/s 2024-09-19T07:41:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:02 smithi174 ceph-mon[35370]: pgmap v94: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 41 MiB/s wr, 1.87k op/s 2024-09-19T07:41:02.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:02 smithi103 ceph-mon[31201]: pgmap v94: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 41 MiB/s wr, 1.87k op/s 2024-09-19T07:41:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:03 smithi134 ceph-mon[35414]: pgmap v95: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.9 MiB/s wr, 781 op/s 2024-09-19T07:41:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:03 smithi174 ceph-mon[35370]: pgmap v95: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.9 MiB/s wr, 781 op/s 2024-09-19T07:41:03.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:03 smithi103 ceph-mon[31201]: pgmap v95: 97 pgs: 97 active+clean; 1.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.9 MiB/s wr, 781 op/s 2024-09-19T07:41:04.950 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:41:05.293 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:05.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 150.0 s, 2290.4 tps, lat 43.675 ms stddev 73.091 2024-09-19T07:41:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:05 smithi174 ceph-mon[35370]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 29 MiB/s wr, 2.22k op/s 2024-09-19T07:41:05.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:05 smithi103 ceph-mon[31201]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 29 MiB/s wr, 2.22k op/s 2024-09-19T07:41:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:05 smithi134 ceph-mon[35414]: pgmap v96: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 29 MiB/s wr, 2.22k op/s 2024-09-19T07:41:06.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 -- 172.21.15.103:0/2662175905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 msgr2=0x7f786810e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 --2- 172.21.15.103:0/2662175905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 0x7f786810e080 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f785c009900 tx=0x7f785c02a410 comp rx=0 tx=0).stop 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 -- 172.21.15.103:0/2662175905 shutdown_connections 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 --2- 172.21.15.103:0/2662175905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786810e7b0 0x7f7868115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 --2- 172.21.15.103:0/2662175905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 0x7f786810e080 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 --2- 172.21.15.103:0/2662175905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 0x7f7868109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 -- 172.21.15.103:0/2662175905 >> 172.21.15.103:0/2662175905 conn(0x7f7868077c10 msgr2=0x7f786807a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.482+0000 7f786e94a640 1 -- 172.21.15.103:0/2662175905 shutdown_connections 2024-09-19T07:41:06.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.483+0000 7f786e94a640 1 -- 172.21.15.103:0/2662175905 wait complete. 2024-09-19T07:41:06.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.483+0000 7f786e94a640 1 Processor -- start 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786e94a640 1 -- start start 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786e94a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 0x7f78681a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786e94a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 0x7f78681a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786e94a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786810e7b0 0x7f786819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786e94a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78681185b0 con 0x7f7868109c40 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786e94a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7868118430 con 0x7f786810e7b0 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786e94a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7868118730 con 0x7f7868109290 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f78677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 0x7f78681a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 0x7f78681a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786cee7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786810e7b0 0x7f786819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 0x7f78681a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:32866/0 (socket says 172.21.15.103:32866) 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f78677fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 0x7f78681a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40260/0 (socket says 172.21.15.103:40260) 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 -- 172.21.15.103:0/3038598918 learned_addr learned my addr 172.21.15.103:0/3038598918 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 -- 172.21.15.103:0/3038598918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786810e7b0 msgr2=0x7f786819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786810e7b0 0x7f786819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 -- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 msgr2=0x7f78681a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 0x7f78681a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f7867fff640 1 -- 172.21.15.103:0/3038598918 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786819efb0 con 0x7f7868109290 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.484+0000 7f786cee7640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786810e7b0 0x7f786819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.485+0000 7f7867fff640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 0x7f78681a46c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f785800e9f0 tx=0x7f785800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:06.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.485+0000 7f78657fa640 1 -- 172.21.15.103:0/3038598918 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785800cd80 con 0x7f7868109290 2024-09-19T07:41:06.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.485+0000 7f78657fa640 1 -- 172.21.15.103:0/3038598918 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7858005500 con 0x7f7868109290 2024-09-19T07:41:06.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.485+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786819f2a0 con 0x7f7868109290 2024-09-19T07:41:06.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.485+0000 7f78657fa640 1 -- 172.21.15.103:0/3038598918 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858010690 con 0x7f7868109290 2024-09-19T07:41:06.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.485+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78681ab590 con 0x7f7868109290 2024-09-19T07:41:06.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.488+0000 7f78657fa640 1 -- 172.21.15.103:0/3038598918 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f78580046c0 con 0x7f7868109290 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 Processor -- start 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 -- start start 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14cf4a0 0x55f9e14cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14da680 0x55f9e14dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9e14352e0 con 0x55f9e14cf4a0 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f9e13dcce0 con 0x55f9e14da680 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7870d23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9e14312a0 con 0x55f9e14cfdc0 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f78657fa640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78500a3430 0x7f78500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7867fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f786cee7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14cf4a0 0x55f9e14cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f78677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14da680 0x55f9e14dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7867fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14da140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:32876/0 (socket says 172.21.15.103:32876) 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f786cee7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14cf4a0 0x55f9e14cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40262/0 (socket says 172.21.15.103:40262) 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7867fff640 1 -- 172.21.15.103:0/2845898741 learned_addr learned my addr 172.21.15.103:0/2845898741 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f78657fa640 1 -- 172.21.15.103:0/3038598918 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f78580d0090 con 0x7f7868109290 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f786cee7640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78500a3430 0x7f78500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7867fff640 1 -- 172.21.15.103:0/2845898741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14da680 msgr2=0x55f9e14dca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7867fff640 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14da680 0x55f9e14dca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7867fff640 1 -- 172.21.15.103:0/2845898741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14cf4a0 msgr2=0x55f9e14cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.489+0000 7f7867fff640 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14cf4a0 0x55f9e14cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.490+0000 7f7867fff640 1 -- 172.21.15.103:0/2845898741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e13c1440 con 0x55f9e14cfdc0 2024-09-19T07:41:06.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.490+0000 7f78677fe640 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14da680 0x55f9e14dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.490+0000 7f7867fff640 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14da140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f785809c080 tx=0x7f7858004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52dc6b243e8c01a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.490+0000 7f784e7fc640 1 -- 172.21.15.103:0/2845898741 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785809c380 con 0x55f9e14cfdc0 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.490+0000 7f784e7fc640 1 -- 172.21.15.103:0/2845898741 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f785809ca80 con 0x55f9e14cfdc0 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f784e7fc640 1 -- 172.21.15.103:0/2845898741 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785809c5e0 con 0x55f9e14cfdc0 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 -- 172.21.15.103:0/2845898741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 msgr2=0x55f9e14da140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14da140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f785809c080 tx=0x7f7858004b60 comp rx=0 tx=0).stop 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f786cee7640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78500a3430 0x7f78500a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7860005fd0 tx=0x7f7860004ab0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 -- 172.21.15.103:0/2845898741 shutdown_connections 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14da680 0x55f9e14dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14da140 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 --2- 172.21.15.103:0/2845898741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14cf4a0 0x55f9e14cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 -- 172.21.15.103:0/2845898741 >> 172.21.15.103:0/2845898741 conn(0x55f9e14cba30 msgr2=0x55f9e14cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:06.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.491+0000 7f7870d23740 1 -- 172.21.15.103:0/2845898741 shutdown_connections 2024-09-19T07:41:06.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.492+0000 7f7870d23740 1 -- 172.21.15.103:0/2845898741 wait complete. 2024-09-19T07:41:06.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.493+0000 7f7870d23740 1 -- start start 2024-09-19T07:41:06.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.493+0000 7f7870d23740 1 Processor -- start 2024-09-19T07:41:06.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.493+0000 7f7870d23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cf4a0 0x55f9e14d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.493+0000 7f7870d23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14d71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7870d23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14d76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7870d23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9e14dd290 con 0x55f9e14da680 2024-09-19T07:41:06.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7870d23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f9e13f0050 con 0x55f9e14cf4a0 2024-09-19T07:41:06.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7870d23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9e14dd410 con 0x55f9e14cfdc0 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7870d23740 10 client.0 _ll_register_callbacks cb 0x55f9e1430160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7867fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cf4a0 0x55f9e14d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f786cee7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14d76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78677fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14d71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f786cee7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14d76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40268/0 (socket says 172.21.15.103:40268) 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7867fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cf4a0 0x55f9e14d9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:49350/0 (socket says 172.21.15.103:49350) 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f786cee7640 1 -- 172.21.15.103:0/2229215306 learned_addr learned my addr 172.21.15.103:0/2229215306 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78677fe640 1 -- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cf4a0 msgr2=0x55f9e14d9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78677fe640 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cf4a0 0x55f9e14d9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78677fe640 1 -- 172.21.15.103:0/2229215306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 msgr2=0x55f9e14d76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78677fe640 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14d76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78677fe640 1 -- 172.21.15.103:0/2229215306 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e11a63c0 con 0x55f9e14cfdc0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f7867fff640 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cf4a0 0x55f9e14d9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f786cee7640 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14d76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78677fe640 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14d71b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f785c02a930 tx=0x7f785c0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88427f87ea7dcca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.494+0000 7f78477fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f78477fe640 1 -- 172.21.15.103:0/2229215306 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c035040 con 0x55f9e14cfdc0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f78477fe640 1 -- 172.21.15.103:0/2229215306 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f785c005450 con 0x55f9e14cfdc0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f785809ca80 con 0x55f9e14cfdc0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f78477fe640 1 -- 172.21.15.103:0/2229215306 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f785c005650 con 0x55f9e14cfdc0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 10 client.25576 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9e14e8bf0 con 0x55f9e14cfdc0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f78477fe640 1 -- 172.21.15.103:0/2229215306 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f785809ca80 con 0x55f9e14cfdc0 2024-09-19T07:41:06.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f78477fe640 1 -- 172.21.15.103:0/2229215306 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f785c003990 con 0x55f9e14cfdc0 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 10 client.25576 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 20 client.25576 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e14ea070 0x55f9e14ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 4 client.25576 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.495+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9e1442850 con 0x55f9e14ea070 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.496+0000 7f7867fff640 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e14ea070 0x55f9e14ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.496+0000 7f7867fff640 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e14ea070 0x55f9e14ec4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72787792bc42547 server_cookie=ea3e0863f43d5b51 in_seq=0 out_seq=0 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.496+0000 7f78477fe640 10 client.25576 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f78477fe640 1 -- 172.21.15.103:0/2229215306 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9e1442850 con 0x55f9e14ea070 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f78477fe640 10 client.25576 handle_command_reply: tid=0 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f7870d23740 1 client.25576 shutdown 2024-09-19T07:41:06.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 msgr2=0x55f9e14d71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f7870d23740 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14d71b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f785c02a930 tx=0x7f785c0043b0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 shutdown_connections 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f7870d23740 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e14ea070 0x55f9e14ec4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.497+0000 7f7870d23740 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14d76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.498+0000 7f7870d23740 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cfdc0 0x55f9e14d71b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.498+0000 7f7870d23740 1 --2- 172.21.15.103:0/2229215306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cf4a0 0x55f9e14d9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.498+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 >> 172.21.15.103:0/2229215306 conn(0x55f9e14cba30 msgr2=0x55f9e14d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.498+0000 7f78477fe640 0 client.25576 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.498+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 shutdown_connections 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.498+0000 7f7870d23740 1 -- 172.21.15.103:0/2229215306 wait complete. 2024-09-19T07:41:06.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.498+0000 7f7870d23740 20 client.25576 trim_cache size 0 max 16384 2024-09-19T07:41:06.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.518+0000 7f7870d23740 1 Processor -- start 2024-09-19T07:41:06.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7870d23740 1 -- start start 2024-09-19T07:41:06.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7870d23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7870d23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7870d23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7870d23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78681b3e10 con 0x55f9e14da680 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7870d23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78681b5aa0 con 0x55f9e14cfdc0 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7870d23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78681b44e0 con 0x55f9e14cf4a0 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f786cee7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f786cee7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40280/0 (socket says 172.21.15.103:40280) 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f786cee7640 1 -- 172.21.15.103:0/425523360 learned_addr learned my addr 172.21.15.103:0/425523360 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f78677fe640 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7867fff640 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f78677fe640 1 -- 172.21.15.103:0/425523360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 msgr2=0x55f9e14e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f78677fe640 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f78677fe640 1 -- 172.21.15.103:0/425523360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 msgr2=0x55f9e14f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f78677fe640 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f78677fe640 1 -- 172.21.15.103:0/425523360 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e14da270 con 0x55f9e14cfdc0 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f7867fff640 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.519+0000 7f78677fe640 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14f1e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f785c031550 tx=0x7f785c031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c07331ec964bb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.520+0000 7f784dffb640 1 -- 172.21.15.103:0/425523360 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c035030 con 0x55f9e14cfdc0 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.520+0000 7f784dffb640 1 -- 172.21.15.103:0/425523360 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f785c041070 con 0x55f9e14cfdc0 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.520+0000 7f784dffb640 1 -- 172.21.15.103:0/425523360 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c000e60 con 0x55f9e14cfdc0 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.520+0000 7f7870d23740 1 -- 172.21.15.103:0/425523360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 msgr2=0x55f9e14f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.520+0000 7f7870d23740 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14f1e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f785c031550 tx=0x7f785c031400 comp rx=0 tx=0).stop 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- 172.21.15.103:0/425523360 shutdown_connections 2024-09-19T07:41:06.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e14f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14f1e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 --2- 172.21.15.103:0/425523360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- 172.21.15.103:0/425523360 >> 172.21.15.103:0/425523360 conn(0x55f9e14cba30 msgr2=0x55f9e14d62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:06.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- 172.21.15.103:0/425523360 shutdown_connections 2024-09-19T07:41:06.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- 172.21.15.103:0/425523360 wait complete. 2024-09-19T07:41:06.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- start start 2024-09-19T07:41:06.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 Processor -- start 2024-09-19T07:41:06.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14dddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14de330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e154cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78681b5310 con 0x55f9e14da680 2024-09-19T07:41:06.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78681b4bb0 con 0x55f9e14cfdc0 2024-09-19T07:41:06.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.521+0000 7f7870d23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78681b1df0 con 0x55f9e14cf4a0 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7870d23740 10 client.0 _ll_register_callbacks cb 0x7f785c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14dddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:32906/0 (socket says 172.21.15.103:32906) 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 -- 172.21.15.103:0/574445759 learned_addr learned my addr 172.21.15.103:0/574445759 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f78677fe640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14de330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f786cee7640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e154cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 -- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 msgr2=0x55f9e14de330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14de330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 -- 172.21.15.103:0/574445759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 msgr2=0x55f9e154cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e154cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 -- 172.21.15.103:0/574445759 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e154d440 con 0x55f9e14cf4a0 2024-09-19T07:41:06.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f786cee7640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e154cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7867fff640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14dddf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78580a4ac0 tx=0x7f78580d0450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9e9eb0bd1190d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f78477fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f78477fe640 1 -- 172.21.15.103:0/574445759 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785809ca80 con 0x55f9e14cf4a0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f78477fe640 1 -- 172.21.15.103:0/574445759 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f78580cda70 con 0x55f9e14cf4a0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.522+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f785c041070 con 0x55f9e14cf4a0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.523+0000 7f78477fe640 1 -- 172.21.15.103:0/574445759 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f78580cdc70 con 0x55f9e14cf4a0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.523+0000 7f7870d23740 10 client.25582 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.523+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f9e1554860 con 0x55f9e14cf4a0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.523+0000 7f78477fe640 1 -- 172.21.15.103:0/574445759 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f785c041070 con 0x55f9e14cf4a0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f78477fe640 1 -- 172.21.15.103:0/574445759 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f785809b7d0 con 0x55f9e14cf4a0 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f7870d23740 10 client.25582 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f7870d23740 10 client.25582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:06.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f7870d23740 20 client.25582 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f7870d23740 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e1554bf0 0x55f9e1556fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f7870d23740 4 client.25582 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9e143cd60 con 0x55f9e1554bf0 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.524+0000 7f78677fe640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e1554bf0 0x55f9e1556fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f78677fe640 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e1554bf0 0x55f9e1556fe0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b1c4c1a9154e2 server_cookie=40a52abc349dd740 in_seq=0 out_seq=0 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f78477fe640 10 client.25582 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f78477fe640 1 -- 172.21.15.103:0/574445759 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f9e143cd60 con 0x55f9e1554bf0 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f78477fe640 10 client.25582 handle_command_reply: tid=0 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 client.25582 shutdown 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 msgr2=0x55f9e14dddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14dddf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78580a4ac0 tx=0x7f78580d0450 comp rx=0 tx=0).stop 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 shutdown_connections 2024-09-19T07:41:06.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f9e1554bf0 0x55f9e1556fe0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9e14da680 0x55f9e154cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9e14cfdc0 0x55f9e14de330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 --2- 172.21.15.103:0/574445759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9e14cf4a0 0x55f9e14dddf0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78580a4ac0 tx=0x7f78580d0450 comp rx=0 tx=0).stop 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 >> 172.21.15.103:0/574445759 conn(0x55f9e14cba30 msgr2=0x55f9e14ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.525+0000 7f78477fe640 0 client.25582 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.526+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 shutdown_connections 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.526+0000 7f7870d23740 1 -- 172.21.15.103:0/574445759 wait complete. 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.526+0000 7f7870d23740 20 client.25582 trim_cache size 0 max 16384 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:41:06.530 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:41:06.531 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:41:06.531 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:41:06.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78500a3430 msgr2=0x7f78500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78500a3430 0x7f78500a58f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7860005fd0 tx=0x7f7860004ab0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 msgr2=0x7f78681a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:06.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 0x7f78681a46c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f785800e9f0 tx=0x7f785800edf0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 shutdown_connections 2024-09-19T07:41:06.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f78500a3430 0x7f78500a58f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f786810e7b0 0x7f786819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7868109c40 0x7f78681a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 --2- 172.21.15.103:0/3038598918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7868109290 0x7f78681a46c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:06.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 >> 172.21.15.103:0/3038598918 conn(0x7f7868077c10 msgr2=0x7f786810c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:06.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 shutdown_connections 2024-09-19T07:41:06.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:06.527+0000 7f786e94a640 1 -- 172.21.15.103:0/3038598918 wait complete. 2024-09-19T07:41:07.285 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:41:07.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:41:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:07.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:07 smithi174 ceph-mon[35370]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 1.90k op/s 2024-09-19T07:41:07.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:07 smithi103 ceph-mon[31201]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 1.90k op/s 2024-09-19T07:41:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:07 smithi134 ceph-mon[35414]: pgmap v97: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 1.90k op/s 2024-09-19T07:41:08.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 -- 172.21.15.103:0/864349533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109450 msgr2=0x7f8320110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 --2- 172.21.15.103:0/864349533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109450 0x7f8320110250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8310009900 tx=0x7f831002a410 comp rx=0 tx=0).stop 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 -- 172.21.15.103:0/864349533 shutdown_connections 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 --2- 172.21.15.103:0/864349533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f832011b140 0x7f832011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 --2- 172.21.15.103:0/864349533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8320109830 0x7f8320109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 --2- 172.21.15.103:0/864349533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109450 0x7f8320110250 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 -- 172.21.15.103:0/864349533 >> 172.21.15.103:0/864349533 conn(0x7f832006e140 msgr2=0x7f832006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.172+0000 7f8325482640 1 -- 172.21.15.103:0/864349533 shutdown_connections 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.173+0000 7f8325482640 1 -- 172.21.15.103:0/864349533 wait complete. 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.173+0000 7f8325482640 1 Processor -- start 2024-09-19T07:41:08.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.173+0000 7f8325482640 1 -- start start 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f8325482640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8320109450 0x7f83201a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f8325482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109830 0x7f83201a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f8325482640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f832011b140 0x7f83201aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f8325482640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8320120da0 con 0x7f8320109830 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f8325482640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8320120c20 con 0x7f8320109450 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f8325482640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8320120f20 con 0x7f832011b140 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109830 0x7f83201a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109830 0x7f83201a7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32894/0 (socket says 172.21.15.103:32894) 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 -- 172.21.15.103:0/1703205789 learned_addr learned my addr 172.21.15.103:0/1703205789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831effd640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8320109450 0x7f83201a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:08.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 -- 172.21.15.103:0/1703205789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f832011b140 msgr2=0x7f83201aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831f7fe640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f832011b140 0x7f83201aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f832011b140 0x7f83201aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 -- 172.21.15.103:0/1703205789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8320109450 msgr2=0x7f83201a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8320109450 0x7f83201a71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 -- 172.21.15.103:0/1703205789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83201ac170 con 0x7f8320109830 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831effd640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8320109450 0x7f83201a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831f7fe640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f832011b140 0x7f83201aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.174+0000 7f831e7fc640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109830 0x7f83201a7700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f831400ea60 tx=0x7f831400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.175+0000 7f8307fff640 1 -- 172.21.15.103:0/1703205789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400cde0 con 0x7f8320109830 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.175+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83201ac460 con 0x7f8320109830 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.175+0000 7f8307fff640 1 -- 172.21.15.103:0/1703205789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83140035f0 con 0x7f8320109830 2024-09-19T07:41:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.175+0000 7f8307fff640 1 -- 172.21.15.103:0/1703205789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010670 con 0x7f8320109830 2024-09-19T07:41:08.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.175+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201b3d40 con 0x7f8320109830 2024-09-19T07:41:08.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.176+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83201a8520 con 0x7f8320109830 2024-09-19T07:41:08.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.178+0000 7f8307fff640 1 -- 172.21.15.103:0/1703205789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8314004020 con 0x7f8320109830 2024-09-19T07:41:08.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.179+0000 7f8307fff640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83080a3430 0x7f83080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:08.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.179+0000 7f8307fff640 1 -- 172.21.15.103:0/1703205789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f83140d0170 con 0x7f8320109830 2024-09-19T07:41:08.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.179+0000 7f831effd640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83080a3430 0x7f83080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:08.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.180+0000 7f831effd640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83080a3430 0x7f83080a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8310005ab0 tx=0x7f8310005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:08.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.185+0000 7f8307fff640 1 -- 172.21.15.103:0/1703205789 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f831409cb00 con 0x7f8320109830 2024-09-19T07:41:08.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.525+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f832010b550 con 0x7f8320109830 2024-09-19T07:41:08.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.526+0000 7f8307fff640 1 -- 172.21.15.103:0/1703205789 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12003 (secure 0 0 0) 0x7f831409c400 con 0x7f8320109830 2024-09-19T07:41:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:41:08.526 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":32,"btime":"2024-09-19T07:40:55:290656+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:40:54.957956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:41:08.526 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 32 2024-09-19T07:41:08.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.529+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83080a3430 msgr2=0x7f83080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:08.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.529+0000 7f8325482640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83080a3430 0x7f83080a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8310005ab0 tx=0x7f8310005a20 comp rx=0 tx=0).stop 2024-09-19T07:41:08.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.529+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109830 msgr2=0x7f83201a7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:08.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.529+0000 7f8325482640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109830 0x7f83201a7700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f831400ea60 tx=0x7f831400ef20 comp rx=0 tx=0).stop 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 shutdown_connections 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83080a3430 0x7f83080a58f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f832011b140 0x7f83201aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8320109830 0x7f83201a7700 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 --2- 172.21.15.103:0/1703205789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8320109450 0x7f83201a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 >> 172.21.15.103:0/1703205789 conn(0x7f832006e140 msgr2=0x7f832011be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 shutdown_connections 2024-09-19T07:41:08.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:08.530+0000 7f8325482640 1 -- 172.21.15.103:0/1703205789 wait complete. 2024-09-19T07:41:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:08 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:41:08.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:08 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:41:08.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:08 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:41:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:09 smithi174 ceph-mon[35370]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 1.90k op/s 2024-09-19T07:41:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:09 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1703205789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:09.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:09 smithi103 ceph-mon[31201]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 1.90k op/s 2024-09-19T07:41:09.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:09 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1703205789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:09.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:09 smithi134 ceph-mon[35414]: pgmap v98: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 MiB/s wr, 1.90k op/s 2024-09-19T07:41:09.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:09 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1703205789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 155.0 s, 3439.6 tps, lat 29.050 ms stddev 30.674 2024-09-19T07:41:10.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:41:10.940 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:11.603 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:11 smithi103 ceph-mon[31201]: pgmap v99: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 2.32k op/s 2024-09-19T07:41:11.604 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:11 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:41:11.604 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:11 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:41:11.604 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:11 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:41:11.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 -- 172.21.15.103:0/1315235853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc101430 msgr2=0x7f84bc10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/1315235853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc101430 0x7f84bc10d960 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009c10 tx=0x7f84b402a6f0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 -- 172.21.15.103:0/1315235853 shutdown_connections 2024-09-19T07:41:11.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/1315235853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc101430 0x7f84bc10d960 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/1315235853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84bc100a90 0x7f84bc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/1315235853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc106a90 0x7f84bc106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 -- 172.21.15.103:0/1315235853 >> 172.21.15.103:0/1315235853 conn(0x7f84bc0fc7d0 msgr2=0x7f84bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:11.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 -- 172.21.15.103:0/1315235853 shutdown_connections 2024-09-19T07:41:11.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.624+0000 7f84c1bc7640 1 -- 172.21.15.103:0/1315235853 wait complete. 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.625+0000 7f84c1bc7640 1 Processor -- start 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.625+0000 7f84c1bc7640 1 -- start start 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84c1bc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84bc100a90 0x7f84bc19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84c1bc7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc101430 0x7f84bc19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84c1bc7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc106a90 0x7f84bc1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84c1bc7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc110e40 con 0x7f84bc106a90 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84c1bc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc110cc0 con 0x7f84bc100a90 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84c1bc7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc110fc0 con 0x7f84bc101430 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc101430 0x7f84bc19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc101430 0x7f84bc19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40682/0 (socket says 172.21.15.103:40682) 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 -- 172.21.15.103:0/2055815549 learned_addr learned my addr 172.21.15.103:0/2055815549 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc106a90 0x7f84bc1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84bc100a90 0x7f84bc19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 -- 172.21.15.103:0/2055815549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84bc100a90 msgr2=0x7f84bc19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84bc100a90 0x7f84bc19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 -- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc106a90 msgr2=0x7f84bc1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc106a90 0x7f84bc1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.626+0000 7f84baffd640 1 -- 172.21.15.103:0/2055815549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc196c90 con 0x7f84bc101430 2024-09-19T07:41:11.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.627+0000 7f84baffd640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc101430 0x7f84bc19e530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d950 tx=0x7f84b000dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:11.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.627+0000 7f84b8ff9640 1 -- 172.21.15.103:0/2055815549 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0017070 con 0x7f84bc101430 2024-09-19T07:41:11.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.627+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc196f80 con 0x7f84bc101430 2024-09-19T07:41:11.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.627+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1b3d10 con 0x7f84bc101430 2024-09-19T07:41:11.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.627+0000 7f84b8ff9640 1 -- 172.21.15.103:0/2055815549 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b000de80 con 0x7f84bc101430 2024-09-19T07:41:11.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.627+0000 7f84b8ff9640 1 -- 172.21.15.103:0/2055815549 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000f620 con 0x7f84bc101430 2024-09-19T07:41:11.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.630+0000 7f84b8ff9640 1 -- 172.21.15.103:0/2055815549 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f84b000f7c0 con 0x7f84bc101430 2024-09-19T07:41:11.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 Processor -- start 2024-09-19T07:41:11.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 -- start start 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f69760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f088ec22f0 con 0x55f088f5c5d0 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f088e69cf0 con 0x55f088f5db40 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f088ebe2b0 con 0x55f088f67370 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.631+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f5ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32936/0 (socket says 172.21.15.103:32936) 2024-09-19T07:41:11.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bb7fe640 1 -- 172.21.15.103:0/3910695456 learned_addr learned my addr 172.21.15.103:0/3910695456 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84b8ff9640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84a40a3580 0x7f84a40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84baffd640 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f69760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bbfff640 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bb7fe640 1 -- 172.21.15.103:0/3910695456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 msgr2=0x55f088f69760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f69760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bb7fe640 1 -- 172.21.15.103:0/3910695456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 msgr2=0x55f088f5c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f5c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bb7fe640 1 -- 172.21.15.103:0/3910695456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f088e4e3a0 con 0x55f088f5c5d0 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84b8ff9640 1 -- 172.21.15.103:0/2055815549 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f84b00cf7c0 con 0x7f84bc101430 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bbfff640 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f5c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84baffd640 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.632+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84a40a3580 0x7f84a40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.633+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f5ca10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009c50 tx=0x7f84ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551b7b65aa17f9bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:11.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.633+0000 7f84a1ffb640 1 -- 172.21.15.103:0/3910695456 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac02a420 con 0x55f088f5c5d0 2024-09-19T07:41:11.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.633+0000 7f84a1ffb640 1 -- 172.21.15.103:0/3910695456 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac02aaa0 con 0x55f088f5c5d0 2024-09-19T07:41:11.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.633+0000 7f84a1ffb640 1 -- 172.21.15.103:0/3910695456 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac029be0 con 0x55f088f5c5d0 2024-09-19T07:41:11.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84a40a3580 0x7f84a40a5a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f84b4004af0 tx=0x7f84b4009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:11.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 -- 172.21.15.103:0/3910695456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 msgr2=0x55f088f5ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f5ca10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009c50 tx=0x7f84ac011810 comp rx=0 tx=0).stop 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 -- 172.21.15.103:0/3910695456 shutdown_connections 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f5ca10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/3910695456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f5c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 -- 172.21.15.103:0/3910695456 >> 172.21.15.103:0/3910695456 conn(0x55f088f25310 msgr2=0x55f088f58c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 -- 172.21.15.103:0/3910695456 shutdown_connections 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.634+0000 7f84c3fa0740 1 -- 172.21.15.103:0/3910695456 wait complete. 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 -- start start 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 Processor -- start 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f66c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5db40 0x55f088f63eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f67370 0x55f088f643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f088f6a070 con 0x55f088f5c5d0 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f088e7d060 con 0x55f088f67370 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f088f6a1f0 con 0x55f088f5db40 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84c3fa0740 10 client.0 _ll_register_callbacks cb 0x55f088bd08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84bbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f67370 0x55f088f643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84baffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5db40 0x55f088f63eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84bbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f67370 0x55f088f643f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60278/0 (socket says 172.21.15.103:60278) 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84baffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5db40 0x55f088f63eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40706/0 (socket says 172.21.15.103:40706) 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84bbfff640 1 -- 172.21.15.103:0/4279268796 learned_addr learned my addr 172.21.15.103:0/4279268796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.635+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84baffd640 1 -- 172.21.15.103:0/4279268796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f67370 msgr2=0x55f088f643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84baffd640 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f67370 0x55f088f643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84baffd640 1 -- 172.21.15.103:0/4279268796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 msgr2=0x55f088f66c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84baffd640 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f66c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84baffd640 1 -- 172.21.15.103:0/4279268796 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f088c333c0 con 0x55f088f5db40 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84baffd640 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5db40 0x55f088f63eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f84b00ccec0 tx=0x7f84b006b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886df6bb350d5a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f849affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f849affd640 1 -- 172.21.15.103:0/4279268796 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0020040 con 0x55f088f5db40 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ac02aaa0 con 0x55f088f5db40 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f849affd640 1 -- 172.21.15.103:0/4279268796 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f84b0009c30 con 0x55f088f5db40 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f849affd640 1 -- 172.21.15.103:0/4279268796 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84b00a46c0 con 0x55f088f5db40 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84c3fa0740 10 client.25591 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:11.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.636+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f088f75a10 con 0x55f088f5db40 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f849affd640 1 -- 172.21.15.103:0/4279268796 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac02aaa0 con 0x55f088f5db40 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f849affd640 1 -- 172.21.15.103:0/4279268796 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f84b009aea0 con 0x55f088f5db40 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f84c3fa0740 10 client.25591 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f84c3fa0740 10 client.25591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f84c3fa0740 20 client.25591 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088f76e20 0x55f088f79290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f84c3fa0740 4 client.25591 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f088ecf860 con 0x55f088f76e20 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.637+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088f76e20 0x55f088f79290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.638+0000 7f849affd640 10 client.25591 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:41:11.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.639+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088f76e20 0x55f088f79290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed9e1d9518db91f1 server_cookie=bb6219495d50017 in_seq=0 out_seq=0 2024-09-19T07:41:11.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.640+0000 7f849affd640 1 -- 172.21.15.103:0/4279268796 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f088ecf860 con 0x55f088f76e20 2024-09-19T07:41:11.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.640+0000 7f849affd640 10 client.25591 handle_command_reply: tid=0 2024-09-19T07:41:11.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.640+0000 7f84c3fa0740 1 client.25591 shutdown 2024-09-19T07:41:11.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5db40 msgr2=0x55f088f63eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5db40 0x55f088f63eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f84b00ccec0 tx=0x7f84b006b850 comp rx=0 tx=0).stop 2024-09-19T07:41:11.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 shutdown_connections 2024-09-19T07:41:11.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088f76e20 0x55f088f79290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f67370 0x55f088f643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5db40 0x55f088f63eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/4279268796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f5c5d0 0x55f088f66c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f849affd640 0 client.25591 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:41:11.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 >> 172.21.15.103:0/4279268796 conn(0x55f088f25310 msgr2=0x55f088f5e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:11.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 shutdown_connections 2024-09-19T07:41:11.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 1 -- 172.21.15.103:0/4279268796 wait complete. 2024-09-19T07:41:11.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.641+0000 7f84c3fa0740 20 client.25591 trim_cache size 0 max 16384 2024-09-19T07:41:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 Processor -- start 2024-09-19T07:41:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 -- start start 2024-09-19T07:41:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5c5d0 0x55f088f733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f67370 0x55f088f68ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800e790 con 0x55f088f67370 2024-09-19T07:41:11.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8010420 con 0x55f088f5db40 2024-09-19T07:41:11.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.661+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800ee60 con 0x55f088f5c5d0 2024-09-19T07:41:11.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f67370 0x55f088f68ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f67370 0x55f088f68ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32960/0 (socket says 172.21.15.103:32960) 2024-09-19T07:41:11.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 -- 172.21.15.103:0/2729552140 learned_addr learned my addr 172.21.15.103:0/2729552140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:11.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84baffd640 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 -- 172.21.15.103:0/2729552140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5c5d0 msgr2=0x55f088f733a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:41:11.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5c5d0 0x55f088f733a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 -- 172.21.15.103:0/2729552140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 msgr2=0x55f088f68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 -- 172.21.15.103:0/2729552140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f088f5f120 con 0x55f088f67370 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.662+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f67370 0x55f088f68ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009be0 tx=0x7f84b403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72de10fbd6e39aea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.663+0000 7f84a17fa640 1 -- 172.21.15.103:0/2729552140 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4036050 con 0x55f088f67370 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.663+0000 7f84a17fa640 1 -- 172.21.15.103:0/2729552140 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b4050070 con 0x55f088f67370 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.663+0000 7f84a17fa640 1 -- 172.21.15.103:0/2729552140 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4031680 con 0x55f088f67370 2024-09-19T07:41:11.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.664+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2729552140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f67370 msgr2=0x55f088f68ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.664+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f67370 0x55f088f68ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009be0 tx=0x7f84b403f000 comp rx=0 tx=0).stop 2024-09-19T07:41:11.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.665+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2729552140 shutdown_connections 2024-09-19T07:41:11.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.665+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f67370 0x55f088f68ca0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.665+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.665+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2729552140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f5c5d0 0x55f088f733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.665+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2729552140 >> 172.21.15.103:0/2729552140 conn(0x55f088f25310 msgr2=0x55f088f63e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:11 smithi174 ceph-mon[35370]: pgmap v99: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 2.32k op/s 2024-09-19T07:41:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:11 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:41:11.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:11 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:41:11.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:11 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:41:11.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.667+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2729552140 shutdown_connections 2024-09-19T07:41:11.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.668+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2729552140 wait complete. 2024-09-19T07:41:11.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.668+0000 7f84c3fa0740 1 -- start start 2024-09-19T07:41:11.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.669+0000 7f84c3fa0740 1 Processor -- start 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.669+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f6f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.669+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84c3fa0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f6ffa0 0x55f088f6a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f6f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f6f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60290/0 (socket says 172.21.15.103:60290) 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84c3fa0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800fc90 con 0x55f088f6ffa0 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84bb7fe640 1 -- 172.21.15.103:0/2465310952 learned_addr learned my addr 172.21.15.103:0/2465310952 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:11.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800f530 con 0x55f088f5db40 2024-09-19T07:41:11.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84a800c770 con 0x55f088f67370 2024-09-19T07:41:11.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84c3fa0740 10 client.0 _ll_register_callbacks cb 0x7f84b0097d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:11.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84baffd640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.670+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f6ffa0 0x55f088f6a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.671+0000 7f84bb7fe640 1 -- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 msgr2=0x55f088f6fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:11.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.671+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f6fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.671+0000 7f84bb7fe640 1 -- 172.21.15.103:0/2465310952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f6ffa0 msgr2=0x55f088f6a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.671+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f6ffa0 0x55f088f6a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.671+0000 7f84bb7fe640 1 -- 172.21.15.103:0/2465310952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f088f6ac10 con 0x55f088f5db40 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.671+0000 7f84bbfff640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f6ffa0 0x55f088f6a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.672+0000 7f84bb7fe640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f6f520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84ac029790 tx=0x7f84ac029720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946301b78e81c94b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.672+0000 7f849affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.672+0000 7f849affd640 1 -- 172.21.15.103:0/2465310952 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac029580 con 0x55f088f5db40 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.672+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b4050070 con 0x55f088f5db40 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.672+0000 7f849affd640 1 -- 172.21.15.103:0/2465310952 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f84ac004220 con 0x55f088f5db40 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.672+0000 7f849affd640 1 -- 172.21.15.103:0/2465310952 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84ac004420 con 0x55f088f5db40 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.673+0000 7f84c3fa0740 10 client.25637 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:11.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.673+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f088fe2570 con 0x55f088f5db40 2024-09-19T07:41:11.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.673+0000 7f849affd640 1 -- 172.21.15.103:0/2465310952 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b4050070 con 0x55f088f5db40 2024-09-19T07:41:11.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.673+0000 7f849affd640 1 -- 172.21.15.103:0/2465310952 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f84ac02a420 con 0x55f088f5db40 2024-09-19T07:41:11.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.674+0000 7f84c3fa0740 10 client.25637 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:11.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.674+0000 7f84c3fa0740 10 client.25637 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:41:11.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.674+0000 7f84c3fa0740 20 client.25637 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:11.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.674+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088fe2b00 0x55f088fe4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:11.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.674+0000 7f84c3fa0740 4 client.25637 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:41:11.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.674+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f088ec9d70 con 0x55f088fe2b00 2024-09-19T07:41:11.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.675+0000 7f84baffd640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088fe2b00 0x55f088fe4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:11.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.675+0000 7f84baffd640 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088fe2b00 0x55f088fe4ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=365d5a247c1d771d server_cookie=f64efaaa1d1d41ed in_seq=0 out_seq=0 2024-09-19T07:41:11.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.675+0000 7f849affd640 10 client.25637 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:41:11.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.675+0000 7f849affd640 1 -- 172.21.15.103:0/2465310952 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f088ec9d70 con 0x55f088fe2b00 2024-09-19T07:41:11.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.675+0000 7f849affd640 10 client.25637 handle_command_reply: tid=0 2024-09-19T07:41:11.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.676+0000 7f84c3fa0740 1 client.25637 shutdown 2024-09-19T07:41:11.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.679+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 msgr2=0x55f088f6f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.679+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f6f520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84ac029790 tx=0x7f84ac029720 comp rx=0 tx=0).stop 2024-09-19T07:41:11.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.680+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 shutdown_connections 2024-09-19T07:41:11.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.680+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f088fe2b00 0x55f088fe4ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.680+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f088f6ffa0 0x55f088f6a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.681+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f088f67370 0x55f088f6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.681+0000 7f84c3fa0740 1 --2- 172.21.15.103:0/2465310952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f088f5db40 0x55f088f6f520 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.681+0000 7f849affd640 0 client.25637 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:41:11.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.681+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 >> 172.21.15.103:0/2465310952 conn(0x55f088f25310 msgr2=0x55f088f62340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:11.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.681+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 shutdown_connections 2024-09-19T07:41:11.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.681+0000 7f84c3fa0740 1 -- 172.21.15.103:0/2465310952 wait complete. 2024-09-19T07:41:11.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.681+0000 7f84c3fa0740 20 client.25637 trim_cache size 0 max 16384 2024-09-19T07:41:11.682 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84a40a3580 msgr2=0x7f84a40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84a40a3580 0x7f84a40a5a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f84b4004af0 tx=0x7f84b4009500 comp rx=0 tx=0).stop 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc101430 msgr2=0x7f84bc19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc101430 0x7f84bc19e530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d950 tx=0x7f84b000dd50 comp rx=0 tx=0).stop 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 shutdown_connections 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84a40a3580 0x7f84a40a5a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84bc106a90 0x7f84bc1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc101430 0x7f84bc19e530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 --2- 172.21.15.103:0/2055815549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84bc100a90 0x7f84bc19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:11.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.682+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 >> 172.21.15.103:0/2055815549 conn(0x7f84bc0fc7d0 msgr2=0x7f84bc10c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:11.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.683+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 shutdown_connections 2024-09-19T07:41:11.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:11.683+0000 7f84c1bc7640 1 -- 172.21.15.103:0/2055815549 wait complete. 2024-09-19T07:41:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:11 smithi134 ceph-mon[35414]: pgmap v99: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 2.32k op/s 2024-09-19T07:41:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:11 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:41:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:11 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:41:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:11 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:41:12.537 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:41:12.893 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:13.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.446+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/2330200673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf4101df0 msgr2=0x7f0bf41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.446+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/2330200673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf4101df0 0x7f0bf41021d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009860 tx=0x7f0be802a360 comp rx=0 tx=0).stop 2024-09-19T07:41:13.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.446+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/2330200673 shutdown_connections 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.446+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/2330200673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf4106f20 0x7f0bf410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.446+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/2330200673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf41027a0 0x7f0bf41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.446+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/2330200673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf4101df0 0x7f0bf41021d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.446+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/2330200673 >> 172.21.15.103:0/2330200673 conn(0x7f0bf40fd8b0 msgr2=0x7f0bf40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.447+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/2330200673 shutdown_connections 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.447+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/2330200673 wait complete. 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.448+0000 7f0bfb1e4640 1 Processor -- start 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.448+0000 7f0bfb1e4640 1 -- start start 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bfb1e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf4101df0 0x7f0bf419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bfb1e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41027a0 0x7f0bf419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bfb1e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf4106f20 0x7f0bf419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41027a0 0x7f0bf419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41027a0 0x7f0bf419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40740/0 (socket says 172.21.15.103:40740) 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bf9781640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf4106f20 0x7f0bf419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bf9781640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf4106f20 0x7f0bf419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60302/0 (socket says 172.21.15.103:60302) 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bf3fff640 1 -- 172.21.15.103:0/25793457 learned_addr learned my addr 172.21.15.103:0/25793457 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4111fa0 con 0x7f0bf4101df0 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4111e20 con 0x7f0bf4106f20 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.449+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4112120 con 0x7f0bf41027a0 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf3fff640 1 -- 172.21.15.103:0/25793457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf4106f20 msgr2=0x7f0bf419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf4106f20 0x7f0bf419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf3fff640 1 -- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf4101df0 msgr2=0x7f0bf419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf4101df0 0x7f0bf419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf3fff640 1 -- 172.21.15.103:0/25793457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf419f670 con 0x7f0bf41027a0 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41027a0 0x7f0bf419ac00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e9c0 tx=0x7f0be000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf1ffb640 1 -- 172.21.15.103:0/25793457 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be000ccc0 con 0x7f0bf41027a0 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf1ffb640 1 -- 172.21.15.103:0/25793457 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be0005500 con 0x7f0bf41027a0 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bf1ffb640 1 -- 172.21.15.103:0/25793457 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0010430 con 0x7f0bf41027a0 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.450+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf419f960 con 0x7f0bf41027a0 2024-09-19T07:41:13.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.451+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41089f0 con 0x7f0bf41027a0 2024-09-19T07:41:13.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.453+0000 7f0bf1ffb640 1 -- 172.21.15.103:0/25793457 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0be0005020 con 0x7f0bf41027a0 2024-09-19T07:41:13.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.454+0000 7f0bfd5bd740 1 Processor -- start 2024-09-19T07:41:13.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.454+0000 7f0bfd5bd740 1 -- start start 2024-09-19T07:41:13.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589ea4402f0 con 0x5589ea4dbb40 2024-09-19T07:41:13.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589ea3e7cf0 con 0x5589ea4e5370 2024-09-19T07:41:13.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589ea43c2b0 con 0x5589ea4da5d0 2024-09-19T07:41:13.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4da090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32994/0 (socket says 172.21.15.103:32994) 2024-09-19T07:41:13.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 -- 172.21.15.103:0/598104743 learned_addr learned my addr 172.21.15.103:0/598104743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:13.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 -- 172.21.15.103:0/598104743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 msgr2=0x5589ea4daa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4daa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 -- 172.21.15.103:0/598104743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 msgr2=0x5589ea4e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.455+0000 7f0bf9781640 1 -- 172.21.15.103:0/598104743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ea3cc3a0 con 0x5589ea4dbb40 2024-09-19T07:41:13.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.456+0000 7f0bf1ffb640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0be40abd90 0x7f0be40ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.456+0000 7f0bf1ffb640 1 -- 172.21.15.103:0/25793457 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0be00d0050 con 0x7f0bf41027a0 2024-09-19T07:41:13.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.457+0000 7f0bf9781640 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4da090 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00a430 tx=0x7f0bec009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bab1b46ee6d9255 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:13.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.457+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0be40abd90 0x7f0be40ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.457+0000 7f0bdeffd640 1 -- 172.21.15.103:0/598104743 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec013790 con 0x5589ea4dbb40 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.458+0000 7f0bdeffd640 1 -- 172.21.15.103:0/598104743 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec019070 con 0x5589ea4dbb40 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.458+0000 7f0bdeffd640 1 -- 172.21.15.103:0/598104743 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec014de0 con 0x5589ea4dbb40 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.458+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/598104743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 msgr2=0x5589ea4da090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.458+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4da090 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00a430 tx=0x7f0bec009650 comp rx=0 tx=0).stop 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.458+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0be40abd90 0x7f0be40ae250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0be802a950 tx=0x7f0be8005a90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/598104743 shutdown_connections 2024-09-19T07:41:13.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4daa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/598104743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4da090 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/598104743 >> 172.21.15.103:0/598104743 conn(0x5589ea4a3310 msgr2=0x5589ea4d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/598104743 shutdown_connections 2024-09-19T07:41:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/598104743 wait complete. 2024-09-19T07:41:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 -- start start 2024-09-19T07:41:13.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.459+0000 7f0bfd5bd740 1 Processor -- start 2024-09-19T07:41:13.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589ea4e80c0 con 0x5589ea4dbb40 2024-09-19T07:41:13.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589ea3fb060 con 0x5589ea4e5370 2024-09-19T07:41:13.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589ea4e8240 con 0x5589ea4da5d0 2024-09-19T07:41:13.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bfd5bd740 10 client.0 _ll_register_callbacks cb 0x5589ea14e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4e1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33000/0 (socket says 172.21.15.103:33000) 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bf3fff640 1 -- 172.21.15.103:0/468201605 learned_addr learned my addr 172.21.15.103:0/468201605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.460+0000 7f0bf9781640 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bf9781640 1 -- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 msgr2=0x5589ea4e4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bf9781640 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4e4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bf9781640 1 -- 172.21.15.103:0/468201605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 msgr2=0x5589ea4e1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bf9781640 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4e1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bf9781640 1 -- 172.21.15.103:0/468201605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ea1b13c0 con 0x5589ea4e5370 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bf9781640 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e25d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bec004b90 tx=0x7f0bec009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3eabe9b2172fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bd3fff640 1 -- 172.21.15.103:0/468201605 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0139a0 con 0x5589ea4e5370 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bd3fff640 1 -- 172.21.15.103:0/468201605 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0bec013b40 con 0x5589ea4e5370 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.461+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec019070 con 0x5589ea4e5370 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.462+0000 7f0bd3fff640 1 -- 172.21.15.103:0/468201605 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0bec014420 con 0x5589ea4e5370 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.462+0000 7f0bfd5bd740 10 client.25646 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.462+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589ea4f3b20 con 0x5589ea4e5370 2024-09-19T07:41:13.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bd3fff640 1 -- 172.21.15.103:0/468201605 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec019070 con 0x5589ea4e5370 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bd3fff640 1 -- 172.21.15.103:0/468201605 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f0bec028ba0 con 0x5589ea4e5370 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bfd5bd740 10 client.25646 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bfd5bd740 10 client.25646 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bfd5bd740 20 client.25646 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea4f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bfd5bd740 4 client.25646 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.463+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ea44d860 con 0x5589ea4f4f20 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.464+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea4f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.464+0000 7f0bd3fff640 10 client.25646 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.464+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea4f7310 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98c2c4e95312d03c server_cookie=b9e9dce769a71d4f in_seq=0 out_seq=0 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bd3fff640 1 -- 172.21.15.103:0/468201605 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589ea44d860 con 0x5589ea4f4f20 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bd3fff640 10 client.25646 handle_command_reply: tid=0 2024-09-19T07:41:13.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 client.25646 shutdown 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 msgr2=0x5589ea4e25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e25d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0bec004b90 tx=0x7f0bec009b70 comp rx=0 tx=0).stop 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 shutdown_connections 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea4f7310 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4e5370 0x5589ea4e25d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bd3fff640 0 client.25646 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/468201605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4da5d0 0x5589ea4e4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 >> 172.21.15.103:0/468201605 conn(0x5589ea4a3310 msgr2=0x5589ea4dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 shutdown_connections 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/468201605 wait complete. 2024-09-19T07:41:13.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.465+0000 7f0bfd5bd740 20 client.25646 trim_cache size 0 max 16384 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 Processor -- start 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 -- start start 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4dbb40 0x5589ea4df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4e5370 0x5589ea4dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41ab780 con 0x5589ea4e5370 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41ad410 con 0x5589ea4da5d0 2024-09-19T07:41:13.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.482+0000 7f0bfd5bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41abe50 con 0x5589ea4dbb40 2024-09-19T07:41:13.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf9781640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4e5370 0x5589ea4dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf9781640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4e5370 0x5589ea4dc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33004/0 (socket says 172.21.15.103:33004) 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4df240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60338/0 (socket says 172.21.15.103:60338) 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf9781640 1 -- 172.21.15.103:0/2189965546 learned_addr learned my addr 172.21.15.103:0/2189965546 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4dbb40 0x5589ea4df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 -- 172.21.15.103:0/2189965546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4dbb40 msgr2=0x5589ea4df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4dbb40 0x5589ea4df780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 -- 172.21.15.103:0/2189965546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4e5370 msgr2=0x5589ea4dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4e5370 0x5589ea4dc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 -- 172.21.15.103:0/2189965546 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ea4d6510 con 0x5589ea4da5d0 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4dbb40 0x5589ea4df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:41:13.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4df240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5589ea4d7ac0 tx=0x7f0be803b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1edc7476bb52f0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bde7fc640 1 -- 172.21.15.103:0/2189965546 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8037040 con 0x5589ea4da5d0 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bde7fc640 1 -- 172.21.15.103:0/2189965546 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be8005500 con 0x5589ea4da5d0 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.483+0000 7f0bde7fc640 1 -- 172.21.15.103:0/2189965546 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8004b70 con 0x5589ea4da5d0 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.484+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/2189965546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 msgr2=0x5589ea4df240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.484+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4df240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5589ea4d7ac0 tx=0x7f0be803b000 comp rx=0 tx=0).stop 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/2189965546 shutdown_connections 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4e5370 0x5589ea4dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4dbb40 0x5589ea4df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/2189965546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4df240 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/2189965546 >> 172.21.15.103:0/2189965546 conn(0x5589ea4a3310 msgr2=0x5589ea4e10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:13.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/2189965546 shutdown_connections 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/2189965546 wait complete. 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.485+0000 7f0bfd5bd740 1 -- start start 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.486+0000 7f0bfd5bd740 1 Processor -- start 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.486+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.486+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bfd5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4e5370 0x5589ea4fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bf8f80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bf8f80640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60344/0 (socket says 172.21.15.103:60344) 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bf8f80640 1 -- 172.21.15.103:0/1098462524 learned_addr learned my addr 172.21.15.103:0/1098462524 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41acc80 con 0x5589ea4dbb40 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41ac520 con 0x5589ea4da5d0 2024-09-19T07:41:13.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41a9780 con 0x5589ea4e5370 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bf9781640 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4e5370 0x5589ea4fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bfd5bd740 10 client.0 _ll_register_callbacks cb 0x7f0be00d3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.487+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.488+0000 7f0bf3fff640 1 -- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4e5370 msgr2=0x5589ea4fa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.488+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4e5370 0x5589ea4fa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.488+0000 7f0bf3fff640 1 -- 172.21.15.103:0/1098462524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 msgr2=0x5589ea4f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.488+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.488+0000 7f0bf3fff640 1 -- 172.21.15.103:0/1098462524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ea4fa780 con 0x5589ea4dbb40 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.488+0000 7f0bf3fff640 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4f8090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0be0098740 tx=0x7f0be0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e9e71648dfc2a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.488+0000 7f0bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.489+0000 7f0bd3fff640 1 -- 172.21.15.103:0/1098462524 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be009be20 con 0x5589ea4dbb40 2024-09-19T07:41:13.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.489+0000 7f0bd3fff640 1 -- 172.21.15.103:0/1098462524 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0be00d20b0 con 0x5589ea4dbb40 2024-09-19T07:41:13.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.489+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be8005500 con 0x5589ea4dbb40 2024-09-19T07:41:13.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.489+0000 7f0bd3fff640 1 -- 172.21.15.103:0/1098462524 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0be009bc40 con 0x5589ea4dbb40 2024-09-19T07:41:13.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.489+0000 7f0bfd5bd740 10 client.16137 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:13.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.489+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589ea55f3a0 con 0x5589ea4dbb40 2024-09-19T07:41:13.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.489+0000 7f0bd3fff640 1 -- 172.21.15.103:0/1098462524 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be8005500 con 0x5589ea4dbb40 2024-09-19T07:41:13.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bd3fff640 1 -- 172.21.15.103:0/1098462524 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f0be00a4ac0 con 0x5589ea4dbb40 2024-09-19T07:41:13.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bfd5bd740 10 client.16137 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:13.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bfd5bd740 10 client.16137 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:41:13.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bfd5bd740 20 client.16137 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:13.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea561b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bfd5bd740 4 client.16137 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589ea447d70 con 0x5589ea4f4f20 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.490+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea561b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.491+0000 7f0bf8f80640 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea561b40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aaf7a0e622ecf10b server_cookie=4fce7b823ae49209 in_seq=0 out_seq=0 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.491+0000 7f0bd3fff640 10 client.16137 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.491+0000 7f0bd3fff640 1 -- 172.21.15.103:0/1098462524 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589ea447d70 con 0x5589ea4f4f20 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.491+0000 7f0bd3fff640 10 client.16137 handle_command_reply: tid=0 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.491+0000 7f0bfd5bd740 1 client.16137 shutdown 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.491+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 msgr2=0x5589ea4f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.491+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4f8090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0be0098740 tx=0x7f0be0009f90 comp rx=0 tx=0).stop 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 shutdown_connections 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5589ea4f4f20 0x5589ea561b40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589ea4e5370 0x5589ea4fa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589ea4dbb40 0x5589ea4f8090 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bd3fff640 0 client.16137 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:41:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 --2- 172.21.15.103:0/1098462524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589ea4da5d0 0x5589ea4f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 >> 172.21.15.103:0/1098462524 conn(0x5589ea4a3310 msgr2=0x5589ea4d54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 shutdown_connections 2024-09-19T07:41:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 1 -- 172.21.15.103:0/1098462524 wait complete. 2024-09-19T07:41:13.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfd5bd740 20 client.16137 trim_cache size 0 max 16384 2024-09-19T07:41:13.495 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:41:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0be40abd90 msgr2=0x7f0be40ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.492+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0be40abd90 0x7f0be40ae250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0be802a950 tx=0x7f0be8005a90 comp rx=0 tx=0).stop 2024-09-19T07:41:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41027a0 msgr2=0x7f0bf419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41027a0 0x7f0bf419ac00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e9c0 tx=0x7f0be000ee80 comp rx=0 tx=0).stop 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 shutdown_connections 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0be40abd90 0x7f0be40ae250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bf4106f20 0x7f0bf419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41027a0 0x7f0bf419ac00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 --2- 172.21.15.103:0/25793457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bf4101df0 0x7f0bf419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 >> 172.21.15.103:0/25793457 conn(0x7f0bf40fd8b0 msgr2=0x7f0bf40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 shutdown_connections 2024-09-19T07:41:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:13.493+0000 7f0bfb1e4640 1 -- 172.21.15.103:0/25793457 wait complete. 2024-09-19T07:41:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:13 smithi174 ceph-mon[35370]: pgmap v100: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 1.86k op/s 2024-09-19T07:41:13.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:13 smithi103 ceph-mon[31201]: pgmap v100: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 1.86k op/s 2024-09-19T07:41:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:13 smithi134 ceph-mon[35414]: pgmap v100: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 1.86k op/s 2024-09-19T07:41:14.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:41:14.684 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:15.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/3611316773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd8101380 msgr2=0x7f7dd810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/3611316773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd8101380 0x7f7dd810dfb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009900 tx=0x7f7dc802a410 comp rx=0 tx=0).stop 2024-09-19T07:41:15.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/3611316773 shutdown_connections 2024-09-19T07:41:15.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/3611316773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dd810e4f0 0x7f7dd8110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/3611316773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd8101380 0x7f7dd810dfb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/3611316773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd81009d0 0x7f7dd8100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/3611316773 >> 172.21.15.103:0/3611316773 conn(0x7f7dd80fc7d0 msgr2=0x7f7dd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/3611316773 shutdown_connections 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.201+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/3611316773 wait complete. 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 Processor -- start 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 -- start start 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dd81009d0 0x7f7dd819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd8101380 0x7f7dd819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd810e4f0 0x7f7dd819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8114240 con 0x7f7dd81009d0 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81140c0 con 0x7f7dd810e4f0 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7ddcbe7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81143c0 con 0x7f7dd8101380 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7dd5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd8101380 0x7f7dd819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dd81009d0 0x7f7dd819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7dd5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd8101380 0x7f7dd819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40798/0 (socket says 172.21.15.103:40798) 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.202+0000 7f7dd5d9b640 1 -- 172.21.15.103:0/2646812599 learned_addr learned my addr 172.21.15.103:0/2646812599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:15.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.203+0000 7f7dd6d9d640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd810e4f0 0x7f7dd819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.203+0000 7f7dd659c640 1 -- 172.21.15.103:0/2646812599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd8101380 msgr2=0x7f7dd819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.203+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd8101380 0x7f7dd819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.203+0000 7f7dd659c640 1 -- 172.21.15.103:0/2646812599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd810e4f0 msgr2=0x7f7dd819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.203+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd810e4f0 0x7f7dd819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.203+0000 7f7dd659c640 1 -- 172.21.15.103:0/2646812599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd819f6f0 con 0x7f7dd81009d0 2024-09-19T07:41:15.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.204+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dd81009d0 0x7f7dd819a740 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00d6d0 tx=0x7f7dcc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:15.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.204+0000 7f7dc77fe640 1 -- 172.21.15.103:0/2646812599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc017070 con 0x7f7dd81009d0 2024-09-19T07:41:15.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.205+0000 7f7dc77fe640 1 -- 172.21.15.103:0/2646812599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dcc009820 con 0x7f7dd81009d0 2024-09-19T07:41:15.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.205+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd819f9e0 con 0x7f7dd81009d0 2024-09-19T07:41:15.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.205+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd81a72c0 con 0x7f7dd81009d0 2024-09-19T07:41:15.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.206+0000 7f7dc77fe640 1 -- 172.21.15.103:0/2646812599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc004d00 con 0x7f7dd81009d0 2024-09-19T07:41:15.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.208+0000 7f7dc77fe640 1 -- 172.21.15.103:0/2646812599 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7dcc005220 con 0x7f7dd81009d0 2024-09-19T07:41:15.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.208+0000 7f7dc77fe640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7dc00a3360 0x7f7dc00a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7dc00a3360 0x7f7dc00a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 Processor -- start 2024-09-19T07:41:15.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7dc77fe640 1 -- 172.21.15.103:0/2646812599 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7dcc0cf1e0 con 0x7f7dd81009d0 2024-09-19T07:41:15.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 -- start start 2024-09-19T07:41:15.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc205760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cbc15e2f0 con 0x562cbc203370 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562cbc105cf0 con 0x562cbc1f9b40 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.209+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562cbc15a2b0 con 0x562cbc1f85d0 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60360/0 (socket says 172.21.15.103:60360) 2024-09-19T07:41:15.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 -- 172.21.15.103:0/2683198791 learned_addr learned my addr 172.21.15.103:0/2683198791 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7dc00a3360 0x7f7dc00a5820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8031550 tx=0x7f7dc8005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 -- 172.21.15.103:0/2683198791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 msgr2=0x562cbc1f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd6d9d640 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc205760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 -- 172.21.15.103:0/2683198791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 msgr2=0x562cbc205760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc205760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.210+0000 7f7dd659c640 1 -- 172.21.15.103:0/2683198791 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbc0ea3a0 con 0x562cbc1f9b40 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.211+0000 7f7dd6d9d640 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc205760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.211+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.211+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1f8090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc099780 tx=0x7f7dcc09b100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31ca3b9ae12a928 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.211+0000 7f7dc4ff9640 1 -- 172.21.15.103:0/2683198791 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc020040 con 0x562cbc1f9b40 2024-09-19T07:41:15.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.211+0000 7f7dc4ff9640 1 -- 172.21.15.103:0/2683198791 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dcc09f880 con 0x562cbc1f9b40 2024-09-19T07:41:15.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.211+0000 7f7dc4ff9640 1 -- 172.21.15.103:0/2683198791 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc09ab80 con 0x562cbc1f9b40 2024-09-19T07:41:15.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.212+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2683198791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 msgr2=0x562cbc1f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.212+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1f8090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc099780 tx=0x7f7dcc09b100 comp rx=0 tx=0).stop 2024-09-19T07:41:15.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.213+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2683198791 shutdown_connections 2024-09-19T07:41:15.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.213+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc205760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.213+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.213+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2683198791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1f8090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.213+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2683198791 >> 172.21.15.103:0/2683198791 conn(0x562cbc1c1310 msgr2=0x562cbc1f4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:15.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.214+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2683198791 shutdown_connections 2024-09-19T07:41:15.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.214+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2683198791 wait complete. 2024-09-19T07:41:15.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.214+0000 7f7ddefc0740 1 -- start start 2024-09-19T07:41:15.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 1 Processor -- start 2024-09-19T07:41:15.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f85d0 0x562cbc202d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f9b40 0x562cbc200040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc203370 0x562cbc200650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cbc217080 con 0x562cbc1f85d0 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562cbc119060 con 0x562cbc203370 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562cbc217200 con 0x562cbc1f9b40 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7ddefc0740 10 client.0 _ll_register_callbacks cb 0x562cbbe6c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f85d0 0x562cbc202d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc203370 0x562cbc200650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f9b40 0x562cbc200040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f85d0 0x562cbc202d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33058/0 (socket says 172.21.15.103:33058) 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 -- 172.21.15.103:0/3645657075 learned_addr learned my addr 172.21.15.103:0/3645657075 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 -- 172.21.15.103:0/3645657075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f9b40 msgr2=0x562cbc200040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f9b40 0x562cbc200040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 -- 172.21.15.103:0/3645657075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc203370 msgr2=0x562cbc200650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc203370 0x562cbc200650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.215+0000 7f7dd659c640 1 -- 172.21.15.103:0/3645657075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbbecf3c0 con 0x562cbc1f85d0 2024-09-19T07:41:15.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.216+0000 7f7dd659c640 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f85d0 0x562cbc202d90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc097930 tx=0x7f7dcc0cc930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c80b7d13b46ccf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:15.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.216+0000 7f7dbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:41:15.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.216+0000 7f7dbdffb640 1 -- 172.21.15.103:0/3645657075 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0b8070 con 0x562cbc1f85d0 2024-09-19T07:41:15.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.216+0000 7f7dbdffb640 1 -- 172.21.15.103:0/3645657075 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7dcc09a920 con 0x562cbc1f85d0 2024-09-19T07:41:15.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.216+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dcc09f880 con 0x562cbc1f85d0 2024-09-19T07:41:15.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7dbdffb640 1 -- 172.21.15.103:0/3645657075 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7dcc020040 con 0x562cbc1f85d0 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 10 client.16143 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562cbc222ac0 con 0x562cbc1f85d0 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7dbdffb640 1 -- 172.21.15.103:0/3645657075 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dcc09f880 con 0x562cbc1f85d0 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7dbdffb640 1 -- 172.21.15.103:0/3645657075 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f7dcc023060 con 0x562cbc1f85d0 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 10 client.16143 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 10 client.16143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 20 client.16143 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc226200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 4 client.16143 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.217+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cbc16b860 con 0x562cbc223e10 2024-09-19T07:41:15.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.218+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc226200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.218+0000 7f7dbdffb640 10 client.16143 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:41:15.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.218+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc226200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b29aaaad460ee30c server_cookie=5a2b1feca1cee499 in_seq=0 out_seq=0 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7dbdffb640 1 -- 172.21.15.103:0/3645657075 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cbc16b860 con 0x562cbc223e10 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7dbdffb640 10 client.16143 handle_command_reply: tid=0 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 client.16143 shutdown 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f85d0 msgr2=0x562cbc202d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f85d0 0x562cbc202d90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc097930 tx=0x7f7dcc0cc930 comp rx=0 tx=0).stop 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 shutdown_connections 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc226200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc203370 0x562cbc200650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7dbdffb640 0 client.16143 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f9b40 0x562cbc200040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3645657075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f85d0 0x562cbc202d90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.220+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 >> 172.21.15.103:0/3645657075 conn(0x562cbc1c1310 msgr2=0x562cbc1fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:15.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.221+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 shutdown_connections 2024-09-19T07:41:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.221+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3645657075 wait complete. 2024-09-19T07:41:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.221+0000 7f7ddefc0740 20 client.16143 trim_cache size 0 max 16384 2024-09-19T07:41:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.241+0000 7f7ddefc0740 1 Processor -- start 2024-09-19T07:41:15.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.241+0000 7f7ddefc0740 1 -- start start 2024-09-19T07:41:15.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.242+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.242+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.242+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc1fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.242+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8074000 con 0x562cbc203370 2024-09-19T07:41:15.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.242+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81a8d60 con 0x562cbc1f9b40 2024-09-19T07:41:15.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.242+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81a77a0 con 0x562cbc1f85d0 2024-09-19T07:41:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.245+0000 7f7dd5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.245+0000 7f7dd5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60388/0 (socket says 172.21.15.103:60388) 2024-09-19T07:41:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.245+0000 7f7dd5d9b640 1 -- 172.21.15.103:0/3223719667 learned_addr learned my addr 172.21.15.103:0/3223719667 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.245+0000 7f7dd659c640 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.247+0000 7f7dd5d9b640 1 -- 172.21.15.103:0/3223719667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 msgr2=0x562cbc1fb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.247+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1fb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.247+0000 7f7dd5d9b640 1 -- 172.21.15.103:0/3223719667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 msgr2=0x562cbc1fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.247+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc1fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.247+0000 7f7dd5d9b640 1 -- 172.21.15.103:0/3223719667 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbc203100 con 0x562cbc1f9b40 2024-09-19T07:41:15.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.247+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1fb660 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8000c00 tx=0x7f7dc8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307c9af2474f96a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.249+0000 7f7dbffff640 1 -- 172.21.15.103:0/3223719667 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8035040 con 0x562cbc1f9b40 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.249+0000 7f7dbffff640 1 -- 172.21.15.103:0/3223719667 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc8058070 con 0x562cbc1f9b40 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.249+0000 7f7dbffff640 1 -- 172.21.15.103:0/3223719667 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc80049b0 con 0x562cbc1f9b40 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.250+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3223719667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 msgr2=0x562cbc1fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.250+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1fb660 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8000c00 tx=0x7f7dc8040000 comp rx=0 tx=0).stop 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.250+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3223719667 shutdown_connections 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.250+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc203370 0x562cbc1fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.250+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f9b40 0x562cbc1fb660 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.250+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/3223719667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc1f85d0 0x562cbc1fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.250+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3223719667 >> 172.21.15.103:0/3223719667 conn(0x562cbc1c1310 msgr2=0x562cbc1f3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:15.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.254+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3223719667 shutdown_connections 2024-09-19T07:41:15.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.255+0000 7f7ddefc0740 1 -- 172.21.15.103:0/3223719667 wait complete. 2024-09-19T07:41:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.255+0000 7f7ddefc0740 1 -- start start 2024-09-19T07:41:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 1 Processor -- start 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f85d0 0x562cbc289730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f9b40 0x562cbc289c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc203370 0x562cbc28a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81a85d0 con 0x562cbc1f9b40 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd81a7e70 con 0x562cbc1f85d0 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8071fe0 con 0x562cbc203370 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.256+0000 7f7ddefc0740 10 client.0 _ll_register_callbacks cb 0x7f7dcc097860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f85d0 0x562cbc289730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f85d0 0x562cbc289730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60390/0 (socket says 172.21.15.103:60390) 2024-09-19T07:41:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 -- 172.21.15.103:0/2397382656 learned_addr learned my addr 172.21.15.103:0/2397382656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:15.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd6d9d640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc203370 0x562cbc28a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f9b40 0x562cbc289c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 -- 172.21.15.103:0/2397382656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc203370 msgr2=0x562cbc28a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc203370 0x562cbc28a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 -- 172.21.15.103:0/2397382656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f9b40 msgr2=0x562cbc289c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f9b40 0x562cbc289c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd659c640 1 -- 172.21.15.103:0/2397382656 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbc227270 con 0x562cbc1f85d0 2024-09-19T07:41:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.257+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f9b40 0x562cbc289c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.259+0000 7f7dd659c640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f85d0 0x562cbc289730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc09a650 tx=0x7f7dcc0a0ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3506f59b63b24b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.260+0000 7f7dbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.260+0000 7f7dbdffb640 1 -- 172.21.15.103:0/2397382656 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0b8040 con 0x562cbc1f85d0 2024-09-19T07:41:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.261+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dc8058070 con 0x562cbc1f85d0 2024-09-19T07:41:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.261+0000 7f7dbdffb640 1 -- 172.21.15.103:0/2397382656 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f7dcc097220 con 0x562cbc1f85d0 2024-09-19T07:41:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.262+0000 7f7dbdffb640 1 -- 172.21.15.103:0/2397382656 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7dcc09bbd0 con 0x562cbc1f85d0 2024-09-19T07:41:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.263+0000 7f7ddefc0740 10 client.25670 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.263+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562cbc227270 con 0x562cbc1f85d0 2024-09-19T07:41:15.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.264+0000 7f7dbdffb640 1 -- 172.21.15.103:0/2397382656 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc8058070 con 0x562cbc1f85d0 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.264+0000 7f7dbdffb640 1 -- 172.21.15.103:0/2397382656 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f7dcc097420 con 0x562cbc1f85d0 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.265+0000 7f7ddefc0740 10 client.25670 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.265+0000 7f7ddefc0740 10 client.25670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.265+0000 7f7ddefc0740 20 client.25670 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.265+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc2996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.265+0000 7f7ddefc0740 4 client.25670 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:41:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.265+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cbc165d70 con 0x562cbc223e10 2024-09-19T07:41:15.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.266+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc2996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:15.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.268+0000 7f7dd5d9b640 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc2996d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a6e7a390c7cd904 server_cookie=5ffd60f267db86a8 in_seq=0 out_seq=0 2024-09-19T07:41:15.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.269+0000 7f7dbdffb640 10 client.25670 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:41:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.269+0000 7f7dbdffb640 1 -- 172.21.15.103:0/2397382656 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cbc165d70 con 0x562cbc223e10 2024-09-19T07:41:15.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.269+0000 7f7dbdffb640 10 client.25670 handle_command_reply: tid=0 2024-09-19T07:41:15.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.269+0000 7f7ddefc0740 1 client.25670 shutdown 2024-09-19T07:41:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f85d0 msgr2=0x562cbc289730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f85d0 0x562cbc289730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc09a650 tx=0x7f7dcc0a0ed0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 shutdown_connections 2024-09-19T07:41:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x562cbc223e10 0x562cbc2996d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cbc203370 0x562cbc28a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cbc1f9b40 0x562cbc289c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 --2- 172.21.15.103:0/2397382656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cbc1f85d0 0x562cbc289730 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 >> 172.21.15.103:0/2397382656 conn(0x562cbc1c1310 msgr2=0x562cbc2018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:15.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.271+0000 7f7dbdffb640 0 client.25670 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:41:15.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.273+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 shutdown_connections 2024-09-19T07:41:15.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.273+0000 7f7ddefc0740 1 -- 172.21.15.103:0/2397382656 wait complete. 2024-09-19T07:41:15.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.274+0000 7f7ddefc0740 20 client.25670 trim_cache size 0 max 16384 2024-09-19T07:41:15.275 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:41:15.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.276+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7dc00a3360 msgr2=0x7f7dc00a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.276+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7dc00a3360 0x7f7dc00a5820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8031550 tx=0x7f7dc8005a20 comp rx=0 tx=0).stop 2024-09-19T07:41:15.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.276+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dd81009d0 msgr2=0x7f7dd819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:15.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.276+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dd81009d0 0x7f7dd819a740 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00d6d0 tx=0x7f7dcc00db90 comp rx=0 tx=0).stop 2024-09-19T07:41:15.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.277+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 shutdown_connections 2024-09-19T07:41:15.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.277+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f7dc00a3360 0x7f7dc00a5820 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.277+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dd810e4f0 0x7f7dd819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.277+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd8101380 0x7f7dd819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.277+0000 7f7ddcbe7640 1 --2- 172.21.15.103:0/2646812599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dd81009d0 0x7f7dd819a740 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:15.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.277+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 >> 172.21.15.103:0/2646812599 conn(0x7f7dd80fc7d0 msgr2=0x7f7dd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:15.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.277+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 shutdown_connections 2024-09-19T07:41:15.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:15.278+0000 7f7ddcbe7640 1 -- 172.21.15.103:0/2646812599 wait complete. 2024-09-19T07:41:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 160.0 s, 3320.8 tps, lat 30.054 ms stddev 31.466 2024-09-19T07:41:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:15 smithi174 ceph-mon[35370]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 2.17k op/s 2024-09-19T07:41:15.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:15 smithi103 ceph-mon[31201]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 2.17k op/s 2024-09-19T07:41:15.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:15 smithi134 ceph-mon[35414]: pgmap v101: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 2.17k op/s 2024-09-19T07:41:17.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:41:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:17 smithi103 ceph-mon[31201]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T07:41:17.551 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:17.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:17 smithi174 ceph-mon[35370]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T07:41:17.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:17 smithi134 ceph-mon[35414]: pgmap v102: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T07:41:18.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 -- 172.21.15.103:0/1891934662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 msgr2=0x7ff16810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 --2- 172.21.15.103:0/1891934662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16810da40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff160009930 tx=0x7ff16002a4d0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 -- 172.21.15.103:0/1891934662 shutdown_connections 2024-09-19T07:41:18.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 --2- 172.21.15.103:0/1891934662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16810da40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 --2- 172.21.15.103:0/1891934662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff168102570 0x7ff1681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 --2- 172.21.15.103:0/1891934662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff168100e20 0x7ff168101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 -- 172.21.15.103:0/1891934662 >> 172.21.15.103:0/1891934662 conn(0x7ff1680fc7d0 msgr2=0x7ff1680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:18.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 -- 172.21.15.103:0/1891934662 shutdown_connections 2024-09-19T07:41:18.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.351+0000 7ff16d069640 1 -- 172.21.15.103:0/1891934662 wait complete. 2024-09-19T07:41:18.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.352+0000 7ff16d069640 1 Processor -- start 2024-09-19T07:41:18.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.352+0000 7ff16d069640 1 -- start start 2024-09-19T07:41:18.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16d069640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff168100e20 0x7ff1681a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16d069640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff168102570 0x7ff1681a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16d069640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16d069640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810fe80 con 0x7ff168100e20 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16d069640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810fd00 con 0x7ff168102570 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16d069640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff168110000 con 0x7ff168102ef0 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff166d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff168100e20 0x7ff1681a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16759e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff166d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff168100e20 0x7ff1681a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51816/0 (socket says 172.21.15.103:51816) 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff16759e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52126/0 (socket says 172.21.15.103:52126) 2024-09-19T07:41:18.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.353+0000 7ff166d9d640 1 -- 172.21.15.103:0/2880313825 learned_addr learned my addr 172.21.15.103:0/2880313825 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:18.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.354+0000 7ff16659c640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff168102570 0x7ff1681a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.354+0000 7ff16759e640 1 -- 172.21.15.103:0/2880313825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff168102570 msgr2=0x7ff1681a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.354+0000 7ff16759e640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff168102570 0x7ff1681a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.354+0000 7ff16759e640 1 -- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff168100e20 msgr2=0x7ff1681a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.354+0000 7ff16759e640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff168100e20 0x7ff1681a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.354+0000 7ff16759e640 1 -- 172.21.15.103:0/2880313825 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16819f030 con 0x7ff168102ef0 2024-09-19T07:41:18.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.355+0000 7ff16759e640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16819e900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff160009900 tx=0x7ff160004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:18.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.355+0000 7ff14ffff640 1 -- 172.21.15.103:0/2880313825 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16003f030 con 0x7ff168102ef0 2024-09-19T07:41:18.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.355+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16819f2c0 con 0x7ff168102ef0 2024-09-19T07:41:18.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.355+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681ab610 con 0x7ff168102ef0 2024-09-19T07:41:18.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.355+0000 7ff14ffff640 1 -- 172.21.15.103:0/2880313825 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff160004440 con 0x7ff168102ef0 2024-09-19T07:41:18.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.355+0000 7ff14ffff640 1 -- 172.21.15.103:0/2880313825 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160035070 con 0x7ff168102ef0 2024-09-19T07:41:18.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff14ffff640 1 -- 172.21.15.103:0/2880313825 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff160042030 con 0x7ff168102ef0 2024-09-19T07:41:18.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 Processor -- start 2024-09-19T07:41:18.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 -- start start 2024-09-19T07:41:18.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d4eb34d2f0 con 0x55d4eb3e8b40 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4eb2f4cf0 con 0x55d4eb3e75d0 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.359+0000 7ff16f442740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4eb3492b0 con 0x55d4eb3f2370 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff166d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff16659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff16659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52140/0 (socket says 172.21.15.103:52140) 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff16659c640 1 -- 172.21.15.103:0/3330693016 learned_addr learned my addr 172.21.15.103:0/3330693016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff16659c640 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff166d9d640 1 -- 172.21.15.103:0/3330693016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3f2370 msgr2=0x55d4eb3f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff166d9d640 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff166d9d640 1 -- 172.21.15.103:0/3330693016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 msgr2=0x55d4eb3e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff166d9d640 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3e7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff166d9d640 1 -- 172.21.15.103:0/3330693016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4eb2d93a0 con 0x55d4eb3e75d0 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff14ffff640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1500a3500 0x7ff1500a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.360+0000 7ff14ffff640 1 -- 172.21.15.103:0/2880313825 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff1600efad0 con 0x7ff168102ef0 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.361+0000 7ff16659c640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1500a3500 0x7ff1500a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.361+0000 7ff166d9d640 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3e7a10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff158005fd0 tx=0x7ff15800b500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec241c96bd3e132 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.362+0000 7ff14d7fa640 1 -- 172.21.15.103:0/3330693016 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158004230 con 0x55d4eb3e75d0 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.362+0000 7ff14d7fa640 1 -- 172.21.15.103:0/3330693016 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1580043d0 con 0x55d4eb3e75d0 2024-09-19T07:41:18.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.362+0000 7ff14d7fa640 1 -- 172.21.15.103:0/3330693016 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158004230 con 0x55d4eb3e75d0 2024-09-19T07:41:18.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.363+0000 7ff16659c640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1500a3500 0x7ff1500a59c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55d4eb3e8070 tx=0x7ff15c009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:18.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.363+0000 7ff16f442740 1 -- 172.21.15.103:0/3330693016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 msgr2=0x55d4eb3e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.363+0000 7ff16f442740 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3e7a10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff158005fd0 tx=0x7ff15800b500 comp rx=0 tx=0).stop 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 -- 172.21.15.103:0/3330693016 shutdown_connections 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3e7a10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 --2- 172.21.15.103:0/3330693016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 -- 172.21.15.103:0/3330693016 >> 172.21.15.103:0/3330693016 conn(0x55d4eb3b0310 msgr2=0x55d4eb3e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 -- 172.21.15.103:0/3330693016 shutdown_connections 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 -- 172.21.15.103:0/3330693016 wait complete. 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.364+0000 7ff16f442740 1 -- start start 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 1 Processor -- start 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3f2370 0x55d4eb3ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d4eb3f5070 con 0x55d4eb3e8b40 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4eb308060 con 0x55d4eb3f2370 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4eb3f51f0 con 0x55d4eb3e75d0 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16f442740 10 client.0 _ll_register_callbacks cb 0x55d4eb05b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51834/0 (socket says 172.21.15.103:51834) 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16659c640 1 -- 172.21.15.103:0/438138695 learned_addr learned my addr 172.21.15.103:0/438138695 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.365+0000 7ff16759e640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3f2370 0x55d4eb3ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff166d9d640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff16759e640 1 -- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 msgr2=0x55d4eb3f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff16759e640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff16759e640 1 -- 172.21.15.103:0/438138695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 msgr2=0x55d4eb3eee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff16759e640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff16759e640 1 -- 172.21.15.103:0/438138695 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4eb0be3c0 con 0x55d4eb3f2370 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff166d9d640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff16759e640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3f2370 0x55d4eb3ef340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff1600ed230 tx=0x7ff1600bb900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a5a01868ccd973e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff14aff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff14aff5640 1 -- 172.21.15.103:0/438138695 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160049040 con 0x55d4eb3f2370 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.366+0000 7ff14aff5640 1 -- 172.21.15.103:0/438138695 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff160031bd0 con 0x55d4eb3f2370 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.367+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1580043d0 con 0x55d4eb3f2370 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.367+0000 7ff14aff5640 1 -- 172.21.15.103:0/438138695 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1600d9050 con 0x55d4eb3f2370 2024-09-19T07:41:18.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.367+0000 7ff16f442740 10 client.25679 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.367+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4eb4009a0 con 0x55d4eb3f2370 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.367+0000 7ff14aff5640 1 -- 172.21.15.103:0/438138695 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1580043d0 con 0x55d4eb3f2370 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.368+0000 7ff14aff5640 1 -- 172.21.15.103:0/438138695 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7ff16004c060 con 0x55d4eb3f2370 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.368+0000 7ff16f442740 10 client.25679 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.368+0000 7ff16f442740 10 client.25679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.368+0000 7ff16f442740 20 client.25679 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.368+0000 7ff16f442740 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb401f50 0x55d4eb4043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.368+0000 7ff16f442740 4 client.25679 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.368+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4eb35a860 con 0x55d4eb401f50 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.369+0000 7ff166d9d640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb401f50 0x55d4eb4043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff166d9d640 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb401f50 0x55d4eb4043d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e71fe4b22142f0aa server_cookie=7c3809f8154ed2ff in_seq=0 out_seq=0 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff14aff5640 10 client.25679 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff14aff5640 1 -- 172.21.15.103:0/438138695 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4eb35a860 con 0x55d4eb401f50 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff14aff5640 10 client.25679 handle_command_reply: tid=0 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 client.25679 shutdown 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3f2370 msgr2=0x55d4eb3ef340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3f2370 0x55d4eb3ef340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff1600ed230 tx=0x7ff1600bb900 comp rx=0 tx=0).stop 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 shutdown_connections 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb401f50 0x55d4eb4043d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3f2370 0x55d4eb3ef340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 --2- 172.21.15.103:0/438138695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 >> 172.21.15.103:0/438138695 conn(0x55d4eb3b0310 msgr2=0x55d4eb3e9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.370+0000 7ff14aff5640 0 client.25679 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.371+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 shutdown_connections 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.371+0000 7ff16f442740 1 -- 172.21.15.103:0/438138695 wait complete. 2024-09-19T07:41:18.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.371+0000 7ff16f442740 20 client.25679 trim_cache size 0 max 16384 2024-09-19T07:41:18.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.387+0000 7ff16f442740 1 Processor -- start 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16f442740 1 -- start start 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16f442740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16f442740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3ef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16f442740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16f442740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff15400e7c0 con 0x55d4eb3f2370 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16f442740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff154010450 con 0x55d4eb3e8b40 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16f442740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff15400ee90 con 0x55d4eb3e75d0 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52154/0 (socket says 172.21.15.103:52154) 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 -- 172.21.15.103:0/2674762129 learned_addr learned my addr 172.21.15.103:0/2674762129 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16759e640 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff16659c640 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 -- 172.21.15.103:0/2674762129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e8b40 msgr2=0x55d4eb3ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 -- 172.21.15.103:0/2674762129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 msgr2=0x55d4eb3f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.388+0000 7ff166d9d640 1 -- 172.21.15.103:0/2674762129 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4eb3e34f0 con 0x55d4eb3e75d0 2024-09-19T07:41:18.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.391+0000 7ff166d9d640 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eedc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff15800b600 tx=0x7ff15800b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da974ce79c1a63c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.394+0000 7ff14cff9640 1 -- 172.21.15.103:0/2674762129 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15802f070 con 0x55d4eb3e75d0 2024-09-19T07:41:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.394+0000 7ff14cff9640 1 -- 172.21.15.103:0/2674762129 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff158005510 con 0x55d4eb3e75d0 2024-09-19T07:41:18.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.394+0000 7ff14cff9640 1 -- 172.21.15.103:0/2674762129 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158004230 con 0x55d4eb3e75d0 2024-09-19T07:41:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.395+0000 7ff16f442740 1 -- 172.21.15.103:0/2674762129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 msgr2=0x55d4eb3eedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.395+0000 7ff16f442740 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eedc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff15800b600 tx=0x7ff15800b590 comp rx=0 tx=0).stop 2024-09-19T07:41:18.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.396+0000 7ff16f442740 1 -- 172.21.15.103:0/2674762129 shutdown_connections 2024-09-19T07:41:18.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.396+0000 7ff16f442740 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.396+0000 7ff16f442740 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.396+0000 7ff16f442740 1 --2- 172.21.15.103:0/2674762129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eedc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.396+0000 7ff16f442740 1 -- 172.21.15.103:0/2674762129 >> 172.21.15.103:0/2674762129 conn(0x55d4eb3b0310 msgr2=0x55d4eb3f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:18.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.396+0000 7ff16f442740 1 -- 172.21.15.103:0/2674762129 shutdown_connections 2024-09-19T07:41:18.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.400+0000 7ff16f442740 1 -- 172.21.15.103:0/2674762129 wait complete. 2024-09-19T07:41:18.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.400+0000 7ff16f442740 1 -- start start 2024-09-19T07:41:18.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.400+0000 7ff16f442740 1 Processor -- start 2024-09-19T07:41:18.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.401+0000 7ff16f442740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.401+0000 7ff16f442740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.401+0000 7ff16f442740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.401+0000 7ff166d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.401+0000 7ff166d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56916/0 (socket says 172.21.15.103:56916) 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff16659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff16659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52166/0 (socket says 172.21.15.103:52166) 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff166d9d640 1 -- 172.21.15.103:0/2075455948 learned_addr learned my addr 172.21.15.103:0/2075455948 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff15400fcc0 con 0x55d4eb3f2370 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff15400f560 con 0x55d4eb3e75d0 2024-09-19T07:41:18.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff15400c7a0 con 0x55d4eb3e8b40 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff16759e640 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff16f442740 10 client.0 _ll_register_callbacks cb 0x7ff1600b7960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff166d9d640 1 -- 172.21.15.103:0/2075455948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e8b40 msgr2=0x55d4eb3eb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff166d9d640 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.402+0000 7ff166d9d640 1 -- 172.21.15.103:0/2075455948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 msgr2=0x55d4eb3f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.403+0000 7ff166d9d640 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.403+0000 7ff166d9d640 1 -- 172.21.15.103:0/2075455948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4eb3f9a50 con 0x55d4eb3e75d0 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.403+0000 7ff166d9d640 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eb430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff15802d040 tx=0x7ff15800ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720b8b2cdf9040f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.404+0000 7ff14aff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.405+0000 7ff14aff5640 1 -- 172.21.15.103:0/2075455948 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158037020 con 0x55d4eb3e75d0 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.405+0000 7ff14aff5640 1 -- 172.21.15.103:0/2075455948 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff158003670 con 0x55d4eb3e75d0 2024-09-19T07:41:18.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.405+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d4eb354d70 con 0x55d4eb3e75d0 2024-09-19T07:41:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.407+0000 7ff14aff5640 1 -- 172.21.15.103:0/2075455948 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff15802f040 con 0x55d4eb3e75d0 2024-09-19T07:41:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.407+0000 7ff16f442740 10 client.25685 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:18.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.407+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4eb3fad10 con 0x55d4eb3e75d0 2024-09-19T07:41:18.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff14aff5640 1 -- 172.21.15.103:0/2075455948 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff158003cd0 con 0x55d4eb3e75d0 2024-09-19T07:41:18.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff14aff5640 1 -- 172.21.15.103:0/2075455948 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7ff158032070 con 0x55d4eb3e75d0 2024-09-19T07:41:18.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff16f442740 10 client.25685 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:18.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff16f442740 10 client.25685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:41:18.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff16f442740 20 client.25685 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:18.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff16f442740 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb46cc70 0x55d4eb46f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff16f442740 4 client.25685 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:41:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.408+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4eb354d70 con 0x55d4eb46cc70 2024-09-19T07:41:18.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.409+0000 7ff16659c640 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb46cc70 0x55d4eb46f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:18.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.412+0000 7ff16659c640 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb46cc70 0x55d4eb46f060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0fb0d5fa06c7ed5 server_cookie=fad083463376d94a in_seq=0 out_seq=0 2024-09-19T07:41:18.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.412+0000 7ff14aff5640 10 client.25685 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:41:18.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff14aff5640 1 -- 172.21.15.103:0/2075455948 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4eb354d70 con 0x55d4eb46cc70 2024-09-19T07:41:18.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff14aff5640 10 client.25685 handle_command_reply: tid=0 2024-09-19T07:41:18.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 client.25685 shutdown 2024-09-19T07:41:18.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 msgr2=0x55d4eb3eb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eb430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff15802d040 tx=0x7ff15800ab40 comp rx=0 tx=0).stop 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 shutdown_connections 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55d4eb46cc70 0x55d4eb46f060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d4eb3f2370 0x55d4eb3f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4eb3e8b40 0x55d4eb3eb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff14aff5640 0 client.25685 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 --2- 172.21.15.103:0/2075455948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4eb3e75d0 0x55d4eb3eb430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.413+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 >> 172.21.15.103:0/2075455948 conn(0x55d4eb3b0310 msgr2=0x55d4eb3f0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.414+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 shutdown_connections 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.414+0000 7ff16f442740 1 -- 172.21.15.103:0/2075455948 wait complete. 2024-09-19T07:41:18.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.414+0000 7ff16f442740 20 client.25685 trim_cache size 0 max 16384 2024-09-19T07:41:18.417 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1500a3500 msgr2=0x7ff1500a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1500a3500 0x7ff1500a59c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55d4eb3e8070 tx=0x7ff15c009500 comp rx=0 tx=0).stop 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 msgr2=0x7ff16819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16819e900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff160009900 tx=0x7ff160004040 comp rx=0 tx=0).stop 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 shutdown_connections 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1500a3500 0x7ff1500a59c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102ef0 0x7ff16819e900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff168102570 0x7ff1681a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 --2- 172.21.15.103:0/2880313825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff168100e20 0x7ff1681a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.415+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 >> 172.21.15.103:0/2880313825 conn(0x7ff1680fc7d0 msgr2=0x7ff168104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:18.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.416+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 shutdown_connections 2024-09-19T07:41:18.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:18.416+0000 7ff16d069640 1 -- 172.21.15.103:0/2880313825 wait complete. 2024-09-19T07:41:19.275 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:41:19.604 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:19 smithi103 ceph-mon[31201]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T07:41:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:19 smithi134 ceph-mon[35414]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T07:41:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:19 smithi174 ceph-mon[35370]: pgmap v103: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 737 op/s 2024-09-19T07:41:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 -- 172.21.15.103:0/1549550813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c100a80 msgr2=0x7fac0c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 --2- 172.21.15.103:0/1549550813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c100a80 0x7fac0c100ee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac00009900 tx=0x7fac0002a400 comp rx=0 tx=0).stop 2024-09-19T07:41:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 -- 172.21.15.103:0/1549550813 shutdown_connections 2024-09-19T07:41:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 --2- 172.21.15.103:0/1549550813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c101420 0x7fac0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 --2- 172.21.15.103:0/1549550813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c100a80 0x7fac0c100ee0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 --2- 172.21.15.103:0/1549550813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac0c106a80 0x7fac0c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 -- 172.21.15.103:0/1549550813 >> 172.21.15.103:0/1549550813 conn(0x7fac0c0fc7a0 msgr2=0x7fac0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 -- 172.21.15.103:0/1549550813 shutdown_connections 2024-09-19T07:41:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 -- 172.21.15.103:0/1549550813 wait complete. 2024-09-19T07:41:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 Processor -- start 2024-09-19T07:41:20.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.136+0000 7fac11778640 1 -- start start 2024-09-19T07:41:20.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac11778640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c100a80 0x7fac0c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac11778640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac0c101420 0x7fac0c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac11778640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c106a80 0x7fac0c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac11778640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c10fdd0 con 0x7fac0c100a80 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac11778640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c10fc50 con 0x7fac0c106a80 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac11778640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c10ff50 con 0x7fac0c101420 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c100a80 0x7fac0c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c106a80 0x7fac0c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c100a80 0x7fac0c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51858/0 (socket says 172.21.15.103:51858) 2024-09-19T07:41:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c106a80 0x7fac0c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56946/0 (socket says 172.21.15.103:56946) 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 -- 172.21.15.103:0/3203125397 learned_addr learned my addr 172.21.15.103:0/3203125397 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0a7fc640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac0c101420 0x7fac0c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 -- 172.21.15.103:0/3203125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac0c101420 msgr2=0x7fac0c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac0c101420 0x7fac0c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 -- 172.21.15.103:0/3203125397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c106a80 msgr2=0x7fac0c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c106a80 0x7fac0c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 -- 172.21.15.103:0/3203125397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0c19f5f0 con 0x7fac0c100a80 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.137+0000 7fac0affd640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c100a80 0x7fac0c19a640 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fac0400bfd0 tx=0x7fac0400ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.138+0000 7fabf3fff640 1 -- 172.21.15.103:0/3203125397 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04019070 con 0x7fac0c100a80 2024-09-19T07:41:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.138+0000 7fabf3fff640 1 -- 172.21.15.103:0/3203125397 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac04014410 con 0x7fac0c100a80 2024-09-19T07:41:20.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.138+0000 7fabf3fff640 1 -- 172.21.15.103:0/3203125397 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04013610 con 0x7fac0c100a80 2024-09-19T07:41:20.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.138+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0c19f8e0 con 0x7fac0c100a80 2024-09-19T07:41:20.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.138+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac0c1a71c0 con 0x7fac0c100a80 2024-09-19T07:41:20.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.141+0000 7fabf3fff640 1 -- 172.21.15.103:0/3203125397 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fac0401c070 con 0x7fac0c100a80 2024-09-19T07:41:20.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.142+0000 7fabf3fff640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fabf40a34b0 0x7fabf40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.142+0000 7fabf3fff640 1 -- 172.21.15.103:0/3203125397 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fac040cff50 con 0x7fac0c100a80 2024-09-19T07:41:20.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.142+0000 7fac0a7fc640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fabf40a34b0 0x7fabf40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac0a7fc640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fabf40a34b0 0x7fabf40a5970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac00005b50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 Processor -- start 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 -- start start 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb7b40 0x563b5abb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb65d0 0x563b5abb6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b5ab1c2f0 con 0x563b5abb65d0 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b5aac3cf0 con 0x563b5abb7b40 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac13b51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b5ab182b0 con 0x563b5abc1370 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac0a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb65d0 0x563b5abb6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.143+0000 7fac0affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb7b40 0x563b5abb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb7b40 0x563b5abb6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56956/0 (socket says 172.21.15.103:56956) 2024-09-19T07:41:20.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0affd640 1 -- 172.21.15.103:0/1560233822 learned_addr learned my addr 172.21.15.103:0/1560233822 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:20.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0a7fc640 1 -- 172.21.15.103:0/1560233822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 msgr2=0x563b5abc3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0a7fc640 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abc3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0a7fc640 1 -- 172.21.15.103:0/1560233822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb7b40 msgr2=0x563b5abb6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0a7fc640 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb7b40 0x563b5abb6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0a7fc640 1 -- 172.21.15.103:0/1560233822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5aaa83a0 con 0x563b5abb65d0 2024-09-19T07:41:20.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.144+0000 7fac0a7fc640 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb65d0 0x563b5abb6a10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fac0002a9f0 tx=0x7fac0003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21840eb2c3b23cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:20.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.145+0000 7fabf17fa640 1 -- 172.21.15.103:0/1560233822 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00035040 con 0x563b5abb65d0 2024-09-19T07:41:20.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.145+0000 7fabf17fa640 1 -- 172.21.15.103:0/1560233822 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0004e070 con 0x563b5abb65d0 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.145+0000 7fabf17fa640 1 -- 172.21.15.103:0/1560233822 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00003e50 con 0x563b5abb65d0 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.146+0000 7fac13b51740 1 -- 172.21.15.103:0/1560233822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb65d0 msgr2=0x563b5abb6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.146+0000 7fac13b51740 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb65d0 0x563b5abb6a10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fac0002a9f0 tx=0x7fac0003d000 comp rx=0 tx=0).stop 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.147+0000 7fac13b51740 1 -- 172.21.15.103:0/1560233822 shutdown_connections 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.147+0000 7fac13b51740 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.147+0000 7fac13b51740 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb65d0 0x563b5abb6a10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.147+0000 7fac13b51740 1 --2- 172.21.15.103:0/1560233822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb7b40 0x563b5abb6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.147+0000 7fac13b51740 1 -- 172.21.15.103:0/1560233822 >> 172.21.15.103:0/1560233822 conn(0x563b5ab7f310 msgr2=0x563b5abb2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:20.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.148+0000 7fac13b51740 1 -- 172.21.15.103:0/1560233822 shutdown_connections 2024-09-19T07:41:20.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.148+0000 7fac13b51740 1 -- 172.21.15.103:0/1560233822 wait complete. 2024-09-19T07:41:20.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.148+0000 7fac13b51740 1 -- start start 2024-09-19T07:41:20.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.148+0000 7fac13b51740 1 Processor -- start 2024-09-19T07:41:20.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac13b51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abc0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac13b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac13b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abbe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac0a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac0affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abc0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac0a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51876/0 (socket says 172.21.15.103:51876) 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac0a7fc640 1 -- 172.21.15.103:0/2611007050 learned_addr learned my addr 172.21.15.103:0/2611007050 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b5abc4010 con 0x563b5abb7b40 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563b5aad7060 con 0x563b5abb65d0 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b5abc4190 con 0x563b5abc1370 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abbe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.149+0000 7fac13b51740 10 client.0 _ll_register_callbacks cb 0x563b5a82a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:20.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fac0b7fe640 1 -- 172.21.15.103:0/2611007050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 msgr2=0x563b5abc0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abc0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fac0b7fe640 1 -- 172.21.15.103:0/2611007050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 msgr2=0x563b5abbe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbe070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fac0b7fe640 1 -- 172.21.15.103:0/2611007050 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5a88d3c0 con 0x563b5abc1370 2024-09-19T07:41:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0004e070 con 0x563b5abc1370 2024-09-19T07:41:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abbe800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabfc005fd0 tx=0x7fabfc013e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5bfc8012ccc6326 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:20.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fabeeff5640 10 client.25660 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fabeeff5640 1 -- 172.21.15.103:0/2611007050 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc004b20 con 0x563b5abc1370 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.150+0000 7fabeeff5640 1 -- 172.21.15.103:0/2611007050 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fabfc004cc0 con 0x563b5abc1370 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fabeeff5640 1 -- 172.21.15.103:0/2611007050 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabfc003970 con 0x563b5abc1370 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 10 client.25660 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=32}) -- 0x563b5abcfaf0 con 0x563b5abc1370 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fabeeff5640 1 -- 172.21.15.103:0/2611007050 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fabfc01a070 con 0x563b5abc1370 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 10 client.25660 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 10 client.25660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 20 client.25660 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5abd3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 4 client.25660 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b5ab29860 con 0x563b5abd0f20 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.151+0000 7fac0affd640 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5abd3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fabeeff5640 10 client.25660 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac0affd640 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5abd3340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3686d11dbffb87e7 server_cookie=c3638685689e4a5a in_seq=0 out_seq=0 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fabeeff5640 1 -- 172.21.15.103:0/2611007050 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b5ab29860 con 0x563b5abd0f20 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fabeeff5640 10 client.25660 handle_command_reply: tid=0 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 client.25660 shutdown 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 msgr2=0x563b5abbe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abbe800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fabfc005fd0 tx=0x7fabfc013e80 comp rx=0 tx=0).stop 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 shutdown_connections 2024-09-19T07:41:20.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5abd3340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abc1370 0x563b5abbe800 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fabeeff5640 0 client.25660 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 --2- 172.21.15.103:0/2611007050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abc0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.152+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 >> 172.21.15.103:0/2611007050 conn(0x563b5ab7f310 msgr2=0x563b5abb8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.153+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 shutdown_connections 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.153+0000 7fac13b51740 1 -- 172.21.15.103:0/2611007050 wait complete. 2024-09-19T07:41:20.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.153+0000 7fac13b51740 20 client.25660 trim_cache size 0 max 16384 2024-09-19T07:41:20.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.170+0000 7fac13b51740 1 Processor -- start 2024-09-19T07:41:20.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.170+0000 7fac13b51740 1 -- start start 2024-09-19T07:41:20.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.170+0000 7fac13b51740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abb2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.170+0000 7fac13b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb7b40 0x563b5abbdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.170+0000 7fac13b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abc1370 0x563b5abb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb7b40 0x563b5abbdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb7b40 0x563b5abbdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52216/0 (socket says 172.21.15.103:52216) 2024-09-19T07:41:20.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0a7fc640 1 -- 172.21.15.103:0/2055345195 learned_addr learned my addr 172.21.15.103:0/2055345195 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:20.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abc1370 0x563b5abb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0affd640 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abb2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1af920 con 0x563b5abc1370 2024-09-19T07:41:20.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1b15b0 con 0x563b5abb65d0 2024-09-19T07:41:20.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1afff0 con 0x563b5abb7b40 2024-09-19T07:41:20.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0affd640 1 -- 172.21.15.103:0/2055345195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb7b40 msgr2=0x563b5abbdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0affd640 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb7b40 0x563b5abbdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0affd640 1 -- 172.21.15.103:0/2055345195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abc1370 msgr2=0x563b5abb1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0affd640 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abc1370 0x563b5abb1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.171+0000 7fac0affd640 1 -- 172.21.15.103:0/2055345195 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5abc1130 con 0x563b5abb65d0 2024-09-19T07:41:20.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.172+0000 7fac0affd640 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abb2e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac04098c60 tx=0x7fac04014d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe122f73bf8dea48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:20.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.172+0000 7fabf0ff9640 1 -- 172.21.15.103:0/2055345195 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac040a4e50 con 0x563b5abb65d0 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.172+0000 7fabf0ff9640 1 -- 172.21.15.103:0/2055345195 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0409a330 con 0x563b5abb65d0 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.172+0000 7fabf0ff9640 1 -- 172.21.15.103:0/2055345195 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac040cd8d0 con 0x563b5abb65d0 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 msgr2=0x563b5abb2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abb2e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac04098c60 tx=0x7fac04014d20 comp rx=0 tx=0).stop 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 shutdown_connections 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abc1370 0x563b5abb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb7b40 0x563b5abbdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 --2- 172.21.15.103:0/2055345195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abb65d0 0x563b5abb2e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 >> 172.21.15.103:0/2055345195 conn(0x563b5ab7f310 msgr2=0x563b5abb1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:20.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.173+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 shutdown_connections 2024-09-19T07:41:20.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.174+0000 7fac13b51740 1 -- 172.21.15.103:0/2055345195 wait complete. 2024-09-19T07:41:20.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.174+0000 7fac13b51740 1 -- start start 2024-09-19T07:41:20.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.174+0000 7fac13b51740 1 Processor -- start 2024-09-19T07:41:20.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac13b51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb65d0 0x563b5abbb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac13b51740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac0affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb65d0 0x563b5abbb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac0affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb65d0 0x563b5abbb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52218/0 (socket says 172.21.15.103:52218) 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac0a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac0a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51886/0 (socket says 172.21.15.103:51886) 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac0affd640 1 -- 172.21.15.103:0/1624263147 learned_addr learned my addr 172.21.15.103:0/1624263147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac13b51740 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abc1370 0x563b5abbba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1b0e20 con 0x563b5abb7b40 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1b06c0 con 0x563b5abc1370 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1ad900 con 0x563b5abb65d0 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.175+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abc1370 0x563b5abbba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac13b51740 10 client.0 _ll_register_callbacks cb 0x7fabfc009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac0b7fe640 1 -- 172.21.15.103:0/1624263147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb65d0 msgr2=0x563b5abbb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb65d0 0x563b5abbb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac0b7fe640 1 -- 172.21.15.103:0/1624263147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 msgr2=0x563b5abbb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac0b7fe640 1 -- 172.21.15.103:0/1624263147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5ac3abe0 con 0x563b5abc1370 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac0b7fe640 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abc1370 0x563b5abbba90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fabfc000f80 tx=0x7fabfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=267f620ccfff67e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fabeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fabeeff5640 1 -- 172.21.15.103:0/1624263147 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc033060 con 0x563b5abc1370 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fabeeff5640 1 -- 172.21.15.103:0/1624263147 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fabfc004d30 con 0x563b5abc1370 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0409a330 con 0x563b5abc1370 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.176+0000 7fabeeff5640 1 -- 172.21.15.103:0/1624263147 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fabfc003fb0 con 0x563b5abc1370 2024-09-19T07:41:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 10 client.25703 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b5ac3bde0 con 0x563b5abc1370 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fabeeff5640 1 -- 172.21.15.103:0/1624263147 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0409a330 con 0x563b5abc1370 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fabeeff5640 1 -- 172.21.15.103:0/1624263147 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fabfc01a070 con 0x563b5abc1370 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 10 client.25703 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 20 client.25703 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5ac3e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 4 client.25703 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b5ab23d70 con 0x563b5abd0f20 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.177+0000 7fac0affd640 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5ac3e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.178+0000 7fac0affd640 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5ac3e5f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee0b8510b476dcc server_cookie=fe0ac7fe7378e1c9 in_seq=0 out_seq=0 2024-09-19T07:41:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.178+0000 7fabeeff5640 10 client.25703 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.178+0000 7fabeeff5640 1 -- 172.21.15.103:0/1624263147 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b5ab23d70 con 0x563b5abd0f20 2024-09-19T07:41:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.178+0000 7fabeeff5640 10 client.25703 handle_command_reply: tid=0 2024-09-19T07:41:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.178+0000 7fac13b51740 1 client.25703 shutdown 2024-09-19T07:41:20.186 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:41:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.178+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abc1370 msgr2=0x563b5abbba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.178+0000 7fac13b51740 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abc1370 0x563b5abbba90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fabfc000f80 tx=0x7fabfc004a40 comp rx=0 tx=0).stop 2024-09-19T07:41:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 shutdown_connections 2024-09-19T07:41:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563b5abd0f20 0x563b5ac3e5f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563b5abc1370 0x563b5abbba90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b5abb7b40 0x563b5abbb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 --2- 172.21.15.103:0/1624263147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b5abb65d0 0x563b5abbb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 >> 172.21.15.103:0/1624263147 conn(0x563b5ab7f310 msgr2=0x563b5abcc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fabeeff5640 0 client.25703 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 shutdown_connections 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 1 -- 172.21.15.103:0/1624263147 wait complete. 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.179+0000 7fac13b51740 20 client.25703 trim_cache size 0 max 16384 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fabf40a34b0 msgr2=0x7fabf40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fabf40a34b0 0x7fabf40a5970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac00005b50 comp rx=0 tx=0).stop 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c100a80 msgr2=0x7fac0c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c100a80 0x7fac0c19a640 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fac0400bfd0 tx=0x7fac0400ae10 comp rx=0 tx=0).stop 2024-09-19T07:41:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 shutdown_connections 2024-09-19T07:41:20.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fabf40a34b0 0x7fabf40a5970 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac0c106a80 0x7fac0c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac0c101420 0x7fac0c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 --2- 172.21.15.103:0/3203125397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac0c100a80 0x7fac0c19a640 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:20.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 >> 172.21.15.103:0/3203125397 conn(0x7fac0c0fc7a0 msgr2=0x7fac0c0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:20.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 shutdown_connections 2024-09-19T07:41:20.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:20.180+0000 7fac11778640 1 -- 172.21.15.103:0/3203125397 wait complete. 2024-09-19T07:41:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 165.0 s, 3668.4 tps, lat 27.332 ms stddev 28.018 2024-09-19T07:41:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:21 smithi103 ceph-mon[31201]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-09-19T07:41:21.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:21 smithi134 ceph-mon[35414]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-09-19T07:41:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:21 smithi174 ceph-mon[35370]: pgmap v104: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-09-19T07:41:22.007 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:41:22.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:41:22.364 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 -- 172.21.15.103:0/2943731874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 msgr2=0x7f9ff8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/2943731874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8103170 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009930 tx=0x7f9fec02a420 comp rx=0 tx=0).stop 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 -- 172.21.15.103:0/2943731874 shutdown_connections 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/2943731874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff8108290 0x7f9ff810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/2943731874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff8103740 0x7f9ff8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/2943731874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8103170 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 -- 172.21.15.103:0/2943731874 >> 172.21.15.103:0/2943731874 conn(0x7f9ff80fc7d0 msgr2=0x7f9ff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:22.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 -- 172.21.15.103:0/2943731874 shutdown_connections 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.951+0000 7f9fff0e0640 1 -- 172.21.15.103:0/2943731874 wait complete. 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 Processor -- start 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 -- start start 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8117230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff8103740 0x7f9ff8117770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff8108290 0x7f9ff81112b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810fed0 con 0x7f9ff8102d90 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810fd50 con 0x7f9ff8108290 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9fff0e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8110050 con 0x7f9ff8103740 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9ffce7c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8117230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9ffce7c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8117230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51904/0 (socket says 172.21.15.103:51904) 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9ffce7c640 1 -- 172.21.15.103:0/3957260445 learned_addr learned my addr 172.21.15.103:0/3957260445 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9ffd67d640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff8108290 0x7f9ff81112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.952+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff8103740 0x7f9ff8117770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ffd67d640 1 -- 172.21.15.103:0/3957260445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff8103740 msgr2=0x7f9ff8117770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ffd67d640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff8103740 0x7f9ff8117770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ffd67d640 1 -- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 msgr2=0x7f9ff8117230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ffd67d640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8117230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ffd67d640 1 -- 172.21.15.103:0/3957260445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff8111b40 con 0x7f9ff8108290 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ffd67d640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff8108290 0x7f9ff81112b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e940 tx=0x7f9ff000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ff5ffb640 1 -- 172.21.15.103:0/3957260445 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0004d20 con 0x7f9ff8108290 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff8111e30 con 0x7f9ff8108290 2024-09-19T07:41:22.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ff5ffb640 1 -- 172.21.15.103:0/3957260445 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ff0005020 con 0x7f9ff8108290 2024-09-19T07:41:22.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9ff5ffb640 1 -- 172.21.15.103:0/3957260445 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00116b0 con 0x7f9ff8108290 2024-09-19T07:41:22.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.953+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff81b3e30 con 0x7f9ff8108290 2024-09-19T07:41:22.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.956+0000 7f9ff5ffb640 1 -- 172.21.15.103:0/3957260445 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9ff00054d0 con 0x7f9ff8108290 2024-09-19T07:41:22.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 Processor -- start 2024-09-19T07:41:22.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 -- start start 2024-09-19T07:41:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f400800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f400d40 0x55fd3f403130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fd3f35b280 con 0x55fd3f400d40 2024-09-19T07:41:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fd3f302b10 con 0x55fd3f3f6480 2024-09-19T07:41:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7fa0014b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fd3f357240 con 0x55fd3f3f5b60 2024-09-19T07:41:22.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ff5ffb640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9fe80a3400 0x7f9fe80a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f400d40 0x55fd3f403130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ffce7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f400800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f400d40 0x55fd3f403130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51914/0 (socket says 172.21.15.103:51914) 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ff7fff640 1 -- 172.21.15.103:0/763699273 learned_addr learned my addr 172.21.15.103:0/763699273 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ffd67d640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9fe80a3400 0x7f9fe80a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ff5ffb640 1 -- 172.21.15.103:0/3957260445 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9ff00cfd20 con 0x7f9ff8108290 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.957+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9ffce7c640 1 -- 172.21.15.103:0/763699273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f5b60 msgr2=0x55fd3f3f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9ffce7c640 1 -- 172.21.15.103:0/763699273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f400d40 msgr2=0x55fd3f403130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f400d40 0x55fd3f403130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9ffce7c640 1 -- 172.21.15.103:0/763699273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3f30f1f0 con 0x55fd3f3f6480 2024-09-19T07:41:22.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f400800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009850 tx=0x7f9fec00b100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5950a8329763d84f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:22.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9ffd67d640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9fe80a3400 0x7f9fe80a58c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00cd530 tx=0x7f9ff007c040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:22.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9fe2ffd640 1 -- 172.21.15.103:0/763699273 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec040070 con 0x55fd3f3f6480 2024-09-19T07:41:22.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9fe2ffd640 1 -- 172.21.15.103:0/763699273 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec005520 con 0x55fd3f3f6480 2024-09-19T07:41:22.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.958+0000 7f9fe2ffd640 1 -- 172.21.15.103:0/763699273 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec035040 con 0x55fd3f3f6480 2024-09-19T07:41:22.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.960+0000 7fa0014b9740 1 -- 172.21.15.103:0/763699273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 msgr2=0x55fd3f400800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:22.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.960+0000 7fa0014b9740 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f400800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009850 tx=0x7f9fec00b100 comp rx=0 tx=0).stop 2024-09-19T07:41:22.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.960+0000 7fa0014b9740 1 -- 172.21.15.103:0/763699273 shutdown_connections 2024-09-19T07:41:22.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.960+0000 7fa0014b9740 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f400d40 0x55fd3f403130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.960+0000 7fa0014b9740 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f400800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.960+0000 7fa0014b9740 1 --2- 172.21.15.103:0/763699273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.960+0000 7fa0014b9740 1 -- 172.21.15.103:0/763699273 >> 172.21.15.103:0/763699273 conn(0x55fd3f3f20f0 msgr2=0x55fd3f3f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:22.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.961+0000 7fa0014b9740 1 -- 172.21.15.103:0/763699273 shutdown_connections 2024-09-19T07:41:22.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.961+0000 7fa0014b9740 1 -- 172.21.15.103:0/763699273 wait complete. 2024-09-19T07:41:22.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.961+0000 7fa0014b9740 1 -- start start 2024-09-19T07:41:22.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.961+0000 7fa0014b9740 1 Processor -- start 2024-09-19T07:41:22.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f3fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 0x55fd3f3fe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7fa0014b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fd3f403910 con 0x55fd3f3f5b60 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7fa0014b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fd3f315f10 con 0x55fd3f3f6480 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7fa0014b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fd3f403a90 con 0x55fd3f400d40 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7fa0014b9740 10 client.0 _ll_register_callbacks cb 0x55fd3f2dea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3fdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51918/0 (socket says 172.21.15.103:51918) 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.962+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1983456910 learned_addr learned my addr 172.21.15.103:0/1983456910 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1983456910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 msgr2=0x55fd3f3fe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 0x55fd3f3fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1983456910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 msgr2=0x55fd3f3fe0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f3fe0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1983456910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3f0cc6a0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3fdba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400bff0 tx=0x7f9fe4010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc5aad4178f9519 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9fd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1983456910 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4004230 con 0x55fd3f3f5b60 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1983456910 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9fe40043d0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.963+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fec005520 con 0x55fd3f3f5b60 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1983456910 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9fe400f7f0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 10 client.16191 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd3f40f560 con 0x55fd3f3f5b60 2024-09-19T07:41:22.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1983456910 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe40035f0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1983456910 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f9fe401c070 con 0x55fd3f3f5b60 2024-09-19T07:41:22.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 10 client.16191 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:22.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:22.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 20 client.16191 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f412ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 4 client.16191 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9fec005520 con 0x55fd3f410ac0 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.964+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f412ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7f9fd7fff640 10 client.16191 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f412ef0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17660293f6ced46f server_cookie=d056dbd92b9efdeb in_seq=0 out_seq=0 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1983456910 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9fec005520 con 0x55fd3f410ac0 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7f9fd7fff640 10 client.16191 handle_command_reply: tid=0 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 client.16191 shutdown 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 msgr2=0x55fd3f3fdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3fdba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400bff0 tx=0x7f9fe4010ae0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 shutdown_connections 2024-09-19T07:41:22.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f412ef0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7f9fd7fff640 0 client.16191 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 0x55fd3f3fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f3fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.965+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1983456910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3fdba0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.966+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 >> 172.21.15.103:0/1983456910 conn(0x55fd3f3f20f0 msgr2=0x55fd3f3f1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.966+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 shutdown_connections 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.966+0000 7fa0014b9740 1 -- 172.21.15.103:0/1983456910 wait complete. 2024-09-19T07:41:22.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.966+0000 7fa0014b9740 20 client.16191 trim_cache size 0 max 16384 2024-09-19T07:41:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 Processor -- start 2024-09-19T07:41:22.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 -- start start 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f3fe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 0x55fd3f418800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81bc6a0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81be330 con 0x55fd3f3f6480 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7fa0014b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81bcd70 con 0x55fd3f400d40 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7f9ffce7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f3fe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51928/0 (socket says 172.21.15.103:51928) 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7f9ff7fff640 1 -- 172.21.15.103:0/14781331 learned_addr learned my addr 172.21.15.103:0/14781331 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.989+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 0x55fd3f418800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9ff7fff640 1 -- 172.21.15.103:0/14781331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 msgr2=0x55fd3f418800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:22.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 0x55fd3f418800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9ff7fff640 1 -- 172.21.15.103:0/14781331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 msgr2=0x55fd3f3fe470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:22.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f3fe470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9ff7fff640 1 -- 172.21.15.103:0/14781331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3f3f1f20 con 0x55fd3f3f5b60 2024-09-19T07:41:22.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f85c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4012010 tx=0x7f9fe400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9972aa630858528f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:22.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9fe27fc640 1 -- 172.21.15.103:0/14781331 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4012da0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9fe27fc640 1 -- 172.21.15.103:0/14781331 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe4005020 con 0x55fd3f3f5b60 2024-09-19T07:41:22.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.990+0000 7f9fe27fc640 1 -- 172.21.15.103:0/14781331 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4004a90 con 0x55fd3f3f5b60 2024-09-19T07:41:22.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.991+0000 7fa0014b9740 1 -- 172.21.15.103:0/14781331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 msgr2=0x55fd3f3f85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:22.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.991+0000 7fa0014b9740 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f85c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4012010 tx=0x7f9fe400b920 comp rx=0 tx=0).stop 2024-09-19T07:41:22.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.992+0000 7fa0014b9740 1 -- 172.21.15.103:0/14781331 shutdown_connections 2024-09-19T07:41:22.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.992+0000 7fa0014b9740 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f400d40 0x55fd3f418800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.992+0000 7fa0014b9740 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f3f6480 0x55fd3f3fe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.992+0000 7fa0014b9740 1 --2- 172.21.15.103:0/14781331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f3f85c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.992+0000 7fa0014b9740 1 -- 172.21.15.103:0/14781331 >> 172.21.15.103:0/14781331 conn(0x55fd3f3f20f0 msgr2=0x55fd3f3f1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:22.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.992+0000 7fa0014b9740 1 -- 172.21.15.103:0/14781331 shutdown_connections 2024-09-19T07:41:22.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.993+0000 7fa0014b9740 1 -- 172.21.15.103:0/14781331 wait complete. 2024-09-19T07:41:22.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.993+0000 7fa0014b9740 1 -- start start 2024-09-19T07:41:22.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.993+0000 7fa0014b9740 1 Processor -- start 2024-09-19T07:41:22.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f404ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f6480 0x55fd3f405420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7fa0014b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f400d40 0x55fd3f47a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:22.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7fa0014b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81bdba0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7fa0014b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81bd440 con 0x55fd3f400d40 2024-09-19T07:41:22.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7fa0014b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81ba680 con 0x55fd3f3f6480 2024-09-19T07:41:22.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f404ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7f9ff7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f404ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51934/0 (socket says 172.21.15.103:51934) 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1753792375 learned_addr learned my addr 172.21.15.103:0/1753792375 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.994+0000 7fa0014b9740 10 client.0 _ll_register_callbacks cb 0x7f9fe400b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f400d40 0x55fd3f47a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f6480 0x55fd3f405420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1753792375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f6480 msgr2=0x55fd3f405420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f6480 0x55fd3f405420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f400d40 msgr2=0x55fd3f47a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f400d40 0x55fd3f47a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ff7fff640 1 -- 172.21.15.103:0/1753792375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd3f47ad10 con 0x55fd3f3f5b60 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f6480 0x55fd3f405420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:22.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9ff7fff640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f404ee0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005cd0 tx=0x7f9fe400ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd37d286e2e3604 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:22.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9fd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:41:22.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1753792375 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40053d0 con 0x55fd3f3f5b60 2024-09-19T07:41:22.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1753792375 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9fe4005570 con 0x55fd3f3f5b60 2024-09-19T07:41:22.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe4005020 con 0x55fd3f3f5b60 2024-09-19T07:41:22.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1753792375 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9fe4004810 con 0x55fd3f3f5b60 2024-09-19T07:41:22.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7fa0014b9740 10 client.16197 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:22.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.995+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fd3f47bf10 con 0x55fd3f3f5b60 2024-09-19T07:41:22.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1753792375 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe4005020 con 0x55fd3f3f5b60 2024-09-19T07:41:22.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1753792375 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f9fe402f030 con 0x55fd3f3f5b60 2024-09-19T07:41:22.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7fa0014b9740 10 client.16197 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:22.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7fa0014b9740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:22.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7fa0014b9740 20 client.16197 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f47e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7fa0014b9740 4 client.16197 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fd3f3687f0 con 0x55fd3f410ac0 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.996+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f47e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.998+0000 7f9ffce7c640 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f47e640 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387f0a595705879f server_cookie=7e77a196464682f9 in_seq=0 out_seq=0 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.998+0000 7f9fd7fff640 10 client.16197 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.998+0000 7f9fd7fff640 1 -- 172.21.15.103:0/1753792375 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fd3f3687f0 con 0x55fd3f410ac0 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.998+0000 7f9fd7fff640 10 client.16197 handle_command_reply: tid=0 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.998+0000 7fa0014b9740 1 client.16197 shutdown 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 msgr2=0x55fd3f404ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:23.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f404ee0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005cd0 tx=0x7f9fe400ff10 comp rx=0 tx=0).stop 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 shutdown_connections 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fd3f410ac0 0x55fd3f47e640 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd3f400d40 0x55fd3f47a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd3f3f6480 0x55fd3f405420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 --2- 172.21.15.103:0/1753792375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fd3f3f5b60 0x55fd3f404ee0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7f9fd7fff640 0 client.16197 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 >> 172.21.15.103:0/1753792375 conn(0x55fd3f3f20f0 msgr2=0x55fd3f40bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:23.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 shutdown_connections 2024-09-19T07:41:23.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 1 -- 172.21.15.103:0/1753792375 wait complete. 2024-09-19T07:41:23.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:22.999+0000 7fa0014b9740 20 client.16197 trim_cache size 0 max 16384 2024-09-19T07:41:23.002 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:41:23.002 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:41:23.002 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "02687aae-4ea4-4460-9b81-9efbe839ca0d", 2024-09-19T07:41:23.002 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:41:23.002 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9fe80a3400 msgr2=0x7f9fe80a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9fe80a3400 0x7f9fe80a58c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00cd530 tx=0x7f9ff007c040 comp rx=0 tx=0).stop 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff8108290 msgr2=0x7f9ff81112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff8108290 0x7f9ff81112b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e940 tx=0x7f9ff000ee00 comp rx=0 tx=0).stop 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 shutdown_connections 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9fe80a3400 0x7f9fe80a58c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ff8108290 0x7f9ff81112b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff8103740 0x7f9ff8117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 --2- 172.21.15.103:0/3957260445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff8102d90 0x7f9ff8117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 >> 172.21.15.103:0/3957260445 conn(0x7f9ff80fc7d0 msgr2=0x7f9ff80fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 shutdown_connections 2024-09-19T07:41:23.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:23.000+0000 7f9fff0e0640 1 -- 172.21.15.103:0/3957260445 wait complete. 2024-09-19T07:41:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:23 smithi103 ceph-mon[31201]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 710 op/s 2024-09-19T07:41:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:23 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:41:23.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:23 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:23 smithi134 ceph-mon[35414]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 710 op/s 2024-09-19T07:41:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:23 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:41:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:23 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:23 smithi174 ceph-mon[35370]: pgmap v105: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 710 op/s 2024-09-19T07:41:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:23 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:41:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:23 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:41:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:24 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:41:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:41:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:41:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:41:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:41:24.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:41:24.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:24 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.844 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:24.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:41:24.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:24.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:24 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:41:25.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.393+0000 7f5303fff640 1 -- 172.21.15.103:0/2976738866 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53040319b0 con 0x7f53101049a0 2024-09-19T07:41:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 -- 172.21.15.103:0/2976738866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53101049a0 msgr2=0x7f53101089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 --2- 172.21.15.103:0/2976738866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53101049a0 0x7f53101089f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5304009900 tx=0x7f530402a400 comp rx=0 tx=0).stop 2024-09-19T07:41:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 -- 172.21.15.103:0/2976738866 shutdown_connections 2024-09-19T07:41:25.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 --2- 172.21.15.103:0/2976738866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5310109120 0x7f5310110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 --2- 172.21.15.103:0/2976738866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53101049a0 0x7f53101089f0 secure :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5304009900 tx=0x7f530402a400 comp rx=0 tx=0).stop 2024-09-19T07:41:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 --2- 172.21.15.103:0/2976738866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5310103ff0 0x7f53101043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 -- 172.21.15.103:0/2976738866 >> 172.21.15.103:0/2976738866 conn(0x7f53100ffab0 msgr2=0x7f5310101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:25.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.394+0000 7f531751e640 1 -- 172.21.15.103:0/2976738866 shutdown_connections 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.395+0000 7f531751e640 1 -- 172.21.15.103:0/2976738866 wait complete. 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.395+0000 7f531751e640 1 Processor -- start 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.396+0000 7f531751e640 1 -- start start 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.396+0000 7f531751e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310103ff0 0x7f531010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.396+0000 7f531751e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5310109120 0x7f5310073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.396+0000 7f531751e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5310073f70 0x7f531010e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.396+0000 7f531751e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f531006c220 con 0x7f5310103ff0 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.396+0000 7f531751e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f531006c0a0 con 0x7f5310109120 2024-09-19T07:41:25.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.396+0000 7f531751e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f531006c3a0 con 0x7f5310073f70 2024-09-19T07:41:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f53152ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310103ff0 0x7f531010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f53152ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310103ff0 0x7f531010de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51956/0 (socket says 172.21.15.103:51956) 2024-09-19T07:41:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f53152ba640 1 -- 172.21.15.103:0/812510355 learned_addr learned my addr 172.21.15.103:0/812510355 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f5315abb640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5310073f70 0x7f531010e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f5314ab9640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5310109120 0x7f5310073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f5315abb640 1 -- 172.21.15.103:0/812510355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5310109120 msgr2=0x7f5310073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f5315abb640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5310109120 0x7f5310073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f5315abb640 1 -- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310103ff0 msgr2=0x7f531010de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f5315abb640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310103ff0 0x7f531010de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.397+0000 7f5315abb640 1 -- 172.21.15.103:0/812510355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531010e890 con 0x7f5310073f70 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.398+0000 7f5315abb640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5310073f70 0x7f531010e350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f530800ca30 tx=0x7f530800cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.398+0000 7f53027fc640 1 -- 172.21.15.103:0/812510355 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308007d90 con 0x7f5310073f70 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.398+0000 7f53027fc640 1 -- 172.21.15.103:0/812510355 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5308005020 con 0x7f5310073f70 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.398+0000 7f53027fc640 1 -- 172.21.15.103:0/812510355 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530800f740 con 0x7f5310073f70 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.399+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531010eb80 con 0x7f5310073f70 2024-09-19T07:41:25.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.400+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101b42d0 con 0x7f5310073f70 2024-09-19T07:41:25.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.405+0000 7f53027fc640 1 -- 172.21.15.103:0/812510355 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f53080051d0 con 0x7f5310073f70 2024-09-19T07:41:25.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 Processor -- start 2024-09-19T07:41:25.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 -- start start 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f303f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f304a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5639f2fa52e0 con 0x5639f303f4a0 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639f2f4cce0 con 0x5639f303fdc0 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53198f7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5639f2fa12a0 con 0x5639f304a680 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f5314ab9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f303f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f53152ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f304a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f5314ab9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f303f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51966/0 (socket says 172.21.15.103:51966) 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f5314ab9640 1 -- 172.21.15.103:0/1781953124 learned_addr learned my addr 172.21.15.103:0/1781953124 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:25.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.406+0000 7f5314ab9640 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53152ba640 1 -- 172.21.15.103:0/1781953124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 msgr2=0x5639f304ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53152ba640 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53152ba640 1 -- 172.21.15.103:0/1781953124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 msgr2=0x5639f303f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53152ba640 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f303f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53152ba640 1 -- 172.21.15.103:0/1781953124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f2f31440 con 0x5639f303fdc0 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53027fc640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52fc0a3580 0x7f52fc0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53152ba640 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f304a140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f530c009c80 tx=0x7f530c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109739222df33c7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f52fb7fe640 1 -- 172.21.15.103:0/1781953124 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c004250 con 0x5639f303fdc0 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f52fb7fe640 1 -- 172.21.15.103:0/1781953124 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c009e30 con 0x5639f303fdc0 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f52fb7fe640 1 -- 172.21.15.103:0/1781953124 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c004e20 con 0x5639f303fdc0 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f5314ab9640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52fc0a3580 0x7f52fc0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.407+0000 7f53027fc640 1 -- 172.21.15.103:0/812510355 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f53080d0160 con 0x7f5310073f70 2024-09-19T07:41:25.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 -- 172.21.15.103:0/1781953124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 msgr2=0x5639f304a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f5314ab9640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52fc0a3580 0x7f52fc0a5a40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5639f303cdb0 tx=0x7f5304005750 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f304a140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f530c009c80 tx=0x7f530c011810 comp rx=0 tx=0).stop 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 -- 172.21.15.103:0/1781953124 shutdown_connections 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f304a140 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 --2- 172.21.15.103:0/1781953124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f303f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 -- 172.21.15.103:0/1781953124 >> 172.21.15.103:0/1781953124 conn(0x5639f303ba30 msgr2=0x5639f303be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 -- 172.21.15.103:0/1781953124 shutdown_connections 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 -- 172.21.15.103:0/1781953124 wait complete. 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 -- start start 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.408+0000 7f53198f7740 1 Processor -- start 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53198f7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303f4a0 0x5639f3049ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53198f7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303fdc0 0x5639f3047170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53198f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 0x5639f30476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53198f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5639f304d250 con 0x5639f304a680 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53198f7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639f2f60050 con 0x5639f303f4a0 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53198f7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5639f304d3d0 con 0x5639f303fdc0 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53198f7740 10 client.0 _ll_register_callbacks cb 0x5639f2fa0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53152ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303f4a0 0x5639f3049ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53152ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303f4a0 0x5639f3049ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57080/0 (socket says 172.21.15.103:57080) 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.409+0000 7f53152ba640 1 -- 172.21.15.103:0/3427422771 learned_addr learned my addr 172.21.15.103:0/3427422771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f5314ab9640 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303fdc0 0x5639f3047170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f53152ba640 1 -- 172.21.15.103:0/3427422771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303fdc0 msgr2=0x5639f3047170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f53152ba640 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303fdc0 0x5639f3047170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f53152ba640 1 -- 172.21.15.103:0/3427422771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 msgr2=0x5639f30476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f53152ba640 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 0x5639f30476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f53152ba640 1 -- 172.21.15.103:0/3427422771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f2d163c0 con 0x5639f303f4a0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f5314ab9640 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303fdc0 0x5639f3047170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f53152ba640 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303f4a0 0x5639f3049ec0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f530c004960 tx=0x7f530c0049c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3cfbdea8875ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f52f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f52f8ff9640 1 -- 172.21.15.103:0/3427422771 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c029070 con 0x5639f303f4a0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f52f8ff9640 1 -- 172.21.15.103:0/3427422771 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f530c003a00 con 0x5639f303f4a0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.410+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f530c009e30 con 0x5639f303f4a0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.411+0000 7f52f8ff9640 1 -- 172.21.15.103:0/3427422771 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f530c035050 con 0x5639f303f4a0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.411+0000 7f53198f7740 10 client.25727 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.411+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639f3058b10 con 0x5639f303f4a0 2024-09-19T07:41:25.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.411+0000 7f52f8ff9640 1 -- 172.21.15.103:0/3427422771 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c0044a0 con 0x5639f303f4a0 2024-09-19T07:41:25.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f52f8ff9640 1 -- 172.21.15.103:0/3427422771 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f530c023430 con 0x5639f303f4a0 2024-09-19T07:41:25.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f53198f7740 10 client.25727 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:25.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f53198f7740 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:25.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f53198f7740 20 client.25727 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f53198f7740 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f3059ec0 0x5639f305c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f53198f7740 4 client.25727 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f530c009e30 con 0x5639f3059ec0 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f5314ab9640 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f3059ec0 0x5639f305c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f52f8ff9640 10 client.25727 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.412+0000 7f5314ab9640 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f3059ec0 0x5639f305c310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38291202c0177ac6 server_cookie=97e8f9afa11fa43e in_seq=0 out_seq=0 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f52f8ff9640 1 -- 172.21.15.103:0/3427422771 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f530c009e30 con 0x5639f3059ec0 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f52f8ff9640 10 client.25727 handle_command_reply: tid=0 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 client.25727 shutdown 2024-09-19T07:41:25.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303f4a0 msgr2=0x5639f3049ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303f4a0 0x5639f3049ec0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f530c004960 tx=0x7f530c0049c0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 shutdown_connections 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f3059ec0 0x5639f305c310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 0x5639f30476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303fdc0 0x5639f3047170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 --2- 172.21.15.103:0/3427422771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303f4a0 0x5639f3049ec0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 >> 172.21.15.103:0/3427422771 conn(0x5639f303ba30 msgr2=0x5639f3041760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f52f8ff9640 0 client.25727 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 shutdown_connections 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 1 -- 172.21.15.103:0/3427422771 wait complete. 2024-09-19T07:41:25.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.413+0000 7f53198f7740 20 client.25727 trim_cache size 0 max 16384 2024-09-19T07:41:25.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 Processor -- start 2024-09-19T07:41:25.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 -- start start 2024-09-19T07:41:25.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303f4a0 0x5639f3042900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3046a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 0x5639f303a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53101bcc20 con 0x5639f304a680 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53101be8b0 con 0x5639f303fdc0 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.439+0000 7f53198f7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53101bd2f0 con 0x5639f303f4a0 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3046a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3046a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57086/0 (socket says 172.21.15.103:57086) 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 -- 172.21.15.103:0/290352701 learned_addr learned my addr 172.21.15.103:0/290352701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f53152ba640 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303f4a0 0x5639f3042900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5315abb640 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 0x5639f303a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 -- 172.21.15.103:0/290352701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303f4a0 msgr2=0x5639f3042900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303f4a0 0x5639f3042900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 -- 172.21.15.103:0/290352701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 msgr2=0x5639f303a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 0x5639f303a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 -- 172.21.15.103:0/290352701 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f303b860 con 0x5639f303fdc0 2024-09-19T07:41:25.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f53152ba640 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303f4a0 0x5639f3042900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:25.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.440+0000 7f5314ab9640 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3046a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5310104f00 tx=0x7f5304038000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d178fb3775ad12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:25.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.441+0000 7f52faffd640 1 -- 172.21.15.103:0/290352701 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304058070 con 0x5639f303fdc0 2024-09-19T07:41:25.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.441+0000 7f52faffd640 1 -- 172.21.15.103:0/290352701 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5304004c10 con 0x5639f303fdc0 2024-09-19T07:41:25.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.441+0000 7f52faffd640 1 -- 172.21.15.103:0/290352701 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304035040 con 0x5639f303fdc0 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 -- 172.21.15.103:0/290352701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 msgr2=0x5639f3046a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3046a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5310104f00 tx=0x7f5304038000 comp rx=0 tx=0).stop 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 -- 172.21.15.103:0/290352701 shutdown_connections 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f304a680 0x5639f303a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3046a50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 --2- 172.21.15.103:0/290352701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f303f4a0 0x5639f3042900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 -- 172.21.15.103:0/290352701 >> 172.21.15.103:0/290352701 conn(0x5639f303ba30 msgr2=0x5639f303a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 -- 172.21.15.103:0/290352701 shutdown_connections 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.442+0000 7f53198f7740 1 -- 172.21.15.103:0/290352701 wait complete. 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.443+0000 7f53198f7740 1 -- start start 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.443+0000 7f53198f7740 1 Processor -- start 2024-09-19T07:41:25.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53198f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f3052770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53198f7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3052cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53198f7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53198f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53101be120 con 0x5639f303f4a0 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53198f7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53101bd9c0 con 0x5639f303fdc0 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53198f7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53101bac00 con 0x5639f304a680 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53198f7740 10 client.0 _ll_register_callbacks cb 0x7f530c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53152ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f3052770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53152ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f3052770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51998/0 (socket says 172.21.15.103:51998) 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f53152ba640 1 -- 172.21.15.103:0/2982543937 learned_addr learned my addr 172.21.15.103:0/2982543937 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5314ab9640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3052cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5315abb640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5314ab9640 1 -- 172.21.15.103:0/2982543937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 msgr2=0x5639f304db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5314ab9640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5314ab9640 1 -- 172.21.15.103:0/2982543937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 msgr2=0x5639f3052770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5314ab9640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f3052770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5314ab9640 1 -- 172.21.15.103:0/2982543937 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f304e3b0 con 0x5639f303fdc0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5315abb640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f5314ab9640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3052cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5639f3040df0 tx=0x7f5304004ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f79adb63d38ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.444+0000 7f52f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f52f8ff9640 1 -- 172.21.15.103:0/2982543937 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304058040 con 0x5639f303fdc0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f52f8ff9640 1 -- 172.21.15.103:0/2982543937 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5304005380 con 0x5639f303fdc0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5304004c10 con 0x5639f303fdc0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f52f8ff9640 1 -- 172.21.15.103:0/2982543937 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5304035040 con 0x5639f303fdc0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 10 client.25733 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639f30c4680 con 0x5639f303fdc0 2024-09-19T07:41:25.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f52f8ff9640 1 -- 172.21.15.103:0/2982543937 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5304004c10 con 0x5639f303fdc0 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f52f8ff9640 1 -- 172.21.15.103:0/2982543937 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7f530403a040 con 0x5639f303fdc0 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 10 client.25733 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 20 client.25733 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f30c4d40 0x5639f30c7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 4 client.25733 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.445+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639f2fb2850 con 0x5639f30c4d40 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.446+0000 7f53152ba640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f30c4d40 0x5639f30c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.446+0000 7f53152ba640 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f30c4d40 0x5639f30c7160 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ea1559cc9ab136 server_cookie=4e597aa3efa0b65f in_seq=0 out_seq=0 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.446+0000 7f52f8ff9640 10 client.25733 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.446+0000 7f52f8ff9640 1 -- 172.21.15.103:0/2982543937 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5639f2fb2850 con 0x5639f30c4d40 2024-09-19T07:41:25.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.446+0000 7f52f8ff9640 10 client.25733 handle_command_reply: tid=0 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 client.25733 shutdown 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 msgr2=0x5639f3052cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3052cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5639f3040df0 tx=0x7f5304004ff0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 shutdown_connections 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639f30c4d40 0x5639f30c7160 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639f304a680 0x5639f304db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f52f8ff9640 0 client.25733 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639f303fdc0 0x5639f3052cb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 --2- 172.21.15.103:0/2982543937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5639f303f4a0 0x5639f3052770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 >> 172.21.15.103:0/2982543937 conn(0x5639f303ba30 msgr2=0x5639f30427f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.447+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 shutdown_connections 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f53198f7740 1 -- 172.21.15.103:0/2982543937 wait complete. 2024-09-19T07:41:25.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f53198f7740 20 client.25733 trim_cache size 0 max 16384 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: "02687aae-4ea4-4460-9b81-9efbe839ca0d": { 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "02687aae-4ea4-4460-9b81-9efbe839ca0d", 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:25.454 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:25.455 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:41:25.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52fc0a3580 msgr2=0x7f52fc0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52fc0a3580 0x7f52fc0a5a40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5639f303cdb0 tx=0x7f5304005750 comp rx=0 tx=0).stop 2024-09-19T07:41:25.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5310073f70 msgr2=0x7f531010e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5310073f70 0x7f531010e350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f530800ca30 tx=0x7f530800cef0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 shutdown_connections 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52fc0a3580 0x7f52fc0a5a40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5310073f70 0x7f531010e350 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5310109120 0x7f5310073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 --2- 172.21.15.103:0/812510355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5310103ff0 0x7f531010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.448+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 >> 172.21.15.103:0/812510355 conn(0x7f53100ffab0 msgr2=0x7f5310100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.449+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 shutdown_connections 2024-09-19T07:41:25.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:25.449+0000 7f531751e640 1 -- 172.21.15.103:0/812510355 wait complete. 2024-09-19T07:41:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 170.0 s, 3227.2 tps, lat 30.886 ms stddev 31.390 2024-09-19T07:41:25.547 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T07:41:25.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:41:25.577 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:25 smithi103 ceph-mon[31201]: pgmap v106: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-19T07:41:25.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:25 smithi134 ceph-mon[35414]: pgmap v106: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-19T07:41:25.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:25 smithi174 ceph-mon[35370]: pgmap v106: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-19T07:41:26.343 INFO:tasks.cephfs.filesystem:scrub status for tag:02687aae-4ea4-4460-9b81-9efbe839ca0d - {'path': '/', 'tag': '02687aae-4ea4-4460-9b81-9efbe839ca0d', 'options': 'recursive,force'} 2024-09-19T07:41:26.441 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.984+0000 7f52d79e9640 1 -- 172.21.15.103:0/2413968659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d010c170 msgr2=0x7f52d010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.984+0000 7f52d79e9640 1 --2- 172.21.15.103:0/2413968659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d010c170 0x7f52d010e610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009860 tx=0x7f52cc02a3d0 comp rx=0 tx=0).stop 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.985+0000 7f52d79e9640 1 -- 172.21.15.103:0/2413968659 shutdown_connections 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.985+0000 7f52d79e9640 1 --2- 172.21.15.103:0/2413968659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d010c170 0x7f52d010e610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.985+0000 7f52d79e9640 1 --2- 172.21.15.103:0/2413968659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52d0103660 0x7f52d010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.985+0000 7f52d79e9640 1 --2- 172.21.15.103:0/2413968659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 0x7f52d0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.985+0000 7f52d79e9640 1 -- 172.21.15.103:0/2413968659 >> 172.21.15.103:0/2413968659 conn(0x7f52d00fc7a0 msgr2=0x7f52d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.985+0000 7f52d79e9640 1 -- 172.21.15.103:0/2413968659 shutdown_connections 2024-09-19T07:41:26.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.985+0000 7f52d79e9640 1 -- 172.21.15.103:0/2413968659 wait complete. 2024-09-19T07:41:26.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 Processor -- start 2024-09-19T07:41:26.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 -- start start 2024-09-19T07:41:26.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 0x7f52d0103bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:26.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0103660 0x7f52d0104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:26.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52d010c170 0x7f52d0104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:26.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52d0111f30 con 0x7f52d0102d40 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52d0111db0 con 0x7f52d010c170 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.986+0000 7f52d79e9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01120b0 con 0x7f52d0103660 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d4f84640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0103660 0x7f52d0104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 0x7f52d0103bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 0x7f52d0103bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52018/0 (socket says 172.21.15.103:52018) 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 -- 172.21.15.103:0/1710157357 learned_addr learned my addr 172.21.15.103:0/1710157357 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5f86640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52d010c170 0x7f52d0104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d4f84640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0103660 0x7f52d0104100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:52360/0 (socket says 172.21.15.103:52360) 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 -- 172.21.15.103:0/1710157357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0103660 msgr2=0x7f52d0104100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0103660 0x7f52d0104100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 -- 172.21.15.103:0/1710157357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52d010c170 msgr2=0x7f52d0104640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:26.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52d010c170 0x7f52d0104640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:26.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.987+0000 7f52d5785640 1 -- 172.21.15.103:0/1710157357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d01aacc0 con 0x7f52d0102d40 2024-09-19T07:41:26.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.988+0000 7f52d5785640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 0x7f52d0103bc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009fa0 tx=0x7f52c800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:26.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.988+0000 7f52c67fc640 1 -- 172.21.15.103:0/1710157357 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c801b070 con 0x7f52d0102d40 2024-09-19T07:41:26.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.988+0000 7f52c67fc640 1 -- 172.21.15.103:0/1710157357 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c800ba50 con 0x7f52d0102d40 2024-09-19T07:41:26.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.988+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d01aafb0 con 0x7f52d0102d40 2024-09-19T07:41:26.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.988+0000 7f52c67fc640 1 -- 172.21.15.103:0/1710157357 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8016c70 con 0x7f52d0102d40 2024-09-19T07:41:26.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.989+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d01ab460 con 0x7f52d0102d40 2024-09-19T07:41:26.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.992+0000 7f52c67fc640 1 -- 172.21.15.103:0/1710157357 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f52c801e070 con 0x7f52d0102d40 2024-09-19T07:41:26.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.990+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d006b070 con 0x7f52d0102d40 2024-09-19T07:41:26.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.992+0000 7f52c67fc640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52bc0a3500 0x7f52bc0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:26.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.993+0000 7f52c67fc640 1 -- 172.21.15.103:0/1710157357 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f52c80cfc00 con 0x7f52d0102d40 2024-09-19T07:41:26.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.993+0000 7f52d4f84640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52bc0a3500 0x7f52bc0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:26.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:26.994+0000 7f52d4f84640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52bc0a3500 0x7f52bc0a59c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f52d0105380 tx=0x7f52c0005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:27.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.000+0000 7f52c67fc640 1 -- 172.21.15.103:0/1710157357 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f52c80d6080 con 0x7f52d0102d40 2024-09-19T07:41:27.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.310+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52d01ab720 con 0x7f52d0102d40 2024-09-19T07:41:27.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.311+0000 7f52c67fc640 1 -- 172.21.15.103:0/1710157357 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+12003 (secure 0 0 0) 0x7f52c809c5f0 con 0x7f52d0102d40 2024-09-19T07:41:27.312 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:41:27.313 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":32,"btime":"2024-09-19T07:40:55:290656+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:40:54.957956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:41:27.313 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 32 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52bc0a3500 msgr2=0x7f52bc0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52bc0a3500 0x7f52bc0a59c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f52d0105380 tx=0x7f52c0005f90 comp rx=0 tx=0).stop 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 msgr2=0x7f52d0103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 0x7f52d0103bc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009fa0 tx=0x7f52c800c800 comp rx=0 tx=0).stop 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 shutdown_connections 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f52bc0a3500 0x7f52bc0a59c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52d010c170 0x7f52d0104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0103660 0x7f52d0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 --2- 172.21.15.103:0/1710157357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d0102d40 0x7f52d0103bc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 >> 172.21.15.103:0/1710157357 conn(0x7f52d00fc7a0 msgr2=0x7f52d010d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 shutdown_connections 2024-09-19T07:41:27.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:27.315+0000 7f52d79e9640 1 -- 172.21.15.103:0/1710157357 wait complete. 2024-09-19T07:41:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:27 smithi103 ceph-mon[31201]: pgmap v107: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-09-19T07:41:27.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:27 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1710157357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:27.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:27 smithi174 ceph-mon[35370]: pgmap v107: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-09-19T07:41:27.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:27 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1710157357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:27.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:27 smithi134 ceph-mon[35414]: pgmap v107: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-09-19T07:41:27.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:27 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1710157357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:27.989 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T07:41:27.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T07:41:28.374 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:28.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/591513714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0106f20 msgr2=0x7fc9d010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/591513714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0106f20 0x7fc9d010eaa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009be0 tx=0x7fc9c802a6d0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/591513714 shutdown_connections 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/591513714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0106f20 0x7fc9d010eaa0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/591513714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d01027a0 0x7fc9d01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/591513714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0101df0 0x7fc9d01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/591513714 >> 172.21.15.103:0/591513714 conn(0x7fc9d00fd8f0 msgr2=0x7fc9d00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/591513714 shutdown_connections 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.914+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/591513714 wait complete. 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 Processor -- start 2024-09-19T07:41:28.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 -- start start 2024-09-19T07:41:28.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0101df0 0x7fc9d01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d01027a0 0x7fc9d0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0106f20 0x7fc9d019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111f30 con 0x7fc9d0101df0 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111db0 con 0x7fc9d0106f20 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d7c2a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01120b0 con 0x7fc9d01027a0 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d59c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0101df0 0x7fc9d01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d59c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0101df0 0x7fc9d01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33010/0 (socket says 172.21.15.103:33010) 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.915+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4233806088 learned_addr learned my addr 172.21.15.103:0/4233806088 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.916+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d01027a0 0x7fc9d0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.916+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0106f20 0x7fc9d019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.916+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4233806088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d01027a0 msgr2=0x7fc9d0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.916+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d01027a0 0x7fc9d0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.916+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4233806088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0106f20 msgr2=0x7fc9d019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.916+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0106f20 0x7fc9d019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.916+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4233806088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d019d3a0 con 0x7fc9d0101df0 2024-09-19T07:41:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.917+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0101df0 0x7fc9d01983f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c890 tx=0x7fc9cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.918+0000 7fc9c6ffd640 1 -- 172.21.15.103:0/4233806088 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc007bb0 con 0x7fc9d0101df0 2024-09-19T07:41:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.918+0000 7fc9c6ffd640 1 -- 172.21.15.103:0/4233806088 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc007d50 con 0x7fc9d0101df0 2024-09-19T07:41:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.919+0000 7fc9c6ffd640 1 -- 172.21.15.103:0/4233806088 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00f450 con 0x7fc9d0101df0 2024-09-19T07:41:28.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.924+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d019d690 con 0x7fc9d0101df0 2024-09-19T07:41:28.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.924+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a4f70 con 0x7fc9d0101df0 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.934+0000 7fc9da003740 1 Processor -- start 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.934+0000 7fc9da003740 1 -- start start 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.934+0000 7fc9da003740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.934+0000 7fc9da003740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c8860 0x5563596c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.934+0000 7fc9da003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c3010 0x5563596c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.935+0000 7fc9da003740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563596282a0 con 0x5563596c7f40 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.937+0000 7fc9da003740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563595cfb10 con 0x5563596c3010 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.937+0000 7fc9da003740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556359624260 con 0x5563596c8860 2024-09-19T07:41:28.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.936+0000 7fc9d51c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c3010 0x5563596c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.937+0000 7fc9d51c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c3010 0x5563596c3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60200/0 (socket says 172.21.15.103:60200) 2024-09-19T07:41:28.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.937+0000 7fc9d51c5640 1 -- 172.21.15.103:0/1791993016 learned_addr learned my addr 172.21.15.103:0/1791993016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:28.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.935+0000 7fc9d61c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.939+0000 7fc9c6ffd640 1 -- 172.21.15.103:0/4233806088 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc9cc004660 con 0x7fc9d0101df0 2024-09-19T07:41:28.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.940+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c8860 0x5563596c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.941+0000 7fc9c6ffd640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9bc0a3310 0x7fc9bc0a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.941+0000 7fc9c6ffd640 1 -- 172.21.15.103:0/4233806088 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc9cc0cfc00 con 0x7fc9d0101df0 2024-09-19T07:41:28.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.943+0000 7fc9d51c5640 1 -- 172.21.15.103:0/1791993016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c8860 msgr2=0x5563596c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.943+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c8860 0x5563596c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.943+0000 7fc9d51c5640 1 -- 172.21.15.103:0/1791993016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 msgr2=0x5563596c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.943+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.943+0000 7fc9d51c5640 1 -- 172.21.15.103:0/1791993016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556359399b90 con 0x5563596c3010 2024-09-19T07:41:28.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.944+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9bc0a3310 0x7fc9bc0a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.946+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9bc0a3310 0x7fc9bc0a57d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80092f0 tx=0x7fc9c802a6d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:28.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.946+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c3010 0x5563596c3490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0004620 tx=0x7fc9c000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57dc86a03ea46c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:28.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.947+0000 7fc9bbfff640 1 -- 172.21.15.103:0/1791993016 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0003bd0 con 0x5563596c3010 2024-09-19T07:41:28.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.947+0000 7fc9bbfff640 1 -- 172.21.15.103:0/1791993016 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c0037070 con 0x5563596c3010 2024-09-19T07:41:28.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.947+0000 7fc9bbfff640 1 -- 172.21.15.103:0/1791993016 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00336b0 con 0x5563596c3010 2024-09-19T07:41:28.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 -- 172.21.15.103:0/1791993016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c3010 msgr2=0x5563596c3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:28.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c3010 0x5563596c3490 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0004620 tx=0x7fc9c000bb20 comp rx=0 tx=0).stop 2024-09-19T07:41:28.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 -- 172.21.15.103:0/1791993016 shutdown_connections 2024-09-19T07:41:28.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c3010 0x5563596c3490 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c8860 0x5563596c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 --2- 172.21.15.103:0/1791993016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 -- 172.21.15.103:0/1791993016 >> 172.21.15.103:0/1791993016 conn(0x5563596be7a0 msgr2=0x5563596beb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:28.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 -- 172.21.15.103:0/1791993016 shutdown_connections 2024-09-19T07:41:28.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.948+0000 7fc9da003740 1 -- 172.21.15.103:0/1791993016 wait complete. 2024-09-19T07:41:28.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.949+0000 7fc9da003740 1 -- start start 2024-09-19T07:41:28.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.949+0000 7fc9da003740 1 Processor -- start 2024-09-19T07:41:28.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9da003740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9da003740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9da003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x5563596d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9da003740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563596cc530 con 0x5563596c7f40 2024-09-19T07:41:28.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9da003740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563595b06e0 con 0x5563596c8860 2024-09-19T07:41:28.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9da003740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563596cc6b0 con 0x5563596c3010 2024-09-19T07:41:28.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9da003740 10 client.0 _ll_register_callbacks cb 0x5563595d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9d59c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9d59c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56980/0 (socket says 172.21.15.103:56980) 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4062715867 learned_addr learned my addr 172.21.15.103:0/4062715867 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.950+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x5563596d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 msgr2=0x5563596d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x5563596d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4062715867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 msgr2=0x5563596cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9d59c6640 1 -- 172.21.15.103:0/4062715867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563596d27a0 con 0x5563596c3010 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cda50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc007590 tx=0x7fc9cc09c590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7b61fd24cf1ba93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.951+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4062715867 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0cd780 con 0x5563596c3010 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.952+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c0037070 con 0x5563596c3010 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.952+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4062715867 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc9cc09b2e0 con 0x5563596c3010 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.952+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4062715867 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc0a4b00 con 0x5563596c3010 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.952+0000 7fc9da003740 10 client.25708 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:28.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.952+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563596dc3a0 con 0x5563596c3010 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.952+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4062715867 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c0037070 con 0x5563596c3010 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.952+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4062715867 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fc9cc09ad50 con 0x5563596c3010 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.953+0000 7fc9da003740 10 client.25708 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.953+0000 7fc9da003740 10 client.25708 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.953+0000 7fc9da003740 20 client.25708 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.953+0000 7fc9da003740 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563596ddd40 0x5563596e0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.953+0000 7fc9da003740 4 client.25708 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.953+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563596359d0 con 0x5563596ddd40 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.954+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563596ddd40 0x5563596e0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.955+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563596ddd40 0x5563596e0200 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=affb81e5922ebe8b server_cookie=3ba0ea0714463d22 in_seq=0 out_seq=0 2024-09-19T07:41:28.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.955+0000 7fc9b97fa640 10 client.25708 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:28.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4062715867 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563596359d0 con 0x5563596ddd40 2024-09-19T07:41:28.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9b97fa640 10 client.25708 handle_command_reply: tid=0 2024-09-19T07:41:28.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 client.25708 shutdown 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 msgr2=0x5563596cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cda50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc007590 tx=0x7fc9cc09c590 comp rx=0 tx=0).stop 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 shutdown_connections 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5563596ddd40 0x5563596e0200 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x5563596d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 --2- 172.21.15.103:0/4062715867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cda50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 >> 172.21.15.103:0/4062715867 conn(0x5563596be7a0 msgr2=0x5563596cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9b97fa640 0 client.25708 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 shutdown_connections 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.957+0000 7fc9da003740 1 -- 172.21.15.103:0/4062715867 wait complete. 2024-09-19T07:41:28.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.958+0000 7fc9da003740 20 client.25708 trim_cache size 0 max 16384 2024-09-19T07:41:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 Processor -- start 2024-09-19T07:41:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 -- start start 2024-09-19T07:41:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c7f40 0x5563596c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c8860 0x5563596e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ab420 con 0x5563596c8860 2024-09-19T07:41:28.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ad0b0 con 0x5563596c7f40 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.980+0000 7fc9da003740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01abaf0 con 0x5563596c3010 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c7f40 0x5563596c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c7f40 0x5563596c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60216/0 (socket says 172.21.15.103:60216) 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 -- 172.21.15.103:0/3016340067 learned_addr learned my addr 172.21.15.103:0/3016340067 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c8860 0x5563596e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 -- 172.21.15.103:0/3016340067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 msgr2=0x5563596cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 -- 172.21.15.103:0/3016340067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c8860 msgr2=0x5563596e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c8860 0x5563596e5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 -- 172.21.15.103:0/3016340067 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563596ba260 con 0x5563596c7f40 2024-09-19T07:41:28.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.981+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c7f40 0x5563596c9e50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00045f0 tx=0x7fc9c0035040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51fc6657db90700 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.982+0000 7fc9bb7fe640 1 -- 172.21.15.103:0/3016340067 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0003dd0 con 0x5563596c7f40 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.983+0000 7fc9bb7fe640 1 -- 172.21.15.103:0/3016340067 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c0037040 con 0x5563596c7f40 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.983+0000 7fc9bb7fe640 1 -- 172.21.15.103:0/3016340067 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c003c900 con 0x5563596c7f40 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 -- 172.21.15.103:0/3016340067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c7f40 msgr2=0x5563596c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c7f40 0x5563596c9e50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00045f0 tx=0x7fc9c0035040 comp rx=0 tx=0).stop 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 -- 172.21.15.103:0/3016340067 shutdown_connections 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c8860 0x5563596e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c7f40 0x5563596c9e50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 --2- 172.21.15.103:0/3016340067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 -- 172.21.15.103:0/3016340067 >> 172.21.15.103:0/3016340067 conn(0x5563596be7a0 msgr2=0x5563596cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 -- 172.21.15.103:0/3016340067 shutdown_connections 2024-09-19T07:41:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.984+0000 7fc9da003740 1 -- 172.21.15.103:0/3016340067 wait complete. 2024-09-19T07:41:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.985+0000 7fc9da003740 1 -- start start 2024-09-19T07:41:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.985+0000 7fc9da003740 1 Processor -- start 2024-09-19T07:41:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.985+0000 7fc9da003740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9da003740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9da003740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x556359747aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9da003740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ac920 con 0x5563596c7f40 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9da003740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ac1c0 con 0x5563596c8860 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9da003740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01a9400 con 0x5563596c3010 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9da003740 10 client.0 _ll_register_callbacks cb 0x7fc9cc09bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x556359747aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x556359747aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:60220/0 (socket says 172.21.15.103:60220) 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 -- 172.21.15.103:0/4002189185 learned_addr learned my addr 172.21.15.103:0/4002189185 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d51c5640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 -- 172.21.15.103:0/4002189185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 msgr2=0x5563596d58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596d58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 -- 172.21.15.103:0/4002189185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 msgr2=0x5563596d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d61c7640 1 -- 172.21.15.103:0/4002189185 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563596ce300 con 0x5563596c8860 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.986+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.987+0000 7fc9d61c7640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x556359747aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009be0 tx=0x7fc9c8009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f4033e143518f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.987+0000 7fc9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.987+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4002189185 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8036040 con 0x5563596c8860 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.987+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4002189185 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc9c8004250 con 0x5563596c8860 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.987+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c0037040 con 0x5563596c8860 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4002189185 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc9c8004450 con 0x5563596c8860 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 10 client.25751 fetch_fsmap learned FSMap version 32 2024-09-19T07:41:28.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563596c2040 con 0x5563596c8860 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4002189185 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c8044040 con 0x5563596c8860 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4002189185 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8390+0+0 (secure 0 0 0) 0x7fc9c803c040 con 0x5563596c8860 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 10 client.25751 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 10 client.25751 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 20 client.25751 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556359749720 0x55635974bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 4 client.25751 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T07:41:28.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.988+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x7fc9c0037040 con 0x556359749720 2024-09-19T07:41:28.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.989+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556359749720 0x55635974bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:28.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.990+0000 7fc9b97fa640 10 client.25751 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:28.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:28.990+0000 7fc9d59c6640 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556359749720 0x55635974bb40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9a9b09ee08a47c server_cookie=11c5d74e605dbc13 in_seq=0 out_seq=0 2024-09-19T07:41:29.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9b97fa640 1 -- 172.21.15.103:0/4002189185 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fc9c0037040 con 0x556359749720 2024-09-19T07:41:29.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9b97fa640 10 client.25751 handle_command_reply: tid=0 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 client.25751 shutdown 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 msgr2=0x556359747aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x556359747aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009be0 tx=0x7fc9c8009280 comp rx=0 tx=0).stop 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 shutdown_connections 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556359749720 0x55635974bb40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563596c8860 0x556359747aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563596c7f40 0x5563596d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 --2- 172.21.15.103:0/4002189185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563596c3010 0x5563596d58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 >> 172.21.15.103:0/4002189185 conn(0x5563596be7a0 msgr2=0x5563596c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9b97fa640 0 client.25751 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 shutdown_connections 2024-09-19T07:41:29.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 1 -- 172.21.15.103:0/4002189185 wait complete. 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.136+0000 7fc9da003740 20 client.25751 trim_cache size 0 max 16384 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 32, 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 14, 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "99f1493b": { 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 14, 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:41:29.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 136.6, 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:41:29.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:29.141 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:29.141 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:29.141 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:29.141 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:29.141 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:41:29.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9bc0a3310 msgr2=0x7fc9bc0a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9bc0a3310 0x7fc9bc0a57d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80092f0 tx=0x7fc9c802a6d0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0101df0 msgr2=0x7fc9d01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0101df0 0x7fc9d01983f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c890 tx=0x7fc9cc00cd50 comp rx=0 tx=0).stop 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 shutdown_connections 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9bc0a3310 0x7fc9bc0a57d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d0106f20 0x7fc9d019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d01027a0 0x7fc9d0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 --2- 172.21.15.103:0/4233806088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9d0101df0 0x7fc9d01983f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 >> 172.21.15.103:0/4233806088 conn(0x7fc9d00fd8f0 msgr2=0x7fc9d00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 shutdown_connections 2024-09-19T07:41:29.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:29.137+0000 7fc9d7c2a640 1 -- 172.21.15.103:0/4233806088 wait complete. 2024-09-19T07:41:29.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:29 smithi103 ceph-mon[31201]: pgmap v108: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-09-19T07:41:29.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:41:29.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:41:29.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:41:29.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:41:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:29 smithi134 ceph-mon[35414]: pgmap v108: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-09-19T07:41:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:41:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:41:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:41:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:41:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:29 smithi174 ceph-mon[35370]: pgmap v108: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 709 op/s 2024-09-19T07:41:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:41:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:41:29.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:41:29.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:41:29.941 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '99f1493b', quiesce took 4.4 seconds. Will release after: 12.200000000000001 2024-09-19T07:41:30.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 175.0 s, 2282.3 tps, lat 31.268 ms stddev 32.787 2024-09-19T07:41:30.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:30 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:41:30.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:30 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:41:30.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:30 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:41:31.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:31 smithi103 ceph-mon[31201]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-09-19T07:41:31.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:31 smithi134 ceph-mon[35414]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-09-19T07:41:31.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:31 smithi174 ceph-mon[35370]: pgmap v109: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-09-19T07:41:33.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:33 smithi103 ceph-mon[31201]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-09-19T07:41:33.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:33 smithi134 ceph-mon[35414]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-09-19T07:41:33.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:33 smithi174 ceph-mon[35370]: pgmap v110: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2024-09-19T07:41:35.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 180.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:41:35.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:35 smithi103 ceph-mon[31201]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 612 op/s 2024-09-19T07:41:35.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:35 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:35.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:35 smithi134 ceph-mon[35414]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 612 op/s 2024-09-19T07:41:35.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:35 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:35.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:35 smithi174 ceph-mon[35370]: pgmap v111: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 612 op/s 2024-09-19T07:41:35.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:35 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:37.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:37 smithi103 ceph-mon[31201]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T07:41:37.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:37 smithi134 ceph-mon[35414]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T07:41:37.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:37 smithi174 ceph-mon[35370]: pgmap v112: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T07:41:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:39 smithi103 ceph-mon[31201]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T07:41:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:39 smithi134 ceph-mon[35414]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T07:41:39.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:39 smithi174 ceph-mon[35370]: pgmap v113: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-09-19T07:41:40.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 185.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:41:40.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:40 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:40.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:40 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:40.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:40 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:41.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:41 smithi134 ceph-mon[35414]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.5 MiB/s wr, 335 op/s 2024-09-19T07:41:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:41 smithi174 ceph-mon[35370]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.5 MiB/s wr, 335 op/s 2024-09-19T07:41:41.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:41 smithi103 ceph-mon[31201]: pgmap v114: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.5 MiB/s wr, 335 op/s 2024-09-19T07:41:42.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:41:42.403 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 -- 172.21.15.103:0/2877897536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600101420 msgr2=0x7fc60010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 --2- 172.21.15.103:0/2877897536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600101420 0x7fc60010d950 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009cd0 tx=0x7fc5f802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 -- 172.21.15.103:0/2877897536 shutdown_connections 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 --2- 172.21.15.103:0/2877897536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600101420 0x7fc60010d950 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 --2- 172.21.15.103:0/2877897536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc600100a80 0x7fc600100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 --2- 172.21.15.103:0/2877897536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc600106a80 0x7fc600106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 -- 172.21.15.103:0/2877897536 >> 172.21.15.103:0/2877897536 conn(0x7fc6000fc7a0 msgr2=0x7fc6000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 -- 172.21.15.103:0/2877897536 shutdown_connections 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.241+0000 7fc6070f1640 1 -- 172.21.15.103:0/2877897536 wait complete. 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 Processor -- start 2024-09-19T07:41:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 -- start start 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc600100a80 0x7fc600198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc600101420 0x7fc600198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600106a80 0x7fc60019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc600110f20 con 0x7fc600106a80 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc600110da0 con 0x7fc600100a80 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc6070f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001110a0 con 0x7fc600101420 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600106a80 0x7fc60019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600106a80 0x7fc60019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50312/0 (socket says 172.21.15.103:50312) 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 -- 172.21.15.103:0/1335549489 learned_addr learned my addr 172.21.15.103:0/1335549489 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc604e8d640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc600100a80 0x7fc600198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc5fffff640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc600101420 0x7fc600198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 -- 172.21.15.103:0/1335549489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc600101420 msgr2=0x7fc600198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc600101420 0x7fc600198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 -- 172.21.15.103:0/1335549489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc600100a80 msgr2=0x7fc600198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc600100a80 0x7fc600198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.242+0000 7fc60568e640 1 -- 172.21.15.103:0/1335549489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60019d4c0 con 0x7fc600106a80 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.243+0000 7fc60568e640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600106a80 0x7fc60019cde0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8006010 tx=0x7fc5f8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.243+0000 7fc5fdffb640 1 -- 172.21.15.103:0/1335549489 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8004450 con 0x7fc600106a80 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.243+0000 7fc5fdffb640 1 -- 172.21.15.103:0/1335549489 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8003e10 con 0x7fc600106a80 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.243+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60019d750 con 0x7fc600106a80 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.243+0000 7fc5fdffb640 1 -- 172.21.15.103:0/1335549489 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8046480 con 0x7fc600106a80 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.243+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6001a5090 con 0x7fc600106a80 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.244+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc600100f60 con 0x7fc600106a80 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.245+0000 7fc5fdffb640 1 -- 172.21.15.103:0/1335549489 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc5f80045f0 con 0x7fc600106a80 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.246+0000 7fc5fdffb640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc5f00a3430 0x7fc5f00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.246+0000 7fc604e8d640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc5f00a3430 0x7fc5f00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.246+0000 7fc5fdffb640 1 -- 172.21.15.103:0/1335549489 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc5f802e030 con 0x7fc600106a80 2024-09-19T07:41:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.246+0000 7fc604e8d640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc5f00a3430 0x7fc5f00a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009990 tx=0x7fc5f4009110 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.250+0000 7fc5fdffb640 1 -- 172.21.15.103:0/1335549489 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc5f804b060 con 0x7fc600106a80 2024-09-19T07:41:43.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.453+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc60002d7c0 con 0x7fc600106a80 2024-09-19T07:41:43.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.454+0000 7fc5fdffb640 1 -- 172.21.15.103:0/1335549489 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11997 (secure 0 0 0) 0x7fc5f8039380 con 0x7fc600106a80 2024-09-19T07:41:43.455 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:41:43.456 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":34,"btime":"2024-09-19T07:41:39:459221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:41:38.519254+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:41:43.456 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 34 2024-09-19T07:41:43.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.456+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc5f00a3430 msgr2=0x7fc5f00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:43.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.456+0000 7fc6070f1640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc5f00a3430 0x7fc5f00a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4009990 tx=0x7fc5f4009110 comp rx=0 tx=0).stop 2024-09-19T07:41:43.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.456+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600106a80 msgr2=0x7fc60019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.456+0000 7fc6070f1640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600106a80 0x7fc60019cde0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8006010 tx=0x7fc5f8003c20 comp rx=0 tx=0).stop 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 shutdown_connections 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc5f00a3430 0x7fc5f00a58f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc600106a80 0x7fc60019cde0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc600101420 0x7fc600198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 --2- 172.21.15.103:0/1335549489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc600100a80 0x7fc600198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 >> 172.21.15.103:0/1335549489 conn(0x7fc6000fc7a0 msgr2=0x7fc60006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 shutdown_connections 2024-09-19T07:41:43.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:43.457+0000 7fc6070f1640 1 -- 172.21.15.103:0/1335549489 wait complete. 2024-09-19T07:41:43.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:43 smithi103 ceph-mon[31201]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 723 KiB/s wr, 39 op/s 2024-09-19T07:41:43.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:43 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1335549489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:43.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:43 smithi134 ceph-mon[35414]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 723 KiB/s wr, 39 op/s 2024-09-19T07:41:43.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:43 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1335549489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:43 smithi174 ceph-mon[35370]: pgmap v115: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 723 KiB/s wr, 39 op/s 2024-09-19T07:41:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:43 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1335549489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:41:44.260 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 99f1493b --release --await' 2024-09-19T07:41:44.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 99f1493b --release --await 2024-09-19T07:41:44.510 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.989+0000 7f3b2f893640 1 -- 172.21.15.103:0/862892936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28102790 msgr2=0x7f3b281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.989+0000 7f3b2f893640 1 --2- 172.21.15.103:0/862892936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28102790 0x7f3b281067e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009900 tx=0x7f3b1802a410 comp rx=0 tx=0).stop 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.989+0000 7f3b2f893640 1 -- 172.21.15.103:0/862892936 shutdown_connections 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.989+0000 7f3b2f893640 1 --2- 172.21.15.103:0/862892936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b28106f10 0x7f3b2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.989+0000 7f3b2f893640 1 --2- 172.21.15.103:0/862892936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28102790 0x7f3b281067e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.989+0000 7f3b2f893640 1 --2- 172.21.15.103:0/862892936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b28101de0 0x7f3b281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.989+0000 7f3b2f893640 1 -- 172.21.15.103:0/862892936 >> 172.21.15.103:0/862892936 conn(0x7f3b280fd8c0 msgr2=0x7f3b280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:45.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 -- 172.21.15.103:0/862892936 shutdown_connections 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 -- 172.21.15.103:0/862892936 wait complete. 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 Processor -- start 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 -- start start 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b28101de0 0x7f3b2819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b28102790 0x7f3b2819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28106f10 0x7f3b2819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28111f30 con 0x7f3b28106f10 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28111db0 con 0x7f3b28102790 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2f893640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281120b0 con 0x7f3b28101de0 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2de30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28106f10 0x7f3b2819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2de30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28106f10 0x7f3b2819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50328/0 (socket says 172.21.15.103:50328) 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.990+0000 7f3b2de30640 1 -- 172.21.15.103:0/2942149163 learned_addr learned my addr 172.21.15.103:0/2942149163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b28102790 0x7f3b2819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b28101de0 0x7f3b2819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2de30640 1 -- 172.21.15.103:0/2942149163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b28101de0 msgr2=0x7f3b2819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2de30640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b28101de0 0x7f3b2819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2de30640 1 -- 172.21.15.103:0/2942149163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b28102790 msgr2=0x7f3b2819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2de30640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b28102790 0x7f3b2819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2de30640 1 -- 172.21.15.103:0/2942149163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2819f5e0 con 0x7f3b28106f10 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b28101de0 0x7f3b2819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b28102790 0x7f3b2819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2de30640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28106f10 0x7f3b2819ef00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000cad0 tx=0x7f3b2000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b1e7fc640 1 -- 172.21.15.103:0/2942149163 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000ae20 con 0x7f3b28106f10 2024-09-19T07:41:45.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b1e7fc640 1 -- 172.21.15.103:0/2942149163 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b20005020 con 0x7f3b28106f10 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b1e7fc640 1 -- 172.21.15.103:0/2942149163 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2000f6f0 con 0x7f3b28106f10 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2819f8d0 con 0x7f3b28106f10 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.991+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b281a71b0 con 0x7f3b28106f10 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.993+0000 7f3b1e7fc640 1 -- 172.21.15.103:0/2942149163 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3b200051d0 con 0x7f3b28106f10 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 Processor -- start 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 -- start start 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178060 0x55ba5d178440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178980 0x55ba5d182d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d185630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d0dd620 con 0x55ba5d183240 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d084e50 con 0x55ba5d178980 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d0d95e0 con 0x55ba5d178060 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b1e7fc640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b140a33b0 0x7f3b140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d185630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d185630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50338/0 (socket says 172.21.15.103:50338) 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/402582823 learned_addr learned my addr 172.21.15.103:0/402582823 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178980 0x55ba5d182d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b2de30640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b140a33b0 0x7f3b140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.994+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178060 0x55ba5d178440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b1e7fc640 1 -- 172.21.15.103:0/2942149163 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3b200cfdf0 con 0x7f3b28106f10 2024-09-19T07:41:45.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/402582823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178060 msgr2=0x55ba5d178440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178060 0x55ba5d178440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/402582823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178980 msgr2=0x55ba5d182d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178980 0x55ba5d182d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/402582823 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba5d075d40 con 0x55ba5d183240 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178980 0x55ba5d182d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d185630 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b1802a930 tx=0x7f3b18031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425ec0692bdaa407 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b2de30640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b140a33b0 0x7f3b140a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b200cdb50 tx=0x7f3b20005960 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b137fe640 1 -- 172.21.15.103:0/402582823 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18035040 con 0x55ba5d183240 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b137fe640 1 -- 172.21.15.103:0/402582823 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b180319f0 con 0x55ba5d183240 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.995+0000 7f3b137fe640 1 -- 172.21.15.103:0/402582823 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180049a0 con 0x55ba5d183240 2024-09-19T07:41:45.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 -- 172.21.15.103:0/402582823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 msgr2=0x55ba5d185630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d185630 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b1802a930 tx=0x7f3b18031400 comp rx=0 tx=0).stop 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 -- 172.21.15.103:0/402582823 shutdown_connections 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d185630 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178980 0x55ba5d182d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/402582823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178060 0x55ba5d178440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 -- 172.21.15.103:0/402582823 >> 172.21.15.103:0/402582823 conn(0x55ba5d1745f0 msgr2=0x55ba5d1749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 -- 172.21.15.103:0/402582823 shutdown_connections 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 -- 172.21.15.103:0/402582823 wait complete. 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 -- start start 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.996+0000 7f3b31c6c740 1 Processor -- start 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 0x55ba5d180140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d180680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d180bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d185e10 con 0x55ba5d183240 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d065f10 con 0x55ba5d178060 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba5d185f90 con 0x55ba5d178980 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b31c6c740 10 client.0 _ll_register_callbacks cb 0x55ba5d089db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d180bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d180bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50348/0 (socket says 172.21.15.103:50348) 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3531316023 learned_addr learned my addr 172.21.15.103:0/3531316023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:45.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d180680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 0x55ba5d180140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3531316023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 msgr2=0x55ba5d180680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d180680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 msgr2=0x55ba5d180140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 0x55ba5d180140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3531316023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba5cdec580 con 0x55ba5d183240 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d180bc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b180045b0 tx=0x7f3b180044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eebb98ef01927e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.997+0000 7f3b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3531316023 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18000e60 con 0x55ba5d183240 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3531316023 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3b180049b0 con 0x55ba5d183240 2024-09-19T07:41:45.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b180319f0 con 0x55ba5d183240 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3531316023 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b1805c050 con 0x55ba5d183240 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 10 client.16254 fetch_fsmap learned FSMap version 35 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba5ceb6960 con 0x55ba5d183240 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3531316023 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b180319f0 con 0x55ba5d183240 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3531316023 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f3b1803a040 con 0x55ba5d183240 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 10 client.16254 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 10 client.16254 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 20 client.16254 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d192c50 0x55ba5d195040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 4 client.16254 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba5d0eab90 con 0x55ba5d192c50 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.998+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d192c50 0x55ba5d195040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.999+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d192c50 0x55ba5d195040 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3dae2ac5f7da31 server_cookie=6ef7d0b3e3702a1a in_seq=0 out_seq=0 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:44.999+0000 7f3b10ff9640 10 client.16254 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3531316023 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba5d0eab90 con 0x55ba5d192c50 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b10ff9640 10 client.16254 handle_command_reply: tid=0 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 client.16254 shutdown 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 msgr2=0x55ba5d180bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d180bc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b180045b0 tx=0x7f3b180044b0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 shutdown_connections 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d192c50 0x55ba5d195040 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d180bc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d180680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b10ff9640 0 client.16254 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3531316023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 0x55ba5d180140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 >> 172.21.15.103:0/3531316023 conn(0x55ba5d1745f0 msgr2=0x55ba5d17a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 shutdown_connections 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3531316023 wait complete. 2024-09-19T07:41:45.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.000+0000 7f3b31c6c740 20 client.16254 trim_cache size 0 max 16384 2024-09-19T07:41:45.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 Processor -- start 2024-09-19T07:41:45.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 -- start start 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d178060 0x55ba5d198060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d183240 0x55ba5d17d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281af9c0 con 0x55ba5d178060 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281b1590 con 0x55ba5d183240 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281b0090 con 0x55ba5d178980 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d178060 0x55ba5d198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d178060 0x55ba5d198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50358/0 (socket says 172.21.15.103:50358) 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/90540302 learned_addr learned my addr 172.21.15.103:0/90540302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d183240 0x55ba5d17d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/90540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 msgr2=0x55ba5d1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/90540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d183240 msgr2=0x55ba5d17d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d183240 0x55ba5d17d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/90540302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba5d175280 con 0x55ba5d178060 2024-09-19T07:41:45.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.018+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d178060 0x55ba5d198060 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55ba5d181500 tx=0x7f3b18005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adc3dd0174bfd63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:45.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b12ffd640 1 -- 172.21.15.103:0/90540302 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18035040 con 0x55ba5d178060 2024-09-19T07:41:45.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b12ffd640 1 -- 172.21.15.103:0/90540302 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b180316b0 con 0x55ba5d178060 2024-09-19T07:41:45.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b12ffd640 1 -- 172.21.15.103:0/90540302 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1803d6e0 con 0x55ba5d178060 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b31c6c740 1 -- 172.21.15.103:0/90540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d178060 msgr2=0x55ba5d198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d178060 0x55ba5d198060 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55ba5d181500 tx=0x7f3b18005990 comp rx=0 tx=0).stop 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b31c6c740 1 -- 172.21.15.103:0/90540302 shutdown_connections 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d183240 0x55ba5d17d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/90540302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d178060 0x55ba5d198060 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.019+0000 7f3b31c6c740 1 -- 172.21.15.103:0/90540302 >> 172.21.15.103:0/90540302 conn(0x55ba5d1745f0 msgr2=0x55ba5d17a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 -- 172.21.15.103:0/90540302 shutdown_connections 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 -- 172.21.15.103:0/90540302 wait complete. 2024-09-19T07:41:45.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 -- start start 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 Processor -- start 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 0x55ba5d1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d1fc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281b0ec0 con 0x55ba5d183240 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281b0760 con 0x55ba5d178060 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281ad9a0 con 0x55ba5d178980 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b31c6c740 10 client.0 _ll_register_callbacks cb 0x7f3b18031940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d1fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b2ce2e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d1fc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50364/0 (socket says 172.21.15.103:50364) 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.020+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3287182607 learned_addr learned my addr 172.21.15.103:0/3287182607 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3287182607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 msgr2=0x55ba5d1969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3287182607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 msgr2=0x55ba5d1964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 0x55ba5d1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b2ce2e640 1 -- 172.21.15.103:0/3287182607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba5d1fcad0 con 0x55ba5d183240 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b2ce2e640 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d1fc590 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006010 tx=0x7f3b180045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fcbe5dde7b91d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3287182607 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18035070 con 0x55ba5d183240 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3287182607 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f3b180319f0 con 0x55ba5d183240 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b180316b0 con 0x55ba5d183240 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3287182607 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b1806d050 con 0x55ba5d183240 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b31c6c740 10 client.16260 fetch_fsmap learned FSMap version 35 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba5d1fdd70 con 0x55ba5d183240 2024-09-19T07:41:45.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.021+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3287182607 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b180316b0 con 0x55ba5d183240 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.022+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3287182607 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f3b18068030 con 0x55ba5d183240 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.022+0000 7f3b31c6c740 10 client.16260 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.022+0000 7f3b31c6c740 10 client.16260 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.022+0000 7f3b31c6c740 20 client.16260 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.022+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d1fe1f0 0x55ba5d200610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.022+0000 7f3b31c6c740 4 client.16260 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "99f1493b", "release": true, "await": true}] 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.022+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "99f1493b", "release": true, "await": true}) -- 0x55ba5d0e50a0 con 0x55ba5d1fe1f0 2024-09-19T07:41:45.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.023+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d1fe1f0 0x55ba5d200610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:45.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.024+0000 7f3b2d62f640 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d1fe1f0 0x55ba5d200610 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed284582293ec5a server_cookie=699216144589175 in_seq=0 out_seq=0 2024-09-19T07:41:45.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.024+0000 7f3b10ff9640 10 client.16260 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:45.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.047+0000 7f3b10ff9640 1 -- 172.21.15.103:0/3287182607 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ba5d0e50a0 con 0x55ba5d1fe1f0 2024-09-19T07:41:45.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.047+0000 7f3b10ff9640 10 client.16260 handle_command_reply: tid=0 2024-09-19T07:41:45.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.047+0000 7f3b31c6c740 1 client.16260 shutdown 2024-09-19T07:41:45.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 msgr2=0x55ba5d1fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d1fc590 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006010 tx=0x7f3b180045b0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 shutdown_connections 2024-09-19T07:41:45.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ba5d1fe1f0 0x55ba5d200610 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ba5d183240 0x55ba5d1fc590 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba5d178980 0x55ba5d1969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 --2- 172.21.15.103:0/3287182607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba5d178060 0x55ba5d1964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b10ff9640 0 client.16260 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:45.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.050+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 >> 172.21.15.103:0/3287182607 conn(0x55ba5d1745f0 msgr2=0x55ba5d180100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:45.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.051+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 shutdown_connections 2024-09-19T07:41:45.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.051+0000 7f3b31c6c740 1 -- 172.21.15.103:0/3287182607 wait complete. 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.052+0000 7f3b31c6c740 20 client.16260 trim_cache size 0 max 16384 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 35, 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 16, 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stdout: "99f1493b": { 2024-09-19T07:41:45.053 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 16, 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 136.6, 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:41:45.054 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:41:45.055 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b140a33b0 msgr2=0x7f3b140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b140a33b0 0x7f3b140a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b200cdb50 tx=0x7f3b20005960 comp rx=0 tx=0).stop 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28106f10 msgr2=0x7f3b2819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28106f10 0x7f3b2819ef00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000cad0 tx=0x7f3b2000cf90 comp rx=0 tx=0).stop 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 shutdown_connections 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b140a33b0 0x7f3b140a5870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b28106f10 0x7f3b2819ef00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b28102790 0x7f3b2819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 --2- 172.21.15.103:0/2942149163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b28101de0 0x7f3b2819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:45.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 >> 172.21.15.103:0/2942149163 conn(0x7f3b280fd8c0 msgr2=0x7f3b280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:45.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.053+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 shutdown_connections 2024-09-19T07:41:45.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:45.054+0000 7f3b2f893640 1 -- 172.21.15.103:0/2942149163 wait complete. 2024-09-19T07:41:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 190.0 s, 271.0 tps, lat 1213.309 ms stddev 4197.559 2024-09-19T07:41:45.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:45 smithi103 ceph-mon[31201]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 353 op/s 2024-09-19T07:41:45.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:45 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:45.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:45 smithi134 ceph-mon[35414]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 353 op/s 2024-09-19T07:41:45.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:45 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:45.831 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '99f1493b', total seconds elapsed: 20.3 2024-09-19T07:41:45.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:45 smithi174 ceph-mon[35370]: pgmap v116: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.3 MiB/s wr, 353 op/s 2024-09-19T07:41:45.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:45 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:41:47.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:47 smithi134 ceph-mon[35414]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 348 op/s 2024-09-19T07:41:47.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:47 smithi174 ceph-mon[35370]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 348 op/s 2024-09-19T07:41:47.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:47 smithi103 ceph-mon[31201]: pgmap v117: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 348 op/s 2024-09-19T07:41:49.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:49 smithi134 ceph-mon[35414]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 348 op/s 2024-09-19T07:41:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:49 smithi174 ceph-mon[35370]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 348 op/s 2024-09-19T07:41:49.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:49 smithi103 ceph-mon[31201]: pgmap v118: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 348 op/s 2024-09-19T07:41:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 195.0 s, 3373.0 tps, lat 29.770 ms stddev 32.457 2024-09-19T07:41:51.771 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:51 smithi103 ceph-mon[31201]: pgmap v119: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.9 MiB/s wr, 620 op/s 2024-09-19T07:41:51.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:51 smithi134 ceph-mon[35414]: pgmap v119: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.9 MiB/s wr, 620 op/s 2024-09-19T07:41:51.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:51 smithi174 ceph-mon[35370]: pgmap v119: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.9 MiB/s wr, 620 op/s 2024-09-19T07:41:53.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:53 smithi134 ceph-mon[35414]: pgmap v120: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.2 MiB/s wr, 587 op/s 2024-09-19T07:41:53.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:53 smithi174 ceph-mon[35370]: pgmap v120: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.2 MiB/s wr, 587 op/s 2024-09-19T07:41:53.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:53 smithi103 ceph-mon[31201]: pgmap v120: 97 pgs: 97 active+clean; 2.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.2 MiB/s wr, 587 op/s 2024-09-19T07:41:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 200.0 s, 3507.6 tps, lat 28.402 ms stddev 29.346 2024-09-19T07:41:55.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:55 smithi134 ceph-mon[35414]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 902 op/s 2024-09-19T07:41:55.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:55 smithi174 ceph-mon[35370]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 902 op/s 2024-09-19T07:41:55.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:55 smithi103 ceph-mon[31201]: pgmap v121: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 14 MiB/s wr, 902 op/s 2024-09-19T07:41:56.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:41:56.713 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.220+0000 7f381f18a640 1 -- 172.21.15.103:0/3972310541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818102570 msgr2=0x7f38181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.220+0000 7f381f18a640 1 --2- 172.21.15.103:0/3972310541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818102570 0x7f38181029b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f380c009900 tx=0x7f380c02a480 comp rx=0 tx=0).stop 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.220+0000 7f381f18a640 1 -- 172.21.15.103:0/3972310541 shutdown_connections 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.220+0000 7f381f18a640 1 --2- 172.21.15.103:0/3972310541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818102ef0 0x7f381810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.220+0000 7f381f18a640 1 --2- 172.21.15.103:0/3972310541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818102570 0x7f38181029b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.220+0000 7f381f18a640 1 --2- 172.21.15.103:0/3972310541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818100e20 0x7f3818101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.220+0000 7f381f18a640 1 -- 172.21.15.103:0/3972310541 >> 172.21.15.103:0/3972310541 conn(0x7f38180fc7d0 msgr2=0x7f38180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:57.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.221+0000 7f381f18a640 1 -- 172.21.15.103:0/3972310541 shutdown_connections 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.221+0000 7f381f18a640 1 -- 172.21.15.103:0/3972310541 wait complete. 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 Processor -- start 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 -- start start 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818100e20 0x7f3818198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818102570 0x7f38181989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818102ef0 0x7f381819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38181120a0 con 0x7f3818100e20 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3818111f20 con 0x7f3818102570 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381f18a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3818112220 con 0x7f3818102ef0 2024-09-19T07:41:57.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381cf26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818100e20 0x7f3818198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381cf26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818100e20 0x7f3818198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48590/0 (socket says 172.21.15.103:48590) 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818102ef0 0x7f381819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818102ef0 0x7f381819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:51574/0 (socket says 172.21.15.103:51574) 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.222+0000 7f381d727640 1 -- 172.21.15.103:0/3604771447 learned_addr learned my addr 172.21.15.103:0/3604771447 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.223+0000 7f381d727640 1 -- 172.21.15.103:0/3604771447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818102570 msgr2=0x7f38181989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.223+0000 7f381d727640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818102570 0x7f38181989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.223+0000 7f381d727640 1 -- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818100e20 msgr2=0x7f3818198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.223+0000 7f381d727640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818100e20 0x7f3818198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.223+0000 7f381d727640 1 -- 172.21.15.103:0/3604771447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381819d420 con 0x7f3818102ef0 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.223+0000 7f381d727640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818102ef0 0x7f381819cd40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3810007f30 tx=0x7f381000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.224+0000 7f3815ffb640 1 -- 172.21.15.103:0/3604771447 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810020460 con 0x7f3818102ef0 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.224+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381819d710 con 0x7f3818102ef0 2024-09-19T07:41:57.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.224+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381806bc10 con 0x7f3818102ef0 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.224+0000 7f3815ffb640 1 -- 172.21.15.103:0/3604771447 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38100328d0 con 0x7f3818102ef0 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.224+0000 7f3815ffb640 1 -- 172.21.15.103:0/3604771447 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381001f830 con 0x7f3818102ef0 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.227+0000 7f3815ffb640 1 -- 172.21.15.103:0/3604771447 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3810032a80 con 0x7f3818102ef0 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.228+0000 7f3815ffb640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f38000a33b0 0x7f38000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f381cf26640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f38000a33b0 0x7f38000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3815ffb640 1 -- 172.21.15.103:0/3604771447 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f38100e11e0 con 0x7f3818102ef0 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 Processor -- start 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f381cf26640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f38000a33b0 0x7f38000a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f38080058c0 tx=0x7f380800a2d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 -- start start 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed012d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0138140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0138680 0x557ed013aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ed00932e0 con 0x557ed012d4a0 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557ed003ace0 con 0x557ed0138680 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3821563740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ed008f2a0 con 0x557ed012ddc0 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f381cf26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed012d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f381cf26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed012d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48594/0 (socket says 172.21.15.103:48594) 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f381cf26640 1 -- 172.21.15.103:0/60355428 learned_addr learned my addr 172.21.15.103:0/60355428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f381d727640 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0138680 0x557ed013aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.229+0000 7f3817fff640 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f381cf26640 1 -- 172.21.15.103:0/60355428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 msgr2=0x557ed0138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f381cf26640 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0138140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f381cf26640 1 -- 172.21.15.103:0/60355428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0138680 msgr2=0x557ed013aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f381cf26640 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0138680 0x557ed013aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f381cf26640 1 -- 172.21.15.103:0/60355428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed001f440 con 0x557ed012d4a0 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f381cf26640 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed012d880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f380800ff40 tx=0x7f3808012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de60192f74ae0ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f37feffd640 1 -- 172.21.15.103:0/60355428 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808016070 con 0x557ed012d4a0 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f37feffd640 1 -- 172.21.15.103:0/60355428 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380801b900 con 0x557ed012d4a0 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.230+0000 7f37feffd640 1 -- 172.21.15.103:0/60355428 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380801caa0 con 0x557ed012d4a0 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 -- 172.21.15.103:0/60355428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 msgr2=0x557ed012d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed012d880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f380800ff40 tx=0x7f3808012000 comp rx=0 tx=0).stop 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 -- 172.21.15.103:0/60355428 shutdown_connections 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0138680 0x557ed013aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 --2- 172.21.15.103:0/60355428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed012d880 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 -- 172.21.15.103:0/60355428 >> 172.21.15.103:0/60355428 conn(0x557ed0129a30 msgr2=0x557ed0129e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 -- 172.21.15.103:0/60355428 shutdown_connections 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.231+0000 7f3821563740 1 -- 172.21.15.103:0/60355428 wait complete. 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 -- start start 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 Processor -- start 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed0137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012ddc0 0x557ed0135250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed0138680 0x557ed0135790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ed013b250 con 0x557ed012d4a0 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557ed004e050 con 0x557ed012ddc0 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ed013b3d0 con 0x557ed0138680 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.232+0000 7f3821563740 10 client.0 _ll_register_callbacks cb 0x557ed008e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:57.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f3817fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012ddc0 0x557ed0135250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed0138680 0x557ed0135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381cf26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed0137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed0138680 0x557ed0135790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:51588/0 (socket says 172.21.15.103:51588) 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 -- 172.21.15.103:0/948727485 learned_addr learned my addr 172.21.15.103:0/948727485 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f3817fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012ddc0 0x557ed0135250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:36234/0 (socket says 172.21.15.103:36234) 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381cf26640 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed0137fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48602/0 (socket says 172.21.15.103:48602) 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 -- 172.21.15.103:0/948727485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012ddc0 msgr2=0x557ed0135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012ddc0 0x557ed0135250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 -- 172.21.15.103:0/948727485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 msgr2=0x557ed0137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed0137fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 -- 172.21.15.103:0/948727485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ecfe043c0 con 0x557ed0138680 2024-09-19T07:41:57.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f381d727640 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed0138680 0x557ed0135790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f38100ad1d0 tx=0x7f38100adc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5e56705b80b45a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f37f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f37f7fff640 1 -- 172.21.15.103:0/948727485 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38100ca020 con 0x557ed0138680 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.233+0000 7f37f7fff640 1 -- 172.21.15.103:0/948727485 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f38100b5eb0 con 0x557ed0138680 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.234+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f380801b900 con 0x557ed0138680 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.234+0000 7f37f7fff640 1 -- 172.21.15.103:0/948727485 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f38100b6140 con 0x557ed0138680 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.234+0000 7f3821563740 10 client.25735 fetch_fsmap learned FSMap version 35 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.234+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ed0146c30 con 0x557ed0138680 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.234+0000 7f37f7fff640 1 -- 172.21.15.103:0/948727485 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380801b900 con 0x557ed0138680 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.234+0000 7f37f7fff640 1 -- 172.21.15.103:0/948727485 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f38100a9220 con 0x557ed0138680 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.235+0000 7f3821563740 10 client.25735 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.235+0000 7f3821563740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.235+0000 7f3821563740 20 client.25735 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:57.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.235+0000 7f3821563740 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed01480f0 0x557ed014a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.235+0000 7f3821563740 4 client.25735 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:41:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.235+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed00a0850 con 0x557ed01480f0 2024-09-19T07:41:57.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.235+0000 7f381cf26640 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed01480f0 0x557ed014a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.237+0000 7f37f7fff640 10 client.25735 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:57.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.237+0000 7f381cf26640 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed01480f0 0x557ed014a4e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9676541b8b40c854 server_cookie=ebe99e7e3da60e8b in_seq=0 out_seq=0 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f37f7fff640 1 -- 172.21.15.103:0/948727485 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ed00a0850 con 0x557ed01480f0 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f37f7fff640 10 client.25735 handle_command_reply: tid=0 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 client.25735 shutdown 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed0138680 msgr2=0x557ed0135790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed0138680 0x557ed0135790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f38100ad1d0 tx=0x7f38100adc50 comp rx=0 tx=0).stop 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 shutdown_connections 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed01480f0 0x557ed014a4e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed0138680 0x557ed0135790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f37f7fff640 0 client.25735 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012ddc0 0x557ed0135250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 --2- 172.21.15.103:0/948727485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed0137fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 >> 172.21.15.103:0/948727485 conn(0x557ed0129a30 msgr2=0x557ed012f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:57.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 shutdown_connections 2024-09-19T07:41:57.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 1 -- 172.21.15.103:0/948727485 wait complete. 2024-09-19T07:41:57.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.238+0000 7f3821563740 20 client.25735 trim_cache size 0 max 16384 2024-09-19T07:41:57.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.261+0000 7f3821563740 1 Processor -- start 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.261+0000 7f3821563740 1 -- start start 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f3821563740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0129a30 0x557ed01371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f3821563740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f3821563740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0150250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f3821563740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3818072310 con 0x557ed012d4a0 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f3821563740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3818073ee0 con 0x557ed0129a30 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f3821563740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38180729e0 con 0x557ed012ddc0 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381d727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:51598/0 (socket says 172.21.15.103:51598) 2024-09-19T07:41:57.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381d727640 1 -- 172.21.15.103:0/3151427773 learned_addr learned my addr 172.21.15.103:0/3151427773 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:57.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381cf26640 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0129a30 0x557ed01371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f3817fff640 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381cf26640 1 -- 172.21.15.103:0/3151427773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 msgr2=0x557ed0150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381cf26640 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381cf26640 1 -- 172.21.15.103:0/3151427773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 msgr2=0x557ed014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381cf26640 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed014fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.262+0000 7f381cf26640 1 -- 172.21.15.103:0/3151427773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed0138310 con 0x557ed0129a30 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.263+0000 7f3817fff640 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:41:57.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.263+0000 7f381cf26640 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0129a30 0x557ed01371c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f380800f790 tx=0x7f380801c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc00f53415e43ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.263+0000 7f37fe7fc640 1 -- 172.21.15.103:0/3151427773 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808016070 con 0x557ed0129a30 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.263+0000 7f37fe7fc640 1 -- 172.21.15.103:0/3151427773 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380801c770 con 0x557ed0129a30 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.263+0000 7f37fe7fc640 1 -- 172.21.15.103:0/3151427773 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380802a040 con 0x557ed0129a30 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 -- 172.21.15.103:0/3151427773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0129a30 msgr2=0x557ed01371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0129a30 0x557ed01371c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f380800f790 tx=0x7f380801c640 comp rx=0 tx=0).stop 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 -- 172.21.15.103:0/3151427773 shutdown_connections 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed0150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed012d4a0 0x557ed014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 --2- 172.21.15.103:0/3151427773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed0129a30 0x557ed01371c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 -- 172.21.15.103:0/3151427773 >> 172.21.15.103:0/3151427773 conn(0x557ed01358f0 msgr2=0x557ed0136ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 -- 172.21.15.103:0/3151427773 shutdown_connections 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 -- 172.21.15.103:0/3151427773 wait complete. 2024-09-19T07:41:57.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 -- start start 2024-09-19T07:41:57.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.264+0000 7f3821563740 1 Processor -- start 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3821563740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed0129a30 0x557ed0140a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3821563740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012d4a0 0x557ed013ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3821563740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed013e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3821563740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3818073810 con 0x557ed0129a30 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3821563740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38180730b0 con 0x557ed012d4a0 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3821563740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38180702f0 con 0x557ed012ddc0 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3821563740 10 client.0 _ll_register_callbacks cb 0x7f3810007e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:41:57.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381cf26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed0129a30 0x557ed0140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381cf26640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed0129a30 0x557ed0140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48620/0 (socket says 172.21.15.103:48620) 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381cf26640 1 -- 172.21.15.103:0/3703930772 learned_addr learned my addr 172.21.15.103:0/3703930772 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381d727640 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed013e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f3817fff640 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012d4a0 0x557ed013ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381d727640 1 -- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012d4a0 msgr2=0x557ed013ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381d727640 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012d4a0 0x557ed013ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381d727640 1 -- 172.21.15.103:0/3703930772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed0129a30 msgr2=0x557ed0140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381d727640 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed0129a30 0x557ed0140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.265+0000 7f381d727640 1 -- 172.21.15.103:0/3703930772 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed013e830 con 0x557ed012ddc0 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.266+0000 7f381d727640 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed013e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38100a9ea0 tx=0x7f38100b5ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5d4a637a0a3419 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.266+0000 7f37f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.266+0000 7f37f7fff640 1 -- 172.21.15.103:0/3703930772 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38100ca040 con 0x557ed012ddc0 2024-09-19T07:41:57.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.266+0000 7f37f7fff640 1 -- 172.21.15.103:0/3703930772 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f38100782c0 con 0x557ed012ddc0 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.266+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f380801c770 con 0x557ed012ddc0 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f37f7fff640 1 -- 172.21.15.103:0/3703930772 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f38100ace30 con 0x557ed012ddc0 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 10 client.25741 fetch_fsmap learned FSMap version 35 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ed01b2960 con 0x557ed012ddc0 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f37f7fff640 1 -- 172.21.15.103:0/3703930772 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380801c770 con 0x557ed012ddc0 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f37f7fff640 1 -- 172.21.15.103:0/3703930772 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f38100b2b30 con 0x557ed012ddc0 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 10 client.25741 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 10 client.25741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 20 client.25741 populate_metadata read hostname 'smithi103' 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed0138680 0x557ed01b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 4 client.25741 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.267+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ed009ad60 con 0x557ed0138680 2024-09-19T07:41:57.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f381cf26640 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed0138680 0x557ed01b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f37f7fff640 10 client.25741 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f381cf26640 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed0138680 0x557ed01b5090 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06a281951376605 server_cookie=81c9f4e9dcb698ab in_seq=0 out_seq=0 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f37f7fff640 1 -- 172.21.15.103:0/3703930772 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557ed009ad60 con 0x557ed0138680 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f37f7fff640 10 client.25741 handle_command_reply: tid=0 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f3821563740 1 client.25741 shutdown 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 msgr2=0x557ed013e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.268+0000 7f3821563740 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed013e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38100a9ea0 tx=0x7f38100b5ee0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 shutdown_connections 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557ed0138680 0x557ed01b5090 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed012ddc0 0x557ed013e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ed012d4a0 0x557ed013ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 --2- 172.21.15.103:0/3703930772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed0129a30 0x557ed0140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 >> 172.21.15.103:0/3703930772 conn(0x557ed01358f0 msgr2=0x557ed0128350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:57.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f37f7fff640 0 client.25741 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:41:57.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 shutdown_connections 2024-09-19T07:41:57.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 1 -- 172.21.15.103:0/3703930772 wait complete. 2024-09-19T07:41:57.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.269+0000 7f3821563740 20 client.25741 trim_cache size 0 max 16384 2024-09-19T07:41:57.273 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:41:57.274 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:41:57.274 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:41:57.274 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:41:57.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f38000a33b0 msgr2=0x7f38000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f38000a33b0 0x7f38000a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f38080058c0 tx=0x7f380800a2d0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818102ef0 msgr2=0x7f381819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818102ef0 0x7f381819cd40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3810007f30 tx=0x7f381000a620 comp rx=0 tx=0).stop 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 shutdown_connections 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f38000a33b0 0x7f38000a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3818102ef0 0x7f381819cd40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3818102570 0x7f38181989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 --2- 172.21.15.103:0/3604771447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3818100e20 0x7f3818198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.270+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 >> 172.21.15.103:0/3604771447 conn(0x7f38180fc7d0 msgr2=0x7f3818106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.271+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 shutdown_connections 2024-09-19T07:41:57.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:57.271+0000 7f381f18a640 1 -- 172.21.15.103:0/3604771447 wait complete. 2024-09-19T07:41:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:57 smithi134 ceph-mon[35414]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-09-19T07:41:57.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:57 smithi103 ceph-mon[31201]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-09-19T07:41:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:57 smithi174 ceph-mon[35370]: pgmap v122: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-09-19T07:41:58.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:41:58.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:41:58.426 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:41:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 -- 172.21.15.103:0/14926883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc112bc0 msgr2=0x7f9afc114fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:59.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 --2- 172.21.15.103:0/14926883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc112bc0 0x7f9afc114fb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009be0 tx=0x7f9af402a6b0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 -- 172.21.15.103:0/14926883 shutdown_connections 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 --2- 172.21.15.103:0/14926883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc112bc0 0x7f9afc114fb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 --2- 172.21.15.103:0/14926883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9afc074eb0 0x7f9afc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 --2- 172.21.15.103:0/14926883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc074590 0x7f9afc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 -- 172.21.15.103:0/14926883 >> 172.21.15.103:0/14926883 conn(0x7f9afc0fd8d0 msgr2=0x7f9afc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 -- 172.21.15.103:0/14926883 shutdown_connections 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.289+0000 7f9b024d1640 1 -- 172.21.15.103:0/14926883 wait complete. 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 Processor -- start 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 -- start start 2024-09-19T07:41:59.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc074590 0x7f9afc1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc074eb0 0x7f9afc1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9afc112bc0 0x7f9afc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc1185a0 con 0x7f9afc074590 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc118420 con 0x7f9afc112bc0 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9b024d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc118720 con 0x7f9afc074eb0 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9afbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc074590 0x7f9afc1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9afbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc074590 0x7f9afc1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40218/0 (socket says 172.21.15.103:40218) 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9afbfff640 1 -- 172.21.15.103:0/3184669386 learned_addr learned my addr 172.21.15.103:0/3184669386 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.290+0000 7f9afb7fe640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc074eb0 0x7f9afc1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9b00a6e640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9afc112bc0 0x7f9afc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9afbfff640 1 -- 172.21.15.103:0/3184669386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc074eb0 msgr2=0x7f9afc1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:59.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9afbfff640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc074eb0 0x7f9afc1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9afbfff640 1 -- 172.21.15.103:0/3184669386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9afc112bc0 msgr2=0x7f9afc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9afbfff640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9afc112bc0 0x7f9afc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9afbfff640 1 -- 172.21.15.103:0/3184669386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc19efa0 con 0x7f9afc074590 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9afb7fe640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc074eb0 0x7f9afc1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9afbfff640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc074590 0x7f9afc1a46b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e9c0 tx=0x7f9aec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9af97fa640 1 -- 172.21.15.103:0/3184669386 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00ccc0 con 0x7f9afc074590 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9af97fa640 1 -- 172.21.15.103:0/3184669386 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec005500 con 0x7f9afc074590 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9afc19f290 con 0x7f9afc074590 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9af97fa640 1 -- 172.21.15.103:0/3184669386 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec010430 con 0x7f9afc074590 2024-09-19T07:41:59.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.291+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9afc1ab580 con 0x7f9afc074590 2024-09-19T07:41:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.293+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9afc074970 con 0x7f9afc074590 2024-09-19T07:41:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.294+0000 7f9af97fa640 1 -- 172.21.15.103:0/3184669386 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9aec0046c0 con 0x7f9afc074590 2024-09-19T07:41:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.295+0000 7f9af97fa640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ae00a3430 0x7f9ae00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:41:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.295+0000 7f9afb7fe640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ae00a3430 0x7f9ae00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:41:59.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.295+0000 7f9af97fa640 1 -- 172.21.15.103:0/3184669386 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9aec0d0070 con 0x7f9afc074590 2024-09-19T07:41:59.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.295+0000 7f9afb7fe640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ae00a3430 0x7f9ae00a58f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0ffed0 tx=0x7f9ae8005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:41:59.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.302+0000 7f9af97fa640 1 -- 172.21.15.103:0/3184669386 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9aec09ca00 con 0x7f9afc074590 2024-09-19T07:41:59.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.602+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9afc19fda0 con 0x7f9afc074590 2024-09-19T07:41:59.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.603+0000 7f9af97fa640 1 -- 172.21.15.103:0/3184669386 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11996 (secure 0 0 0) 0x7f9aec09c300 con 0x7f9afc074590 2024-09-19T07:41:59.603 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:41:59.603 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":35,"btime":"2024-09-19T07:41:44:478751+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:41:43.518367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:41:59.604 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 35 2024-09-19T07:41:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ae00a3430 msgr2=0x7f9ae00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ae00a3430 0x7f9ae00a58f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0ffed0 tx=0x7f9ae8005f90 comp rx=0 tx=0).stop 2024-09-19T07:41:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc074590 msgr2=0x7f9afc1a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:41:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc074590 0x7f9afc1a46b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e9c0 tx=0x7f9aec00ee80 comp rx=0 tx=0).stop 2024-09-19T07:41:59.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 shutdown_connections 2024-09-19T07:41:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ae00a3430 0x7f9ae00a58f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9afc112bc0 0x7f9afc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc074eb0 0x7f9afc1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 --2- 172.21.15.103:0/3184669386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9afc074590 0x7f9afc1a46b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:41:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 >> 172.21.15.103:0/3184669386 conn(0x7f9afc0fd8d0 msgr2=0x7f9afc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:41:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.605+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 shutdown_connections 2024-09-19T07:41:59.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:41:59.606+0000 7f9b024d1640 1 -- 172.21.15.103:0/3184669386 wait complete. 2024-09-19T07:41:59.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:41:59 smithi134 ceph-mon[35414]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-09-19T07:41:59.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:41:59 smithi174 ceph-mon[35370]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-09-19T07:41:59.944 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:41:59 smithi103 ceph-mon[31201]: pgmap v123: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.1 MiB/s wr, 587 op/s 2024-09-19T07:42:00.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:42:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 205.0 s, 2926.0 tps, lat 34.153 ms stddev 38.780 2024-09-19T07:42:00.719 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:00.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:00 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3184669386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:00.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:00 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3184669386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:00.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:00 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3184669386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:01.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1850723359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8101470 msgr2=0x7fe0b8101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1850723359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8101470 0x7fe0b8101850 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009900 tx=0x7fe0ac02a410 comp rx=0 tx=0).stop 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1850723359 shutdown_connections 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1850723359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0b8105de0 0x7fe0b810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1850723359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101e20 0x7fe0b81056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1850723359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8101470 0x7fe0b8101850 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1850723359 >> 172.21.15.103:0/1850723359 conn(0x7fe0b8077230 msgr2=0x7fe0b80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1850723359 shutdown_connections 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.198+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1850723359 wait complete. 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 Processor -- start 2024-09-19T07:42:01.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 -- start start 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101470 0x7fe0b81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0b8101e20 0x7fe0b81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8105de0 0x7fe0b819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8110e40 con 0x7fe0b8105de0 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8110cc0 con 0x7fe0b8101470 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bf1f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8110fc0 con 0x7fe0b8101e20 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bd794640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8105de0 0x7fe0b819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101470 0x7fe0b81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bd794640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8105de0 0x7fe0b819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40244/0 (socket says 172.21.15.103:40244) 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101470 0x7fe0b81984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45810/0 (socket says 172.21.15.103:45810) 2024-09-19T07:42:01.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0bd794640 1 -- 172.21.15.103:0/1416073490 learned_addr learned my addr 172.21.15.103:0/1416073490 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.199+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0b8101e20 0x7fe0b81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bd794640 1 -- 172.21.15.103:0/1416073490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0b8101e20 msgr2=0x7fe0b81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bd794640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0b8101e20 0x7fe0b81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bd794640 1 -- 172.21.15.103:0/1416073490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101470 msgr2=0x7fe0b81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bd794640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101470 0x7fe0b81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bd794640 1 -- 172.21.15.103:0/1416073490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b819d460 con 0x7fe0b8105de0 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101470 0x7fe0b81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bd794640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8105de0 0x7fe0b819cd80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009f60 tx=0x7fe0b0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0b5ffb640 1 -- 172.21.15.103:0/1416073490 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000cd80 con 0x7fe0b8105de0 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0b5ffb640 1 -- 172.21.15.103:0/1416073490 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b0013770 con 0x7fe0b8105de0 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0b5ffb640 1 -- 172.21.15.103:0/1416073490 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0005580 con 0x7fe0b8105de0 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b819d750 con 0x7fe0b8105de0 2024-09-19T07:42:01.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.200+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b80fc630 con 0x7fe0b8105de0 2024-09-19T07:42:01.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.203+0000 7fe0b5ffb640 1 -- 172.21.15.103:0/1416073490 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe0b0013920 con 0x7fe0b8105de0 2024-09-19T07:42:01.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 Processor -- start 2024-09-19T07:42:01.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 -- start start 2024-09-19T07:42:01.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e85f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e86c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557b6e7c52f0 con 0x557b6e86a370 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557b6e76ccf0 con 0x557b6e860b40 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b6e7c12b0 con 0x557b6e85f5d0 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e85f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0b5ffb640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe0a80a33b0 0x7fe0a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e85f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45816/0 (socket says 172.21.15.103:45816) 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48054/0 (socket says 172.21.15.103:48054) 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2651511578 learned_addr learned my addr 172.21.15.103:0/2651511578 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:01.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e86c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.204+0000 7fe0b5ffb640 1 -- 172.21.15.103:0/1416073490 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe0b00d3d70 con 0x7fe0b8105de0 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bcf93640 1 -- 172.21.15.103:0/2651511578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 msgr2=0x557b6e85f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e85f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bcf93640 1 -- 172.21.15.103:0/2651511578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 msgr2=0x557b6e86c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e86c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bcf93640 1 -- 172.21.15.103:0/2651511578 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6e7513a0 con 0x557b6e85f5d0 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bd794640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe0a80a33b0 0x7fe0a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e85f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85fa10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac005940 tx=0x7fe0ac005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24521de00506eb4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0bd794640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe0a80a33b0 0x7fe0a80a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0b009cbf0 tx=0x7fe0b0027000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0a2ffd640 1 -- 172.21.15.103:0/2651511578 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac035040 con 0x557b6e85f5d0 2024-09-19T07:42:01.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0a2ffd640 1 -- 172.21.15.103:0/2651511578 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac0318b0 con 0x557b6e85f5d0 2024-09-19T07:42:01.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.205+0000 7fe0a2ffd640 1 -- 172.21.15.103:0/2651511578 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0049b0 con 0x557b6e85f5d0 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2651511578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 msgr2=0x557b6e85fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85fa10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac005940 tx=0x7fe0ac005e80 comp rx=0 tx=0).stop 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2651511578 shutdown_connections 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e86c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85fa10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2651511578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e85f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2651511578 >> 172.21.15.103:0/2651511578 conn(0x557b6e828310 msgr2=0x557b6e85bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2651511578 shutdown_connections 2024-09-19T07:42:01.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2651511578 wait complete. 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.206+0000 7fe0c15d0740 1 -- start start 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 1 Processor -- start 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e85f5d0 0x557b6e869d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e860b40 0x557b6e867040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e867650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557b6e86d010 con 0x557b6e860b40 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557b6e780060 con 0x557b6e85f5d0 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b6e86d190 con 0x557b6e86a370 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0c15d0740 10 client.0 _ll_register_callbacks cb 0x557b6e4d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e85f5d0 0x557b6e869d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e860b40 0x557b6e867040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e85f5d0 0x557b6e869d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45820/0 (socket says 172.21.15.103:45820) 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0b7fff640 1 -- 172.21.15.103:0/4100220000 learned_addr learned my addr 172.21.15.103:0/4100220000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.207+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e867650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe0b7fff640 1 -- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 msgr2=0x557b6e867650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e867650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe0b7fff640 1 -- 172.21.15.103:0/4100220000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e860b40 msgr2=0x557b6e867040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e860b40 0x557b6e867040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe0b7fff640 1 -- 172.21.15.103:0/4100220000 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6e5363c0 con 0x557b6e85f5d0 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e85f5d0 0x557b6e869d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4003670 tx=0x7fe0a4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a872176012f39bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe097fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe097fff640 1 -- 172.21.15.103:0/4100220000 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4011af0 con 0x557b6e85f5d0 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe097fff640 1 -- 172.21.15.103:0/4100220000 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe0a4011c90 con 0x557b6e85f5d0 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.208+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac0318b0 con 0x557b6e85f5d0 2024-09-19T07:42:01.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe097fff640 1 -- 172.21.15.103:0/4100220000 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe0a4005020 con 0x557b6e85f5d0 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe0c15d0740 10 client.25796 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b6e878b90 con 0x557b6e85f5d0 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe097fff640 1 -- 172.21.15.103:0/4100220000 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac0318b0 con 0x557b6e85f5d0 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe097fff640 1 -- 172.21.15.103:0/4100220000 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fe0a4004cf0 con 0x557b6e85f5d0 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe0c15d0740 10 client.25796 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe0c15d0740 10 client.25796 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe0c15d0740 20 client.25796 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.209+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e87a050 0x557b6e87c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.210+0000 7fe0c15d0740 4 client.25796 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.210+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b6e7d2860 con 0x557b6e87a050 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.210+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e87a050 0x557b6e87c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.210+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e87a050 0x557b6e87c440 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5573b80deaa3bc4c server_cookie=d6f19647c1bfb5c8 in_seq=0 out_seq=0 2024-09-19T07:42:01.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.210+0000 7fe097fff640 10 client.25796 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe097fff640 1 -- 172.21.15.103:0/4100220000 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b6e7d2860 con 0x557b6e87a050 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe097fff640 10 client.25796 handle_command_reply: tid=0 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 client.25796 shutdown 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e85f5d0 msgr2=0x557b6e869d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e85f5d0 0x557b6e869d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4003670 tx=0x7fe0a4005f90 comp rx=0 tx=0).stop 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 shutdown_connections 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e87a050 0x557b6e87c440 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e867650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e860b40 0x557b6e867040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/4100220000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e85f5d0 0x557b6e869d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 >> 172.21.15.103:0/4100220000 conn(0x557b6e828310 msgr2=0x557b6e861960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:01.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe097fff640 0 client.25796 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:01.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 shutdown_connections 2024-09-19T07:42:01.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 1 -- 172.21.15.103:0/4100220000 wait complete. 2024-09-19T07:42:01.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.211+0000 7fe0c15d0740 20 client.25796 trim_cache size 0 max 16384 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 Processor -- start 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 -- start start 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e8762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e862120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b81ab710 con 0x557b6e86a370 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b81ad2e0 con 0x557b6e860b40 2024-09-19T07:42:01.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b81abde0 con 0x557b6e85f5d0 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.226+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e862120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e8762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e862120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40288/0 (socket says 172.21.15.103:40288) 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2417372018 learned_addr learned my addr 172.21.15.103:0/2417372018 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2417372018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 msgr2=0x557b6e85be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2417372018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 msgr2=0x557b6e8762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e8762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2417372018 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b6e86a100 con 0x557b6e86a370 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e862120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40227e0 tx=0x7fe0a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9756c9582820e3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0a27fc640 1 -- 172.21.15.103:0/2417372018 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a402f040 con 0x557b6e86a370 2024-09-19T07:42:01.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0a27fc640 1 -- 172.21.15.103:0/2417372018 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a4011230 con 0x557b6e86a370 2024-09-19T07:42:01.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.227+0000 7fe0a27fc640 1 -- 172.21.15.103:0/2417372018 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4011c70 con 0x557b6e86a370 2024-09-19T07:42:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2417372018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 msgr2=0x557b6e862120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e862120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40227e0 tx=0x7fe0a4000bc0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2417372018 shutdown_connections 2024-09-19T07:42:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e86a370 0x557b6e862120 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e8762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2417372018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e85f5d0 0x557b6e85be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2417372018 >> 172.21.15.103:0/2417372018 conn(0x557b6e828310 msgr2=0x557b6e85b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2417372018 shutdown_connections 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2417372018 wait complete. 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 -- start start 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.228+0000 7fe0c15d0740 1 Processor -- start 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e85f5d0 0x557b6e872420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e872960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0c15d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e863350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b81acc10 con 0x557b6e85f5d0 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b81ac4b0 con 0x557b6e860b40 2024-09-19T07:42:01.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0c15d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b81a96f0 con 0x557b6e86a370 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0c15d0740 10 client.0 _ll_register_callbacks cb 0x7fe0a4004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e85f5d0 0x557b6e872420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e872960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0bcf93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e872960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45832/0 (socket says 172.21.15.103:45832) 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0bcf93640 1 -- 172.21.15.103:0/2608095522 learned_addr learned my addr 172.21.15.103:0/2608095522 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e863350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 msgr2=0x557b6e863350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e863350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2608095522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 msgr2=0x557b6e872960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e872960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0b7fff640 1 -- 172.21.15.103:0/2608095522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b6e873020 con 0x557b6e85f5d0 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.229+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e872960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:01.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe0b7fff640 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e85f5d0 0x557b6e872420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x557b6e857620 tx=0x7fe0a4023f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd06cfc72078454 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe097fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe097fff640 1 -- 172.21.15.103:0/2608095522 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a402f050 con 0x557b6e85f5d0 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe097fff640 1 -- 172.21.15.103:0/2608095522 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe0a40051c0 con 0x557b6e85f5d0 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a4011230 con 0x557b6e85f5d0 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe097fff640 1 -- 172.21.15.103:0/2608095522 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe0a40113e0 con 0x557b6e85f5d0 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe0c15d0740 10 client.16299 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b6e8e4210 con 0x557b6e85f5d0 2024-09-19T07:42:01.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.230+0000 7fe097fff640 1 -- 172.21.15.103:0/2608095522 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a4011230 con 0x557b6e85f5d0 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe097fff640 1 -- 172.21.15.103:0/2608095522 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fe0a4035070 con 0x557b6e85f5d0 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0c15d0740 10 client.16299 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0c15d0740 10 client.16299 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0c15d0740 20 client.16299 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e8e48d0 0x557b6e8e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0c15d0740 4 client.16299 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b6e7ccd70 con 0x557b6e8e48d0 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e8e48d0 0x557b6e8e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe0bcf93640 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e8e48d0 0x557b6e8e6cc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad503512a45a8dd0 server_cookie=b738c3a082867706 in_seq=0 out_seq=0 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.231+0000 7fe097fff640 10 client.16299 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe097fff640 1 -- 172.21.15.103:0/2608095522 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b6e7ccd70 con 0x557b6e8e48d0 2024-09-19T07:42:01.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe097fff640 10 client.16299 handle_command_reply: tid=0 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 client.16299 shutdown 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e85f5d0 msgr2=0x557b6e872420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e85f5d0 0x557b6e872420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x557b6e857620 tx=0x7fe0a4023f60 comp rx=0 tx=0).stop 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 shutdown_connections 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557b6e8e48d0 0x557b6e8e6cc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b6e86a370 0x557b6e863350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b6e860b40 0x557b6e872960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 --2- 172.21.15.103:0/2608095522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557b6e85f5d0 0x557b6e872420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 >> 172.21.15.103:0/2608095522 conn(0x557b6e828310 msgr2=0x557b6e85a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe097fff640 0 client.16299 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 shutdown_connections 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 1 -- 172.21.15.103:0/2608095522 wait complete. 2024-09-19T07:42:01.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.232+0000 7fe0c15d0740 20 client.16299 trim_cache size 0 max 16384 2024-09-19T07:42:01.238 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe0a80a33b0 msgr2=0x7fe0a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe0a80a33b0 0x7fe0a80a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0b009cbf0 tx=0x7fe0b0027000 comp rx=0 tx=0).stop 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8105de0 msgr2=0x7fe0b819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8105de0 0x7fe0b819cd80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009f60 tx=0x7fe0b0009f90 comp rx=0 tx=0).stop 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 shutdown_connections 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe0a80a33b0 0x7fe0a80a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0b8105de0 0x7fe0b819cd80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0b8101e20 0x7fe0b81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 --2- 172.21.15.103:0/1416073490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0b8101470 0x7fe0b81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 >> 172.21.15.103:0/1416073490 conn(0x7fe0b8077230 msgr2=0x7fe0b81002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:01.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 shutdown_connections 2024-09-19T07:42:01.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:01.233+0000 7fe0bf1f7640 1 -- 172.21.15.103:0/1416073490 wait complete. 2024-09-19T07:42:01.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:01 smithi134 ceph-mon[35414]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 891 op/s 2024-09-19T07:42:01.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:01 smithi174 ceph-mon[35370]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 891 op/s 2024-09-19T07:42:01.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:01 smithi103 ceph-mon[31201]: pgmap v124: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 891 op/s 2024-09-19T07:42:02.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:42:02.320 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:02.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 -- 172.21.15.103:0/2199489255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc100a90 msgr2=0x7f4cfc100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 --2- 172.21.15.103:0/2199489255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc100a90 0x7f4cfc100ef0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009930 tx=0x7f4ce402a420 comp rx=0 tx=0).stop 2024-09-19T07:42:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 -- 172.21.15.103:0/2199489255 shutdown_connections 2024-09-19T07:42:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 --2- 172.21.15.103:0/2199489255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cfc101430 0x7f4cfc10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 --2- 172.21.15.103:0/2199489255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc100a90 0x7f4cfc100ef0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 --2- 172.21.15.103:0/2199489255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc106a90 0x7f4cfc106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 -- 172.21.15.103:0/2199489255 >> 172.21.15.103:0/2199489255 conn(0x7f4cfc0fc7d0 msgr2=0x7f4cfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 -- 172.21.15.103:0/2199489255 shutdown_connections 2024-09-19T07:42:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.884+0000 7f4d0147c640 1 -- 172.21.15.103:0/2199489255 wait complete. 2024-09-19T07:42:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 Processor -- start 2024-09-19T07:42:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 -- start start 2024-09-19T07:42:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc100a90 0x7f4cfc1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc101430 0x7f4cfc199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cfc106a90 0x7f4cfc1a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc110e90 con 0x7f4cfc101430 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc110d10 con 0x7f4cfc106a90 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4d0147c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc111010 con 0x7f4cfc100a90 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4cfaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc100a90 0x7f4cfc1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc101430 0x7f4cfc199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4cfaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc100a90 0x7f4cfc1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48094/0 (socket says 172.21.15.103:48094) 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4cfaffd640 1 -- 172.21.15.103:0/856663121 learned_addr learned my addr 172.21.15.103:0/856663121 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:02.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc101430 0x7f4cfc199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40312/0 (socket says 172.21.15.103:40312) 2024-09-19T07:42:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.885+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cfc106a90 0x7f4cfc1a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf3fff640 1 -- 172.21.15.103:0/856663121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc100a90 msgr2=0x7f4cfc1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc100a90 0x7f4cfc1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf3fff640 1 -- 172.21.15.103:0/856663121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cfc106a90 msgr2=0x7f4cfc1a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cfc106a90 0x7f4cfc1a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf3fff640 1 -- 172.21.15.103:0/856663121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc1a48a0 con 0x7f4cfc101430 2024-09-19T07:42:02.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc100a90 0x7f4cfc1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cfc106a90 0x7f4cfc1a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc101430 0x7f4cfc199820 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005940 tx=0x7f4ce4031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf8ff9640 1 -- 172.21.15.103:0/856663121 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce403f070 con 0x7f4cfc101430 2024-09-19T07:42:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf8ff9640 1 -- 172.21.15.103:0/856663121 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4003ad0 con 0x7f4cfc101430 2024-09-19T07:42:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc1a4a70 con 0x7f4cfc101430 2024-09-19T07:42:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4cf8ff9640 1 -- 172.21.15.103:0/856663121 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4035070 con 0x7f4cfc101430 2024-09-19T07:42:02.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.886+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc1a4f20 con 0x7f4cfc101430 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.888+0000 7f4cf8ff9640 1 -- 172.21.15.103:0/856663121 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4ce40035f0 con 0x7f4cfc101430 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 Processor -- start 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 -- start start 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56362424a2f0 con 0x5636242e5b40 2024-09-19T07:42:02.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5636241f1cf0 con 0x5636242f3780 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.889+0000 7f4d03855740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5636242462b0 con 0x5636242e45d0 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf8ff9640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4ce80a3430 0x7f4ce80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf8ff9640 1 -- 172.21.15.103:0/856663121 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4ce40f00f0 con 0x7f4cfc101430 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cfb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cfaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cfb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242e4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40324/0 (socket says 172.21.15.103:40324) 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cfb7fe640 1 -- 172.21.15.103:0/147191706 learned_addr learned my addr 172.21.15.103:0/147191706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4ce80a3430 0x7f4ce80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf3fff640 1 -- 172.21.15.103:0/147191706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 msgr2=0x5636242e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf3fff640 1 -- 172.21.15.103:0/147191706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 msgr2=0x5636242e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cf3fff640 1 -- 172.21.15.103:0/147191706 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636241d63a0 con 0x5636242f3780 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.890+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.891+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242f5b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40ed850 tx=0x7f4ce4003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbae5b8e9943c41b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.891+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4ce80a3430 0x7f4ce80a58f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40099e0 tx=0x7f4cf4009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.891+0000 7f4cf17fa640 1 -- 172.21.15.103:0/147191706 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4048070 con 0x5636242f3780 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.891+0000 7f4cf17fa640 1 -- 172.21.15.103:0/147191706 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce40c5050 con 0x5636242f3780 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.891+0000 7f4cf17fa640 1 -- 172.21.15.103:0/147191706 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40b8b00 con 0x5636242f3780 2024-09-19T07:42:02.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 -- 172.21.15.103:0/147191706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 msgr2=0x5636242f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242f5b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40ed850 tx=0x7f4ce4003f60 comp rx=0 tx=0).stop 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 -- 172.21.15.103:0/147191706 shutdown_connections 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242f5b70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 --2- 172.21.15.103:0/147191706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 -- 172.21.15.103:0/147191706 >> 172.21.15.103:0/147191706 conn(0x5636242ad310 msgr2=0x5636242e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 -- 172.21.15.103:0/147191706 shutdown_connections 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 -- 172.21.15.103:0/147191706 wait complete. 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 -- start start 2024-09-19T07:42:02.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.892+0000 7f4d03855740 1 Processor -- start 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4d03855740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e45d0 0x5636242f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4d03855740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e5b40 0x5636242ec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4d03855740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4d03855740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636241f1cf0 con 0x5636242e45d0 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4d03855740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563624205060 con 0x5636242f3780 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4d03855740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5636242e7310 con 0x5636242e5b40 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4d03855740 10 client.0 _ll_register_callbacks cb 0x563623f588c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e5b40 0x5636242ec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e5b40 0x5636242ec2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48118/0 (socket says 172.21.15.103:48118) 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cf3fff640 1 -- 172.21.15.103:0/1939037458 learned_addr learned my addr 172.21.15.103:0/1939037458 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e45d0 0x5636242f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfb7fe640 1 -- 172.21.15.103:0/1939037458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e5b40 msgr2=0x5636242ec2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e5b40 0x5636242ec2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfb7fe640 1 -- 172.21.15.103:0/1939037458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e45d0 msgr2=0x5636242f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e45d0 0x5636242f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfb7fe640 1 -- 172.21.15.103:0/1939037458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563623fbb3c0 con 0x5636242f3780 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.893+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e45d0 0x5636242f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242ec820 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4003e90 tx=0x7f4cf4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba294c78f2372f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4ce27fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4ce27fc640 1 -- 172.21.15.103:0/1939037458 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf402d060 con 0x5636242f3780 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4ce27fc640 1 -- 172.21.15.103:0/1939037458 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4cf4020870 con 0x5636242f3780 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce40c5050 con 0x5636242f3780 2024-09-19T07:42:02.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4ce27fc640 1 -- 172.21.15.103:0/1939037458 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4cf4004b20 con 0x5636242f3780 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4d03855740 10 client.25814 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5636242fd8c0 con 0x5636242f3780 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.894+0000 7f4ce27fc640 1 -- 172.21.15.103:0/1939037458 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce40c5050 con 0x5636242f3780 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4ce27fc640 1 -- 172.21.15.103:0/1939037458 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f4cf4027030 con 0x5636242f3780 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4d03855740 10 client.25814 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4d03855740 10 client.25814 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4d03855740 20 client.25814 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4d03855740 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5636242fec60 0x563624301090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4d03855740 4 client.25814 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:02.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563624257860 con 0x5636242fec60 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5636242fec60 0x563624301090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4ce27fc640 10 client.25814 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.895+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5636242fec60 0x563624301090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20fcc93264076396 server_cookie=ffeec6142a659816 in_seq=0 out_seq=0 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4ce27fc640 1 -- 172.21.15.103:0/1939037458 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563624257860 con 0x5636242fec60 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4ce27fc640 10 client.25814 handle_command_reply: tid=0 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 client.25814 shutdown 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 msgr2=0x5636242ec820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242ec820 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4003e90 tx=0x7f4cf4004a40 comp rx=0 tx=0).stop 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 shutdown_connections 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5636242fec60 0x563624301090 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x5636242ec820 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e5b40 0x5636242ec2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 --2- 172.21.15.103:0/1939037458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e45d0 0x5636242f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 >> 172.21.15.103:0/1939037458 conn(0x5636242ad310 msgr2=0x5636242e69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:02.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4ce27fc640 0 client.25814 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:02.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 shutdown_connections 2024-09-19T07:42:02.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 1 -- 172.21.15.103:0/1939037458 wait complete. 2024-09-19T07:42:02.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.896+0000 7f4d03855740 20 client.25814 trim_cache size 0 max 16384 2024-09-19T07:42:02.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.912+0000 7f4d03855740 1 Processor -- start 2024-09-19T07:42:02.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.912+0000 7f4d03855740 1 -- start start 2024-09-19T07:42:02.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4d03855740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4d03855740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x563624306960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4d03855740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x563624306ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4d03855740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1ab670 con 0x5636242e5b40 2024-09-19T07:42:02.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4d03855740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1ad240 con 0x5636242f3780 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4d03855740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1abd40 con 0x5636242e45d0 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x563624306960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x563624306960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40346/0 (socket says 172.21.15.103:40346) 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 -- 172.21.15.103:0/3749219857 learned_addr learned my addr 172.21.15.103:0/3749219857 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x563624306ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 -- 172.21.15.103:0/3749219857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 msgr2=0x5636242eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 -- 172.21.15.103:0/3749219857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 msgr2=0x563624306ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x563624306ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 -- 172.21.15.103:0/3749219857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636242ea830 con 0x5636242e5b40 2024-09-19T07:42:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.913+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x563624306960 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40bc420 tx=0x7f4ce4005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6e2731c1e8b6e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.914+0000 7f4cf0ff9640 1 -- 172.21.15.103:0/3749219857 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4048070 con 0x5636242e5b40 2024-09-19T07:42:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.914+0000 7f4cf0ff9640 1 -- 172.21.15.103:0/3749219857 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce403b460 con 0x5636242e5b40 2024-09-19T07:42:02.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.914+0000 7f4cf0ff9640 1 -- 172.21.15.103:0/3749219857 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40edc40 con 0x5636242e5b40 2024-09-19T07:42:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- 172.21.15.103:0/3749219857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 msgr2=0x563624306960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x563624306960 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40bc420 tx=0x7f4ce4005b70 comp rx=0 tx=0).stop 2024-09-19T07:42:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- 172.21.15.103:0/3749219857 shutdown_connections 2024-09-19T07:42:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242f3780 0x563624306ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x563624306960 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 --2- 172.21.15.103:0/3749219857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242e45d0 0x5636242eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- 172.21.15.103:0/3749219857 >> 172.21.15.103:0/3749219857 conn(0x5636242ad310 msgr2=0x5636242eabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- 172.21.15.103:0/3749219857 shutdown_connections 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- 172.21.15.103:0/3749219857 wait complete. 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- start start 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 Processor -- start 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242e45d0 0x5636242f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242f3780 0x563624367920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1acb70 con 0x5636242e5b40 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1ac410 con 0x5636242e45d0 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.915+0000 7f4d03855740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1a9650 con 0x5636242f3780 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242e45d0 0x5636242f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40358/0 (socket says 172.21.15.103:40358) 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cf3fff640 1 -- 172.21.15.103:0/3735041317 learned_addr learned my addr 172.21.15.103:0/3735041317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4d03855740 10 client.0 _ll_register_callbacks cb 0x7f4cf400ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:02.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfb7fe640 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242f3780 0x563624367920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfaffd640 1 -- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242f3780 msgr2=0x563624367920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242f3780 0x563624367920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfaffd640 1 -- 172.21.15.103:0/3735041317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 msgr2=0x5636242f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:02.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfaffd640 1 -- 172.21.15.103:0/3735041317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563624367e60 con 0x5636242e45d0 2024-09-19T07:42:02.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:02.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4cfaffd640 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242e45d0 0x5636242f6870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5636243018a0 tx=0x7f4cec010e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479656111f80e3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:02.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4ce27fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:02.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4ce27fc640 1 -- 172.21.15.103:0/3735041317 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00fda0 con 0x5636242e45d0 2024-09-19T07:42:02.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4ce27fc640 1 -- 172.21.15.103:0/3735041317 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4cec027440 con 0x5636242e45d0 2024-09-19T07:42:02.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.916+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce403b460 con 0x5636242e45d0 2024-09-19T07:42:02.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4ce27fc640 1 -- 172.21.15.103:0/3735041317 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4cec027640 con 0x5636242e45d0 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 10 client.25820 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563624369140 con 0x5636242e45d0 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4ce27fc640 1 -- 172.21.15.103:0/3735041317 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce403b460 con 0x5636242e45d0 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4ce27fc640 1 -- 172.21.15.103:0/3735041317 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f4cec025020 con 0x5636242e45d0 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 10 client.25820 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 10 client.25820 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 20 client.25820 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x563624369860 0x56362436bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 4 client.25820 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563624251d70 con 0x563624369860 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.917+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x563624369860 0x56362436bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4ce27fc640 10 client.25820 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4cf3fff640 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x563624369860 0x56362436bc80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b297ef13df451c2a server_cookie=bc77b9933e94eda6 in_seq=0 out_seq=0 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4ce27fc640 1 -- 172.21.15.103:0/3735041317 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563624251d70 con 0x563624369860 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4ce27fc640 10 client.25820 handle_command_reply: tid=0 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 client.25820 shutdown 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242e45d0 msgr2=0x5636242f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242e45d0 0x5636242f6870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5636243018a0 tx=0x7f4cec010e90 comp rx=0 tx=0).stop 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 shutdown_connections 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x563624369860 0x56362436bc80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636242f3780 0x563624367920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636242e5b40 0x5636242f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 --2- 172.21.15.103:0/3735041317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636242e45d0 0x5636242f6870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 >> 172.21.15.103:0/3735041317 conn(0x5636242ad310 msgr2=0x5636242e6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4ce27fc640 0 client.25820 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 shutdown_connections 2024-09-19T07:42:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 1 -- 172.21.15.103:0/3735041317 wait complete. 2024-09-19T07:42:02.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.918+0000 7f4d03855740 20 client.25820 trim_cache size 0 max 16384 2024-09-19T07:42:02.928 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:02.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4ce80a3430 msgr2=0x7f4ce80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4ce80a3430 0x7f4ce80a58f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40099e0 tx=0x7f4cf4009500 comp rx=0 tx=0).stop 2024-09-19T07:42:02.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc101430 msgr2=0x7f4cfc199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:02.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc101430 0x7f4cfc199820 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005940 tx=0x7f4ce4031b50 comp rx=0 tx=0).stop 2024-09-19T07:42:02.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 shutdown_connections 2024-09-19T07:42:02.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4ce80a3430 0x7f4ce80a58f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cfc106a90 0x7f4cfc1a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4cfc101430 0x7f4cfc199820 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 --2- 172.21.15.103:0/856663121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cfc100a90 0x7f4cfc1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:02.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 >> 172.21.15.103:0/856663121 conn(0x7f4cfc0fc7d0 msgr2=0x7f4cfc0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:02.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 shutdown_connections 2024-09-19T07:42:02.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:02.919+0000 7f4d0147c640 1 -- 172.21.15.103:0/856663121 wait complete. 2024-09-19T07:42:03.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:42:03.773 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:03 smithi103 ceph-mon[31201]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.5 MiB/s wr, 619 op/s 2024-09-19T07:42:03.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:03 smithi134 ceph-mon[35414]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.5 MiB/s wr, 619 op/s 2024-09-19T07:42:03.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:03 smithi174 ceph-mon[35370]: pgmap v125: 97 pgs: 97 active+clean; 2.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.5 MiB/s wr, 619 op/s 2024-09-19T07:42:04.235 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:05.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.021+0000 7fb959ed0640 1 -- 172.21.15.103:0/2311233350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954100e20 msgr2=0x7fb954101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.021+0000 7fb959ed0640 1 --2- 172.21.15.103:0/2311233350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954100e20 0x7fb954101200 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb944009900 tx=0x7fb94402a480 comp rx=0 tx=0).stop 2024-09-19T07:42:05.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.021+0000 7fb959ed0640 1 -- 172.21.15.103:0/2311233350 shutdown_connections 2024-09-19T07:42:05.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.021+0000 7fb959ed0640 1 --2- 172.21.15.103:0/2311233350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 0x7fb95410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.021+0000 7fb959ed0640 1 --2- 172.21.15.103:0/2311233350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb954102570 0x7fb9541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.021+0000 7fb959ed0640 1 --2- 172.21.15.103:0/2311233350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954100e20 0x7fb954101200 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.021+0000 7fb959ed0640 1 -- 172.21.15.103:0/2311233350 >> 172.21.15.103:0/2311233350 conn(0x7fb9540fc7d0 msgr2=0x7fb9540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.022+0000 7fb959ed0640 1 -- 172.21.15.103:0/2311233350 shutdown_connections 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.022+0000 7fb959ed0640 1 -- 172.21.15.103:0/2311233350 wait complete. 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.022+0000 7fb959ed0640 1 Processor -- start 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb959ed0640 1 -- start start 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb959ed0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb954100e20 0x7fb9541a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb959ed0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954102570 0x7fb9541a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb959ed0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 0x7fb95419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb959ed0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541142d0 con 0x7fb954102570 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb959ed0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb954114150 con 0x7fb954100e20 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb959ed0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb954114450 con 0x7fb954102ef0 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 0x7fb95419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb9537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb954100e20 0x7fb9541a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb952ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954102570 0x7fb9541a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 0x7fb95419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48160/0 (socket says 172.21.15.103:48160) 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 -- 172.21.15.103:0/4086370706 learned_addr learned my addr 172.21.15.103:0/4086370706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 -- 172.21.15.103:0/4086370706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb954100e20 msgr2=0x7fb9541a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb954100e20 0x7fb9541a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 -- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954102570 msgr2=0x7fb9541a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954102570 0x7fb9541a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb953fff640 1 -- 172.21.15.103:0/4086370706 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95419acd0 con 0x7fb954102ef0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.023+0000 7fb952ffd640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954102570 0x7fb9541a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.024+0000 7fb953fff640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 0x7fb95419a5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00eaa0 tx=0x7fb94c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.024+0000 7fb950ff9640 1 -- 172.21.15.103:0/4086370706 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00cdb0 con 0x7fb954102ef0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.024+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95419afc0 con 0x7fb954102ef0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.024+0000 7fb950ff9640 1 -- 172.21.15.103:0/4086370706 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb94c005020 con 0x7fb954102ef0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.024+0000 7fb950ff9640 1 -- 172.21.15.103:0/4086370706 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0116d0 con 0x7fb954102ef0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.024+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9541a72b0 con 0x7fb954102ef0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.027+0000 7fb95c2a9740 1 Processor -- start 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.027+0000 7fb950ff9640 1 -- 172.21.15.103:0/4086370706 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb94c011870 con 0x7fb954102ef0 2024-09-19T07:42:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.027+0000 7fb95c2a9740 1 -- start start 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.028+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7cb40 0x561fe5c7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.028+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7b5d0 0x561fe5c7ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.028+0000 7fb952ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7cb40 0x561fe5c7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.028+0000 7fb950ff9640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb93c0a33b0 0x7fb93c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.028+0000 7fb952ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7cb40 0x561fe5c7b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48162/0 (socket says 172.21.15.103:48162) 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.029+0000 7fb950ff9640 1 -- 172.21.15.103:0/4086370706 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb94c0cfe30 con 0x7fb954102ef0 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.029+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c86370 0x561fe5c88760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.029+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fe5be12f0 con 0x561fe5c7b5d0 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.029+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fe5b88cf0 con 0x561fe5c86370 2024-09-19T07:42:05.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.029+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fe5bdd2b0 con 0x561fe5c7cb40 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.029+0000 7fb952ffd640 1 -- 172.21.15.103:0/2115206042 learned_addr learned my addr 172.21.15.103:0/2115206042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb953fff640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb93c0a33b0 0x7fb93c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb952ffd640 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c86370 0x561fe5c88760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb952ffd640 1 -- 172.21.15.103:0/2115206042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c86370 msgr2=0x561fe5c88760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb952ffd640 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c86370 0x561fe5c88760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb952ffd640 1 -- 172.21.15.103:0/2115206042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7b5d0 msgr2=0x561fe5c7ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb952ffd640 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7b5d0 0x561fe5c7ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb952ffd640 1 -- 172.21.15.103:0/2115206042 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe5b6d3a0 con 0x561fe5c7cb40 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb952ffd640 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7cb40 0x561fe5c7b090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb94800bfd0 tx=0x7fb94800bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01f1238af50df44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb953fff640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb93c0a33b0 0x7fb93c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0a4860 tx=0x7fb94c009270 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb939ffb640 1 -- 172.21.15.103:0/2115206042 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94801a030 con 0x561fe5c7cb40 2024-09-19T07:42:05.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.030+0000 7fb939ffb640 1 -- 172.21.15.103:0/2115206042 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb948014170 con 0x561fe5c7cb40 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb939ffb640 1 -- 172.21.15.103:0/2115206042 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948003850 con 0x561fe5c7cb40 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 -- 172.21.15.103:0/2115206042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7cb40 msgr2=0x561fe5c7b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7cb40 0x561fe5c7b090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb94800bfd0 tx=0x7fb94800bcf0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 -- 172.21.15.103:0/2115206042 shutdown_connections 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c86370 0x561fe5c88760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7b5d0 0x561fe5c7ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/2115206042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7cb40 0x561fe5c7b090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 -- 172.21.15.103:0/2115206042 >> 172.21.15.103:0/2115206042 conn(0x561fe5c44310 msgr2=0x561fe5c77c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 -- 172.21.15.103:0/2115206042 shutdown_connections 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.031+0000 7fb95c2a9740 1 -- 172.21.15.103:0/2115206042 wait complete. 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 -- start start 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 Processor -- start 2024-09-19T07:42:05.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c85d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c82fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5c83560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fe5c91920 con 0x561fe5c7cb40 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fe5b9c060 con 0x561fe5c7b5d0 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fe5c91aa0 con 0x561fe5c86370 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb95c2a9740 10 client.0 _ll_register_callbacks cb 0x561fe58ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb952ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c85d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb952ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c85d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45936/0 (socket says 172.21.15.103:45936) 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb952ffd640 1 -- 172.21.15.103:0/1256393318 learned_addr learned my addr 172.21.15.103:0/1256393318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:05.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.032+0000 7fb9537fe640 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c82fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb952ffd640 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5c83560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb952ffd640 1 -- 172.21.15.103:0/1256393318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 msgr2=0x561fe5c83560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb952ffd640 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5c83560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb952ffd640 1 -- 172.21.15.103:0/1256393318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 msgr2=0x561fe5c82fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb952ffd640 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c82fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb952ffd640 1 -- 172.21.15.103:0/1256393318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe59523c0 con 0x561fe5c7b5d0 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb952ffd640 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c85d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb94800bbc0 tx=0x7fb948005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc48235dd6f72816 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb932ffd640 1 -- 172.21.15.103:0/1256393318 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94801a070 con 0x561fe5c7b5d0 2024-09-19T07:42:05.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb948014170 con 0x561fe5c7b5d0 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb932ffd640 1 -- 172.21.15.103:0/1256393318 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb948004430 con 0x561fe5c7b5d0 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.033+0000 7fb932ffd640 1 -- 172.21.15.103:0/1256393318 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb948031050 con 0x561fe5c7b5d0 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.034+0000 7fb95c2a9740 10 client.25832 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.034+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561fe5c9d370 con 0x561fe5c7b5d0 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.034+0000 7fb932ffd640 1 -- 172.21.15.103:0/1256393318 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb948014170 con 0x561fe5c7b5d0 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.034+0000 7fb932ffd640 1 -- 172.21.15.103:0/1256393318 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb948011740 con 0x561fe5c7b5d0 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.034+0000 7fb95c2a9740 10 client.25832 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb95c2a9740 10 client.25832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb95c2a9740 20 client.25832 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5ca0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb95c2a9740 4 client.25832 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe5bee860 con 0x561fe5c9e720 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb9537fe640 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5ca0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb9537fe640 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5ca0b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebbe3957fc585f63 server_cookie=e23abbd17319f79a in_seq=0 out_seq=0 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.035+0000 7fb932ffd640 10 client.25832 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb932ffd640 1 -- 172.21.15.103:0/1256393318 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fe5bee860 con 0x561fe5c9e720 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb932ffd640 10 client.25832 handle_command_reply: tid=0 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 client.25832 shutdown 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 msgr2=0x561fe5c85d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c85d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb94800bbc0 tx=0x7fb948005fb0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 shutdown_connections 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5ca0b10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb932ffd640 0 client.25832 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:05.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5c83560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c82fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/1256393318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c85d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 >> 172.21.15.103:0/1256393318 conn(0x561fe5c44310 msgr2=0x561fe5c7d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:05.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 shutdown_connections 2024-09-19T07:42:05.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 1 -- 172.21.15.103:0/1256393318 wait complete. 2024-09-19T07:42:05.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.036+0000 7fb95c2a9740 20 client.25832 trim_cache size 0 max 16384 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 Processor -- start 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 -- start start 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c9abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c76c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5ca61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541ab740 con 0x561fe5c7cb40 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541ad310 con 0x561fe5c7b5d0 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541abe10 con 0x561fe5c86370 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb9537fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c76c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb9537fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c76c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40402/0 (socket says 172.21.15.103:40402) 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb9537fe640 1 -- 172.21.15.103:0/729857109 learned_addr learned my addr 172.21.15.103:0/729857109 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb952ffd640 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c9abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb952ffd640 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5ca61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb952ffd640 1 -- 172.21.15.103:0/729857109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 msgr2=0x561fe5ca61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb952ffd640 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5ca61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb952ffd640 1 -- 172.21.15.103:0/729857109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 msgr2=0x561fe5c76c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb952ffd640 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c76c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb952ffd640 1 -- 172.21.15.103:0/729857109 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe5c78960 con 0x561fe5c7b5d0 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.053+0000 7fb9537fe640 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c76c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.054+0000 7fb952ffd640 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c9abc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb94800b5a0 tx=0x7fb948017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7532cd68dfcddbd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.054+0000 7fb9397fa640 1 -- 172.21.15.103:0/729857109 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94801a070 con 0x561fe5c7b5d0 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.054+0000 7fb9397fa640 1 -- 172.21.15.103:0/729857109 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb948011920 con 0x561fe5c7b5d0 2024-09-19T07:42:05.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.054+0000 7fb9397fa640 1 -- 172.21.15.103:0/729857109 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948005020 con 0x561fe5c7b5d0 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.054+0000 7fb95c2a9740 1 -- 172.21.15.103:0/729857109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 msgr2=0x561fe5c9abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.054+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c9abc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb94800b5a0 tx=0x7fb948017040 comp rx=0 tx=0).stop 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 -- 172.21.15.103:0/729857109 shutdown_connections 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c86370 0x561fe5ca61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c7cb40 0x561fe5c76c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/729857109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7b5d0 0x561fe5c9abc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 -- 172.21.15.103:0/729857109 >> 172.21.15.103:0/729857109 conn(0x561fe5c44310 msgr2=0x561fe5c76500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 -- 172.21.15.103:0/729857109 shutdown_connections 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 -- 172.21.15.103:0/729857109 wait complete. 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 -- start start 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 Processor -- start 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7b5d0 0x561fe5c96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.055+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7cb40 0x561fe5c97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb95c2a9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c86370 0x561fe5c93d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541acc40 con 0x561fe5c86370 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541ac4e0 con 0x561fe5c7cb40 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb95c2a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9541a9740 con 0x561fe5c7b5d0 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb95c2a9740 10 client.0 _ll_register_callbacks cb 0x7fb948014850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7cb40 0x561fe5c97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7cb40 0x561fe5c97270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45964/0 (socket says 172.21.15.103:45964) 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 -- 172.21.15.103:0/385282832 learned_addr learned my addr 172.21.15.103:0/385282832 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb952ffd640 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c86370 0x561fe5c93d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb952ffd640 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7b5d0 0x561fe5c96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 -- 172.21.15.103:0/385282832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7b5d0 msgr2=0x561fe5c96d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7b5d0 0x561fe5c96d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 -- 172.21.15.103:0/385282832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c86370 msgr2=0x561fe5c93d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c86370 0x561fe5c93d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.056+0000 7fb9537fe640 1 -- 172.21.15.103:0/385282832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe5c94240 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb9537fe640 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7cb40 0x561fe5c97270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb94402a9a0 tx=0x7fb944004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc355ea9f6e1cd82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb932ffd640 1 -- 172.21.15.103:0/385282832 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944035040 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb932ffd640 1 -- 172.21.15.103:0/385282832 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb944005510 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb948011920 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb932ffd640 1 -- 172.21.15.103:0/385282832 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9440046c0 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb95c2a9740 10 client.25838 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561fe5d09860 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb932ffd640 1 -- 172.21.15.103:0/385282832 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb948011920 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.057+0000 7fb932ffd640 1 -- 172.21.15.103:0/385282832 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb944031a90 con 0x561fe5c7cb40 2024-09-19T07:42:05.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb95c2a9740 10 client.25838 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb95c2a9740 10 client.25838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb95c2a9740 20 client.25838 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5d0bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb95c2a9740 4 client.25838 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fe5be8d70 con 0x561fe5c9e720 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb952ffd640 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5d0bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb952ffd640 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5d0bfb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fecc6a9741a62dca server_cookie=5b438528eea9dd10 in_seq=0 out_seq=0 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb932ffd640 10 client.25838 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb932ffd640 1 -- 172.21.15.103:0/385282832 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fe5be8d70 con 0x561fe5c9e720 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb932ffd640 10 client.25838 handle_command_reply: tid=0 2024-09-19T07:42:05.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.058+0000 7fb95c2a9740 1 client.25838 shutdown 2024-09-19T07:42:05.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7cb40 msgr2=0x561fe5c97270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7cb40 0x561fe5c97270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb94402a9a0 tx=0x7fb944004460 comp rx=0 tx=0).stop 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 shutdown_connections 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561fe5c9e720 0x561fe5d0bfb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe5c86370 0x561fe5c93d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe5c7cb40 0x561fe5c97270 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 --2- 172.21.15.103:0/385282832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe5c7b5d0 0x561fe5c96d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 >> 172.21.15.103:0/385282832 conn(0x561fe5c44310 msgr2=0x561fe5c7d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb932ffd640 0 client.25838 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 shutdown_connections 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 1 -- 172.21.15.103:0/385282832 wait complete. 2024-09-19T07:42:05.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb95c2a9740 20 client.25838 trim_cache size 0 max 16384 2024-09-19T07:42:05.067 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:05.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb93c0a33b0 msgr2=0x7fb93c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb93c0a33b0 0x7fb93c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0a4860 tx=0x7fb94c009270 comp rx=0 tx=0).stop 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 msgr2=0x7fb95419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 0x7fb95419a5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00eaa0 tx=0x7fb94c00ef60 comp rx=0 tx=0).stop 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 shutdown_connections 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb93c0a33b0 0x7fb93c0a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb954102ef0 0x7fb95419a5a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb954102570 0x7fb9541a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 --2- 172.21.15.103:0/4086370706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb954100e20 0x7fb9541a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.059+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 >> 172.21.15.103:0/4086370706 conn(0x7fb9540fc7d0 msgr2=0x7fb954105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.060+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 shutdown_connections 2024-09-19T07:42:05.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:05.060+0000 7fb959ed0640 1 -- 172.21.15.103:0/4086370706 wait complete. 2024-09-19T07:42:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 210.0 s, 827.2 tps, lat 120.720 ms stddev 171.371 2024-09-19T07:42:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:05 smithi134 ceph-mon[35414]: pgmap v126: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 MiB/s wr, 1.52k op/s 2024-09-19T07:42:05.841 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:05 smithi103 ceph-mon[31201]: pgmap v126: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 MiB/s wr, 1.52k op/s 2024-09-19T07:42:05.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:42:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:05 smithi174 ceph-mon[35370]: pgmap v126: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 MiB/s wr, 1.52k op/s 2024-09-19T07:42:06.284 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:07.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.077+0000 7fe3b1403640 1 -- 172.21.15.103:0/3041646960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 msgr2=0x7fe3ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.077+0000 7fe3b1403640 1 --2- 172.21.15.103:0/3041646960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 0x7fe3ac1029b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009900 tx=0x7fe3a002a480 comp rx=0 tx=0).stop 2024-09-19T07:42:07.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 -- 172.21.15.103:0/3041646960 shutdown_connections 2024-09-19T07:42:07.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 --2- 172.21.15.103:0/3041646960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac102ef0 0x7fe3ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 --2- 172.21.15.103:0/3041646960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 0x7fe3ac1029b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 --2- 172.21.15.103:0/3041646960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3ac100e20 0x7fe3ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 -- 172.21.15.103:0/3041646960 >> 172.21.15.103:0/3041646960 conn(0x7fe3ac0fc7d0 msgr2=0x7fe3ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:07.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 -- 172.21.15.103:0/3041646960 shutdown_connections 2024-09-19T07:42:07.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 -- 172.21.15.103:0/3041646960 wait complete. 2024-09-19T07:42:07.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.078+0000 7fe3b1403640 1 Processor -- start 2024-09-19T07:42:07.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3b1403640 1 -- start start 2024-09-19T07:42:07.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3b1403640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac100e20 0x7fe3ac199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3b1403640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 0x7fe3ac199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3b1403640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3ac102ef0 0x7fe3ac1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3b1403640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac110f80 con 0x7fe3ac102570 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3b1403640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac110e00 con 0x7fe3ac102ef0 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3b1403640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac111100 con 0x7fe3ac100e20 2024-09-19T07:42:07.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac100e20 0x7fe3ac199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3aa7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 0x7fe3ac199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac100e20 0x7fe3ac199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48202/0 (socket says 172.21.15.103:48202) 2024-09-19T07:42:07.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.079+0000 7fe3aaffd640 1 -- 172.21.15.103:0/1413667271 learned_addr learned my addr 172.21.15.103:0/1413667271 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:07.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3aa7fc640 1 -- 172.21.15.103:0/1413667271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac100e20 msgr2=0x7fe3ac199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3ab7fe640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3ac102ef0 0x7fe3ac1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac100e20 0x7fe3ac199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3aa7fc640 1 -- 172.21.15.103:0/1413667271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3ac102ef0 msgr2=0x7fe3ac1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3ac102ef0 0x7fe3ac1a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3aa7fc640 1 -- 172.21.15.103:0/1413667271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac197100 con 0x7fe3ac102570 2024-09-19T07:42:07.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac100e20 0x7fe3ac199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:07.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3ab7fe640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3ac102ef0 0x7fe3ac1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:07.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 0x7fe3ac199850 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005940 tx=0x7fe3a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:07.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.080+0000 7fe393fff640 1 -- 172.21.15.103:0/1413667271 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a003f070 con 0x7fe3ac102570 2024-09-19T07:42:07.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.081+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac197390 con 0x7fe3ac102570 2024-09-19T07:42:07.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.081+0000 7fe393fff640 1 -- 172.21.15.103:0/1413667271 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a0031c30 con 0x7fe3ac102570 2024-09-19T07:42:07.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.081+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac1978c0 con 0x7fe3ac102570 2024-09-19T07:42:07.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.081+0000 7fe393fff640 1 -- 172.21.15.103:0/1413667271 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0035030 con 0x7fe3ac102570 2024-09-19T07:42:07.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.084+0000 7fe3b37dc740 1 Processor -- start 2024-09-19T07:42:07.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.084+0000 7fe3b37dc740 1 -- start start 2024-09-19T07:42:07.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe393fff640 1 -- 172.21.15.103:0/1413667271 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe3a0042070 con 0x7fe3ac102570 2024-09-19T07:42:07.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20ea5d0 0x555db20eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20f5370 0x555db20f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555db20502f0 con 0x555db20ea5d0 2024-09-19T07:42:07.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555db1ff7cf0 con 0x555db20f5370 2024-09-19T07:42:07.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555db204c2b0 con 0x555db20ebb40 2024-09-19T07:42:07.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20ea090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48216/0 (socket says 172.21.15.103:48216) 2024-09-19T07:42:07.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3ab7fe640 1 -- 172.21.15.103:0/218248502 learned_addr learned my addr 172.21.15.103:0/218248502 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:07.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20f5370 0x555db20f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.085+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20ea5d0 0x555db20eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3ab7fe640 1 -- 172.21.15.103:0/218248502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20f5370 msgr2=0x555db20f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3ab7fe640 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20f5370 0x555db20f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3ab7fe640 1 -- 172.21.15.103:0/218248502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20ea5d0 msgr2=0x555db20eaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3ab7fe640 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20ea5d0 0x555db20eaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3ab7fe640 1 -- 172.21.15.103:0/218248502 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db1fdc3a0 con 0x555db20ebb40 2024-09-19T07:42:07.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20ea5d0 0x555db20eaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:07.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3ab7fe640 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20ea090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009e40 tx=0x7fe3a400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=922584cff1d13594 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:07.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe393fff640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3940a33b0 0x7fe3940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3917fa640 1 -- 172.21.15.103:0/218248502 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4004d20 con 0x555db20ebb40 2024-09-19T07:42:07.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3917fa640 1 -- 172.21.15.103:0/218248502 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a4005500 con 0x555db20ebb40 2024-09-19T07:42:07.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3917fa640 1 -- 172.21.15.103:0/218248502 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a40118e0 con 0x555db20ebb40 2024-09-19T07:42:07.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe393fff640 1 -- 172.21.15.103:0/1413667271 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe3a00f0640 con 0x7fe3ac102570 2024-09-19T07:42:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.086+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3940a33b0 0x7fe3940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 -- 172.21.15.103:0/218248502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 msgr2=0x555db20ea090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3940a33b0 0x7fe3940a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x555db20e7ac0 tx=0x7fe39c005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20ea090 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009e40 tx=0x7fe3a400f730 comp rx=0 tx=0).stop 2024-09-19T07:42:07.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 -- 172.21.15.103:0/218248502 shutdown_connections 2024-09-19T07:42:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20f5370 0x555db20f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20ea5d0 0x555db20eaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/218248502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20ea090 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 -- 172.21.15.103:0/218248502 >> 172.21.15.103:0/218248502 conn(0x555db20b3310 msgr2=0x555db20e6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 -- 172.21.15.103:0/218248502 shutdown_connections 2024-09-19T07:42:07.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.087+0000 7fe3b37dc740 1 -- 172.21.15.103:0/218248502 wait complete. 2024-09-19T07:42:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.088+0000 7fe3b37dc740 1 -- start start 2024-09-19T07:42:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.088+0000 7fe3b37dc740 1 Processor -- start 2024-09-19T07:42:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.088+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.088+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.088+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555db20f8070 con 0x555db20f5370 2024-09-19T07:42:07.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555db200b060 con 0x555db20ea5d0 2024-09-19T07:42:07.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555db20f81f0 con 0x555db20ebb40 2024-09-19T07:42:07.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3b37dc740 10 client.0 _ll_register_callbacks cb 0x555db1d5e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:07.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:46002/0 (socket says 172.21.15.103:46002) 2024-09-19T07:42:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aaffd640 1 -- 172.21.15.103:0/1824124518 learned_addr learned my addr 172.21.15.103:0/1824124518 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aa7fc640 1 -- 172.21.15.103:0/1824124518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 msgr2=0x555db20f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aa7fc640 1 -- 172.21.15.103:0/1824124518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 msgr2=0x555db20f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aa7fc640 1 -- 172.21.15.103:0/1824124518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db1dc13c0 con 0x555db20ebb40 2024-09-19T07:42:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3ab7fe640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f1fa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00edda0 tx=0x7fe3a00b85d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d932c615c4118ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:07.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe38eff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:42:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.089+0000 7fe38eff5640 1 -- 172.21.15.103:0/1824124518 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0049070 con 0x555db20ebb40 2024-09-19T07:42:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe38eff5640 1 -- 172.21.15.103:0/1824124518 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe3a00f60d0 con 0x555db20ebb40 2024-09-19T07:42:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a4005500 con 0x555db20ebb40 2024-09-19T07:42:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe38eff5640 1 -- 172.21.15.103:0/1824124518 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe3a00b8c30 con 0x555db20ebb40 2024-09-19T07:42:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 10 client.25813 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555db20f8070 con 0x555db20ebb40 2024-09-19T07:42:07.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe38eff5640 1 -- 172.21.15.103:0/1824124518 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a4005500 con 0x555db20ebb40 2024-09-19T07:42:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe38eff5640 1 -- 172.21.15.103:0/1824124518 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fe3a003b460 con 0x555db20ebb40 2024-09-19T07:42:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 10 client.25813 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 10 client.25813 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:42:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 20 client.25813 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2104f70 0x555db21073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 4 client.25813 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:07.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.090+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db205d860 con 0x555db2104f70 2024-09-19T07:42:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.091+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2104f70 0x555db21073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.091+0000 7fe38eff5640 10 client.25813 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:42:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.092+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2104f70 0x555db21073b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9700c4fb8b0809a2 server_cookie=c2ea31ba2f27e630 in_seq=0 out_seq=0 2024-09-19T07:42:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.093+0000 7fe38eff5640 1 -- 172.21.15.103:0/1824124518 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555db205d860 con 0x555db2104f70 2024-09-19T07:42:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.093+0000 7fe38eff5640 10 client.25813 handle_command_reply: tid=0 2024-09-19T07:42:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.093+0000 7fe3b37dc740 1 client.25813 shutdown 2024-09-19T07:42:07.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.093+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 msgr2=0x555db20f1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f1fa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00edda0 tx=0x7fe3a00b85d0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 shutdown_connections 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2104f70 0x555db21073b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f1fa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/1824124518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 >> 172.21.15.103:0/1824124518 conn(0x555db20b3310 msgr2=0x555db20ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe38eff5640 0 client.25813 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 shutdown_connections 2024-09-19T07:42:07.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 1 -- 172.21.15.103:0/1824124518 wait complete. 2024-09-19T07:42:07.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.094+0000 7fe3b37dc740 20 client.25813 trim_cache size 0 max 16384 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.112+0000 7fe3b37dc740 1 Processor -- start 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.112+0000 7fe3b37dc740 1 -- start start 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.112+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db2101370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.112+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db210cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800e650 con 0x555db20f5370 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe398010220 con 0x555db20ea5d0 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800ed20 con 0x555db20ebb40 2024-09-19T07:42:07.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db2101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db210cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db2101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:46014/0 (socket says 172.21.15.103:46014) 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 -- 172.21.15.103:0/3368919045 learned_addr learned my addr 172.21.15.103:0/3368919045 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 -- 172.21.15.103:0/3368919045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 msgr2=0x555db20e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 -- 172.21.15.103:0/3368919045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 msgr2=0x555db210cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db210cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 -- 172.21.15.103:0/3368919045 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db20f2890 con 0x555db20ea5d0 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3ab7fe640 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db210cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.113+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db2101370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe39c012f20 tx=0x7fe39c01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e24e238166c5008e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:07.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.114+0000 7fe390ff9640 1 -- 172.21.15.103:0/3368919045 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00fe50 con 0x555db20ea5d0 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.114+0000 7fe390ff9640 1 -- 172.21.15.103:0/3368919045 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39c005500 con 0x555db20ea5d0 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.114+0000 7fe390ff9640 1 -- 172.21.15.103:0/3368919045 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c004a00 con 0x555db20ea5d0 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 -- 172.21.15.103:0/3368919045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 msgr2=0x555db2101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db2101370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe39c012f20 tx=0x7fe39c01a000 comp rx=0 tx=0).stop 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 -- 172.21.15.103:0/3368919045 shutdown_connections 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db210cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/3368919045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db2101370 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 -- 172.21.15.103:0/3368919045 >> 172.21.15.103:0/3368919045 conn(0x555db20b3310 msgr2=0x555db20e5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 -- 172.21.15.103:0/3368919045 shutdown_connections 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 -- 172.21.15.103:0/3368919045 wait complete. 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 -- start start 2024-09-19T07:42:07.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.115+0000 7fe3b37dc740 1 Processor -- start 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3b37dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800fb50 con 0x555db20f5370 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800f3f0 con 0x555db20ea5d0 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3b37dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800c610 con 0x555db20ebb40 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3b37dc740 10 client.0 _ll_register_callbacks cb 0x7fe3a004df90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40432/0 (socket says 172.21.15.103:40432) 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3ab7fe640 1 -- 172.21.15.103:0/2484577147 learned_addr learned my addr 172.21.15.103:0/2484577147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.116+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.117+0000 7fe3aaffd640 1 -- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 msgr2=0x555db20f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.117+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.117+0000 7fe3aaffd640 1 -- 172.21.15.103:0/2484577147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 msgr2=0x555db20f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.117+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.117+0000 7fe3aaffd640 1 -- 172.21.15.103:0/2484577147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db216ee50 con 0x555db20ea5d0 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.117+0000 7fe3aaffd640 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f9460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000c00 tx=0x7fe39c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8b8eb41c5da35df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.118+0000 7fe38eff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.118+0000 7fe38eff5640 1 -- 172.21.15.103:0/2484577147 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c01f9f0 con 0x555db20ea5d0 2024-09-19T07:42:07.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.118+0000 7fe38eff5640 1 -- 172.21.15.103:0/2484577147 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe39c01fb90 con 0x555db20ea5d0 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.118+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39c005500 con 0x555db20ea5d0 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.118+0000 7fe38eff5640 1 -- 172.21.15.103:0/2484577147 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe39c00fe50 con 0x555db20ea5d0 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.118+0000 7fe3b37dc740 10 client.25853 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.118+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555db21700f0 con 0x555db20ea5d0 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe38eff5640 1 -- 172.21.15.103:0/2484577147 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39c03b070 con 0x555db20ea5d0 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe38eff5640 1 -- 172.21.15.103:0/2484577147 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fe39c0407d0 con 0x555db20ea5d0 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe3b37dc740 10 client.25853 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe3b37dc740 10 client.25853 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe3b37dc740 20 client.25853 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:07.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2170780 0x555db2172ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe3b37dc740 4 client.25853 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe39c005500 con 0x555db2170780 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.119+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2170780 0x555db2172ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.120+0000 7fe38eff5640 10 client.25853 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.120+0000 7fe3aa7fc640 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2170780 0x555db2172ba0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3f948aeb643a773 server_cookie=d4c0d638ad608233 in_seq=0 out_seq=0 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.120+0000 7fe38eff5640 1 -- 172.21.15.103:0/2484577147 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe39c005500 con 0x555db2170780 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.120+0000 7fe38eff5640 10 client.25853 handle_command_reply: tid=0 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.120+0000 7fe3b37dc740 1 client.25853 shutdown 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.120+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 msgr2=0x555db20f9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.120+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f9460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000c00 tx=0x7fe39c0095c0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 shutdown_connections 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x555db2170780 0x555db2172ba0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555db20f5370 0x555db20f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555db20ebb40 0x555db20f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 --2- 172.21.15.103:0/2484577147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555db20ea5d0 0x555db20f9460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 >> 172.21.15.103:0/2484577147 conn(0x555db20b3310 msgr2=0x555db20f4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe38eff5640 0 client.25853 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 shutdown_connections 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 1 -- 172.21.15.103:0/2484577147 wait complete. 2024-09-19T07:42:07.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.121+0000 7fe3b37dc740 20 client.25853 trim_cache size 0 max 16384 2024-09-19T07:42:07.131 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:07.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3940a33b0 msgr2=0x7fe3940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3940a33b0 0x7fe3940a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x555db20e7ac0 tx=0x7fe39c005f90 comp rx=0 tx=0).stop 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 msgr2=0x7fe3ac199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 0x7fe3ac199850 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005940 tx=0x7fe3a0003c10 comp rx=0 tx=0).stop 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 shutdown_connections 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3940a33b0 0x7fe3940a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3ac102ef0 0x7fe3ac1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3ac102570 0x7fe3ac199850 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 --2- 172.21.15.103:0/1413667271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3ac100e20 0x7fe3ac199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 >> 172.21.15.103:0/1413667271 conn(0x7fe3ac0fc7d0 msgr2=0x7fe3ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 shutdown_connections 2024-09-19T07:42:07.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:07.122+0000 7fe3b1403640 1 -- 172.21.15.103:0/1413667271 wait complete. 2024-09-19T07:42:07.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:07 smithi103 ceph-mon[31201]: pgmap v127: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.20k op/s 2024-09-19T07:42:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:07 smithi134 ceph-mon[35414]: pgmap v127: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.20k op/s 2024-09-19T07:42:07.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:07 smithi174 ceph-mon[35370]: pgmap v127: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.20k op/s 2024-09-19T07:42:07.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:42:08.336 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:08.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.868+0000 7f85b3476640 1 -- 172.21.15.103:0/2190374403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 msgr2=0x7f85ac1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.868+0000 7f85b3476640 1 --2- 172.21.15.103:0/2190374403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 0x7f85ac1089f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c02a420 comp rx=0 tx=0).stop 2024-09-19T07:42:08.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.869+0000 7f85b3476640 1 -- 172.21.15.103:0/2190374403 shutdown_connections 2024-09-19T07:42:08.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.869+0000 7f85b3476640 1 --2- 172.21.15.103:0/2190374403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 0x7f85ac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.869+0000 7f85b3476640 1 --2- 172.21.15.103:0/2190374403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 0x7f85ac1089f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.869+0000 7f85b3476640 1 --2- 172.21.15.103:0/2190374403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ac103ff0 0x7f85ac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.869+0000 7f85b3476640 1 -- 172.21.15.103:0/2190374403 >> 172.21.15.103:0/2190374403 conn(0x7f85ac0ffab0 msgr2=0x7f85ac101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.870+0000 7f85b3476640 1 -- 172.21.15.103:0/2190374403 shutdown_connections 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.870+0000 7f85b3476640 1 -- 172.21.15.103:0/2190374403 wait complete. 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 Processor -- start 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 -- start start 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ac103ff0 0x7f85ac076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 0x7f85ac076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 0x7f85ac077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1141c0 con 0x7f85ac109120 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac114040 con 0x7f85ac1049a0 2024-09-19T07:42:08.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.871+0000 7f85b3476640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac114340 con 0x7f85ac103ff0 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 0x7f85ac077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 0x7f85ac077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59012/0 (socket says 172.21.15.103:59012) 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 -- 172.21.15.103:0/2517900358 learned_addr learned my addr 172.21.15.103:0/2517900358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1212640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ac103ff0 0x7f85ac076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 0x7f85ac076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 -- 172.21.15.103:0/2517900358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ac103ff0 msgr2=0x7f85ac076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ac103ff0 0x7f85ac076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 -- 172.21.15.103:0/2517900358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 msgr2=0x7f85ac076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 0x7f85ac076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 -- 172.21.15.103:0/2517900358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac1aabb0 con 0x7f85ac109120 2024-09-19T07:42:08.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b1a13640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 0x7f85ac077180 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f85a400eb40 tx=0x7f85a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:08.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85a27fc640 1 -- 172.21.15.103:0/2517900358 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4004b20 con 0x7f85ac109120 2024-09-19T07:42:08.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac1aaea0 con 0x7f85ac109120 2024-09-19T07:42:08.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85a27fc640 1 -- 172.21.15.103:0/2517900358 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a4004cc0 con 0x7f85ac109120 2024-09-19T07:42:08.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.872+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1ab3b0 con 0x7f85ac109120 2024-09-19T07:42:08.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.873+0000 7f85a27fc640 1 -- 172.21.15.103:0/2517900358 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40106e0 con 0x7f85ac109120 2024-09-19T07:42:08.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.876+0000 7f85b584f740 1 Processor -- start 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.876+0000 7f85a27fc640 1 -- 172.21.15.103:0/2517900358 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f85a4005020 con 0x7f85ac109120 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b584f740 1 -- start start 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b584f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22a8b40 0x55b3b22a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b584f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b584f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22b2370 0x55b3b22b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b584f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b3b220d2f0 con 0x55b3b22b2370 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b584f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3b21b4cf0 con 0x55b3b22a75d0 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b584f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b3b22092b0 con 0x55b3b22a8b40 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:49174/0 (socket says 172.21.15.103:49174) 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b0a11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22a8b40 0x55b3b22a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 -- 172.21.15.103:0/2563930447 learned_addr learned my addr 172.21.15.103:0/2563930447 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22b2370 0x55b3b22b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 -- 172.21.15.103:0/2563930447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22a8b40 msgr2=0x55b3b22a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85a27fc640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f85980a33b0 0x7f85980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22a8b40 0x55b3b22a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 -- 172.21.15.103:0/2563930447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22b2370 msgr2=0x55b3b22b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22b2370 0x55b3b22b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b1212640 1 -- 172.21.15.103:0/2563930447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b21993a0 con 0x55b3b22a75d0 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.877+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22a8b40 0x55b3b22a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.878+0000 7f85b1a13640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f85980a33b0 0x7f85980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.878+0000 7f85b1212640 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22a7a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009a40 tx=0x7f85a8009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a8ff65eb8e7c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.878+0000 7f85a27fc640 1 -- 172.21.15.103:0/2517900358 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f85a40d0300 con 0x7f85ac109120 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.878+0000 7f85977fe640 1 -- 172.21.15.103:0/2563930447 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8017070 con 0x55b3b22a75d0 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.878+0000 7f85977fe640 1 -- 172.21.15.103:0/2563930447 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a8013470 con 0x55b3b22a75d0 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.878+0000 7f85977fe640 1 -- 172.21.15.103:0/2563930447 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8012610 con 0x55b3b22a75d0 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.879+0000 7f85b584f740 1 -- 172.21.15.103:0/2563930447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 msgr2=0x55b3b22a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.879+0000 7f85b584f740 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22a7a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009a40 tx=0x7f85a8009f00 comp rx=0 tx=0).stop 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.879+0000 7f85b1a13640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f85980a33b0 0x7f85980a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85a40ce060 tx=0x7f85a4009a70 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.879+0000 7f85b584f740 1 -- 172.21.15.103:0/2563930447 shutdown_connections 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.879+0000 7f85b584f740 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22b2370 0x55b3b22b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.879+0000 7f85b584f740 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22a7a10 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009a40 tx=0x7f85a8009f00 comp rx=0 tx=0).stop 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.880+0000 7f85b584f740 1 --2- 172.21.15.103:0/2563930447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22a8b40 0x55b3b22a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.880+0000 7f85b584f740 1 -- 172.21.15.103:0/2563930447 >> 172.21.15.103:0/2563930447 conn(0x55b3b2270310 msgr2=0x55b3b22a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.880+0000 7f85b584f740 1 -- 172.21.15.103:0/2563930447 shutdown_connections 2024-09-19T07:42:08.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.880+0000 7f85b584f740 1 -- 172.21.15.103:0/2563930447 wait complete. 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.881+0000 7f85b584f740 1 -- start start 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.881+0000 7f85b584f740 1 Processor -- start 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.881+0000 7f85b584f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b584f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b584f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22b2370 0x55b3b22af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b0a11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b1212640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b0a11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:49180/0 (socket says 172.21.15.103:49180) 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b0a11640 1 -- 172.21.15.103:0/2520350370 learned_addr learned my addr 172.21.15.103:0/2520350370 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:08.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b3b22b5070 con 0x55b3b22a8b40 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3b21c8060 con 0x55b3b22a75d0 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b3b22b51f0 con 0x55b3b22b2370 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b584f740 10 client.0 _ll_register_callbacks cb 0x55b3b1f1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22b2370 0x55b3b22af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b1212640 1 -- 172.21.15.103:0/2520350370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22b2370 msgr2=0x55b3b22af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b1212640 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22b2370 0x55b3b22af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b1212640 1 -- 172.21.15.103:0/2520350370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 msgr2=0x55b3b22b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b1212640 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b1212640 1 -- 172.21.15.103:0/2520350370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3b1f7e3c0 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.882+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22b2370 0x55b3b22af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f85b1212640 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22aef00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3ba95beeec6da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f8594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f8594ff9640 1 -- 172.21.15.103:0/2520350370 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8017050 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a8013470 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f8594ff9640 1 -- 172.21.15.103:0/2520350370 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f85a801c450 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f8594ff9640 1 -- 172.21.15.103:0/2520350370 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f85a801c670 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f85b584f740 10 client.16362 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b3b22c0a10 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f8594ff9640 1 -- 172.21.15.103:0/2520350370 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a8013470 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.883+0000 7f8594ff9640 1 -- 172.21.15.103:0/2520350370 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f85a802f9f0 con 0x55b3b22a8b40 2024-09-19T07:42:08.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b584f740 10 client.16362 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b584f740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b584f740 20 client.16362 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b584f740 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22c1d60 0x55b3b22c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b584f740 4 client.16362 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3b221a860 con 0x55b3b22c1d60 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22c1d60 0x55b3b22c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f8594ff9640 10 client.16362 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.884+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22c1d60 0x55b3b22c4150 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc74442338a4085 server_cookie=4c3c8e9486788939 in_seq=0 out_seq=0 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f8594ff9640 1 -- 172.21.15.103:0/2520350370 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3b221a860 con 0x55b3b22c1d60 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f8594ff9640 10 client.16362 handle_command_reply: tid=0 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 client.16362 shutdown 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 msgr2=0x55b3b22aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22aef00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 shutdown_connections 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22c1d60 0x55b3b22c4150 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f8594ff9640 0 client.16362 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b22b2370 0x55b3b22af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22aef00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 --2- 172.21.15.103:0/2520350370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 >> 172.21.15.103:0/2520350370 conn(0x55b3b2270310 msgr2=0x55b3b22a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 shutdown_connections 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.885+0000 7f85b584f740 1 -- 172.21.15.103:0/2520350370 wait complete. 2024-09-19T07:42:08.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.886+0000 7f85b584f740 20 client.16362 trim_cache size 0 max 16384 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.904+0000 7f85b584f740 1 Processor -- start 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b584f740 1 -- start start 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b584f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b584f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a75d0 0x55b3b22b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b584f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a8b40 0x55b3b22b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b584f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b4000 con 0x55b3b22a75d0 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b584f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b5bd0 con 0x55b3b22a8b40 2024-09-19T07:42:08.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b584f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b46d0 con 0x55b3b2270310 2024-09-19T07:42:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b0a11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b1212640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a75d0 0x55b3b22b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b0a11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22be340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49942/0 (socket says 172.21.15.103:49942) 2024-09-19T07:42:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b1212640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a75d0 0x55b3b22b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59042/0 (socket says 172.21.15.103:59042) 2024-09-19T07:42:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b0a11640 1 -- 172.21.15.103:0/2748405353 learned_addr learned my addr 172.21.15.103:0/2748405353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:08.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.905+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a8b40 0x55b3b22b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f85b0a11640 1 -- 172.21.15.103:0/2748405353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a8b40 msgr2=0x55b3b22b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a8b40 0x55b3b22b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f85b0a11640 1 -- 172.21.15.103:0/2748405353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a75d0 msgr2=0x55b3b22b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a75d0 0x55b3b22b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f85b0a11640 1 -- 172.21.15.103:0/2748405353 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3b22a8750 con 0x55b3b2270310 2024-09-19T07:42:08.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22be340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5830f1da89ebe55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:08.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f8596ffd640 1 -- 172.21.15.103:0/2748405353 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c050040 con 0x55b3b2270310 2024-09-19T07:42:08.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f8596ffd640 1 -- 172.21.15.103:0/2748405353 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c004780 con 0x55b3b2270310 2024-09-19T07:42:08.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.906+0000 7f8596ffd640 1 -- 172.21.15.103:0/2748405353 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c035040 con 0x55b3b2270310 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.907+0000 7f85b584f740 1 -- 172.21.15.103:0/2748405353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 msgr2=0x55b3b22be340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.907+0000 7f85b584f740 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22be340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f859c000c00 tx=0x7f859c00b040 comp rx=0 tx=0).stop 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 -- 172.21.15.103:0/2748405353 shutdown_connections 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a8b40 0x55b3b22b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a75d0 0x55b3b22b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 --2- 172.21.15.103:0/2748405353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22be340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 -- 172.21.15.103:0/2748405353 >> 172.21.15.103:0/2748405353 conn(0x55b3b22a71f0 msgr2=0x55b3b22b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 -- 172.21.15.103:0/2748405353 shutdown_connections 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 -- 172.21.15.103:0/2748405353 wait complete. 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.908+0000 7f85b584f740 1 -- start start 2024-09-19T07:42:08.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 1 Processor -- start 2024-09-19T07:42:08.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b5500 con 0x55b3b22a8b40 2024-09-19T07:42:08.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b4da0 con 0x55b3b22a75d0 2024-09-19T07:42:08.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b1fe0 con 0x55b3b2270310 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b584f740 10 client.0 _ll_register_callbacks cb 0x7f85a80098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b0a11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b1212640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b0a11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b1212640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:49206/0 (socket says 172.21.15.103:49206) 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b1212640 1 -- 172.21.15.103:0/2692295407 learned_addr learned my addr 172.21.15.103:0/2692295407 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.909+0000 7f85b1212640 1 -- 172.21.15.103:0/2692295407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 msgr2=0x55b3b22b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b1212640 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b1212640 1 -- 172.21.15.103:0/2692295407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 msgr2=0x55b3b22b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b1212640 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b1212640 1 -- 172.21.15.103:0/2692295407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3b22b8950 con 0x55b3b22a75d0 2024-09-19T07:42:08.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b1212640 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b4000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85a8006a50 tx=0x7f85a8026820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=367f13d547e2b76a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f8594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f8594ff9640 1 -- 172.21.15.103:0/2692295407 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8017050 con 0x55b3b22a75d0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f8594ff9640 1 -- 172.21.15.103:0/2692295407 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f85a8004220 con 0x55b3b22a75d0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859c004780 con 0x55b3b22a75d0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f8594ff9640 1 -- 172.21.15.103:0/2692295407 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f85a8004420 con 0x55b3b22a75d0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b584f740 10 client.25871 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b3b232ca00 con 0x55b3b22a75d0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f8594ff9640 1 -- 172.21.15.103:0/2692295407 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a8013e10 con 0x55b3b22a75d0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f8594ff9640 1 -- 172.21.15.103:0/2692295407 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f85a8003720 con 0x55b3b22a75d0 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b584f740 10 client.25871 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b584f740 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.910+0000 7f85b584f740 20 client.25871 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b584f740 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22b2370 0x55b3b232f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:08.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b584f740 4 client.25871 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:08.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f859c004780 con 0x55b3b22b2370 2024-09-19T07:42:08.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22b2370 0x55b3b232f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:08.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f8594ff9640 10 client.25871 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:08.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b0a11640 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22b2370 0x55b3b232f0c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24bfdd26cdde105 server_cookie=a6524c0530b3403d in_seq=0 out_seq=0 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f8594ff9640 1 -- 172.21.15.103:0/2692295407 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f859c004780 con 0x55b3b22b2370 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f8594ff9640 10 client.25871 handle_command_reply: tid=0 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b584f740 1 client.25871 shutdown 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 msgr2=0x55b3b22b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.911+0000 7f85b584f740 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b4000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85a8006a50 tx=0x7f85a8026820 comp rx=0 tx=0).stop 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 shutdown_connections 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55b3b22b2370 0x55b3b232f0c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b3b22a8b40 0x55b3b22b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3b22a75d0 0x55b3b22b4000 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 --2- 172.21.15.103:0/2692295407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3b2270310 0x55b3b22b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 >> 172.21.15.103:0/2692295407 conn(0x55b3b22a71f0 msgr2=0x55b3b22ad6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:08.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f8594ff9640 0 client.25871 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:08.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 shutdown_connections 2024-09-19T07:42:08.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 1 -- 172.21.15.103:0/2692295407 wait complete. 2024-09-19T07:42:08.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.912+0000 7f85b584f740 20 client.25871 trim_cache size 0 max 16384 2024-09-19T07:42:08.918 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:08.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f85980a33b0 msgr2=0x7f85980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f85980a33b0 0x7f85980a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85a40ce060 tx=0x7f85a4009a70 comp rx=0 tx=0).stop 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 msgr2=0x7f85ac077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 0x7f85ac077180 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f85a400eb40 tx=0x7f85a400c2d0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 shutdown_connections 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f85980a33b0 0x7f85980a5870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85ac109120 0x7f85ac077180 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85ac1049a0 0x7f85ac076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 --2- 172.21.15.103:0/2517900358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ac103ff0 0x7f85ac076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.913+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 >> 172.21.15.103:0/2517900358 conn(0x7f85ac0ffab0 msgr2=0x7f85ac1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.914+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 shutdown_connections 2024-09-19T07:42:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:08.914+0000 7f85b3476640 1 -- 172.21.15.103:0/2517900358 wait complete. 2024-09-19T07:42:09.804 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:09 smithi103 ceph-mon[31201]: pgmap v128: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.20k op/s 2024-09-19T07:42:09.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:09 smithi134 ceph-mon[35414]: pgmap v128: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.20k op/s 2024-09-19T07:42:09.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:09 smithi174 ceph-mon[35370]: pgmap v128: 97 pgs: 97 active+clean; 2.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 MiB/s wr, 1.20k op/s 2024-09-19T07:42:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 215.0 s, 2819.8 tps, lat 35.479 ms stddev 38.181 2024-09-19T07:42:10.806 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:42:10.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:42:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:11.623 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:11 smithi103 ceph-mon[31201]: pgmap v129: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 MiB/s wr, 1.81k op/s 2024-09-19T07:42:11.623 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:11 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:42:11.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.749+0000 7fb830fa6640 1 -- 172.21.15.103:0/2402937451 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c035020 con 0x7fb82c100a90 2024-09-19T07:42:11.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 -- 172.21.15.103:0/2402937451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c100a90 msgr2=0x7fb82c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 --2- 172.21.15.103:0/2402937451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c100a90 0x7fb82c100ef0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009930 tx=0x7fb81c02a420 comp rx=0 tx=0).stop 2024-09-19T07:42:11.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 -- 172.21.15.103:0/2402937451 shutdown_connections 2024-09-19T07:42:11.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 --2- 172.21.15.103:0/2402937451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c101430 0x7fb82c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 --2- 172.21.15.103:0/2402937451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c100a90 0x7fb82c100ef0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 --2- 172.21.15.103:0/2402937451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb82c106a90 0x7fb82c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 -- 172.21.15.103:0/2402937451 >> 172.21.15.103:0/2402937451 conn(0x7fb82c0fc7d0 msgr2=0x7fb82c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:11.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 -- 172.21.15.103:0/2402937451 shutdown_connections 2024-09-19T07:42:11.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.750+0000 7fb83420c640 1 -- 172.21.15.103:0/2402937451 wait complete. 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 Processor -- start 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 -- start start 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c100a90 0x7fb82c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c101430 0x7fb82c1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb82c106a90 0x7fb82c10cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c111fe0 con 0x7fb82c101430 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c111e60 con 0x7fb82c100a90 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.751+0000 7fb83420c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c112160 con 0x7fb82c106a90 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c100a90 0x7fb82c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c100a90 0x7fb82c1a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:49222/0 (socket says 172.21.15.103:49222) 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb8317a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c101430 0x7fb82c1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 -- 172.21.15.103:0/1311148083 learned_addr learned my addr 172.21.15.103:0/1311148083 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb8327a9640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb82c106a90 0x7fb82c10cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 -- 172.21.15.103:0/1311148083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb82c106a90 msgr2=0x7fb82c10cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb82c106a90 0x7fb82c10cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 -- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c101430 msgr2=0x7fb82c1a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c101430 0x7fb82c1a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb831fa8640 1 -- 172.21.15.103:0/1311148083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c10d390 con 0x7fb82c100a90 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.752+0000 7fb8317a7640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c101430 0x7fb82c1a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.753+0000 7fb831fa8640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c100a90 0x7fb82c1a4790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb82800d770 tx=0x7fb82800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.753+0000 7fb822ffd640 1 -- 172.21.15.103:0/1311148083 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828017070 con 0x7fb82c100a90 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.753+0000 7fb822ffd640 1 -- 172.21.15.103:0/1311148083 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb828005500 con 0x7fb82c100a90 2024-09-19T07:42:11.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.753+0000 7fb822ffd640 1 -- 172.21.15.103:0/1311148083 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828004e00 con 0x7fb82c100a90 2024-09-19T07:42:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.753+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c10d680 con 0x7fb82c100a90 2024-09-19T07:42:11.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.753+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c1ab560 con 0x7fb82c100a90 2024-09-19T07:42:11.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.757+0000 7fb8365e5740 1 Processor -- start 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.757+0000 7fb8365e5740 1 -- start start 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.757+0000 7fb822ffd640 1 -- 172.21.15.103:0/1311148083 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb828004fa0 con 0x7fb82c100a90 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48ddb60 0x5625d48ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d48e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48e8d40 0x5625d48eb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8365e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5625d4843280 con 0x5625d48de480 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8365e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625d47eab10 con 0x5625d48e8d40 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8365e5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625d483f240 con 0x5625d48ddb60 2024-09-19T07:42:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8327a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48ddb60 0x5625d48ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8317a7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48e8d40 0x5625d48eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8327a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48ddb60 0x5625d48ddf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49960/0 (socket says 172.21.15.103:49960) 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb8327a9640 1 -- 172.21.15.103:0/4117122326 learned_addr learned my addr 172.21.15.103:0/4117122326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb822ffd640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb8180a34b0 0x7fb8180a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.758+0000 7fb831fa8640 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d48e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb8327a9640 1 -- 172.21.15.103:0/4117122326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48e8d40 msgr2=0x5625d48eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb8327a9640 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48e8d40 0x5625d48eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb8327a9640 1 -- 172.21.15.103:0/4117122326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 msgr2=0x5625d48e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb822ffd640 1 -- 172.21.15.103:0/1311148083 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb8280cf710 con 0x7fb82c100a90 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb8327a9640 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d48e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb8327a9640 1 -- 172.21.15.103:0/4117122326 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d47f71f0 con 0x5625d48ddb60 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb831fa8640 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d48e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:11.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb8327a9640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb8180a34b0 0x7fb8180a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.759+0000 7fb8327a9640 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48ddb60 0x5625d48ddf40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb824010510 tx=0x7fb824009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb10c0fc459b9a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:11.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.761+0000 7fb817fff640 1 -- 172.21.15.103:0/4117122326 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824023070 con 0x5625d48ddb60 2024-09-19T07:42:11.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.761+0000 7fb817fff640 1 -- 172.21.15.103:0/4117122326 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824013bc0 con 0x5625d48ddb60 2024-09-19T07:42:11.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.761+0000 7fb817fff640 1 -- 172.21.15.103:0/4117122326 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82400e740 con 0x5625d48ddb60 2024-09-19T07:42:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 -- 172.21.15.103:0/4117122326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48ddb60 msgr2=0x5625d48ddf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8327a9640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb8180a34b0 0x7fb8180a5970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb824020000 tx=0x7fb82400e9e0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48ddb60 0x5625d48ddf40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb824010510 tx=0x7fb824009f90 comp rx=0 tx=0).stop 2024-09-19T07:42:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 -- 172.21.15.103:0/4117122326 shutdown_connections 2024-09-19T07:42:11.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48e8d40 0x5625d48eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d48e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 --2- 172.21.15.103:0/4117122326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48ddb60 0x5625d48ddf40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 -- 172.21.15.103:0/4117122326 >> 172.21.15.103:0/4117122326 conn(0x5625d48da0f0 msgr2=0x5625d48da4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 -- 172.21.15.103:0/4117122326 shutdown_connections 2024-09-19T07:42:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.762+0000 7fb8365e5740 1 -- 172.21.15.103:0/4117122326 wait complete. 2024-09-19T07:42:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 -- start start 2024-09-19T07:42:11.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 Processor -- start 2024-09-19T07:42:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48de480 0x5625d48e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48e8d40 0x5625d48e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5625d48eb910 con 0x5625d48e8d40 2024-09-19T07:42:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625d47fdf10 con 0x5625d48ddb60 2024-09-19T07:42:11.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.763+0000 7fb8365e5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625d48eba90 con 0x5625d48de480 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48de480 0x5625d48e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb831fa8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48de480 0x5625d48e6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49970/0 (socket says 172.21.15.103:49970) 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 -- 172.21.15.103:0/3358470022 learned_addr learned my addr 172.21.15.103:0/3358470022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8327a9640 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48e8d40 0x5625d48e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8365e5740 10 client.0 _ll_register_callbacks cb 0x5625d47c6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 -- 172.21.15.103:0/3358470022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 msgr2=0x5625d48e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 -- 172.21.15.103:0/3358470022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48e8d40 msgr2=0x5625d48e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48e8d40 0x5625d48e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.764+0000 7fb8317a7640 1 -- 172.21.15.103:0/3358470022 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d45b46a0 con 0x5625d48de480 2024-09-19T07:42:11.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.765+0000 7fb8317a7640 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48de480 0x5625d48e6150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009930 tx=0x7fb81c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b622091b10823ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:11.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.765+0000 7fb8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:42:11.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.766+0000 7fb8157fa640 1 -- 172.21.15.103:0/3358470022 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c035030 con 0x5625d48de480 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.766+0000 7fb8157fa640 1 -- 172.21.15.103:0/3358470022 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb81c0055d0 con 0x5625d48de480 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.766+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb824013bc0 con 0x5625d48de480 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.766+0000 7fb8157fa640 1 -- 172.21.15.103:0/3358470022 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb81c0317a0 con 0x5625d48de480 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.766+0000 7fb8365e5740 10 client.25843 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.766+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625d48f74f0 con 0x5625d48de480 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.766+0000 7fb8157fa640 1 -- 172.21.15.103:0/3358470022 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824013bc0 con 0x5625d48de480 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8157fa640 1 -- 172.21.15.103:0/3358470022 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb81c0035f0 con 0x5625d48de480 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8365e5740 10 client.25843 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8365e5740 10 client.25843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8365e5740 20 client.25843 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d48fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8365e5740 4 client.25843 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:11.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625d48507f0 con 0x5625d48f8a50 2024-09-19T07:42:11.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb831fa8640 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d48fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb8157fa640 10 client.25843 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:11.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.767+0000 7fb831fa8640 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d48fae70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa424bf3a40a4ab8 server_cookie=fef3e2e27f9d54a0 in_seq=0 out_seq=0 2024-09-19T07:42:11.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.768+0000 7fb8157fa640 1 -- 172.21.15.103:0/3358470022 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625d48507f0 con 0x5625d48f8a50 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.768+0000 7fb8157fa640 10 client.25843 handle_command_reply: tid=0 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.768+0000 7fb8365e5740 1 client.25843 shutdown 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48de480 msgr2=0x5625d48e6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48de480 0x5625d48e6150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009930 tx=0x7fb81c031400 comp rx=0 tx=0).stop 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 shutdown_connections 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d48fae70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8157fa640 0 client.25843 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48e8d40 0x5625d48e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48de480 0x5625d48e6150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3358470022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 >> 172.21.15.103:0/3358470022 conn(0x5625d48da0f0 msgr2=0x5625d48d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 shutdown_connections 2024-09-19T07:42:11.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.769+0000 7fb8365e5740 1 -- 172.21.15.103:0/3358470022 wait complete. 2024-09-19T07:42:11.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.770+0000 7fb8365e5740 20 client.25843 trim_cache size 0 max 16384 2024-09-19T07:42:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 Processor -- start 2024-09-19T07:42:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 -- start start 2024-09-19T07:42:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d4900740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d4900c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1b3ef0 con 0x5625d48de480 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1b5ac0 con 0x5625d48ddb60 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8365e5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1b45c0 con 0x5625d48e8d40 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8317a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d4900740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.795+0000 7fb8317a7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d4900740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59110/0 (socket says 172.21.15.103:59110) 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8317a7640 1 -- 172.21.15.103:0/2170736080 learned_addr learned my addr 172.21.15.103:0/2170736080 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8327a9640 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d4900c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb831fa8640 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8317a7640 1 -- 172.21.15.103:0/2170736080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 msgr2=0x5625d4900c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8317a7640 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d4900c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8317a7640 1 -- 172.21.15.103:0/2170736080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 msgr2=0x5625d48e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8317a7640 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8317a7640 1 -- 172.21.15.103:0/2170736080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d48de2b0 con 0x5625d48de480 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8317a7640 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d4900740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009850 tx=0x7fb81c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f6b917d80be5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8177fe640 1 -- 172.21.15.103:0/2170736080 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c035070 con 0x5625d48de480 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8177fe640 1 -- 172.21.15.103:0/2170736080 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c004ba0 con 0x5625d48de480 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.796+0000 7fb8177fe640 1 -- 172.21.15.103:0/2170736080 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c031c30 con 0x5625d48de480 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.797+0000 7fb8365e5740 1 -- 172.21.15.103:0/2170736080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 msgr2=0x5625d4900740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.797+0000 7fb8365e5740 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d4900740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009850 tx=0x7fb81c005050 comp rx=0 tx=0).stop 2024-09-19T07:42:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.797+0000 7fb8365e5740 1 -- 172.21.15.103:0/2170736080 shutdown_connections 2024-09-19T07:42:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.797+0000 7fb8365e5740 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d4900c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.798+0000 7fb8365e5740 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48de480 0x5625d4900740 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.798+0000 7fb8365e5740 1 --2- 172.21.15.103:0/2170736080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48ddb60 0x5625d48e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.798+0000 7fb8365e5740 1 -- 172.21.15.103:0/2170736080 >> 172.21.15.103:0/2170736080 conn(0x5625d48da0f0 msgr2=0x5625d48e7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.798+0000 7fb8365e5740 1 -- 172.21.15.103:0/2170736080 shutdown_connections 2024-09-19T07:42:11.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.798+0000 7fb8365e5740 1 -- 172.21.15.103:0/2170736080 wait complete. 2024-09-19T07:42:11.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.798+0000 7fb8365e5740 1 -- start start 2024-09-19T07:42:11.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.798+0000 7fb8365e5740 1 Processor -- start 2024-09-19T07:42:11.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48ddb60 0x5625d48ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48de480 0x5625d48ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8365e5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d48ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8365e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1b53f0 con 0x5625d48ddb60 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8365e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1b4c90 con 0x5625d48de480 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8365e5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1b1ed0 con 0x5625d48e8d40 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8365e5740 10 client.0 _ll_register_callbacks cb 0x7fb81c004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8327a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d48ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8327a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d48ed6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49988/0 (socket says 172.21.15.103:49988) 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb8327a9640 1 -- 172.21.15.103:0/3377412404 learned_addr learned my addr 172.21.15.103:0/3377412404 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.799+0000 7fb831fa8640 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48ddb60 0x5625d48ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb8317a7640 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48de480 0x5625d48ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb831fa8640 1 -- 172.21.15.103:0/3377412404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 msgr2=0x5625d48ed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb831fa8640 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d48ed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb831fa8640 1 -- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48de480 msgr2=0x5625d48ed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:11.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb831fa8640 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48de480 0x5625d48ed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb831fa8640 1 -- 172.21.15.103:0/3377412404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d4962910 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb831fa8640 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48ddb60 0x5625d48ecc60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb8280cce10 tx=0x7fb828086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fd2619cfacbb29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb8157fa640 1 -- 172.21.15.103:0/3377412404 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8280b3030 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb8157fa640 1 -- 172.21.15.103:0/3377412404 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb828005290 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.800+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c004ba0 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8157fa640 1 -- 172.21.15.103:0/3377412404 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb828020040 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 10 client.16386 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625d4963bf0 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8157fa640 1 -- 172.21.15.103:0/3377412404 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb81c004ba0 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8157fa640 1 -- 172.21.15.103:0/3377412404 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fb8280cd090 con 0x5625d48ddb60 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 10 client.16386 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:11.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:11.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 20 client.16386 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:11.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d49663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:11.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 4 client.16386 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:42:11.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5625d484ad00 con 0x5625d48f8a50 2024-09-19T07:42:11.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.801+0000 7fb8317a7640 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d49663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.802+0000 7fb8317a7640 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d49663f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361fc0eb64e70164 server_cookie=6dabf9cab48a8cc6 in_seq=0 out_seq=0 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.802+0000 7fb8157fa640 10 client.16386 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.804+0000 7fb8157fa640 1 -- 172.21.15.103:0/3377412404 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5625d484ad00 con 0x5625d48f8a50 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.804+0000 7fb8157fa640 10 client.16386 handle_command_reply: tid=0 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.804+0000 7fb8365e5740 1 client.16386 shutdown 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.804+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48ddb60 msgr2=0x5625d48ecc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.804+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48ddb60 0x5625d48ecc60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb8280cce10 tx=0x7fb828086040 comp rx=0 tx=0).stop 2024-09-19T07:42:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.805+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 shutdown_connections 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.805+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5625d48f8a50 0x5625d49663f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.805+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625d48e8d40 0x5625d48ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.805+0000 7fb8157fa640 0 client.16386 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.805+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625d48de480 0x5625d48ed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.805+0000 7fb8365e5740 1 --2- 172.21.15.103:0/3377412404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625d48ddb60 0x5625d48ecc60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.805+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 >> 172.21.15.103:0/3377412404 conn(0x5625d48da0f0 msgr2=0x5625d48d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.806+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 shutdown_connections 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.806+0000 7fb8365e5740 1 -- 172.21.15.103:0/3377412404 wait complete. 2024-09-19T07:42:11.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.806+0000 7fb8365e5740 20 client.16386 trim_cache size 0 max 16384 2024-09-19T07:42:11.809 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:42:11.809 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:42:11.809 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "bff93521-6f6a-4301-9994-af603fd30715", 2024-09-19T07:42:11.809 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:42:11.809 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:42:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb8180a34b0 msgr2=0x7fb8180a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb8180a34b0 0x7fb8180a5970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb824020000 tx=0x7fb82400e9e0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c100a90 msgr2=0x7fb82c1a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c100a90 0x7fb82c1a4790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb82800d770 tx=0x7fb82800dc30 comp rx=0 tx=0).stop 2024-09-19T07:42:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 shutdown_connections 2024-09-19T07:42:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb8180a34b0 0x7fb8180a5970 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb82c106a90 0x7fb82c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb82c101430 0x7fb82c1a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 --2- 172.21.15.103:0/1311148083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c100a90 0x7fb82c1a4790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:11.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 >> 172.21.15.103:0/1311148083 conn(0x7fb82c0fc7d0 msgr2=0x7fb82c0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:11.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 shutdown_connections 2024-09-19T07:42:11.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:11.808+0000 7fb83420c640 1 -- 172.21.15.103:0/1311148083 wait complete. 2024-09-19T07:42:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:11 smithi134 ceph-mon[35414]: pgmap v129: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 MiB/s wr, 1.81k op/s 2024-09-19T07:42:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:11 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:42:11.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:11 smithi174 ceph-mon[35370]: pgmap v129: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 MiB/s wr, 1.81k op/s 2024-09-19T07:42:11.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:11 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:42:12.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:42:12.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:42:12.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:42:12.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:42:12.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:42:12.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:42:12.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:42:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:42:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:42:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:12.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:13 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:42:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:13 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:13 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:42:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:13 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:13 smithi134 ceph-mon[35414]: pgmap v130: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 1.51k op/s 2024-09-19T07:42:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:13 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:42:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:13 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:13 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:42:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:13 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:13 smithi174 ceph-mon[35370]: pgmap v130: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 1.51k op/s 2024-09-19T07:42:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:13 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:42:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:13 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:13 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:42:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:13 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:42:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:13 smithi103 ceph-mon[31201]: pgmap v130: 97 pgs: 97 active+clean; 2.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 MiB/s wr, 1.51k op/s 2024-09-19T07:42:14.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:42:15.316 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 220.0 s, 2763.0 tps, lat 36.147 ms stddev 38.955 2024-09-19T07:42:15.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:15 smithi103 ceph-mon[31201]: pgmap v131: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-09-19T07:42:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.827+0000 7fda68535640 1 -- 172.21.15.103:0/3739639785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 msgr2=0x7fda60106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.827+0000 7fda68535640 1 --2- 172.21.15.103:0/3739639785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 0x7fda60106e50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009860 tx=0x7fda5c02a340 comp rx=0 tx=0).stop 2024-09-19T07:42:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 -- 172.21.15.103:0/3739639785 shutdown_connections 2024-09-19T07:42:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 --2- 172.21.15.103:0/3739639785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60101410 0x7fda6010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 --2- 172.21.15.103:0/3739639785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60100a70 0x7fda60100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 --2- 172.21.15.103:0/3739639785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 0x7fda60106e50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 -- 172.21.15.103:0/3739639785 >> 172.21.15.103:0/3739639785 conn(0x7fda600fc790 msgr2=0x7fda600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:15.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 -- 172.21.15.103:0/3739639785 shutdown_connections 2024-09-19T07:42:15.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 -- 172.21.15.103:0/3739639785 wait complete. 2024-09-19T07:42:15.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:15 smithi134 ceph-mon[35414]: pgmap v131: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-09-19T07:42:15.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 Processor -- start 2024-09-19T07:42:15.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.828+0000 7fda68535640 1 -- start start 2024-09-19T07:42:15.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda68535640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60100a70 0x7fda60198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda68535640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60101410 0x7fda60198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda68535640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 0x7fda6019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda68535640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda60110e70 con 0x7fda60101410 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda68535640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda60110cf0 con 0x7fda60106a70 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda68535640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fda60110ff0 con 0x7fda60100a70 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda65ad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60101410 0x7fda60198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60100a70 0x7fda60198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda66ad2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 0x7fda6019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60100a70 0x7fda60198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50014/0 (socket says 172.21.15.103:50014) 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda65ad0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60101410 0x7fda60198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59142/0 (socket says 172.21.15.103:59142) 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda662d1640 1 -- 172.21.15.103:0/441004250 learned_addr learned my addr 172.21.15.103:0/441004250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda66ad2640 1 -- 172.21.15.103:0/441004250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60100a70 msgr2=0x7fda60198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda66ad2640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60100a70 0x7fda60198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda66ad2640 1 -- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60101410 msgr2=0x7fda60198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda66ad2640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60101410 0x7fda60198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda66ad2640 1 -- 172.21.15.103:0/441004250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6019d3e0 con 0x7fda60106a70 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda662d1640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60100a70 0x7fda60198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda65ad0640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60101410 0x7fda60198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda66ad2640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 0x7fda6019cd00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda58009fd0 tx=0x7fda5800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.829+0000 7fda577fe640 1 -- 172.21.15.103:0/441004250 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5800cd60 con 0x7fda60106a70 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.830+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6019d6d0 con 0x7fda60106a70 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.830+0000 7fda577fe640 1 -- 172.21.15.103:0/441004250 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda58005020 con 0x7fda60106a70 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.830+0000 7fda577fe640 1 -- 172.21.15.103:0/441004250 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58010690 con 0x7fda60106a70 2024-09-19T07:42:15.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.830+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601a4fb0 con 0x7fda60106a70 2024-09-19T07:42:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.833+0000 7fda577fe640 1 -- 172.21.15.103:0/441004250 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fda58010830 con 0x7fda60106a70 2024-09-19T07:42:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.833+0000 7fda6a90e740 1 Processor -- start 2024-09-19T07:42:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda6a90e740 1 -- start start 2024-09-19T07:42:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142360 0x556c14142740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142c80 0x556c14151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14153e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda6a90e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c140a82e0 con 0x556c14151a90 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda6a90e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c1404fce0 con 0x556c14142360 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda6a90e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c140a42a0 con 0x556c14142c80 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142c80 0x556c14151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda65ad0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142360 0x556c14142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142c80 0x556c14151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50030/0 (socket says 172.21.15.103:50030) 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 -- 172.21.15.103:0/1980533221 learned_addr learned my addr 172.21.15.103:0/1980533221 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda65ad0640 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 -- 172.21.15.103:0/1980533221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142360 msgr2=0x556c14142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142360 0x556c14142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 -- 172.21.15.103:0/1980533221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 msgr2=0x556c14153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.834+0000 7fda662d1640 1 -- 172.21.15.103:0/1980533221 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c14034440 con 0x556c14142c80 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.835+0000 7fda577fe640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fda4c0a33b0 0x7fda4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.835+0000 7fda65ad0640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fda4c0a33b0 0x7fda4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.835+0000 7fda577fe640 1 -- 172.21.15.103:0/441004250 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fda580d0340 con 0x7fda60106a70 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.835+0000 7fda662d1640 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142c80 0x556c14151550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009980 tx=0x7fda5c0095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada33cf73964c3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.835+0000 7fda54ff9640 1 -- 172.21.15.103:0/1980533221 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c037040 con 0x556c14142c80 2024-09-19T07:42:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.835+0000 7fda54ff9640 1 -- 172.21.15.103:0/1980533221 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda5c030780 con 0x556c14142c80 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.835+0000 7fda54ff9640 1 -- 172.21.15.103:0/1980533221 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c003cd0 con 0x556c14142c80 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda6a90e740 1 -- 172.21.15.103:0/1980533221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142c80 msgr2=0x556c14151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142c80 0x556c14151550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009980 tx=0x7fda5c0095a0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda65ad0640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fda4c0a33b0 0x7fda4c0a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x556c1413fb20 tx=0x7fda500046c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda6a90e740 1 -- 172.21.15.103:0/1980533221 shutdown_connections 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142c80 0x556c14151550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1980533221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142360 0x556c14142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.836+0000 7fda6a90e740 1 -- 172.21.15.103:0/1980533221 >> 172.21.15.103:0/1980533221 conn(0x556c1413ea30 msgr2=0x556c1413ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.837+0000 7fda6a90e740 1 -- 172.21.15.103:0/1980533221 shutdown_connections 2024-09-19T07:42:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.837+0000 7fda6a90e740 1 -- 172.21.15.103:0/1980533221 wait complete. 2024-09-19T07:42:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 -- start start 2024-09-19T07:42:15.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 Processor -- start 2024-09-19T07:42:15.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c141510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142c80 0x556c1414e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14151a90 0x556c1414eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c14154600 con 0x556c14142c80 2024-09-19T07:42:15.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c14063050 con 0x556c14151a90 2024-09-19T07:42:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c14154780 con 0x556c14142360 2024-09-19T07:42:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda6a90e740 10 client.0 _ll_register_callbacks cb 0x556c140a3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c141510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c141510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50036/0 (socket says 172.21.15.103:50036) 2024-09-19T07:42:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.838+0000 7fda662d1640 1 -- 172.21.15.103:0/2229173966 learned_addr learned my addr 172.21.15.103:0/2229173966 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda66ad2640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14151a90 0x556c1414eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda662d1640 1 -- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14151a90 msgr2=0x556c1414eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda65ad0640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142c80 0x556c1414e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda662d1640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14151a90 0x556c1414eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda662d1640 1 -- 172.21.15.103:0/2229173966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142c80 msgr2=0x556c1414e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda662d1640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142c80 0x556c1414e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda662d1640 1 -- 172.21.15.103:0/2229173966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c13e193c0 con 0x556c14142360 2024-09-19T07:42:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda65ad0640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142c80 0x556c1414e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda662d1640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c141510d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda5c02a930 tx=0x7fda5c0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=715c63dc0bd02e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.839+0000 7fda49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda49ffb640 1 -- 172.21.15.103:0/2229173966 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c037040 con 0x556c14142360 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda49ffb640 1 -- 172.21.15.103:0/2229173966 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fda5c030930 con 0x556c14142360 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda5c030780 con 0x556c14142360 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda49ffb640 1 -- 172.21.15.103:0/2229173966 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fda5c032930 con 0x556c14142360 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 10 client.25861 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556c14154600 con 0x556c14142360 2024-09-19T07:42:15.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda49ffb640 1 -- 172.21.15.103:0/2229173966 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda5c030780 con 0x556c14142360 2024-09-19T07:42:15.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda49ffb640 1 -- 172.21.15.103:0/2229173966 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fda5c03db00 con 0x556c14142360 2024-09-19T07:42:15.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 10 client.25861 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:15.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:15.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 20 client.25861 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:15.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141655c0 0x556c141679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 4 client.25861 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:15.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c140b5850 con 0x556c141655c0 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.840+0000 7fda65ad0640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141655c0 0x556c141679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.841+0000 7fda49ffb640 10 client.25861 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.841+0000 7fda65ad0640 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141655c0 0x556c141679b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7cd51fd1250e372 server_cookie=8705d613043df089 in_seq=0 out_seq=0 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda49ffb640 1 -- 172.21.15.103:0/2229173966 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c140b5850 con 0x556c141655c0 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda49ffb640 10 client.25861 handle_command_reply: tid=0 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 client.25861 shutdown 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 msgr2=0x556c141510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c141510d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda5c02a930 tx=0x7fda5c0035f0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 shutdown_connections 2024-09-19T07:42:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141655c0 0x556c141679b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda49ffb640 0 client.25861 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14151a90 0x556c1414eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142c80 0x556c1414e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2229173966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c141510d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 >> 172.21.15.103:0/2229173966 conn(0x556c1413ea30 msgr2=0x556c14144790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:15.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 shutdown_connections 2024-09-19T07:42:15.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 1 -- 172.21.15.103:0/2229173966 wait complete. 2024-09-19T07:42:15.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.842+0000 7fda6a90e740 20 client.25861 trim_cache size 0 max 16384 2024-09-19T07:42:15.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 Processor -- start 2024-09-19T07:42:15.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 -- start start 2024-09-19T07:42:15.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c1414cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 0x556c14150e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14144560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda601ab650 con 0x556c14151a90 2024-09-19T07:42:15.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda601ad220 con 0x556c14142c80 2024-09-19T07:42:15.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.867+0000 7fda6a90e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fda601abd20 con 0x556c14142360 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c1414cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c1414cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50038/0 (socket says 172.21.15.103:50038) 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 -- 172.21.15.103:0/2688986303 learned_addr learned my addr 172.21.15.103:0/2688986303 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda66ad2640 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 -- 172.21.15.103:0/2688986303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 msgr2=0x556c14150e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 0x556c14150e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 -- 172.21.15.103:0/2688986303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 msgr2=0x556c14144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda662d1640 1 -- 172.21.15.103:0/2688986303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c14142ab0 con 0x556c14142360 2024-09-19T07:42:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.868+0000 7fda66ad2640 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:15.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.873+0000 7fda662d1640 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c1414cec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009980 tx=0x7fda5c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4342c6c19d03c406 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:15.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.874+0000 7fda4bfff640 1 -- 172.21.15.103:0/2688986303 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c037040 con 0x556c14142360 2024-09-19T07:42:15.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.874+0000 7fda4bfff640 1 -- 172.21.15.103:0/2688986303 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda5c003660 con 0x556c14142360 2024-09-19T07:42:15.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.874+0000 7fda4bfff640 1 -- 172.21.15.103:0/2688986303 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c04e950 con 0x556c14142360 2024-09-19T07:42:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.875+0000 7fda6a90e740 1 -- 172.21.15.103:0/2688986303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 msgr2=0x556c1414cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.875+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c1414cec0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009980 tx=0x7fda5c005850 comp rx=0 tx=0).stop 2024-09-19T07:42:15.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.878+0000 7fda6a90e740 1 -- 172.21.15.103:0/2688986303 shutdown_connections 2024-09-19T07:42:15.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.878+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14151a90 0x556c14144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.878+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 0x556c14150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.878+0000 7fda6a90e740 1 --2- 172.21.15.103:0/2688986303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14142360 0x556c1414cec0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.878+0000 7fda6a90e740 1 -- 172.21.15.103:0/2688986303 >> 172.21.15.103:0/2688986303 conn(0x556c1413ea30 msgr2=0x556c14143e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:15.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.878+0000 7fda6a90e740 1 -- 172.21.15.103:0/2688986303 shutdown_connections 2024-09-19T07:42:15.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.878+0000 7fda6a90e740 1 -- 172.21.15.103:0/2688986303 wait complete. 2024-09-19T07:42:15.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.879+0000 7fda6a90e740 1 -- start start 2024-09-19T07:42:15.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.879+0000 7fda6a90e740 1 Processor -- start 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142360 0x556c14162c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 0x556c1415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda6a90e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14151a90 0x556c1415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda6a90e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda601acb50 con 0x556c14142360 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda6a90e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda601ac3f0 con 0x556c14142c80 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda6a90e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fda601a9630 con 0x556c14151a90 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda6a90e740 10 client.0 _ll_register_callbacks cb 0x7fda50011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda66ad2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14151a90 0x556c1415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda66ad2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14151a90 0x556c1415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:50048/0 (socket says 172.21.15.103:50048) 2024-09-19T07:42:15.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.880+0000 7fda66ad2640 1 -- 172.21.15.103:0/1052633116 learned_addr learned my addr 172.21.15.103:0/1052633116 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:15.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.881+0000 7fda662d1640 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142360 0x556c14162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.881+0000 7fda66ad2640 1 -- 172.21.15.103:0/1052633116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 msgr2=0x556c1415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:15.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.881+0000 7fda66ad2640 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 0x556c1415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.881+0000 7fda66ad2640 1 -- 172.21.15.103:0/1052633116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142360 msgr2=0x556c14162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:15.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.881+0000 7fda66ad2640 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142360 0x556c14162c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.881+0000 7fda66ad2640 1 -- 172.21.15.103:0/1052633116 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c1415ed60 con 0x556c14151a90 2024-09-19T07:42:15.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.882+0000 7fda66ad2640 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14151a90 0x556c1415e530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c6a0 tx=0x7fda58005290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98ccca21f6a43c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:15.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.882+0000 7fda49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:42:15.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.882+0000 7fda49ffb640 1 -- 172.21.15.103:0/1052633116 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5809ca30 con 0x556c14151a90 2024-09-19T07:42:15.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.883+0000 7fda49ffb640 1 -- 172.21.15.103:0/1052633116 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fda580a4db0 con 0x556c14151a90 2024-09-19T07:42:15.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.883+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda5c003660 con 0x556c14151a90 2024-09-19T07:42:15.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.883+0000 7fda49ffb640 1 -- 172.21.15.103:0/1052633116 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fda580a5240 con 0x556c14151a90 2024-09-19T07:42:15.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.884+0000 7fda6a90e740 10 client.25867 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:15.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.884+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556c141e11c0 con 0x556c14151a90 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda49ffb640 1 -- 172.21.15.103:0/1052633116 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda5c003660 con 0x556c14151a90 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda49ffb640 1 -- 172.21.15.103:0/1052633116 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fda58010b20 con 0x556c14151a90 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda6a90e740 10 client.25867 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda6a90e740 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda6a90e740 20 client.25867 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141e1890 0x556c141e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda6a90e740 4 client.25867 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.886+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c140afd60 con 0x556c141e1890 2024-09-19T07:42:15.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.887+0000 7fda662d1640 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141e1890 0x556c141e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:15.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.888+0000 7fda662d1640 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141e1890 0x556c141e3c80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bcf94e7e3380a7 server_cookie=bfe5384b4413ba5a in_seq=0 out_seq=0 2024-09-19T07:42:15.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.888+0000 7fda49ffb640 10 client.25867 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:15.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.888+0000 7fda49ffb640 1 -- 172.21.15.103:0/1052633116 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556c140afd60 con 0x556c141e1890 2024-09-19T07:42:15.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.889+0000 7fda49ffb640 10 client.25867 handle_command_reply: tid=0 2024-09-19T07:42:15.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.889+0000 7fda6a90e740 1 client.25867 shutdown 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14151a90 msgr2=0x556c1415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14151a90 0x556c1415e530 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c6a0 tx=0x7fda58005290 comp rx=0 tx=0).stop 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 shutdown_connections 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c141e1890 0x556c141e3c80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c14151a90 0x556c1415e530 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c14142c80 0x556c1415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 --2- 172.21.15.103:0/1052633116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c14142360 0x556c14162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.890+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 >> 172.21.15.103:0/1052633116 conn(0x556c1413ea30 msgr2=0x556c14144670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:15.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.891+0000 7fda49ffb640 0 client.25867 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:15.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.892+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 shutdown_connections 2024-09-19T07:42:15.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.892+0000 7fda6a90e740 1 -- 172.21.15.103:0/1052633116 wait complete. 2024-09-19T07:42:15.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.892+0000 7fda6a90e740 20 client.25867 trim_cache size 0 max 16384 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: "bff93521-6f6a-4301-9994-af603fd30715": { 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "bff93521-6f6a-4301-9994-af603fd30715", 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:15.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:15.896 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:42:15.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fda4c0a33b0 msgr2=0x7fda4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fda4c0a33b0 0x7fda4c0a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x556c1413fb20 tx=0x7fda500046c0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 msgr2=0x7fda6019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:15.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 0x7fda6019cd00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda58009fd0 tx=0x7fda5800ee50 comp rx=0 tx=0).stop 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 shutdown_connections 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fda4c0a33b0 0x7fda4c0a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda60106a70 0x7fda6019cd00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda60101410 0x7fda60198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 --2- 172.21.15.103:0/441004250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda60100a70 0x7fda60198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 >> 172.21.15.103:0/441004250 conn(0x7fda600fc790 msgr2=0x7fda6006a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 shutdown_connections 2024-09-19T07:42:15.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:15.895+0000 7fda68535640 1 -- 172.21.15.103:0/441004250 wait complete. 2024-09-19T07:42:15.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:15 smithi174 ceph-mon[35370]: pgmap v131: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 MiB/s wr, 1.72k op/s 2024-09-19T07:42:16.660 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:42:16.661 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T07:42:16.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:42:16.836 INFO:tasks.cephfs.filesystem:scrub status for tag:bff93521-6f6a-4301-9994-af603fd30715 - {'path': '/', 'tag': 'bff93521-6f6a-4301-9994-af603fd30715', 'options': 'recursive,force'} 2024-09-19T07:42:17.023 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:17.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.563+0000 7feba8c9c640 1 -- 172.21.15.103:0/1627409502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102ef0 msgr2=0x7feba410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.563+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1627409502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102ef0 0x7feba410da40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009ca0 tx=0x7feb9c02a950 comp rx=0 tx=0).stop 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.564+0000 7feba8c9c640 1 -- 172.21.15.103:0/1627409502 shutdown_connections 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.564+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1627409502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102ef0 0x7feba410da40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.564+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1627409502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feba4102570 0x7feba41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.564+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1627409502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feba4100e20 0x7feba4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.564+0000 7feba8c9c640 1 -- 172.21.15.103:0/1627409502 >> 172.21.15.103:0/1627409502 conn(0x7feba40fc7d0 msgr2=0x7feba40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.564+0000 7feba8c9c640 1 -- 172.21.15.103:0/1627409502 shutdown_connections 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.564+0000 7feba8c9c640 1 -- 172.21.15.103:0/1627409502 wait complete. 2024-09-19T07:42:17.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 Processor -- start 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 -- start start 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feba4100e20 0x7feba4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102570 0x7feba4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feba4102ef0 0x7feba419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feba410fe80 con 0x7feba4102570 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feba410fd00 con 0x7feba4102ef0 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba8c9c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feba4110000 con 0x7feba4100e20 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102570 0x7feba4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:17.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102570 0x7feba4196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59192/0 (socket says 172.21.15.103:59192) 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba1d9b640 1 -- 172.21.15.103:0/1220493646 learned_addr learned my addr 172.21.15.103:0/1220493646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba2d9d640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feba4102ef0 0x7feba419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.565+0000 7feba259c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feba4100e20 0x7feba4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba2d9d640 1 -- 172.21.15.103:0/1220493646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feba4100e20 msgr2=0x7feba4196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba2d9d640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feba4100e20 0x7feba4196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba2d9d640 1 -- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102570 msgr2=0x7feba4196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba2d9d640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102570 0x7feba4196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba2d9d640 1 -- 172.21.15.103:0/1220493646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba419b2f0 con 0x7feba4102ef0 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba1d9b640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102570 0x7feba4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba259c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feba4100e20 0x7feba4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:17.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba2d9d640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feba4102ef0 0x7feba419ac10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feb9c02ae70 tx=0x7feb9c030850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:17.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feb937fe640 1 -- 172.21.15.103:0/1220493646 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c048070 con 0x7feba4102ef0 2024-09-19T07:42:17.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feb937fe640 1 -- 172.21.15.103:0/1220493646 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c003e50 con 0x7feba4102ef0 2024-09-19T07:42:17.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba419b580 con 0x7feba4102ef0 2024-09-19T07:42:17.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feb937fe640 1 -- 172.21.15.103:0/1220493646 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c034020 con 0x7feba4102ef0 2024-09-19T07:42:17.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.566+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba41a2ec0 con 0x7feba4102ef0 2024-09-19T07:42:17.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.567+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb84005290 con 0x7feba4102ef0 2024-09-19T07:42:17.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.577+0000 7feb937fe640 1 -- 172.21.15.103:0/1220493646 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7feb9c04b070 con 0x7feba4102ef0 2024-09-19T07:42:17.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.578+0000 7feb937fe640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7feb8c0a5640 0x7feb8c0a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:17.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.579+0000 7feb937fe640 1 -- 172.21.15.103:0/1220493646 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7feb9c0f03c0 con 0x7feba4102ef0 2024-09-19T07:42:17.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.582+0000 7feba259c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7feb8c0a5640 0x7feb8c0a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:17.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.588+0000 7feba259c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7feb8c0a5640 0x7feb8c0a7b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb98009fc0 tx=0x7feb98005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:17.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.589+0000 7feb937fe640 1 -- 172.21.15.103:0/1220493646 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7feb9c080210 con 0x7feba4102ef0 2024-09-19T07:42:17.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:17 smithi103 ceph-mon[31201]: pgmap v132: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-09-19T07:42:17.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:17 smithi134 ceph-mon[35414]: pgmap v132: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-09-19T07:42:17.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.889+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb84005850 con 0x7feba4102ef0 2024-09-19T07:42:17.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.890+0000 7feb937fe640 1 -- 172.21.15.103:0/1220493646 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11996 (secure 0 0 0) 0x7feb9c040070 con 0x7feba4102ef0 2024-09-19T07:42:17.891 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:42:17.891 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":35,"btime":"2024-09-19T07:41:44:478751+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:41:43.518367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:42:17.892 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 35 2024-09-19T07:42:17.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7feb8c0a5640 msgr2=0x7feb8c0a7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:17.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7feb8c0a5640 0x7feb8c0a7b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb98009fc0 tx=0x7feb98005f90 comp rx=0 tx=0).stop 2024-09-19T07:42:17.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feba4102ef0 msgr2=0x7feba419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:17.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feba4102ef0 0x7feba419ac10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feb9c02ae70 tx=0x7feb9c030850 comp rx=0 tx=0).stop 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 shutdown_connections 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7feb8c0a5640 0x7feb8c0a7b00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feba4102ef0 0x7feba419ac10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feba4102570 0x7feba4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 --2- 172.21.15.103:0/1220493646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feba4100e20 0x7feba4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 >> 172.21.15.103:0/1220493646 conn(0x7feba40fc7d0 msgr2=0x7feba41068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 shutdown_connections 2024-09-19T07:42:17.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:17.892+0000 7feba8c9c640 1 -- 172.21.15.103:0/1220493646 wait complete. 2024-09-19T07:42:17.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:17 smithi174 ceph-mon[35370]: pgmap v132: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-09-19T07:42:18.643 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T07:42:18.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T07:42:18.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:18 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1220493646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:18.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:18 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1220493646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:18.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:18 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1220493646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:19.021 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.629+0000 7f0d74d28640 1 -- 172.21.15.103:0/848241500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 msgr2=0x7f0d701021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.629+0000 7f0d74d28640 1 --2- 172.21.15.103:0/848241500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 0x7f0d701021d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009950 tx=0x7f0d6002e850 comp rx=0 tx=0).stop 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 -- 172.21.15.103:0/848241500 shutdown_connections 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 --2- 172.21.15.103:0/848241500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d70106f20 0x7f0d7010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 --2- 172.21.15.103:0/848241500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d701027a0 0x7f0d701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 --2- 172.21.15.103:0/848241500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 0x7f0d701021d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 -- 172.21.15.103:0/848241500 >> 172.21.15.103:0/848241500 conn(0x7f0d700fd8f0 msgr2=0x7f0d700ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 -- 172.21.15.103:0/848241500 shutdown_connections 2024-09-19T07:42:19.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 -- 172.21.15.103:0/848241500 wait complete. 2024-09-19T07:42:19.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.630+0000 7f0d74d28640 1 Processor -- start 2024-09-19T07:42:19.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d74d28640 1 -- start start 2024-09-19T07:42:19.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d74d28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 0x7f0d70196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d74d28640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d701027a0 0x7f0d70196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d74d28640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d70106f20 0x7f0d7019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d74d28640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010fde0 con 0x7f0d70101df0 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d74d28640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010fc60 con 0x7f0d70106f20 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d74d28640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010ff60 con 0x7f0d701027a0 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d6e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 0x7f0d70196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d6e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 0x7f0d70196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58814/0 (socket says 172.21.15.103:58814) 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d65d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d701027a0 0x7f0d70196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d6e59c640 1 -- 172.21.15.103:0/2471953303 learned_addr learned my addr 172.21.15.103:0/2471953303 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d6ed9d640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d70106f20 0x7f0d7019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d65d9b640 1 -- 172.21.15.103:0/2471953303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d70106f20 msgr2=0x7f0d7019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d70106f20 0x7f0d7019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d65d9b640 1 -- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 msgr2=0x7f0d70196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 0x7f0d70196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.631+0000 7f0d65d9b640 1 -- 172.21.15.103:0/2471953303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7019b200 con 0x7f0d701027a0 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.632+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d701027a0 0x7f0d70196790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800e9f0 tx=0x7f0d5800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.632+0000 7f0d67fff640 1 -- 172.21.15.103:0/2471953303 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800ccc0 con 0x7f0d701027a0 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.632+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7019b4f0 con 0x7f0d701027a0 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.632+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701a2dd0 con 0x7f0d701027a0 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.632+0000 7f0d67fff640 1 -- 172.21.15.103:0/2471953303 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d58005020 con 0x7f0d701027a0 2024-09-19T07:42:19.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.632+0000 7f0d67fff640 1 -- 172.21.15.103:0/2471953303 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58004a00 con 0x7f0d701027a0 2024-09-19T07:42:19.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 Processor -- start 2024-09-19T07:42:19.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 -- start start 2024-09-19T07:42:19.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758816b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x557758821400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758821940 0x557758823d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55775877c2a0 con 0x557758817080 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557758723b10 con 0x557758816760 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d77101740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557758778260 con 0x557758821940 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d67fff640 1 -- 172.21.15.103:0/2471953303 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0d58004ba0 con 0x7f0d701027a0 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d6ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758816b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d6ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758816b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:36464/0 (socket says 172.21.15.103:36464) 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d6ed9d640 1 -- 172.21.15.103:0/1096240018 learned_addr learned my addr 172.21.15.103:0/1096240018 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.636+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x557758821400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758821940 0x557758823d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d6ed9d640 1 -- 172.21.15.103:0/1096240018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758821940 msgr2=0x557758823d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:19.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d6ed9d640 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758821940 0x557758823d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d6ed9d640 1 -- 172.21.15.103:0/1096240018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 msgr2=0x557758821400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:19.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d6ed9d640 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x557758821400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d6ed9d640 1 -- 172.21.15.103:0/1096240018 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577584edb90 con 0x557758816760 2024-09-19T07:42:19.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x557758821400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:19.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758821940 0x557758823d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:19.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d6ed9d640 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758816b40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009f60 tx=0x7f0d68009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e77e1fdc74c6cad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:19.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d64d99640 1 -- 172.21.15.103:0/1096240018 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6801e070 con 0x557758816760 2024-09-19T07:42:19.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d64d99640 1 -- 172.21.15.103:0/1096240018 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d68003c30 con 0x557758816760 2024-09-19T07:42:19.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d64d99640 1 -- 172.21.15.103:0/1096240018 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800fa20 con 0x557758816760 2024-09-19T07:42:19.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.637+0000 7f0d67fff640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d5c0a3400 0x7f0d5c0a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.638+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d5c0a3400 0x7f0d5c0a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.638+0000 7f0d67fff640 1 -- 172.21.15.103:0/2471953303 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d580cfb00 con 0x7f0d701027a0 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.638+0000 7f0d77101740 1 -- 172.21.15.103:0/1096240018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 msgr2=0x557758816b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.638+0000 7f0d77101740 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758816b40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009f60 tx=0x7f0d68009f90 comp rx=0 tx=0).stop 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d77101740 1 -- 172.21.15.103:0/1096240018 shutdown_connections 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d77101740 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758821940 0x557758823d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d77101740 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x557758821400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d77101740 1 --2- 172.21.15.103:0/1096240018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758816b40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d77101740 1 -- 172.21.15.103:0/1096240018 >> 172.21.15.103:0/1096240018 conn(0x557758812cf0 msgr2=0x5577588130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d77101740 1 -- 172.21.15.103:0/1096240018 shutdown_connections 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d77101740 1 -- 172.21.15.103:0/1096240018 wait complete. 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.639+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d5c0a3400 0x7f0d5c0a58c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x557758813b50 tx=0x7f0d60005d50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 -- start start 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 Processor -- start 2024-09-19T07:42:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x55775881e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758817080 0x55775881ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d6e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x55775881e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d6e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x55775881e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37820/0 (socket says 172.21.15.103:37820) 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d6e59c640 1 -- 172.21.15.103:0/437079763 learned_addr learned my addr 172.21.15.103:0/437079763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x55775881f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557758824510 con 0x557758821940 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5577587046e0 con 0x557758817080 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.640+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557758824690 con 0x557758816760 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d6e59c640 1 -- 172.21.15.103:0/437079763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758817080 msgr2=0x55775881ece0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758817080 0x55775881ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d6e59c640 1 -- 172.21.15.103:0/437079763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 msgr2=0x55775881f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x55775881f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d6e59c640 1 -- 172.21.15.103:0/437079763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577588361f0 con 0x557758816760 2024-09-19T07:42:19.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d77101740 10 client.0 _ll_register_callbacks cb 0x557758728a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:19.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:42:19.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x55775881e7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d60005e80 tx=0x7f0d60009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be4a3d8cfef7c223 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:19.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d68003c30 con 0x557758816760 2024-09-19T07:42:19.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d55ffb640 1 -- 172.21.15.103:0/437079763 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60050070 con 0x557758816760 2024-09-19T07:42:19.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.641+0000 7f0d55ffb640 1 -- 172.21.15.103:0/437079763 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d600054e0 con 0x557758816760 2024-09-19T07:42:19.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d55ffb640 1 -- 172.21.15.103:0/437079763 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d6005c050 con 0x557758816760 2024-09-19T07:42:19.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d77101740 10 client.25882 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:19.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=35}) -- 0x5577588389b0 con 0x557758816760 2024-09-19T07:42:19.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d55ffb640 1 -- 172.21.15.103:0/437079763 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f0d6000f040 con 0x557758816760 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d77101740 10 client.25882 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d77101740 10 client.25882 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d77101740 20 client.25882 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d77101740 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x55775883c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.642+0000 7f0d77101740 4 client.25882 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.643+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577587899d0 con 0x557758839e70 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.643+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x55775883c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.643+0000 7f0d55ffb640 10 client.25882 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.643+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x55775883c310 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceacb3fa1e0858c0 server_cookie=991b3ba1316549c7 in_seq=0 out_seq=0 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d55ffb640 1 -- 172.21.15.103:0/437079763 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577587899d0 con 0x557758839e70 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d55ffb640 10 client.25882 handle_command_reply: tid=0 2024-09-19T07:42:19.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 client.25882 shutdown 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 msgr2=0x55775881e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x55775881e7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d60005e80 tx=0x7f0d60009260 comp rx=0 tx=0).stop 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 shutdown_connections 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x55775883c310 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x55775881f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758817080 0x55775881ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d55ffb640 0 client.25882 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 --2- 172.21.15.103:0/437079763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x55775881e7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 >> 172.21.15.103:0/437079763 conn(0x557758812cf0 msgr2=0x557758818a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 shutdown_connections 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.644+0000 7f0d77101740 1 -- 172.21.15.103:0/437079763 wait complete. 2024-09-19T07:42:19.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.645+0000 7f0d77101740 20 client.25882 trim_cache size 0 max 16384 2024-09-19T07:42:19.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.670+0000 7f0d77101740 1 Processor -- start 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.670+0000 7f0d77101740 1 -- start start 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.670+0000 7f0d77101740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758819210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d77101740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758817080 0x557758820710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d77101740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x557758820c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d77101740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5000e6c0 con 0x557758821940 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d77101740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50010290 con 0x557758816760 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d77101740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5000ed90 con 0x557758817080 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x557758820c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x557758820c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58850/0 (socket says 172.21.15.103:58850) 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6ed9d640 1 -- 172.21.15.103:0/2987017565 learned_addr learned my addr 172.21.15.103:0/2987017565 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:19.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758819210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758817080 0x557758820710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6e59c640 1 -- 172.21.15.103:0/2987017565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758817080 msgr2=0x557758820710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758817080 0x557758820710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6e59c640 1 -- 172.21.15.103:0/2987017565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 msgr2=0x557758820c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x557758820c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6e59c640 1 -- 172.21.15.103:0/2987017565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775881d750 con 0x557758816760 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6ed9d640 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x557758820c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758817080 0x557758820710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758819210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009920 tx=0x7f0d6004d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997e0175d92e3719 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.671+0000 7f0d57fff640 1 -- 172.21.15.103:0/2987017565 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60050040 con 0x557758816760 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.672+0000 7f0d57fff640 1 -- 172.21.15.103:0/2987017565 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d68003c30 con 0x557758816760 2024-09-19T07:42:19.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.672+0000 7f0d57fff640 1 -- 172.21.15.103:0/2987017565 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6003a040 con 0x557758816760 2024-09-19T07:42:19.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.672+0000 7f0d77101740 1 -- 172.21.15.103:0/2987017565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 msgr2=0x557758819210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.672+0000 7f0d77101740 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758819210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009920 tx=0x7f0d6004d000 comp rx=0 tx=0).stop 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.673+0000 7f0d77101740 1 -- 172.21.15.103:0/2987017565 shutdown_connections 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.673+0000 7f0d77101740 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758821940 0x557758820c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.673+0000 7f0d77101740 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758817080 0x557758820710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.673+0000 7f0d77101740 1 --2- 172.21.15.103:0/2987017565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758816760 0x557758819210 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.673+0000 7f0d77101740 1 -- 172.21.15.103:0/2987017565 >> 172.21.15.103:0/2987017565 conn(0x557758812cf0 msgr2=0x557758820000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.674+0000 7f0d77101740 1 -- 172.21.15.103:0/2987017565 shutdown_connections 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.674+0000 7f0d77101740 1 -- 172.21.15.103:0/2987017565 wait complete. 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.674+0000 7f0d77101740 1 -- start start 2024-09-19T07:42:19.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.675+0000 7f0d77101740 1 Processor -- start 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.676+0000 7f0d77101740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x557758832a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.676+0000 7f0d77101740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x55775882edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d77101740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758821940 0x55775882f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d77101740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5000fbc0 con 0x557758817080 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d77101740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5000f460 con 0x557758821940 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d77101740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5000c6a0 con 0x557758816760 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d77101740 10 client.0 _ll_register_callbacks cb 0x7f0d60005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x557758832a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x557758832a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37838/0 (socket says 172.21.15.103:37838) 2024-09-19T07:42:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 -- 172.21.15.103:0/4035126579 learned_addr learned my addr 172.21.15.103:0/4035126579 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6ed9d640 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758821940 0x55775882f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 -- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758821940 msgr2=0x55775882f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758821940 0x55775882f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 -- 172.21.15.103:0/4035126579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 msgr2=0x55775882edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x55775882edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.677+0000 7f0d6e59c640 1 -- 172.21.15.103:0/4035126579 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577588a4c00 con 0x557758816760 2024-09-19T07:42:19.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.678+0000 7f0d6e59c640 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x557758832a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d600098a0 tx=0x7f0d60005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f58c734bdabe8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:19.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.678+0000 7f0d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:42:19.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.678+0000 7f0d55ffb640 1 -- 172.21.15.103:0/4035126579 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60050040 con 0x557758816760 2024-09-19T07:42:19.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.678+0000 7f0d55ffb640 1 -- 172.21.15.103:0/4035126579 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d60061480 con 0x557758816760 2024-09-19T07:42:19.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.678+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d68003c30 con 0x557758816760 2024-09-19T07:42:19.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.681+0000 7f0d55ffb640 1 -- 172.21.15.103:0/4035126579 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0d60072050 con 0x557758816760 2024-09-19T07:42:19.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.683+0000 7f0d77101740 10 client.25888 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:19.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.683+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5577588a5d40 con 0x557758816760 2024-09-19T07:42:19.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.688+0000 7f0d55ffb640 1 -- 172.21.15.103:0/4035126579 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d60061ba0 con 0x557758816760 2024-09-19T07:42:19.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.689+0000 7f0d55ffb640 1 -- 172.21.15.103:0/4035126579 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f0d60042760 con 0x557758816760 2024-09-19T07:42:19.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.692+0000 7f0d77101740 10 client.25888 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:19.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.692+0000 7f0d77101740 10 client.25888 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:42:19.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.692+0000 7f0d77101740 20 client.25888 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:19.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.692+0000 7f0d77101740 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x5577588a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:19.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.692+0000 7f0d77101740 4 client.25888 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T07:42:19.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.692+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x7f0d68003c30 con 0x557758839e70 2024-09-19T07:42:19.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.692+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x5577588a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:19.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.693+0000 7f0d65d9b640 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x5577588a83c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c649b68189b661d1 server_cookie=4acb8b46c71ca932 in_seq=0 out_seq=0 2024-09-19T07:42:19.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.693+0000 7f0d55ffb640 10 client.25888 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:19 smithi134 ceph-mon[35414]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-09-19T07:42:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:42:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:42:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:42:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:42:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:42:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d55ffb640 1 -- 172.21.15.103:0/4035126579 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f0d68003c30 con 0x557758839e70 2024-09-19T07:42:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d55ffb640 10 client.25888 handle_command_reply: tid=0 2024-09-19T07:42:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 client.25888 shutdown 2024-09-19T07:42:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 msgr2=0x557758832a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x557758832a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d600098a0 tx=0x7f0d60005fd0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 shutdown_connections 2024-09-19T07:42:19.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557758839e70 0x5577588a83c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.885 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:19 smithi103 ceph-mon[31201]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-09-19T07:42:19.885 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:42:19.885 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:42:19.885 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:42:19.885 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:42:19.885 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557758821940 0x55775882f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557758817080 0x55775882edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 --2- 172.21.15.103:0/4035126579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557758816760 0x557758832a80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 >> 172.21.15.103:0/4035126579 conn(0x557758812cf0 msgr2=0x557758820820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d55ffb640 0 client.25888 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 shutdown_connections 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 1 -- 172.21.15.103:0/4035126579 wait complete. 2024-09-19T07:42:19.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.883+0000 7f0d77101740 20 client.25888 trim_cache size 0 max 16384 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 35, 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 18, 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "b80f2a07": { 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 18, 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:42:19.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:19.888 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:19.889 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:19.889 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:19.889 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:19.889 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:42:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d5c0a3400 msgr2=0x7f0d5c0a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d5c0a3400 0x7f0d5c0a58c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x557758813b50 tx=0x7f0d60005d50 comp rx=0 tx=0).stop 2024-09-19T07:42:19.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d701027a0 msgr2=0x7f0d70196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d701027a0 0x7f0d70196790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800e9f0 tx=0x7f0d5800edf0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 shutdown_connections 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d5c0a3400 0x7f0d5c0a58c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d70106f20 0x7f0d7019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d701027a0 0x7f0d70196790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 --2- 172.21.15.103:0/2471953303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d70101df0 0x7f0d70196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 >> 172.21.15.103:0/2471953303 conn(0x7f0d700fd8f0 msgr2=0x7f0d700fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 shutdown_connections 2024-09-19T07:42:19.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:19.884+0000 7f0d74d28640 1 -- 172.21.15.103:0/2471953303 wait complete. 2024-09-19T07:42:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:19 smithi174 ceph-mon[35370]: pgmap v133: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.2 MiB/s wr, 901 op/s 2024-09-19T07:42:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:42:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:42:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:42:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:42:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:42:20.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 225.0 s, 2871.0 tps, lat 29.928 ms stddev 33.640 2024-09-19T07:42:21.125 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b80f2a07', quiesce took 4.5 seconds. Will release after: 25.3 2024-09-19T07:42:21.754 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:21 smithi103 ceph-mon[31201]: pgmap v134: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-09-19T07:42:21.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:21 smithi134 ceph-mon[35414]: pgmap v134: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-09-19T07:42:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:21 smithi174 ceph-mon[35370]: pgmap v134: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.28k op/s 2024-09-19T07:42:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:23 smithi134 ceph-mon[35414]: pgmap v135: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-09-19T07:42:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:23 smithi174 ceph-mon[35370]: pgmap v135: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-09-19T07:42:23.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:23 smithi103 ceph-mon[31201]: pgmap v135: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 675 op/s 2024-09-19T07:42:25.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 230.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:42:25.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:25 smithi134 ceph-mon[35414]: pgmap v136: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.5 MiB/s wr, 713 op/s 2024-09-19T07:42:25.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:25 smithi174 ceph-mon[35370]: pgmap v136: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.5 MiB/s wr, 713 op/s 2024-09-19T07:42:25.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:25 smithi103 ceph-mon[31201]: pgmap v136: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.5 MiB/s wr, 713 op/s 2024-09-19T07:42:27.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:27 smithi174 ceph-mon[35370]: pgmap v137: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-09-19T07:42:27.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:27 smithi103 ceph-mon[31201]: pgmap v137: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-09-19T07:42:28.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:27 smithi134 ceph-mon[35414]: pgmap v137: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-09-19T07:42:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:29 smithi174 ceph-mon[35370]: pgmap v138: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 419 op/s 2024-09-19T07:42:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:29 smithi103 ceph-mon[31201]: pgmap v138: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 419 op/s 2024-09-19T07:42:30.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:29 smithi134 ceph-mon[35414]: pgmap v138: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 419 op/s 2024-09-19T07:42:30.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 235.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:42:31.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:31 smithi174 ceph-mon[35370]: pgmap v139: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 425 op/s 2024-09-19T07:42:31.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:31 smithi103 ceph-mon[31201]: pgmap v139: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 425 op/s 2024-09-19T07:42:32.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:31 smithi134 ceph-mon[35414]: pgmap v139: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 425 op/s 2024-09-19T07:42:33.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:33 smithi174 ceph-mon[35370]: pgmap v140: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 389 KiB/s wr, 44 op/s 2024-09-19T07:42:33.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:33 smithi103 ceph-mon[31201]: pgmap v140: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 389 KiB/s wr, 44 op/s 2024-09-19T07:42:34.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:33 smithi134 ceph-mon[35414]: pgmap v140: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 389 KiB/s wr, 44 op/s 2024-09-19T07:42:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 240.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:42:35.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:35 smithi174 ceph-mon[35370]: pgmap v141: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 396 KiB/s wr, 51 op/s 2024-09-19T07:42:35.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:35 smithi103 ceph-mon[31201]: pgmap v141: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 396 KiB/s wr, 51 op/s 2024-09-19T07:42:36.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:35 smithi134 ceph-mon[35414]: pgmap v141: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 396 KiB/s wr, 51 op/s 2024-09-19T07:42:37.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:37 smithi174 ceph-mon[35370]: pgmap v142: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-19T07:42:37.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:37 smithi103 ceph-mon[31201]: pgmap v142: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-19T07:42:38.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:37 smithi134 ceph-mon[35414]: pgmap v142: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-19T07:42:39.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:39 smithi174 ceph-mon[35370]: pgmap v143: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-19T07:42:39.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:39 smithi103 ceph-mon[31201]: pgmap v143: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-19T07:42:40.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:39 smithi134 ceph-mon[35414]: pgmap v143: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-19T07:42:40.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 245.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:42:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:41 smithi174 ceph-mon[35370]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 723 KiB/s wr, 60 op/s 2024-09-19T07:42:41.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:41 smithi103 ceph-mon[31201]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 723 KiB/s wr, 60 op/s 2024-09-19T07:42:42.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:41 smithi134 ceph-mon[35414]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 723 KiB/s wr, 60 op/s 2024-09-19T07:42:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:43 smithi174 ceph-mon[35370]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 713 KiB/s wr, 54 op/s 2024-09-19T07:42:43.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:43 smithi103 ceph-mon[31201]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 713 KiB/s wr, 54 op/s 2024-09-19T07:42:44.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:43 smithi134 ceph-mon[35414]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 713 KiB/s wr, 54 op/s 2024-09-19T07:42:45.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 250.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:42:45.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:45 smithi174 ceph-mon[35370]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 717 KiB/s wr, 59 op/s 2024-09-19T07:42:45.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:45 smithi103 ceph-mon[31201]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 717 KiB/s wr, 59 op/s 2024-09-19T07:42:46.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:45 smithi134 ceph-mon[35414]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 717 KiB/s wr, 59 op/s 2024-09-19T07:42:46.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:42:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:46.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:42:47.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.258+0000 7f66b3cbc640 1 -- 172.21.15.103:0/1924937301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109450 msgr2=0x7f66ac110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:47.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.258+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/1924937301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109450 0x7f66ac110250 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f669c009900 tx=0x7f669c02a400 comp rx=0 tx=0).stop 2024-09-19T07:42:47.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.259+0000 7f66b3cbc640 1 -- 172.21.15.103:0/1924937301 shutdown_connections 2024-09-19T07:42:47.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.259+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/1924937301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66ac11b140 0x7f66ac11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.259+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/1924937301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac109830 0x7f66ac109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.259+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/1924937301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109450 0x7f66ac110250 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.259+0000 7f66b3cbc640 1 -- 172.21.15.103:0/1924937301 >> 172.21.15.103:0/1924937301 conn(0x7f66ac06e140 msgr2=0x7f66ac06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:47.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.259+0000 7f66b3cbc640 1 -- 172.21.15.103:0/1924937301 shutdown_connections 2024-09-19T07:42:47.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.259+0000 7f66b3cbc640 1 -- 172.21.15.103:0/1924937301 wait complete. 2024-09-19T07:42:47.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.260+0000 7f66b3cbc640 1 Processor -- start 2024-09-19T07:42:47.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.260+0000 7f66b3cbc640 1 -- start start 2024-09-19T07:42:47.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b3cbc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66ac109450 0x7f66ac11a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:47.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b3cbc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109830 0x7f66ac1149b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:47.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b3cbc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac11b140 0x7f66ac114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:47.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b3cbc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac111370 con 0x7f66ac109830 2024-09-19T07:42:47.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b3cbc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1111f0 con 0x7f66ac109450 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b3cbc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1114f0 con 0x7f66ac11b140 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b1257640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109830 0x7f66ac1149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b1257640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109830 0x7f66ac1149b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43742/0 (socket says 172.21.15.103:43742) 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b1257640 1 -- 172.21.15.103:0/511327467 learned_addr learned my addr 172.21.15.103:0/511327467 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b2259640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac11b140 0x7f66ac114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.261+0000 7f66b1a58640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66ac109450 0x7f66ac11a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.262+0000 7f66b2259640 1 -- 172.21.15.103:0/511327467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66ac109450 msgr2=0x7f66ac11a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.262+0000 7f66b2259640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66ac109450 0x7f66ac11a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.262+0000 7f66b2259640 1 -- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109830 msgr2=0x7f66ac1149b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.262+0000 7f66b2259640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109830 0x7f66ac1149b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.262+0000 7f66b2259640 1 -- 172.21.15.103:0/511327467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac115690 con 0x7f66ac11b140 2024-09-19T07:42:47.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.262+0000 7f66b2259640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac11b140 0x7f66ac114ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66a400caf0 tx=0x7f66a40020e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:47.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.269+0000 7f66a2ffd640 1 -- 172.21.15.103:0/511327467 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4016030 con 0x7f66ac11b140 2024-09-19T07:42:47.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.270+0000 7f66a2ffd640 1 -- 172.21.15.103:0/511327467 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a4003e50 con 0x7f66ac11b140 2024-09-19T07:42:47.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.270+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ac1bc140 con 0x7f66ac11b140 2024-09-19T07:42:47.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.270+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ac1bc650 con 0x7f66ac11b140 2024-09-19T07:42:47.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.270+0000 7f66a2ffd640 1 -- 172.21.15.103:0/511327467 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a40045d0 con 0x7f66ac11b140 2024-09-19T07:42:47.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.274+0000 7f66a2ffd640 1 -- 172.21.15.103:0/511327467 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f66a4019070 con 0x7f66ac11b140 2024-09-19T07:42:47.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.275+0000 7f66a2ffd640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66980a3580 0x7f66980a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:47.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.276+0000 7f66a2ffd640 1 -- 172.21.15.103:0/511327467 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f66a40cf450 con 0x7f66ac11b140 2024-09-19T07:42:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.277+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6690005290 con 0x7f66ac11b140 2024-09-19T07:42:47.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.281+0000 7f66b1a58640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66980a3580 0x7f66980a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:47.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.283+0000 7f66b1a58640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66980a3580 0x7f66980a5a40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f669c000c00 tx=0x7f669c009560 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:47.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.285+0000 7f66a2ffd640 1 -- 172.21.15.103:0/511327467 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f66a409bef0 con 0x7f66ac11b140 2024-09-19T07:42:47.394 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:47.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.632+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6690005850 con 0x7f66ac11b140 2024-09-19T07:42:47.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.633+0000 7f66a2ffd640 1 -- 172.21.15.103:0/511327467 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11996 (secure 0 0 0) 0x7f66a409b7f0 con 0x7f66ac11b140 2024-09-19T07:42:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:42:47.633 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":35,"btime":"2024-09-19T07:41:44:478751+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:41:43.518367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:42:47.633 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 35 2024-09-19T07:42:47.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66980a3580 msgr2=0x7f66980a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:47.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66980a3580 0x7f66980a5a40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f669c000c00 tx=0x7f669c009560 comp rx=0 tx=0).stop 2024-09-19T07:42:47.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac11b140 msgr2=0x7f66ac114ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:47.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac11b140 0x7f66ac114ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66a400caf0 tx=0x7f66a40020e0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 shutdown_connections 2024-09-19T07:42:47.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66980a3580 0x7f66980a5a40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac11b140 0x7f66ac114ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66ac109830 0x7f66ac1149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 --2- 172.21.15.103:0/511327467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66ac109450 0x7f66ac11a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:47.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 >> 172.21.15.103:0/511327467 conn(0x7f66ac06e140 msgr2=0x7f66ac070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:47.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 shutdown_connections 2024-09-19T07:42:47.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:47.635+0000 7f66b3cbc640 1 -- 172.21.15.103:0/511327467 wait complete. 2024-09-19T07:42:47.803 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:47 smithi103 ceph-mon[31201]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 710 KiB/s wr, 53 op/s 2024-09-19T07:42:47.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:47 smithi174 ceph-mon[35370]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 710 KiB/s wr, 53 op/s 2024-09-19T07:42:48.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:47 smithi134 ceph-mon[35414]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 710 KiB/s wr, 53 op/s 2024-09-19T07:42:48.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 -- 172.21.15.103:0/3669818850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138100e20 msgr2=0x7f0138101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 --2- 172.21.15.103:0/3669818850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138100e20 0x7f0138101200 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0128009900 tx=0x7f012802a480 comp rx=0 tx=0).stop 2024-09-19T07:42:48.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 -- 172.21.15.103:0/3669818850 shutdown_connections 2024-09-19T07:42:48.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 --2- 172.21.15.103:0/3669818850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0138102ef0 0x7f013810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 --2- 172.21.15.103:0/3669818850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138102570 0x7f01381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 --2- 172.21.15.103:0/3669818850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138100e20 0x7f0138101200 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 -- 172.21.15.103:0/3669818850 >> 172.21.15.103:0/3669818850 conn(0x7f01380fc7d0 msgr2=0x7f01380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 -- 172.21.15.103:0/3669818850 shutdown_connections 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.096+0000 7f013e5b1640 1 -- 172.21.15.103:0/3669818850 wait complete. 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 Processor -- start 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 -- start start 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138100e20 0x7f0138197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0138102570 0x7f01381976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138102ef0 0x7f0138197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0138110fa0 con 0x7f0138102ef0 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0138110e20 con 0x7f0138100e20 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013e5b1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0138111120 con 0x7f0138102570 2024-09-19T07:42:48.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f0137fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138100e20 0x7f0138197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f0137fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138100e20 0x7f0138197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:44426/0 (socket says 172.21.15.103:44426) 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f0137fff640 1 -- 172.21.15.103:0/611689988 learned_addr learned my addr 172.21.15.103:0/611689988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f013cb4e640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138102ef0 0x7f0138197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.097+0000 7f01377fe640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0138102570 0x7f01381976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f0137fff640 1 -- 172.21.15.103:0/611689988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0138102570 msgr2=0x7f01381976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f0137fff640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0138102570 0x7f01381976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f0137fff640 1 -- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138102ef0 msgr2=0x7f0138197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f0137fff640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138102ef0 0x7f0138197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f0137fff640 1 -- 172.21.15.103:0/611689988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01381a69f0 con 0x7f0138100e20 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f0137fff640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138100e20 0x7f0138197180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0128006010 tx=0x7f0128003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f01357fa640 1 -- 172.21.15.103:0/611689988 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012803f070 con 0x7f0138100e20 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01381a6c80 con 0x7f0138100e20 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f01357fa640 1 -- 172.21.15.103:0/611689988 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0128031c30 con 0x7f0138100e20 2024-09-19T07:42:48.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f01357fa640 1 -- 172.21.15.103:0/611689988 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128035030 con 0x7f0138100e20 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.098+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01381a7160 con 0x7f0138100e20 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.100+0000 7f01357fa640 1 -- 172.21.15.103:0/611689988 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0128042070 con 0x7f0138100e20 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 Processor -- start 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 -- start start 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b030880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b03b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b03b680 0x56030b03da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56030af962e0 con 0x56030b03b680 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56030af3dce0 con 0x56030b0304a0 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f014098a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56030af922a0 con 0x56030b030dc0 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f01377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b03b680 0x56030b03da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f013cb4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b030880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f01377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b03b680 0x56030b03da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43772/0 (socket says 172.21.15.103:43772) 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f01377fe640 1 -- 172.21.15.103:0/60774311 learned_addr learned my addr 172.21.15.103:0/60774311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f01357fa640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01200a33b0 0x7f01200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.101+0000 7f0137fff640 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b03b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f01377fe640 1 -- 172.21.15.103:0/60774311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 msgr2=0x56030b03b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f01377fe640 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b03b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f01377fe640 1 -- 172.21.15.103:0/60774311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 msgr2=0x56030b030880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f01377fe640 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b030880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f01357fa640 1 -- 172.21.15.103:0/611689988 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f01280f0400 con 0x7f0138100e20 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f01377fe640 1 -- 172.21.15.103:0/60774311 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030af22440 con 0x56030b03b680 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f0137fff640 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b03b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f013cb4e640 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b030880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f013cb4e640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01200a33b0 0x7f01200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f01377fe640 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b03b680 0x56030b03da70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f012c009c90 tx=0x7f012c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff75bd9598106789 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f011e7fc640 1 -- 172.21.15.103:0/60774311 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c01e070 con 0x56030b03b680 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f011e7fc640 1 -- 172.21.15.103:0/60774311 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012c019b10 con 0x56030b03b680 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.102+0000 7f011e7fc640 1 -- 172.21.15.103:0/60774311 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00ea20 con 0x56030b03b680 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f013cb4e640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01200a33b0 0x7f01200a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f013000abe0 tx=0x7f0130004a40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 -- 172.21.15.103:0/60774311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b03b680 msgr2=0x56030b03da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b03b680 0x56030b03da70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f012c009c90 tx=0x7f012c011810 comp rx=0 tx=0).stop 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 -- 172.21.15.103:0/60774311 shutdown_connections 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b03b680 0x56030b03da70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b03b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 --2- 172.21.15.103:0/60774311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b030880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 -- 172.21.15.103:0/60774311 >> 172.21.15.103:0/60774311 conn(0x56030b02ca30 msgr2=0x56030b02ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 -- 172.21.15.103:0/60774311 shutdown_connections 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 -- 172.21.15.103:0/60774311 wait complete. 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.103+0000 7f014098a740 1 -- start start 2024-09-19T07:42:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 1 Processor -- start 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b0304a0 0x56030b0384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b030dc0 0x56030b038a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b038f60 0x56030b0476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56030b03e250 con 0x56030b038f60 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56030af51050 con 0x56030b030dc0 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56030b03e3d0 con 0x56030b0304a0 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f014098a740 10 client.0 _ll_register_callbacks cb 0x56030af91160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f013cb4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b038f60 0x56030b0476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f01377fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b030dc0 0x56030b038a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f013cb4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b038f60 0x56030b0476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43786/0 (socket says 172.21.15.103:43786) 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f013cb4e640 1 -- 172.21.15.103:0/2718417180 learned_addr learned my addr 172.21.15.103:0/2718417180 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.104+0000 7f0137fff640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b0304a0 0x56030b0384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01377fe640 1 -- 172.21.15.103:0/2718417180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b0304a0 msgr2=0x56030b0384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01377fe640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b0304a0 0x56030b0384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01377fe640 1 -- 172.21.15.103:0/2718417180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b038f60 msgr2=0x56030b0476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01377fe640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b038f60 0x56030b0476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01377fe640 1 -- 172.21.15.103:0/2718417180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56030ad073c0 con 0x56030b030dc0 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f0137fff640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b0304a0 0x56030b0384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f013cb4e640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b038f60 0x56030b0476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01377fe640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b030dc0 0x56030b038a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x56030b02dae0 tx=0x7f012c00e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306fc1ef308c112 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01177fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01177fe640 1 -- 172.21.15.103:0/2718417180 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c029020 con 0x56030b030dc0 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01177fe640 1 -- 172.21.15.103:0/2718417180 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f012c005450 con 0x56030b030dc0 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012c019b10 con 0x56030b030dc0 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f01177fe640 1 -- 172.21.15.103:0/2718417180 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f012c01e050 con 0x56030b030dc0 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f014098a740 10 client.25937 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.105+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56030b04a1e0 con 0x56030b030dc0 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f01177fe640 1 -- 172.21.15.103:0/2718417180 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012c019b10 con 0x56030b030dc0 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f01177fe640 1 -- 172.21.15.103:0/2718417180 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f012c00e9f0 con 0x56030b030dc0 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f014098a740 10 client.25937 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f014098a740 10 client.25937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f014098a740 20 client.25937 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f014098a740 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b04b5f0 0x56030b04d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f014098a740 4 client.25937 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56030afa3850 con 0x56030b04b5f0 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f0137fff640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b04b5f0 0x56030b04d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f0137fff640 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b04b5f0 0x56030b04d9e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92050b1aca266821 server_cookie=217542ebad248031 in_seq=0 out_seq=0 2024-09-19T07:42:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.106+0000 7f01177fe640 10 client.25937 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f01177fe640 1 -- 172.21.15.103:0/2718417180 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56030afa3850 con 0x56030b04b5f0 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f01177fe640 10 client.25937 handle_command_reply: tid=0 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 client.25937 shutdown 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b030dc0 msgr2=0x56030b038a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b030dc0 0x56030b038a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x56030b02dae0 tx=0x7f012c00e640 comp rx=0 tx=0).stop 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 shutdown_connections 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b04b5f0 0x56030b04d9e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b038f60 0x56030b0476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b030dc0 0x56030b038a20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f01177fe640 0 client.25937 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 --2- 172.21.15.103:0/2718417180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b0304a0 0x56030b0384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 >> 172.21.15.103:0/2718417180 conn(0x56030b02ca30 msgr2=0x56030b032790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 shutdown_connections 2024-09-19T07:42:48.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 1 -- 172.21.15.103:0/2718417180 wait complete. 2024-09-19T07:42:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.107+0000 7f014098a740 20 client.25937 trim_cache size 0 max 16384 2024-09-19T07:42:48.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 Processor -- start 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 -- start start 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b0304a0 0x56030b0474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b047a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b038f60 0x56030b0531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01381afa20 con 0x56030b0304a0 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b15f0 con 0x56030b038f60 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f014098a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b00f0 con 0x56030b030dc0 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f0137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b0304a0 0x56030b0474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f0137fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b0304a0 0x56030b0474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43790/0 (socket says 172.21.15.103:43790) 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f0137fff640 1 -- 172.21.15.103:0/3480602209 learned_addr learned my addr 172.21.15.103:0/3480602209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f01377fe640 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b047a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f013cb4e640 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b038f60 0x56030b0531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f0137fff640 1 -- 172.21.15.103:0/3480602209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 msgr2=0x56030b047a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f0137fff640 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b047a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f0137fff640 1 -- 172.21.15.103:0/3480602209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b038f60 msgr2=0x56030b0531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.125+0000 7f0137fff640 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b038f60 0x56030b0531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.126+0000 7f0137fff640 1 -- 172.21.15.103:0/3480602209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56030b03dde0 con 0x56030b0304a0 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.126+0000 7f01377fe640 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b047a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.126+0000 7f0137fff640 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b0304a0 0x56030b0474e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f012802a4c0 tx=0x7f0128036b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181895138b646c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.126+0000 7f011dffb640 1 -- 172.21.15.103:0/3480602209 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128049040 con 0x56030b0304a0 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.126+0000 7f011dffb640 1 -- 172.21.15.103:0/3480602209 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01280bc740 con 0x56030b0304a0 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.126+0000 7f011dffb640 1 -- 172.21.15.103:0/3480602209 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01280b8db0 con 0x56030b0304a0 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 -- 172.21.15.103:0/3480602209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b0304a0 msgr2=0x56030b0474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b0304a0 0x56030b0474e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f012802a4c0 tx=0x7f0128036b60 comp rx=0 tx=0).stop 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 -- 172.21.15.103:0/3480602209 shutdown_connections 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b038f60 0x56030b0531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b030dc0 0x56030b047a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 --2- 172.21.15.103:0/3480602209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b0304a0 0x56030b0474e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 -- 172.21.15.103:0/3480602209 >> 172.21.15.103:0/3480602209 conn(0x56030b02ca30 msgr2=0x56030b03a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 -- 172.21.15.103:0/3480602209 shutdown_connections 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 -- 172.21.15.103:0/3480602209 wait complete. 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.127+0000 7f014098a740 1 -- start start 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 1 Processor -- start 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b043650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b030dc0 0x56030b043b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b038f60 0x56030b034930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b0f20 con 0x56030b030dc0 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b07c0 con 0x56030b0304a0 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01381ada00 con 0x56030b038f60 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f014098a740 10 client.0 _ll_register_callbacks cb 0x7f012c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b030dc0 0x56030b043b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b030dc0 0x56030b043b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43794/0 (socket says 172.21.15.103:43794) 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 -- 172.21.15.103:0/3262772423 learned_addr learned my addr 172.21.15.103:0/3262772423 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f013cb4e640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b038f60 0x56030b034930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f0137fff640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b043650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 -- 172.21.15.103:0/3262772423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b038f60 msgr2=0x56030b034930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b038f60 0x56030b034930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 -- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 msgr2=0x56030b043650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b043650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.128+0000 7f01377fe640 1 -- 172.21.15.103:0/3262772423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56030b034e70 con 0x56030b030dc0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f0137fff640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b043650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f01377fe640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b030dc0 0x56030b043b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f012c00e610 tx=0x7f012c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15530b40a8c399ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f01177fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f01177fe640 1 -- 172.21.15.103:0/3262772423 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c029040 con 0x56030b030dc0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f01177fe640 1 -- 172.21.15.103:0/3262772423 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f012c011d10 con 0x56030b030dc0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01280bc740 con 0x56030b030dc0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f01177fe640 1 -- 172.21.15.103:0/3262772423 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f012c00e3b0 con 0x56030b030dc0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f014098a740 10 client.16443 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56030b0b5920 con 0x56030b030dc0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.129+0000 7f01177fe640 1 -- 172.21.15.103:0/3262772423 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01280bc740 con 0x56030b030dc0 2024-09-19T07:42:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f01177fe640 1 -- 172.21.15.103:0/3262772423 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f012c0041b0 con 0x56030b030dc0 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 10 client.16443 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 20 client.16443 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b0b5fd0 0x56030b0b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 4 client.16443 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56030af9dd60 con 0x56030b0b5fd0 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f0137fff640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b0b5fd0 0x56030b0b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f0137fff640 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b0b5fd0 0x56030b0b83f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e13fd894caf880 server_cookie=45b4157bcd0db64b in_seq=0 out_seq=0 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f01177fe640 10 client.16443 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f01177fe640 1 -- 172.21.15.103:0/3262772423 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56030af9dd60 con 0x56030b0b5fd0 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f01177fe640 10 client.16443 handle_command_reply: tid=0 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 1 client.16443 shutdown 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b030dc0 msgr2=0x56030b043b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.130+0000 7f014098a740 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b030dc0 0x56030b043b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f012c00e610 tx=0x7f012c011810 comp rx=0 tx=0).stop 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 shutdown_connections 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56030b0b5fd0 0x56030b0b83f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56030b038f60 0x56030b034930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56030b030dc0 0x56030b043b90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 --2- 172.21.15.103:0/3262772423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56030b0304a0 0x56030b043650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 >> 172.21.15.103:0/3262772423 conn(0x56030b02ca30 msgr2=0x56030b031ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f01177fe640 0 client.16443 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 shutdown_connections 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 1 -- 172.21.15.103:0/3262772423 wait complete. 2024-09-19T07:42:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f014098a740 20 client.16443 trim_cache size 0 max 16384 2024-09-19T07:42:48.141 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:42:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:42:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:42:48.141 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01200a33b0 msgr2=0x7f01200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f013e5b1640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01200a33b0 0x7f01200a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f013000abe0 tx=0x7f0130004a40 comp rx=0 tx=0).stop 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138100e20 msgr2=0x7f0138197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f013e5b1640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138100e20 0x7f0138197180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0128006010 tx=0x7f0128003c10 comp rx=0 tx=0).stop 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.131+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 shutdown_connections 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.132+0000 7f013e5b1640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01200a33b0 0x7f01200a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.132+0000 7f013e5b1640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0138102ef0 0x7f0138197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.132+0000 7f013e5b1640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0138102570 0x7f01381976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.132+0000 7f013e5b1640 1 --2- 172.21.15.103:0/611689988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0138100e20 0x7f0138197180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.132+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 >> 172.21.15.103:0/611689988 conn(0x7f01380fc7d0 msgr2=0x7f01380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:48.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.132+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 shutdown_connections 2024-09-19T07:42:48.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:48.132+0000 7f013e5b1640 1 -- 172.21.15.103:0/611689988 wait complete. 2024-09-19T07:42:48.679 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id b80f2a07 --release --await' 2024-09-19T07:42:48.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id b80f2a07 --release --await 2024-09-19T07:42:48.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/511327467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:48.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/511327467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:49.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/511327467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:49.477 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:42:49.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:42:49.578 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:49.700 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:49 smithi103 ceph-mon[31201]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 710 KiB/s wr, 53 op/s 2024-09-19T07:42:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:49 smithi174 ceph-mon[35370]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 710 KiB/s wr, 53 op/s 2024-09-19T07:42:50.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.051+0000 7f6d13107640 1 -- 172.21.15.103:0/1634606178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c100e20 msgr2=0x7f6d0c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.051+0000 7f6d13107640 1 --2- 172.21.15.103:0/1634606178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c100e20 0x7f6d0c101200 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009900 tx=0x7f6d0002a480 comp rx=0 tx=0).stop 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- 172.21.15.103:0/1634606178 shutdown_connections 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 --2- 172.21.15.103:0/1634606178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d0c102ef0 0x7f6d0c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 --2- 172.21.15.103:0/1634606178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0c102570 0x7f6d0c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 --2- 172.21.15.103:0/1634606178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c100e20 0x7f6d0c101200 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- 172.21.15.103:0/1634606178 >> 172.21.15.103:0/1634606178 conn(0x7f6d0c0fc7d0 msgr2=0x7f6d0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- 172.21.15.103:0/1634606178 shutdown_connections 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- 172.21.15.103:0/1634606178 wait complete. 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 Processor -- start 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- start start 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0c100e20 0x7f6d0c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c102570 0x7f6d0c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d0c102ef0 0x7f6d0c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c110f80 con 0x7f6d0c102570 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c110e00 con 0x7f6d0c102ef0 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d13107640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c111100 con 0x7f6d0c100e20 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.052+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c102570 0x7f6d0c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c102570 0x7f6d0c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56160/0 (socket says 172.21.15.103:56160) 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 -- 172.21.15.103:0/1026547203 learned_addr learned my addr 172.21.15.103:0/1026547203 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0c100e20 0x7f6d0c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d116a4640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d0c102ef0 0x7f6d0c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 -- 172.21.15.103:0/1026547203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0c100e20 msgr2=0x7f6d0c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0c100e20 0x7f6d0c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 -- 172.21.15.103:0/1026547203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d0c102ef0 msgr2=0x7f6d0c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d0c102ef0 0x7f6d0c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 -- 172.21.15.103:0/1026547203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c19d4d0 con 0x7f6d0c102570 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c102570 0x7f6d0c198a60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800d770 tx=0x7f6cf800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d09ffb640 1 -- 172.21.15.103:0/1026547203 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8017070 con 0x7f6d0c102570 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c19d7c0 con 0x7f6d0c102570 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d09ffb640 1 -- 172.21.15.103:0/1026547203 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf8005500 con 0x7f6d0c102570 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d09ffb640 1 -- 172.21.15.103:0/1026547203 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8004e00 con 0x7f6d0c102570 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.053+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c1a50a0 con 0x7f6d0c102570 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.056+0000 7f6d09ffb640 1 -- 172.21.15.103:0/1026547203 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6cf8004fa0 con 0x7f6d0c102570 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.056+0000 7f6d154e0740 1 Processor -- start 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.056+0000 7f6d154e0740 1 -- start start 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484aa0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0980 0x559484aaad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aab240 0x559484aad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d154e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559484a05620 con 0x559484aab240 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d154e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5594849ace50 con 0x559484aa0980 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d154e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559484a015e0 con 0x559484aa0060 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aab240 0x559484aad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aab240 0x559484aad630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56166/0 (socket says 172.21.15.103:56166) 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2560918535 learned_addr learned my addr 172.21.15.103:0/2560918535 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0980 0x559484aaad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d116a4640 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484aa0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2560918535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 msgr2=0x559484aa0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484aa0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2560918535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0980 msgr2=0x559484aaad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0980 0x559484aaad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2560918535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948499dd40 con 0x559484aab240 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d116a4640 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484aa0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d09ffb640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6cfc0a33b0 0x7f6cfc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aab240 0x559484aad630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80cce60 tx=0x7f6cf80d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dac751af3c75cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.057+0000 7f6d116a4640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6cfc0a33b0 0x7f6cfc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6cf6ffd640 1 -- 172.21.15.103:0/2560918535 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8097d20 con 0x559484aab240 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6cf6ffd640 1 -- 172.21.15.103:0/2560918535 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf80046c0 con 0x559484aab240 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6cf6ffd640 1 -- 172.21.15.103:0/2560918535 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80b86a0 con 0x559484aab240 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d09ffb640 1 -- 172.21.15.103:0/1026547203 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6cf80cf790 con 0x7f6d0c102570 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 -- 172.21.15.103:0/2560918535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aab240 msgr2=0x559484aad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d116a4640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6cfc0a33b0 0x7f6cfc0a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005fd0 tx=0x7f6d040094d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aab240 0x559484aad630 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80cce60 tx=0x7f6cf80d7690 comp rx=0 tx=0).stop 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 -- 172.21.15.103:0/2560918535 shutdown_connections 2024-09-19T07:42:50.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aab240 0x559484aad630 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0980 0x559484aaad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2560918535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484aa0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 -- 172.21.15.103:0/2560918535 >> 172.21.15.103:0/2560918535 conn(0x559484a9c5f0 msgr2=0x559484a9c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 -- 172.21.15.103:0/2560918535 shutdown_connections 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.058+0000 7f6d154e0740 1 -- 172.21.15.103:0/2560918535 wait complete. 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 -- start start 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 Processor -- start 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0060 0x559484aa80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0980 0x559484aa8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aa8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559484aade10 con 0x559484aa0060 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55948498df10 con 0x559484aab240 2024-09-19T07:42:50.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559484aadf90 con 0x559484aa0980 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d154e0740 10 client.0 _ll_register_callbacks cb 0x5594849b1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d10ea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0060 0x559484aa80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d10ea3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0060 0x559484aa80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56168/0 (socket says 172.21.15.103:56168) 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d10ea3640 1 -- 172.21.15.103:0/949722359 learned_addr learned my addr 172.21.15.103:0/949722359 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.059+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0980 0x559484aa8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d116a4640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aa8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d10ea3640 1 -- 172.21.15.103:0/949722359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0980 msgr2=0x559484aa8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0980 0x559484aa8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d10ea3640 1 -- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 msgr2=0x559484aa8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aa8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d10ea3640 1 -- 172.21.15.103:0/949722359 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559484714580 con 0x559484aa0060 2024-09-19T07:42:50.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d116a4640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aa8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0060 0x559484aa80c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6d0002a9a0 tx=0x7f6d00004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e4e0aebb25fffe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6cebfff640 1 -- 172.21.15.103:0/949722359 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00035040 con 0x559484aa0060 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6cebfff640 1 -- 172.21.15.103:0/949722359 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6d00003ac0 con 0x559484aa0060 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559484a12b90 con 0x559484aa0060 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6cebfff640 1 -- 172.21.15.103:0/949722359 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d00003cc0 con 0x559484aa0060 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d154e0740 10 client.16455 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5594847de960 con 0x559484aa0060 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6cebfff640 1 -- 172.21.15.103:0/949722359 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559484a12b90 con 0x559484aa0060 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.060+0000 7f6cebfff640 1 -- 172.21.15.103:0/949722359 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f6d0004f630 con 0x559484aa0060 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d154e0740 10 client.16455 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d154e0740 10 client.16455 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:42:50.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d154e0740 20 client.16455 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d154e0740 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484abac00 0x559484abd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d154e0740 4 client.16455 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559484a0d0a0 con 0x559484abac00 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484abac00 0x559484abd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484abac00 0x559484abd050 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa47c68bf35bc453 server_cookie=35781f156ca1b6dc in_seq=0 out_seq=0 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.061+0000 7f6cebfff640 10 client.16455 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6cebfff640 1 -- 172.21.15.103:0/949722359 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559484a0d0a0 con 0x559484abac00 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6cebfff640 10 client.16455 handle_command_reply: tid=0 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 client.16455 shutdown 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0060 msgr2=0x559484aa80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0060 0x559484aa80c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6d0002a9a0 tx=0x7f6d00004440 comp rx=0 tx=0).stop 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 shutdown_connections 2024-09-19T07:42:50.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484abac00 0x559484abd050 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aa8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6cebfff640 0 client.16455 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0980 0x559484aa8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 --2- 172.21.15.103:0/949722359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0060 0x559484aa80c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 >> 172.21.15.103:0/949722359 conn(0x559484a9c5f0 msgr2=0x559484aa22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 shutdown_connections 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 1 -- 172.21.15.103:0/949722359 wait complete. 2024-09-19T07:42:50.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.062+0000 7f6d154e0740 20 client.16455 trim_cache size 0 max 16384 2024-09-19T07:42:50.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:49 smithi134 ceph-mon[35414]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 710 KiB/s wr, 53 op/s 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 Processor -- start 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 -- start start 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0060 0x559484a9b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ac2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aab240 0x559484ac2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ad850 con 0x559484aa0980 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1af420 con 0x559484aa0060 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d154e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1adf20 con 0x559484aab240 2024-09-19T07:42:50.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ac2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ac2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56184/0 (socket says 172.21.15.103:56184) 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 -- 172.21.15.103:0/574611050 learned_addr learned my addr 172.21.15.103:0/574611050 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0060 0x559484a9b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d116a4640 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aab240 0x559484ac2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 -- 172.21.15.103:0/574611050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aab240 msgr2=0x559484ac2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aab240 0x559484ac2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 -- 172.21.15.103:0/574611050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0060 msgr2=0x559484a9b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0060 0x559484a9b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d0bfff640 1 -- 172.21.15.103:0/574611050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559484a9c420 con 0x559484aa0980 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.080+0000 7f6d116a4640 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aab240 0x559484ac2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.081+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ac2970 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80ccfa0 tx=0x7f6cf80d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d29253bbc20ded server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:50.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.081+0000 7f6cf67fc640 1 -- 172.21.15.103:0/574611050 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8020070 con 0x559484aa0980 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.081+0000 7f6cf67fc640 1 -- 172.21.15.103:0/574611050 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf80d7e90 con 0x559484aa0980 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.081+0000 7f6cf67fc640 1 -- 172.21.15.103:0/574611050 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf80b8870 con 0x559484aa0980 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 -- 172.21.15.103:0/574611050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 msgr2=0x559484ac2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ac2970 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80ccfa0 tx=0x7f6cf80d5550 comp rx=0 tx=0).stop 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 -- 172.21.15.103:0/574611050 shutdown_connections 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aab240 0x559484ac2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ac2970 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 --2- 172.21.15.103:0/574611050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aa0060 0x559484a9b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 -- 172.21.15.103:0/574611050 >> 172.21.15.103:0/574611050 conn(0x559484a9c5f0 msgr2=0x559484a9af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 -- 172.21.15.103:0/574611050 shutdown_connections 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 -- 172.21.15.103:0/574611050 wait complete. 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 -- start start 2024-09-19T07:42:50.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.082+0000 7f6d154e0740 1 Processor -- start 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484ab31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ab3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ab3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1aed50 con 0x559484aa0980 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ae5f0 con 0x559484aab240 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ab830 con 0x559484aa0060 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ab3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56192/0 (socket says 172.21.15.103:56192) 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2133743710 learned_addr learned my addr 172.21.15.103:0/2133743710 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484ab31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 10 client.0 _ll_register_callbacks cb 0x7f6d000040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:50.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d116a4640 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2133743710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 msgr2=0x559484ab31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484ab31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 msgr2=0x559484aae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d0bfff640 1 -- 172.21.15.103:0/2133743710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559484a9d280 con 0x559484aa0980 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.083+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf80d7e90 con 0x559484aa0980 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d0bfff640 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ab3730 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80d5be0 tx=0x7f6cf80cfb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8290f5f9ebc81817 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6cebfff640 10 client.16461 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6cebfff640 1 -- 172.21.15.103:0/2133743710 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8020050 con 0x559484aa0980 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6cebfff640 1 -- 172.21.15.103:0/2133743710 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6cf8099d80 con 0x559484aa0980 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6cebfff640 1 -- 172.21.15.103:0/2133743710 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6cf80b75d0 con 0x559484aa0980 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 10 client.16461 fetch_fsmap learned FSMap version 35 2024-09-19T07:42:50.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=35}) -- 0x559484b25d10 con 0x559484aa0980 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6cebfff640 1 -- 172.21.15.103:0/2133743710 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f6cf80cd170 con 0x559484aa0980 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 10 client.16461 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 10 client.16461 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 20 client.16461 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484b25f80 0x559484b28370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 4 client.16461 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b80f2a07", "release": true, "await": true}] 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.084+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b80f2a07", "release": true, "await": true}) -- 0x5594849895a0 con 0x559484b25f80 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.085+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484b25f80 0x559484b28370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.085+0000 7f6d10ea3640 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484b25f80 0x559484b28370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9976c43422ef3a3 server_cookie=4f03f4125d0b5c7f in_seq=0 out_seq=0 2024-09-19T07:42:50.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.085+0000 7f6cebfff640 10 client.16461 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:50.114 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:50.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.114+0000 7f6cebfff640 1 -- 172.21.15.103:0/2133743710 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5594849895a0 con 0x559484b25f80 2024-09-19T07:42:50.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.114+0000 7f6cebfff640 10 client.16461 handle_command_reply: tid=0 2024-09-19T07:42:50.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.114+0000 7f6d154e0740 1 client.16461 shutdown 2024-09-19T07:42:50.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.116+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 msgr2=0x559484ab3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.116+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ab3730 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80d5be0 tx=0x7f6cf80cfb80 comp rx=0 tx=0).stop 2024-09-19T07:42:50.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.119+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 shutdown_connections 2024-09-19T07:42:50.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.119+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559484b25f80 0x559484b28370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.119+0000 7f6cebfff640 0 client.16461 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:50.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.119+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559484aab240 0x559484aae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.119+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559484aa0980 0x559484ab3730 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.119+0000 7f6d154e0740 1 --2- 172.21.15.103:0/2133743710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559484aa0060 0x559484ab31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.119+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 >> 172.21.15.103:0/2133743710 conn(0x559484a9c5f0 msgr2=0x559484aa87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:50.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.121+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 shutdown_connections 2024-09-19T07:42:50.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.123+0000 7f6d154e0740 1 -- 172.21.15.103:0/2133743710 wait complete. 2024-09-19T07:42:50.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.124+0000 7f6d154e0740 20 client.16461 trim_cache size 0 max 16384 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 35, 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 20, 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "b80f2a07": { 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 20, 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:42:50.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:50.125 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:50.126 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:50.126 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:42:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.124+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6cfc0a33b0 msgr2=0x7f6cfc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.124+0000 7f6d13107640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6cfc0a33b0 0x7f6cfc0a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005fd0 tx=0x7f6d040094d0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.124+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c102570 msgr2=0x7f6d0c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.124+0000 7f6d13107640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c102570 0x7f6d0c198a60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800d770 tx=0x7f6cf800dc30 comp rx=0 tx=0).stop 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 shutdown_connections 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6cfc0a33b0 0x7f6cfc0a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d0c102ef0 0x7f6d0c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d0c102570 0x7f6d0c198a60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 --2- 172.21.15.103:0/1026547203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0c100e20 0x7f6d0c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 >> 172.21.15.103:0/1026547203 conn(0x7f6d0c0fc7d0 msgr2=0x7f6d0c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 shutdown_connections 2024-09-19T07:42:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:50.125+0000 7f6d13107640 1 -- 172.21.15.103:0/1026547203 wait complete. 2024-09-19T07:42:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 255.0 s, 236.8 tps, lat 2596.848 ms stddev 8456.993 2024-09-19T07:42:51.487 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b80f2a07', total seconds elapsed: 34.8 2024-09-19T07:42:51.488 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T07:42:51.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:42:51.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:51 smithi103 ceph-mon[31201]: pgmap v149: 97 pgs: 97 active+clean; 2.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 982 op/s 2024-09-19T07:42:51.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:51 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:42:51.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.736+0000 7f01d35cf640 1 -- 172.21.15.103:0/2070155594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc106f10 msgr2=0x7f01cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:51.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.736+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2070155594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc106f10 0x7f01cc10ea90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ca0 tx=0x7f01c402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:42:51.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.737+0000 7f01d35cf640 1 -- 172.21.15.103:0/2070155594 shutdown_connections 2024-09-19T07:42:51.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.737+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2070155594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc106f10 0x7f01cc10ea90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:51.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.737+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2070155594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc102790 0x7f01cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:51.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.737+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2070155594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01cc101de0 0x7f01cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:51.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.737+0000 7f01d35cf640 1 -- 172.21.15.103:0/2070155594 >> 172.21.15.103:0/2070155594 conn(0x7f01cc0fd8e0 msgr2=0x7f01cc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:51.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.737+0000 7f01d35cf640 1 -- 172.21.15.103:0/2070155594 shutdown_connections 2024-09-19T07:42:51.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.737+0000 7f01d35cf640 1 -- 172.21.15.103:0/2070155594 wait complete. 2024-09-19T07:42:51.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.738+0000 7f01d35cf640 1 Processor -- start 2024-09-19T07:42:51.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.738+0000 7f01d35cf640 1 -- start start 2024-09-19T07:42:51.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d35cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc101de0 0x7f01cc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:51.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d35cf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01cc102790 0x7f01cc10e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:51.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d136b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc101de0 0x7f01cc10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d136b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc101de0 0x7f01cc10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56206/0 (socket says 172.21.15.103:56206) 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d35cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc106f10 0x7f01cc1a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d136b640 1 -- 172.21.15.103:0/2762870550 learned_addr learned my addr 172.21.15.103:0/2762870550 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc111f70 con 0x7f01cc101de0 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d0b6a640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01cc102790 0x7f01cc10e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc111df0 con 0x7f01cc102790 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.739+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1120f0 con 0x7f01cc106f10 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d1b6c640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc106f10 0x7f01cc1a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d0b6a640 1 -- 172.21.15.103:0/2762870550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc106f10 msgr2=0x7f01cc1a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d0b6a640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc106f10 0x7f01cc1a32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d0b6a640 1 -- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc101de0 msgr2=0x7f01cc10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:51.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d0b6a640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc101de0 0x7f01cc10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:51.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d0b6a640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc1a39e0 con 0x7f01cc102790 2024-09-19T07:42:51.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d0b6a640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01cc102790 0x7f01cc10e390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00d6d0 tx=0x7f01bc00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:51.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01c27fc640 1 -- 172.21.15.103:0/2762870550 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc017070 con 0x7f01cc102790 2024-09-19T07:42:51.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc1a3cd0 con 0x7f01cc102790 2024-09-19T07:42:51.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01c27fc640 1 -- 172.21.15.103:0/2762870550 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01bc009820 con 0x7f01cc102790 2024-09-19T07:42:51.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01c27fc640 1 -- 172.21.15.103:0/2762870550 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc004d00 con 0x7f01cc102790 2024-09-19T07:42:51.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.740+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc1ab590 con 0x7f01cc102790 2024-09-19T07:42:51.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.741+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01cc1021c0 con 0x7f01cc102790 2024-09-19T07:42:51.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.743+0000 7f01c27fc640 1 -- 172.21.15.103:0/2762870550 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f01bc0099d0 con 0x7f01cc102790 2024-09-19T07:42:51.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.743+0000 7f01c27fc640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b80a3430 0x7f01b80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:51.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.744+0000 7f01c27fc640 1 -- 172.21.15.103:0/2762870550 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f01bc0cf700 con 0x7f01cc102790 2024-09-19T07:42:51.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.744+0000 7f01d136b640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b80a3430 0x7f01b80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:51.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.746+0000 7f01d136b640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b80a3430 0x7f01b80a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009c90 tx=0x7f01c8005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:51.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:51.746+0000 7f01c27fc640 1 -- 172.21.15.103:0/2762870550 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f01bc09bb20 con 0x7f01cc102790 2024-09-19T07:42:51.864 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:51.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:51 smithi174 ceph-mon[35370]: pgmap v149: 97 pgs: 97 active+clean; 2.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 982 op/s 2024-09-19T07:42:51.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:51 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:42:52.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:51 smithi134 ceph-mon[35414]: pgmap v149: 97 pgs: 97 active+clean; 2.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 982 op/s 2024-09-19T07:42:52.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:51 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:42:52.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.095+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01cc19fd50 con 0x7f01cc102790 2024-09-19T07:42:52.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.096+0000 7f01c27fc640 1 -- 172.21.15.103:0/2762870550 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11994 (secure 0 0 0) 0x7f01cc19fd50 con 0x7f01cc102790 2024-09-19T07:42:52.096 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:42:52.096 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":36,"btime":"2024-09-19T07:42:50:617029+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:42:49.960525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:42:52.096 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 36 2024-09-19T07:42:52.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b80a3430 msgr2=0x7f01b80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b80a3430 0x7f01b80a58f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009c90 tx=0x7f01c8005f90 comp rx=0 tx=0).stop 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01cc102790 msgr2=0x7f01cc10e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01cc102790 0x7f01cc10e390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00d6d0 tx=0x7f01bc00db90 comp rx=0 tx=0).stop 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 shutdown_connections 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b80a3430 0x7f01b80a58f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc106f10 0x7f01cc1a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01cc102790 0x7f01cc10e390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 --2- 172.21.15.103:0/2762870550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01cc101de0 0x7f01cc10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 >> 172.21.15.103:0/2762870550 conn(0x7f01cc0fd8e0 msgr2=0x7f01cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 shutdown_connections 2024-09-19T07:42:52.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:52.099+0000 7f01d35cf640 1 -- 172.21.15.103:0/2762870550 wait complete. 2024-09-19T07:42:52.885 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:52 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2762870550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:52.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:52 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2762870550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:52.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:42:53.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:52 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2762870550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:53.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.156+0000 7f6adfa49640 1 -- 172.21.15.103:0/3028097598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 msgr2=0x7f6ad8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:53.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.156+0000 7f6adfa49640 1 --2- 172.21.15.103:0/3028097598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8100ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009900 tx=0x7f6ac802a420 comp rx=0 tx=0).stop 2024-09-19T07:42:53.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.157+0000 7f6adfa49640 1 -- 172.21.15.103:0/3028097598 shutdown_connections 2024-09-19T07:42:53.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.157+0000 7f6adfa49640 1 --2- 172.21.15.103:0/3028097598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ad8101430 0x7f6ad810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.157+0000 7f6adfa49640 1 --2- 172.21.15.103:0/3028097598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8100ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.157+0000 7f6adfa49640 1 --2- 172.21.15.103:0/3028097598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ad8106a90 0x7f6ad8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.157+0000 7f6adfa49640 1 -- 172.21.15.103:0/3028097598 >> 172.21.15.103:0/3028097598 conn(0x7f6ad80fc7d0 msgr2=0x7f6ad80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:53.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.157+0000 7f6adfa49640 1 -- 172.21.15.103:0/3028097598 shutdown_connections 2024-09-19T07:42:53.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.157+0000 7f6adfa49640 1 -- 172.21.15.103:0/3028097598 wait complete. 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 Processor -- start 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 -- start start 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ad8101430 0x7f6ad8107f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ad8106a90 0x7f6ad8108480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8110f50 con 0x7f6ad8101430 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8110dd0 con 0x7f6ad8100a90 2024-09-19T07:42:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adfa49640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81110d0 con 0x7f6ad8106a90 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6add7e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6add7e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55374/0 (socket says 172.21.15.103:55374) 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6add7e5640 1 -- 172.21.15.103:0/4049207016 learned_addr learned my addr 172.21.15.103:0/4049207016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6adcfe4640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ad8101430 0x7f6ad8107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.158+0000 7f6addfe6640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ad8106a90 0x7f6ad8108480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6add7e5640 1 -- 172.21.15.103:0/4049207016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ad8106a90 msgr2=0x7f6ad8108480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6add7e5640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ad8106a90 0x7f6ad8108480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6add7e5640 1 -- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ad8101430 msgr2=0x7f6ad8107f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6add7e5640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ad8101430 0x7f6ad8107f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6add7e5640 1 -- 172.21.15.103:0/4049207016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad81a6b30 con 0x7f6ad8100a90 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6addfe6640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ad8106a90 0x7f6ad8108480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6add7e5640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8107a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400b450 tx=0x7f6ad400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6ace7fc640 1 -- 172.21.15.103:0/4049207016 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4015020 con 0x7f6ad8100a90 2024-09-19T07:42:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6ace7fc640 1 -- 172.21.15.103:0/4049207016 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad4011d70 con 0x7f6ad8100a90 2024-09-19T07:42:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6ace7fc640 1 -- 172.21.15.103:0/4049207016 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400f340 con 0x7f6ad8100a90 2024-09-19T07:42:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad81a6e20 con 0x7f6ad8100a90 2024-09-19T07:42:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad81a7320 con 0x7f6ad8100a90 2024-09-19T07:42:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.159+0000 7f6adcfe4640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ad8101430 0x7f6ad8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:53.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.161+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad8106e70 con 0x7f6ad8100a90 2024-09-19T07:42:53.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.163+0000 7f6ace7fc640 1 -- 172.21.15.103:0/4049207016 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6ad4018020 con 0x7f6ad8100a90 2024-09-19T07:42:53.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.163+0000 7f6ace7fc640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6ac40a3430 0x7f6ac40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:53.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.164+0000 7f6ace7fc640 1 -- 172.21.15.103:0/4049207016 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f6ad40cfbc0 con 0x7f6ad8100a90 2024-09-19T07:42:53.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.164+0000 7f6adcfe4640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6ac40a3430 0x7f6ac40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:53.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.164+0000 7f6adcfe4640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6ac40a3430 0x7f6ac40a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac80041e0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:53.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.166+0000 7f6ace7fc640 1 -- 172.21.15.103:0/4049207016 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6ad401f020 con 0x7f6ad8100a90 2024-09-19T07:42:53.298 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:53.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.508+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad81a7610 con 0x7f6ad8100a90 2024-09-19T07:42:53.511 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:42:53.511 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":36,"btime":"2024-09-19T07:42:50:617029+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:42:49.960525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:42:53.511 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 36 2024-09-19T07:42:53.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.510+0000 7f6ace7fc640 1 -- 172.21.15.103:0/4049207016 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11994 (secure 0 0 0) 0x7f6ad400d040 con 0x7f6ad8100a90 2024-09-19T07:42:53.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.514+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6ac40a3430 msgr2=0x7f6ac40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:53.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.514+0000 7f6adfa49640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6ac40a3430 0x7f6ac40a58f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac80041e0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.514+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 msgr2=0x7f6ad8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.514+0000 7f6adfa49640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8107a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400b450 tx=0x7f6ad400b910 comp rx=0 tx=0).stop 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.515+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 shutdown_connections 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.515+0000 7f6adfa49640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6ac40a3430 0x7f6ac40a58f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.515+0000 7f6adfa49640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ad8106a90 0x7f6ad8108480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.515+0000 7f6adfa49640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ad8101430 0x7f6ad8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.515+0000 7f6adfa49640 1 --2- 172.21.15.103:0/4049207016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad8100a90 0x7f6ad8107a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.515+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 >> 172.21.15.103:0/4049207016 conn(0x7f6ad80fc7d0 msgr2=0x7f6ad80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.516+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 shutdown_connections 2024-09-19T07:42:53.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:53.516+0000 7f6adfa49640 1 -- 172.21.15.103:0/4049207016 wait complete. 2024-09-19T07:42:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:53 smithi103 ceph-mon[31201]: pgmap v150: 97 pgs: 97 active+clean; 2.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 934 op/s 2024-09-19T07:42:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:53 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4049207016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:53.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:53 smithi174 ceph-mon[35370]: pgmap v150: 97 pgs: 97 active+clean; 2.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 934 op/s 2024-09-19T07:42:53.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:53 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4049207016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:54.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:53 smithi134 ceph-mon[35414]: pgmap v150: 97 pgs: 97 active+clean; 2.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 934 op/s 2024-09-19T07:42:54.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:53 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4049207016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:42:54.378 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T07:42:54.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T07:42:54.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 -- 172.21.15.103:0/2443086901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce01005a0 msgr2=0x7fcce010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 --2- 172.21.15.103:0/2443086901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce01005a0 0x7fcce010ea20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009ca0 tx=0x7fccd802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 -- 172.21.15.103:0/2443086901 shutdown_connections 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 --2- 172.21.15.103:0/2443086901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce01005a0 0x7fcce010ea20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 --2- 172.21.15.103:0/2443086901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce00ffc00 0x7fcce0100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 --2- 172.21.15.103:0/2443086901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce0106b80 0x7fcce0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 -- 172.21.15.103:0/2443086901 >> 172.21.15.103:0/2443086901 conn(0x7fcce00fba90 msgr2=0x7fcce00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.626+0000 7fcce7299640 1 -- 172.21.15.103:0/2443086901 shutdown_connections 2024-09-19T07:42:54.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.627+0000 7fcce7299640 1 -- 172.21.15.103:0/2443086901 wait complete. 2024-09-19T07:42:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.627+0000 7fcce7299640 1 Processor -- start 2024-09-19T07:42:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.627+0000 7fcce7299640 1 -- start start 2024-09-19T07:42:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce7299640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce00ffc00 0x7fcce010e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce7299640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce01005a0 0x7fcce010a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce7299640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce0106b80 0x7fcce010ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce4834640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce01005a0 0x7fcce010a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce4834640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce01005a0 0x7fcce010a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55382/0 (socket says 172.21.15.103:55382) 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce4834640 1 -- 172.21.15.103:0/4162023645 learned_addr learned my addr 172.21.15.103:0/4162023645 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce7299640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0101cc0 con 0x7fcce00ffc00 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce5035640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce00ffc00 0x7fcce010e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0101b40 con 0x7fcce01005a0 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0101e40 con 0x7fcce0106b80 2024-09-19T07:42:54.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.628+0000 7fcce5836640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce0106b80 0x7fcce010ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.629+0000 7fcce4834640 1 -- 172.21.15.103:0/4162023645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce0106b80 msgr2=0x7fcce010ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.629+0000 7fcce4834640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce0106b80 0x7fcce010ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.629+0000 7fcce4834640 1 -- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce00ffc00 msgr2=0x7fcce010e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.629+0000 7fcce4834640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce00ffc00 0x7fcce010e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.629+0000 7fcce4834640 1 -- 172.21.15.103:0/4162023645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce010b100 con 0x7fcce01005a0 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.629+0000 7fcce5035640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce00ffc00 0x7fcce010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.629+0000 7fcce5836640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce0106b80 0x7fcce010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.630+0000 7fcce4834640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce01005a0 0x7fcce010a600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00ea20 tx=0x7fccdc00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.630+0000 7fccd67fc640 1 -- 172.21.15.103:0/4162023645 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00cd80 con 0x7fcce01005a0 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.630+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce010b3f0 con 0x7fcce01005a0 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.630+0000 7fccd67fc640 1 -- 172.21.15.103:0/4162023645 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccdc010910 con 0x7fcce01005a0 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.631+0000 7fccd67fc640 1 -- 172.21.15.103:0/4162023645 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00cd80 con 0x7fcce01005a0 2024-09-19T07:42:54.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.631+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce01ab540 con 0x7fcce01005a0 2024-09-19T07:42:54.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.634+0000 7fccd67fc640 1 -- 172.21.15.103:0/4162023645 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fccdc010ac0 con 0x7fcce01005a0 2024-09-19T07:42:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.634+0000 7fcce9672740 1 Processor -- start 2024-09-19T07:42:54.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce9672740 1 -- start start 2024-09-19T07:42:54.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce9672740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344db40 0x556fa344c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce9672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344c5d0 0x556fa344ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce9672740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa3459760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce9672740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556fa33b22f0 con 0x556fa344c5d0 2024-09-19T07:42:54.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce9672740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556fa3359cf0 con 0x556fa344db40 2024-09-19T07:42:54.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce9672740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556fa33ae2b0 con 0x556fa3457370 2024-09-19T07:42:54.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce5836640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344db40 0x556fa344c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce5836640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344db40 0x556fa344c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55390/0 (socket says 172.21.15.103:55390) 2024-09-19T07:42:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce5836640 1 -- 172.21.15.103:0/769159840 learned_addr learned my addr 172.21.15.103:0/769159840 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce5035640 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344c5d0 0x556fa344ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.635+0000 7fcce5836640 1 -- 172.21.15.103:0/769159840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 msgr2=0x556fa3459760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fccd67fc640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcccc0a3670 0x7fcccc0a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce4834640 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa3459760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce5836640 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa3459760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce5836640 1 -- 172.21.15.103:0/769159840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344c5d0 msgr2=0x556fa344ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce5836640 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344c5d0 0x556fa344ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce5836640 1 -- 172.21.15.103:0/769159840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa333e3a0 con 0x556fa344db40 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce5035640 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344c5d0 0x556fa344ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fccd67fc640 1 -- 172.21.15.103:0/4162023645 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fccdc0cff80 con 0x7fcce01005a0 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce5836640 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344db40 0x556fa344c090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcce01077c0 tx=0x7fccd80098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5294a6a2703eadc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:54.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.636+0000 7fcce5836640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcccc0a3670 0x7fcccc0a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.637+0000 7fcccb7fe640 1 -- 172.21.15.103:0/769159840 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8003b30 con 0x556fa344db40 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.637+0000 7fcccb7fe640 1 -- 172.21.15.103:0/769159840 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd8036040 con 0x556fa344db40 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.637+0000 7fcccb7fe640 1 -- 172.21.15.103:0/769159840 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd802f870 con 0x556fa344db40 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce5836640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcccc0a3670 0x7fcccc0a5b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccd804b000 tx=0x7fccd804c000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 -- 172.21.15.103:0/769159840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344db40 msgr2=0x556fa344c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344db40 0x556fa344c090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcce01077c0 tx=0x7fccd80098b0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 -- 172.21.15.103:0/769159840 shutdown_connections 2024-09-19T07:42:54.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa3459760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344c5d0 0x556fa344ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 --2- 172.21.15.103:0/769159840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344db40 0x556fa344c090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 -- 172.21.15.103:0/769159840 >> 172.21.15.103:0/769159840 conn(0x556fa3415310 msgr2=0x556fa3448c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:54.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 -- 172.21.15.103:0/769159840 shutdown_connections 2024-09-19T07:42:54.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.638+0000 7fcce9672740 1 -- 172.21.15.103:0/769159840 wait complete. 2024-09-19T07:42:54.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.639+0000 7fcce9672740 1 -- start start 2024-09-19T07:42:54.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.639+0000 7fcce9672740 1 Processor -- start 2024-09-19T07:42:54.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.640+0000 7fcce9672740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa3456d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.640+0000 7fcce9672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa3453fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.640+0000 7fcce9672740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa34545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.640+0000 7fcce4834640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa3453fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.640+0000 7fcce4834640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa3453fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56258/0 (socket says 172.21.15.103:56258) 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.640+0000 7fcce4834640 1 -- 172.21.15.103:0/2601611078 learned_addr learned my addr 172.21.15.103:0/2601611078 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.641+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556fa3459ef0 con 0x556fa344db40 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.641+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556fa336d060 con 0x556fa344c5d0 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.641+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556fa345a070 con 0x556fa3457370 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.641+0000 7fcce5035640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa3456d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.641+0000 7fcce5836640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa34545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.641+0000 7fcce9672740 10 client.0 _ll_register_callbacks cb 0x556fa30c08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:54.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fcce4834640 1 -- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 msgr2=0x556fa34545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fcce4834640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa34545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fcce4834640 1 -- 172.21.15.103:0/2601611078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 msgr2=0x556fa3456d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fcce4834640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa3456d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fcce4834640 1 -- 172.21.15.103:0/2601611078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa31233c0 con 0x556fa344db40 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fcce4834640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa3453fe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc09c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cda819b67bb1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fccc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fccc8ff9640 1 -- 172.21.15.103:0/2601611078 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0a0620 con 0x556fa344db40 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fccc8ff9640 1 -- 172.21.15.103:0/2601611078 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fccdc0bc0b0 con 0x556fa344db40 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd8036040 con 0x556fa344db40 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.642+0000 7fccc8ff9640 1 -- 172.21.15.103:0/2601611078 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fccdc0a4ee0 con 0x556fa344db40 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 10 client.16476 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:54.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556fa34658a0 con 0x556fa344db40 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce5035640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa3456d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fccc8ff9640 1 -- 172.21.15.103:0/2601611078 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd8036040 con 0x556fa344db40 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fccc8ff9640 1 -- 172.21.15.103:0/2601611078 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fccdc09b6c0 con 0x556fa344db40 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 10 client.16476 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 10 client.16476 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 20 client.16476 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 4 client.16476 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.643+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fa33bf860 con 0x556fa3466db0 2024-09-19T07:42:54.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.644+0000 7fcce5035640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.644+0000 7fcce5035640 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34691c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e375505ec80f5f8 server_cookie=79959bf2b40f7a8 in_seq=0 out_seq=0 2024-09-19T07:42:54.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.644+0000 7fccc8ff9640 10 client.16476 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fccc8ff9640 1 -- 172.21.15.103:0/2601611078 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fa33bf860 con 0x556fa3466db0 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fccc8ff9640 10 client.16476 handle_command_reply: tid=0 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 client.16476 shutdown 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 msgr2=0x556fa3453fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa3453fe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc09c970 comp rx=0 tx=0).stop 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 shutdown_connections 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34691c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa34545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa3453fe0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fccc8ff9640 0 client.16476 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:54.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 --2- 172.21.15.103:0/2601611078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa3456d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.645+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 >> 172.21.15.103:0/2601611078 conn(0x556fa3415310 msgr2=0x556fa344e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:54.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.646+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 shutdown_connections 2024-09-19T07:42:54.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.646+0000 7fcce9672740 1 -- 172.21.15.103:0/2601611078 wait complete. 2024-09-19T07:42:54.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.646+0000 7fcce9672740 20 client.16476 trim_cache size 0 max 16384 2024-09-19T07:42:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 Processor -- start 2024-09-19T07:42:54.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 -- start start 2024-09-19T07:42:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa344f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa344f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa346ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b3db0 con 0x556fa344db40 2024-09-19T07:42:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b5950 con 0x556fa344c5d0 2024-09-19T07:42:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce9672740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b4480 con 0x556fa3457370 2024-09-19T07:42:54.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce4834640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa344f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce4834640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa344f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56264/0 (socket says 172.21.15.103:56264) 2024-09-19T07:42:54.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce4834640 1 -- 172.21.15.103:0/2758927634 learned_addr learned my addr 172.21.15.103:0/2758927634 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:54.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce5035640 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa344f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.666+0000 7fcce5836640 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa346ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce5035640 1 -- 172.21.15.103:0/2758927634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 msgr2=0x556fa346ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce5035640 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa346ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce5035640 1 -- 172.21.15.103:0/2758927634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 msgr2=0x556fa344f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce5035640 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa344f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce5035640 1 -- 172.21.15.103:0/2758927634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa3454940 con 0x556fa344c5d0 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce5836640 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa346ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce4834640 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa344f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcce5035640 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa344f120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccdc09a520 tx=0x7fccd000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e5d38e3db2cebe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcccaffd640 1 -- 172.21.15.103:0/2758927634 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0005030 con 0x556fa344c5d0 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcccaffd640 1 -- 172.21.15.103:0/2758927634 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd001f030 con 0x556fa344c5d0 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.667+0000 7fcccaffd640 1 -- 172.21.15.103:0/2758927634 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0003850 con 0x556fa344c5d0 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.668+0000 7fcce9672740 1 -- 172.21.15.103:0/2758927634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 msgr2=0x556fa344f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.668+0000 7fcce9672740 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa344f120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccdc09a520 tx=0x7fccd000b390 comp rx=0 tx=0).stop 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.668+0000 7fcce9672740 1 -- 172.21.15.103:0/2758927634 shutdown_connections 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.668+0000 7fcce9672740 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa346ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.668+0000 7fcce9672740 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa344f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.668+0000 7fcce9672740 1 --2- 172.21.15.103:0/2758927634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa344f120 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.668+0000 7fcce9672740 1 -- 172.21.15.103:0/2758927634 >> 172.21.15.103:0/2758927634 conn(0x556fa3415310 msgr2=0x556fa3447d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:54.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.669+0000 7fcce9672740 1 -- 172.21.15.103:0/2758927634 shutdown_connections 2024-09-19T07:42:54.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.669+0000 7fcce9672740 1 -- 172.21.15.103:0/2758927634 wait complete. 2024-09-19T07:42:54.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.670+0000 7fcce9672740 1 -- start start 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.670+0000 7fcce9672740 1 Processor -- start 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.670+0000 7fcce9672740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa345b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce9672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa345b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce9672740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa345bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce9672740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b5280 con 0x556fa344db40 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce9672740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b4b50 con 0x556fa344c5d0 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa345b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce9672740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01b1d90 con 0x556fa3457370 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa345b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55412/0 (socket says 172.21.15.103:55412) 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 -- 172.21.15.103:0/3150940216 learned_addr learned my addr 172.21.15.103:0/3150940216 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce9672740 10 client.0 _ll_register_callbacks cb 0x7fccdc00ef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce4834640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa345b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5836640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa345bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 -- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 msgr2=0x556fa345bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa345bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 -- 172.21.15.103:0/3150940216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 msgr2=0x556fa345b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa345b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 -- 172.21.15.103:0/3150940216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa34d0b10 con 0x556fa344c5d0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce4834640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa345b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fcce5035640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa345b110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccd0014db0 tx=0x7fccd0014de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d51548c57ee549 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.671+0000 7fccc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fccc8ff9640 1 -- 172.21.15.103:0/3150940216 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0031060 con 0x556fa344c5d0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fccc8ff9640 1 -- 172.21.15.103:0/3150940216 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fccd0003b00 con 0x556fa344c5d0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd001f030 con 0x556fa344c5d0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fccc8ff9640 1 -- 172.21.15.103:0/3150940216 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fccd0003d00 con 0x556fa344c5d0 2024-09-19T07:42:54.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 10 client.25988 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:54.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556fa34d1dd0 con 0x556fa344c5d0 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fccc8ff9640 1 -- 172.21.15.103:0/3150940216 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd001f030 con 0x556fa344c5d0 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fccc8ff9640 1 -- 172.21.15.103:0/3150940216 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fccd0004c70 con 0x556fa344c5d0 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 10 client.25988 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 10 client.25988 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 20 client.25988 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34d4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 4 client.25988 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.672+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fa33b9d70 con 0x556fa3466db0 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.674+0000 7fcce4834640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34d4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.675+0000 7fcce4834640 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34d4470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35a34b0e109c965d server_cookie=b6ec281eb8ce3ec9 in_seq=0 out_seq=0 2024-09-19T07:42:54.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.675+0000 7fccc8ff9640 10 client.25988 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.675+0000 7fccc8ff9640 1 -- 172.21.15.103:0/3150940216 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fa33b9d70 con 0x556fa3466db0 2024-09-19T07:42:54.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.675+0000 7fccc8ff9640 10 client.25988 handle_command_reply: tid=0 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 client.25988 shutdown 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 msgr2=0x556fa345b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa345b110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccd0014db0 tx=0x7fccd0014de0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 shutdown_connections 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x556fa3466db0 0x556fa34d4470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa3457370 0x556fa345bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fa344db40 0x556fa345b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fccc8ff9640 0 client.25988 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:42:54.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 --2- 172.21.15.103:0/3150940216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556fa344c5d0 0x556fa345b110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 >> 172.21.15.103:0/3150940216 conn(0x556fa3415310 msgr2=0x556fa3455840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.676+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 shutdown_connections 2024-09-19T07:42:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.677+0000 7fcce9672740 1 -- 172.21.15.103:0/3150940216 wait complete. 2024-09-19T07:42:54.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.677+0000 7fcce9672740 20 client.25988 trim_cache size 0 max 16384 2024-09-19T07:42:54.679 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcccc0a3670 msgr2=0x7fcccc0a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcccc0a3670 0x7fcccc0a5b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccd804b000 tx=0x7fccd804c000 comp rx=0 tx=0).stop 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce01005a0 msgr2=0x7fcce010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce01005a0 0x7fcce010a600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00ea20 tx=0x7fccdc00eee0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 shutdown_connections 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcccc0a3670 0x7fcccc0a5b30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce0106b80 0x7fcce010ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce01005a0 0x7fcce010a600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 --2- 172.21.15.103:0/4162023645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcce00ffc00 0x7fcce010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:54.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 >> 172.21.15.103:0/4162023645 conn(0x7fcce00fba90 msgr2=0x7fcce00fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:54.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 shutdown_connections 2024-09-19T07:42:54.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:54.678+0000 7fcce7299640 1 -- 172.21.15.103:0/4162023645 wait complete. 2024-09-19T07:42:55.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:42:55.465 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 260.0 s, 2675.6 tps, lat 37.211 ms stddev 40.738 2024-09-19T07:42:55.902 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:55 smithi103 ceph-mon[31201]: pgmap v151: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2024-09-19T07:42:55.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:55 smithi174 ceph-mon[35370]: pgmap v151: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2024-09-19T07:42:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.022+0000 7f9b241af640 1 -- 172.21.15.103:0/2369585151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 msgr2=0x7f9b1c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.022+0000 7f9b241af640 1 --2- 172.21.15.103:0/2369585151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 0x7f9b1c10ea20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009ca0 tx=0x7f9b1402a8d0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.022+0000 7f9b241af640 1 -- 172.21.15.103:0/2369585151 shutdown_connections 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.022+0000 7f9b241af640 1 --2- 172.21.15.103:0/2369585151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 0x7f9b1c10ea20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.022+0000 7f9b241af640 1 --2- 172.21.15.103:0/2369585151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b1c100100 0x7f9b1c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.022+0000 7f9b241af640 1 --2- 172.21.15.103:0/2369585151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c106b80 0x7f9b1c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.022+0000 7f9b241af640 1 -- 172.21.15.103:0/2369585151 >> 172.21.15.103:0/2369585151 conn(0x7f9b1c077230 msgr2=0x7f9b1c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.023+0000 7f9b241af640 1 -- 172.21.15.103:0/2369585151 shutdown_connections 2024-09-19T07:42:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.023+0000 7f9b241af640 1 -- 172.21.15.103:0/2369585151 wait complete. 2024-09-19T07:42:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.023+0000 7f9b241af640 1 Processor -- start 2024-09-19T07:42:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.023+0000 7f9b241af640 1 -- start start 2024-09-19T07:42:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b241af640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100100 0x7f9b1c1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b241af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 0x7f9b1c101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b241af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b1c106b80 0x7f9b1c101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b241af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c112050 con 0x7f9b1c100aa0 2024-09-19T07:42:56.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b241af640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c111ed0 con 0x7f9b1c100100 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b241af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1121d0 con 0x7f9b1c106b80 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 0x7f9b1c101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b21f4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100100 0x7f9b1c1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b21f4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100100 0x7f9b1c1012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55434/0 (socket says 172.21.15.103:55434) 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 0x7f9b1c101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56294/0 (socket says 172.21.15.103:56294) 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b21f4b640 1 -- 172.21.15.103:0/2530300348 learned_addr learned my addr 172.21.15.103:0/2530300348 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2274c640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b1c106b80 0x7f9b1c101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2274c640 1 -- 172.21.15.103:0/2530300348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100100 msgr2=0x7f9b1c1012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2274c640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100100 0x7f9b1c1012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2274c640 1 -- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 msgr2=0x7f9b1c101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2274c640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 0x7f9b1c101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.024+0000 7f9b2274c640 1 -- 172.21.15.103:0/2530300348 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c1a68f0 con 0x7f9b1c106b80 2024-09-19T07:42:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.025+0000 7f9b2274c640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b1c106b80 0x7f9b1c101d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b1402aec0 tx=0x7f9b140317d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.025+0000 7f9b12ffd640 1 -- 172.21.15.103:0/2530300348 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14003970 con 0x7f9b1c106b80 2024-09-19T07:42:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.025+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c1a6b80 con 0x7f9b1c106b80 2024-09-19T07:42:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.025+0000 7f9b12ffd640 1 -- 172.21.15.103:0/2530300348 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b14004040 con 0x7f9b1c106b80 2024-09-19T07:42:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.025+0000 7f9b12ffd640 1 -- 172.21.15.103:0/2530300348 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1402f910 con 0x7f9b1c106b80 2024-09-19T07:42:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.025+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c1a7110 con 0x7f9b1c106b80 2024-09-19T07:42:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.028+0000 7f9b12ffd640 1 -- 172.21.15.103:0/2530300348 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9b140041f0 con 0x7f9b1c106b80 2024-09-19T07:42:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.029+0000 7f9b12ffd640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9b080a3400 0x7f9b080a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.029+0000 7f9b12ffd640 1 -- 172.21.15.103:0/2530300348 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9b140f0da0 con 0x7f9b1c106b80 2024-09-19T07:42:56.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.029+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9b080a3400 0x7f9b080a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.030+0000 7f9b26588740 1 Processor -- start 2024-09-19T07:42:56.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.030+0000 7f9b26588740 1 -- start start 2024-09-19T07:42:56.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.030+0000 7f9b26588740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db19b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.030+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9b080a3400 0x7f9b080a58c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009c10 tx=0x7f9b18005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.030+0000 7f9b26588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 0x56540db24400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.030+0000 7f9b21f4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db19b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.030+0000 7f9b26588740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db26d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.031+0000 7f9b21f4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db19b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56296/0 (socket says 172.21.15.103:56296) 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.031+0000 7f9b21f4b640 1 -- 172.21.15.103:0/1853968722 learned_addr learned my addr 172.21.15.103:0/1853968722 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.031+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56540da7f2a0 con 0x56540db19760 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.031+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56540da26b10 con 0x56540db1a080 2024-09-19T07:42:56.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.031+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56540da7b260 con 0x56540db24940 2024-09-19T07:42:56.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.031+0000 7f9b2174a640 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 0x56540db24400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.033+0000 7f9b21f4b640 1 -- 172.21.15.103:0/1853968722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 msgr2=0x56540db26d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.033+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db26d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.033+0000 7f9b21f4b640 1 -- 172.21.15.103:0/1853968722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 msgr2=0x56540db24400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.033+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 0x56540db24400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.034+0000 7f9b21f4b640 1 -- 172.21.15.103:0/1853968722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56540d7f0b90 con 0x56540db19760 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.034+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db19b40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009f40 tx=0x7f9b18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500b5457ef70c224 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.035+0000 7f9b07fff640 1 -- 172.21.15.103:0/1853968722 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004230 con 0x56540db19760 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.035+0000 7f9b07fff640 1 -- 172.21.15.103:0/1853968722 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b180043d0 con 0x56540db19760 2024-09-19T07:42:56.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.035+0000 7f9b07fff640 1 -- 172.21.15.103:0/1853968722 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1800e8f0 con 0x56540db19760 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.036+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 msgr2=0x56540db19b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.036+0000 7f9b26588740 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db19b40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009f40 tx=0x7f9b18014000 comp rx=0 tx=0).stop 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.036+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 shutdown_connections 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.036+0000 7f9b26588740 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db26d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.036+0000 7f9b26588740 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 0x56540db24400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.036+0000 7f9b26588740 1 --2- 172.21.15.103:0/1853968722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db19b40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.036+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 >> 172.21.15.103:0/1853968722 conn(0x56540db15cf0 msgr2=0x56540db160d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.037+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 shutdown_connections 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.037+0000 7f9b26588740 1 -- 172.21.15.103:0/1853968722 wait complete. 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.037+0000 7f9b26588740 1 -- start start 2024-09-19T07:42:56.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.037+0000 7f9b26588740 1 Processor -- start 2024-09-19T07:42:56.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.037+0000 7f9b26588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b26588740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db1a080 0x56540db21df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b26588740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db22330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b26588740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56540db27510 con 0x56540db1a080 2024-09-19T07:42:56.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b26588740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56540da076e0 con 0x56540db19760 2024-09-19T07:42:56.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b26588740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56540db27690 con 0x56540db24940 2024-09-19T07:42:56.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db1a080 0x56540db21df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b26588740 10 client.0 _ll_register_callbacks cb 0x56540da2ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db1a080 0x56540db21df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56298/0 (socket says 172.21.15.103:56298) 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 -- 172.21.15.103:0/2608474189 learned_addr learned my addr 172.21.15.103:0/2608474189 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2274c640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 -- 172.21.15.103:0/2608474189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 msgr2=0x56540db22330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db22330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 -- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 msgr2=0x56540db218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db218b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 -- 172.21.15.103:0/2608474189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56540db30a80 con 0x56540db1a080 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2274c640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:56.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b2174a640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db1a080 0x56540db21df0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00b9f0 tx=0x7f9b0c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a446c1fdb1c6051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.038+0000 7f9b057fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.039+0000 7f9b057fa640 1 -- 172.21.15.103:0/2608474189 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c01b640 con 0x56540db1a080 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.039+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b180043d0 con 0x56540db1a080 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.039+0000 7f9b057fa640 1 -- 172.21.15.103:0/2608474189 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9b0c01b7e0 con 0x56540db1a080 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.039+0000 7f9b057fa640 1 -- 172.21.15.103:0/2608474189 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9b0c01b9e0 con 0x56540db1a080 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.039+0000 7f9b26588740 10 client.16494 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.039+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56540db334c0 con 0x56540db1a080 2024-09-19T07:42:56.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.039+0000 7f9b057fa640 1 -- 172.21.15.103:0/2608474189 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b180043d0 con 0x56540db1a080 2024-09-19T07:42:56.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b057fa640 1 -- 172.21.15.103:0/2608474189 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f9b0c019020 con 0x56540db1a080 2024-09-19T07:42:56.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b26588740 10 client.16494 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:56.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b26588740 10 client.16494 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:42:56.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b26588740 20 client.16494 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:56.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b26588740 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540db36d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b26588740 4 client.16494 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56540da8c9d0 con 0x56540db34920 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.040+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540db36d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.041+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540db36d40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780db279b64bc8a8 server_cookie=88b384c5e0d80b00 in_seq=0 out_seq=0 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.041+0000 7f9b057fa640 10 client.16494 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.041+0000 7f9b057fa640 1 -- 172.21.15.103:0/2608474189 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56540da8c9d0 con 0x56540db34920 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.041+0000 7f9b057fa640 10 client.16494 handle_command_reply: tid=0 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 client.16494 shutdown 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db1a080 msgr2=0x56540db21df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db1a080 0x56540db21df0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00b9f0 tx=0x7f9b0c00beb0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 shutdown_connections 2024-09-19T07:42:56.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540db36d40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b057fa640 0 client.16494 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db1a080 0x56540db21df0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.042+0000 7f9b26588740 1 --2- 172.21.15.103:0/2608474189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.043+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 >> 172.21.15.103:0/2608474189 conn(0x56540db15cf0 msgr2=0x56540db1ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.043+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 shutdown_connections 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.043+0000 7f9b26588740 1 -- 172.21.15.103:0/2608474189 wait complete. 2024-09-19T07:42:56.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.043+0000 7f9b26588740 20 client.16494 trim_cache size 0 max 16384 2024-09-19T07:42:56.074 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 Processor -- start 2024-09-19T07:42:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 -- start start 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db20850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 0x56540db221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db3c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1afa50 con 0x56540db19760 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1b15f0 con 0x56540db1a080 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.074+0000 7f9b26588740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1b0120 con 0x56540db24940 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.076+0000 7f9b2274c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db3c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.076+0000 7f9b2274c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db3c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38762/0 (socket says 172.21.15.103:38762) 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.076+0000 7f9b2274c640 1 -- 172.21.15.103:0/941238837 learned_addr learned my addr 172.21.15.103:0/941238837 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.077+0000 7f9b2274c640 1 -- 172.21.15.103:0/941238837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 msgr2=0x56540db221a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.077+0000 7f9b2274c640 1 --2- 172.21.15.103:0/941238837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 0x56540db221a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.077+0000 7f9b2274c640 1 -- 172.21.15.103:0/941238837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 msgr2=0x56540db20850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.077+0000 7f9b2274c640 1 --2- 172.21.15.103:0/941238837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db20850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.077+0000 7f9b2274c640 1 -- 172.21.15.103:0/941238837 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56540db11260 con 0x56540db24940 2024-09-19T07:42:56.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.078+0000 7f9b2274c640 1 --2- 172.21.15.103:0/941238837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db3c610 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b140bcea0 tx=0x7f9b14009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d30f2e09c70cea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:56.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:55 smithi134 ceph-mon[35414]: pgmap v151: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2024-09-19T07:42:56.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.079+0000 7f9b077fe640 1 -- 172.21.15.103:0/941238837 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140bd1a0 con 0x56540db24940 2024-09-19T07:42:56.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.079+0000 7f9b077fe640 1 -- 172.21.15.103:0/941238837 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b14036040 con 0x56540db24940 2024-09-19T07:42:56.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.079+0000 7f9b077fe640 1 -- 172.21.15.103:0/941238837 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140c1170 con 0x56540db24940 2024-09-19T07:42:56.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.081+0000 7f9b26588740 1 -- 172.21.15.103:0/941238837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 msgr2=0x56540db3c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.081+0000 7f9b26588740 1 --2- 172.21.15.103:0/941238837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db3c610 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b140bcea0 tx=0x7f9b14009980 comp rx=0 tx=0).stop 2024-09-19T07:42:56.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.083+0000 7f9b26588740 1 -- 172.21.15.103:0/941238837 shutdown_connections 2024-09-19T07:42:56.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.083+0000 7f9b26588740 1 --2- 172.21.15.103:0/941238837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db24940 0x56540db3c610 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.083+0000 7f9b26588740 1 --2- 172.21.15.103:0/941238837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db1a080 0x56540db221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.083+0000 7f9b26588740 1 --2- 172.21.15.103:0/941238837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db19760 0x56540db20850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.083+0000 7f9b26588740 1 -- 172.21.15.103:0/941238837 >> 172.21.15.103:0/941238837 conn(0x56540db15cf0 msgr2=0x56540db23930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:56.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.084+0000 7f9b26588740 1 -- 172.21.15.103:0/941238837 shutdown_connections 2024-09-19T07:42:56.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.085+0000 7f9b26588740 1 -- 172.21.15.103:0/941238837 wait complete. 2024-09-19T07:42:56.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.085+0000 7f9b26588740 1 -- start start 2024-09-19T07:42:56.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.085+0000 7f9b26588740 1 Processor -- start 2024-09-19T07:42:56.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b26588740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db1ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b26588740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db1a080 0x56540db1efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b26588740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db24940 0x56540db1f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b26588740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1b0f20 con 0x56540db24940 2024-09-19T07:42:56.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b26588740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1b07f0 con 0x56540db19760 2024-09-19T07:42:56.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b26588740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1ada50 con 0x56540db1a080 2024-09-19T07:42:56.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b26588740 10 client.0 _ll_register_callbacks cb 0x7f9b0c004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:56.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b21f4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db1ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b21f4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db1ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:55482/0 (socket says 172.21.15.103:55482) 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.086+0000 7f9b21f4b640 1 -- 172.21.15.103:0/4114228104 learned_addr learned my addr 172.21.15.103:0/4114228104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.087+0000 7f9b2174a640 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db1a080 0x56540db1efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.087+0000 7f9b2274c640 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db24940 0x56540db1f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.088+0000 7f9b21f4b640 1 -- 172.21.15.103:0/4114228104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db1a080 msgr2=0x56540db1efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.088+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db1a080 0x56540db1efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.088+0000 7f9b21f4b640 1 -- 172.21.15.103:0/4114228104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db24940 msgr2=0x56540db1f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.088+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db24940 0x56540db1f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.088+0000 7f9b21f4b640 1 -- 172.21.15.103:0/4114228104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56540db9e740 con 0x56540db19760 2024-09-19T07:42:56.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.089+0000 7f9b21f4b640 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db1ea90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800e4c0 tx=0x7f9b18005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66bbfdd0653ac165 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:56.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.089+0000 7f9b057fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:42:56.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.090+0000 7f9b057fa640 1 -- 172.21.15.103:0/4114228104 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b180053c0 con 0x56540db19760 2024-09-19T07:42:56.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.090+0000 7f9b057fa640 1 -- 172.21.15.103:0/4114228104 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9b18009040 con 0x56540db19760 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.090+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b14036040 con 0x56540db19760 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b057fa640 1 -- 172.21.15.103:0/4114228104 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9b180035f0 con 0x56540db19760 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 10 client.26003 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56540db9fa00 con 0x56540db19760 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b057fa640 1 -- 172.21.15.103:0/4114228104 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b18003d30 con 0x56540db19760 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b057fa640 1 -- 172.21.15.103:0/4114228104 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f9b180362b0 con 0x56540db19760 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 10 client.26003 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 10 client.26003 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:42:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 20 client.26003 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540dba1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 4 client.26003 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T07:42:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.091+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x7f9b14036040 con 0x56540db34920 2024-09-19T07:42:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.092+0000 7f9b2174a640 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540dba1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:56.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.095+0000 7f9b2174a640 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540dba1e30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7443c18fc7a46321 server_cookie=20b83da53ccde58e in_seq=0 out_seq=0 2024-09-19T07:42:56.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.095+0000 7f9b057fa640 10 client.26003 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:56.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.264+0000 7f9b057fa640 1 -- 172.21.15.103:0/4114228104 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f9b14036040 con 0x56540db34920 2024-09-19T07:42:56.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.264+0000 7f9b057fa640 10 client.26003 handle_command_reply: tid=0 2024-09-19T07:42:56.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 client.26003 shutdown 2024-09-19T07:42:56.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 msgr2=0x56540db1ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db1ea90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800e4c0 tx=0x7f9b18005650 comp rx=0 tx=0).stop 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 shutdown_connections 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56540db34920 0x56540dba1e30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56540db24940 0x56540db1f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56540db1a080 0x56540db1efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b057fa640 0 client.26003 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 --2- 172.21.15.103:0/4114228104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56540db19760 0x56540db1ea90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 >> 172.21.15.103:0/4114228104 conn(0x56540db15cf0 msgr2=0x56540db1b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 shutdown_connections 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 1 -- 172.21.15.103:0/4114228104 wait complete. 2024-09-19T07:42:56.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.265+0000 7f9b26588740 20 client.26003 trim_cache size 0 max 16384 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 36, 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 22, 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "94fd66e0": { 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 22, 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:42:56.268 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:56.269 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:56.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:42:56.270 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:42:56.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9b080a3400 msgr2=0x7f9b080a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9b080a3400 0x7f9b080a58c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009c10 tx=0x7f9b18005f90 comp rx=0 tx=0).stop 2024-09-19T07:42:56.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b1c106b80 msgr2=0x7f9b1c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b1c106b80 0x7f9b1c101d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b1402aec0 tx=0x7f9b140317d0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 shutdown_connections 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9b080a3400 0x7f9b080a58c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b1c106b80 0x7f9b1c101d60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c100aa0 0x7f9b1c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 --2- 172.21.15.103:0/2530300348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100100 0x7f9b1c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 >> 172.21.15.103:0/2530300348 conn(0x7f9b1c077230 msgr2=0x7f9b1c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 shutdown_connections 2024-09-19T07:42:56.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:56.266+0000 7f9b241af640 1 -- 172.21.15.103:0/2530300348 wait complete. 2024-09-19T07:42:57.292 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '94fd66e0', quiesce took 5.8 seconds. Will release after: 24.0 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 -- 172.21.15.103:0/1574079239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0102790 msgr2=0x7f16a01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 --2- 172.21.15.103:0/1574079239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0102790 0x7f16a01067e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1694009900 tx=0x7f169402a410 comp rx=0 tx=0).stop 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 -- 172.21.15.103:0/1574079239 shutdown_connections 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 --2- 172.21.15.103:0/1574079239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a0106f10 0x7f16a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 --2- 172.21.15.103:0/1574079239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0102790 0x7f16a01067e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 --2- 172.21.15.103:0/1574079239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0101de0 0x7f16a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 -- 172.21.15.103:0/1574079239 >> 172.21.15.103:0/1574079239 conn(0x7f16a00fd8c0 msgr2=0x7f16a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 -- 172.21.15.103:0/1574079239 shutdown_connections 2024-09-19T07:42:57.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 -- 172.21.15.103:0/1574079239 wait complete. 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.552+0000 7f16a5dba640 1 Processor -- start 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f16a5dba640 1 -- start start 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f16a5dba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0101de0 0x7f16a01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f16a5dba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a0102790 0x7f16a0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f16a5dba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0106f10 0x7f16a019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f16a5dba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0111f70 con 0x7f16a0106f10 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f16a5dba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0111df0 con 0x7f16a0102790 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f16a5dba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01120f0 con 0x7f16a0101de0 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0106f10 0x7f16a019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0106f10 0x7f16a019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56356/0 (socket says 172.21.15.103:56356) 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 -- 172.21.15.103:0/453178929 learned_addr learned my addr 172.21.15.103:0/453178929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:57.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169effd640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a0102790 0x7f16a0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169f7fe640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0101de0 0x7f16a01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 -- 172.21.15.103:0/453178929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0101de0 msgr2=0x7f16a01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0101de0 0x7f16a01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 -- 172.21.15.103:0/453178929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a0102790 msgr2=0x7f16a0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a0102790 0x7f16a0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.553+0000 7f169ffff640 1 -- 172.21.15.103:0/453178929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a019d380 con 0x7f16a0106f10 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.554+0000 7f169ffff640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0106f10 0x7f16a019cca0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f169800cce0 tx=0x7f169800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.554+0000 7f169cff9640 1 -- 172.21.15.103:0/453178929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169800f450 con 0x7f16a0106f10 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.554+0000 7f169cff9640 1 -- 172.21.15.103:0/453178929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169800fad0 con 0x7f16a0106f10 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.554+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a019d670 con 0x7f16a0106f10 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.554+0000 7f169cff9640 1 -- 172.21.15.103:0/453178929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698018c10 con 0x7f16a0106f10 2024-09-19T07:42:57.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.554+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01a4f50 con 0x7f16a0106f10 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.556+0000 7f169cff9640 1 -- 172.21.15.103:0/453178929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f169800f5f0 con 0x7f16a0106f10 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.556+0000 7f16a8193740 1 Processor -- start 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f16a8193740 1 -- start start 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f16a8193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a55b40 0x55c977a54090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f16a8193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a54a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f16a8193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a5f370 0x55c977a61760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f16a8193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c9779ba2f0 con 0x55c977a5f370 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f16a8193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c977961cf0 con 0x55c977a55b40 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f16a8193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9779b62b0 con 0x55c977a545d0 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169cff9640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16880a33b0 0x7f16880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a5f370 0x55c977a61760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a5f370 0x55c977a61760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56366/0 (socket says 172.21.15.103:56366) 2024-09-19T07:42:57.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 -- 172.21.15.103:0/624859180 learned_addr learned my addr 172.21.15.103:0/624859180 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169ffff640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16880a33b0 0x7f16880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a55b40 0x55c977a54090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169cff9640 1 -- 172.21.15.103:0/453178929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f16980d0290 con 0x7f16a0106f10 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169f7fe640 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a54a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 -- 172.21.15.103:0/624859180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 msgr2=0x55c977a54a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a54a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 -- 172.21.15.103:0/624859180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a55b40 msgr2=0x55c977a54090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a55b40 0x55c977a54090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.557+0000 7f169effd640 1 -- 172.21.15.103:0/624859180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9779463a0 con 0x55c977a5f370 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f169effd640 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a5f370 0x55c977a61760 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1694005ec0 tx=0x7f1694005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fd8fe471ef3563 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f1685ffb640 1 -- 172.21.15.103:0/624859180 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694035070 con 0x55c977a5f370 2024-09-19T07:42:57.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f169ffff640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16880a33b0 0x7f16880a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16980cdaa0 tx=0x7f169806e300 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f1685ffb640 1 -- 172.21.15.103:0/624859180 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1694031800 con 0x55c977a5f370 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f1685ffb640 1 -- 172.21.15.103:0/624859180 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16940047a0 con 0x55c977a5f370 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 -- 172.21.15.103:0/624859180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a5f370 msgr2=0x55c977a61760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a5f370 0x55c977a61760 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1694005ec0 tx=0x7f1694005d90 comp rx=0 tx=0).stop 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 -- 172.21.15.103:0/624859180 shutdown_connections 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a5f370 0x55c977a61760 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a54a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 --2- 172.21.15.103:0/624859180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a55b40 0x55c977a54090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 -- 172.21.15.103:0/624859180 >> 172.21.15.103:0/624859180 conn(0x55c977a1d310 msgr2=0x55c977a50c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.558+0000 7f16a8193740 1 -- 172.21.15.103:0/624859180 shutdown_connections 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 -- 172.21.15.103:0/624859180 wait complete. 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 -- start start 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 Processor -- start 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a5ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a5c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c977a62010 con 0x55c977a55b40 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c977975060 con 0x55c977a545d0 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c977a62190 con 0x55c977a5f370 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f16a8193740 10 client.0 _ll_register_callbacks cb 0x55c9776c88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56382/0 (socket says 172.21.15.103:56382) 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.559+0000 7f169f7fe640 1 -- 172.21.15.103:0/2593204777 learned_addr learned my addr 172.21.15.103:0/2593204777 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169effd640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a5c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169effd640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a5ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169f7fe640 1 -- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 msgr2=0x55c977a5c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169f7fe640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a5c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169f7fe640 1 -- 172.21.15.103:0/2593204777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 msgr2=0x55c977a5ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169f7fe640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a5ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169f7fe640 1 -- 172.21.15.103:0/2593204777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97772b3c0 con 0x55c977a55b40 2024-09-19T07:42:57.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169effd640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a5ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f169f7fe640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5c020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1690012770 tx=0x7f1690012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46d6f50879c1b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f167effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f167effd640 1 -- 172.21.15.103:0/2593204777 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169001f060 con 0x55c977a55b40 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f167effd640 1 -- 172.21.15.103:0/2593204777 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f1690010d60 con 0x55c977a55b40 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1694031800 con 0x55c977a55b40 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f167effd640 1 -- 172.21.15.103:0/2593204777 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f169001a430 con 0x55c977a55b40 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f16a8193740 10 client.16509 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c977a6db10 con 0x55c977a55b40 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.560+0000 7f167effd640 1 -- 172.21.15.103:0/2593204777 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1694031800 con 0x55c977a55b40 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f167effd640 1 -- 172.21.15.103:0/2593204777 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f1690022550 con 0x55c977a55b40 2024-09-19T07:42:57.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f16a8193740 10 client.16509 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f16a8193740 10 client.16509 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f16a8193740 20 client.16509 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f16a8193740 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977a6f020 0x55c977a71410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f16a8193740 4 client.16509 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9779c7860 con 0x55c977a6f020 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f169effd640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977a6f020 0x55c977a71410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f167effd640 10 client.16509 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.561+0000 7f169effd640 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977a6f020 0x55c977a71410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc2a026c83659ac6 server_cookie=3be2d84864405f2f in_seq=0 out_seq=0 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f167effd640 1 -- 172.21.15.103:0/2593204777 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9779c7860 con 0x55c977a6f020 2024-09-19T07:42:57.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f167effd640 10 client.16509 handle_command_reply: tid=0 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 client.16509 shutdown 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 msgr2=0x55c977a5c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5c020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1690012770 tx=0x7f1690012c30 comp rx=0 tx=0).stop 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 shutdown_connections 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977a6f020 0x55c977a71410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a5c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5c020 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f167effd640 0 client.16509 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 --2- 172.21.15.103:0/2593204777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a5ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.562+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 >> 172.21.15.103:0/2593204777 conn(0x55c977a1d310 msgr2=0x55c977a56960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:57.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.563+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 shutdown_connections 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.563+0000 7f16a8193740 1 -- 172.21.15.103:0/2593204777 wait complete. 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.563+0000 7f16a8193740 20 client.16509 trim_cache size 0 max 16384 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.575+0000 7f16a8193740 1 Processor -- start 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.575+0000 7f16a8193740 1 -- start start 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.575+0000 7f16a8193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a4bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16a8193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16a8193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a5f370 0x55c977a6b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16a8193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01ad780 con 0x55c977a55b40 2024-09-19T07:42:57.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16a8193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01af320 con 0x55c977a5f370 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16a8193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01ade50 con 0x55c977a545d0 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56392/0 (socket says 172.21.15.103:56392) 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 -- 172.21.15.103:0/1048033505 learned_addr learned my addr 172.21.15.103:0/1048033505 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169effd640 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a5f370 0x55c977a6b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 -- 172.21.15.103:0/1048033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 msgr2=0x55c977a4bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169effd640 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a4bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a4bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 -- 172.21.15.103:0/1048033505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a5f370 msgr2=0x55c977a6b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a5f370 0x55c977a6b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 -- 172.21.15.103:0/1048033505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c977a5aff0 con 0x55c977a55b40 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169effd640 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a4bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f169f7fe640 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5bff0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1690005e10 tx=0x7f169000a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c566316e5afd58e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16857fa640 1 -- 172.21.15.103:0/1048033505 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690012030 con 0x55c977a55b40 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16857fa640 1 -- 172.21.15.103:0/1048033505 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16900121d0 con 0x55c977a55b40 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.576+0000 7f16857fa640 1 -- 172.21.15.103:0/1048033505 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690004310 con 0x55c977a55b40 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 -- 172.21.15.103:0/1048033505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 msgr2=0x55c977a5bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5bff0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1690005e10 tx=0x7f169000a450 comp rx=0 tx=0).stop 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 -- 172.21.15.103:0/1048033505 shutdown_connections 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a5f370 0x55c977a6b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a5bff0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 --2- 172.21.15.103:0/1048033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a545d0 0x55c977a4bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 -- 172.21.15.103:0/1048033505 >> 172.21.15.103:0/1048033505 conn(0x55c977a1d310 msgr2=0x55c977a5b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:57.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 -- 172.21.15.103:0/1048033505 shutdown_connections 2024-09-19T07:42:57.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 -- 172.21.15.103:0/1048033505 wait complete. 2024-09-19T07:42:57.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.577+0000 7f16a8193740 1 -- start start 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 1 Processor -- start 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a67bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a58e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01aec50 con 0x55c977a55b40 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01ae520 con 0x55c977a545d0 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01ab760 con 0x55c977a5f370 2024-09-19T07:42:57.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f16a8193740 10 client.0 _ll_register_callbacks cb 0x7f169000a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a58e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a58e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56408/0 (socket says 172.21.15.103:56408) 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 -- 172.21.15.103:0/1466909651 learned_addr learned my addr 172.21.15.103:0/1466909651 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169effd640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169effd640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a67bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 -- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 msgr2=0x55c977a593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 -- 172.21.15.103:0/1466909651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 msgr2=0x55c977a67bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:57.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a67bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 -- 172.21.15.103:0/1466909651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c977a71780 con 0x55c977a55b40 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169effd640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a67bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f169f7fe640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a58e40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f16900229f0 tx=0x7f1690013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df088c1c582e8b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.578+0000 7f167effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f167effd640 1 -- 172.21.15.103:0/1466909651 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169001f070 con 0x55c977a55b40 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f167effd640 1 -- 172.21.15.103:0/1466909651 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f1690010d60 con 0x55c977a55b40 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16900121d0 con 0x55c977a55b40 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f167effd640 1 -- 172.21.15.103:0/1466909651 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f169003b050 con 0x55c977a55b40 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 10 client.16515 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:57.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c977ad9b50 con 0x55c977a55b40 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f167effd640 1 -- 172.21.15.103:0/1466909651 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16900121d0 con 0x55c977a55b40 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f167effd640 1 -- 172.21.15.103:0/1466909651 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f16900054b0 con 0x55c977a55b40 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 10 client.16515 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 10 client.16515 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 20 client.16515 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977ada160 0x55c977adc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 4 client.16515 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.579+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9779c1d70 con 0x55c977ada160 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f169effd640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977ada160 0x55c977adc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f167effd640 10 client.16515 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f169effd640 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977ada160 0x55c977adc550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66e576d7aa265295 server_cookie=c66549ae2dfeb24a in_seq=0 out_seq=0 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f167effd640 1 -- 172.21.15.103:0/1466909651 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9779c1d70 con 0x55c977ada160 2024-09-19T07:42:57.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f167effd640 10 client.16515 handle_command_reply: tid=0 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 client.16515 shutdown 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 msgr2=0x55c977a58e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a58e40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f16900229f0 tx=0x7f1690013f90 comp rx=0 tx=0).stop 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 shutdown_connections 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c977ada160 0x55c977adc550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c977a5f370 0x55c977a593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c977a55b40 0x55c977a58e40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f167effd640 0 client.16515 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.580+0000 7f16a8193740 1 --2- 172.21.15.103:0/1466909651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c977a545d0 0x55c977a67bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 >> 172.21.15.103:0/1466909651 conn(0x55c977a1d310 msgr2=0x55c977a5d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 shutdown_connections 2024-09-19T07:42:57.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a8193740 1 -- 172.21.15.103:0/1466909651 wait complete. 2024-09-19T07:42:57.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a8193740 20 client.16515 trim_cache size 0 max 16384 2024-09-19T07:42:57.602 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16880a33b0 msgr2=0x7f16880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16880a33b0 0x7f16880a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16980cdaa0 tx=0x7f169806e300 comp rx=0 tx=0).stop 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0106f10 msgr2=0x7f16a019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0106f10 0x7f16a019cca0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f169800cce0 tx=0x7f169800a620 comp rx=0 tx=0).stop 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 shutdown_connections 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16880a33b0 0x7f16880a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0106f10 0x7f16a019cca0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16a0102790 0x7f16a0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 --2- 172.21.15.103:0/453178929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0101de0 0x7f16a01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 >> 172.21.15.103:0/453178929 conn(0x7f16a00fd8c0 msgr2=0x7f16a00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:57.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 shutdown_connections 2024-09-19T07:42:57.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:57.581+0000 7f16a5dba640 1 -- 172.21.15.103:0/453178929 wait complete. 2024-09-19T07:42:57.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:57 smithi103 ceph-mon[31201]: pgmap v152: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-09-19T07:42:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:57 smithi174 ceph-mon[35370]: pgmap v152: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-09-19T07:42:58.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:57 smithi134 ceph-mon[35414]: pgmap v152: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-09-19T07:42:58.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:42:58.500 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.984+0000 7f217ffb5640 1 -- 172.21.15.103:0/380807913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178100a80 msgr2=0x7f2178100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.984+0000 7f217ffb5640 1 --2- 172.21.15.103:0/380807913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178100a80 0x7f2178100ee0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2168009930 tx=0x7f216802a420 comp rx=0 tx=0).stop 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 -- 172.21.15.103:0/380807913 shutdown_connections 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 --2- 172.21.15.103:0/380807913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2178101420 0x7f217810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 --2- 172.21.15.103:0/380807913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178100a80 0x7f2178100ee0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 --2- 172.21.15.103:0/380807913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2178106a80 0x7f2178106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 -- 172.21.15.103:0/380807913 >> 172.21.15.103:0/380807913 conn(0x7f21780fc7a0 msgr2=0x7f21780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 -- 172.21.15.103:0/380807913 shutdown_connections 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 -- 172.21.15.103:0/380807913 wait complete. 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 Processor -- start 2024-09-19T07:42:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 -- start start 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2178100a80 0x7f21781984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178101420 0x7f2178198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.985+0000 7f217ffb5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2178106a80 0x7f217819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217ffb5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2178110ea0 con 0x7f2178101420 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217ffb5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2178110d20 con 0x7f2178100a80 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217ffb5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2178111020 con 0x7f2178106a80 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178101420 0x7f2178198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178101420 0x7f2178198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39266/0 (socket says 172.21.15.103:39266) 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 -- 172.21.15.103:0/88428783 learned_addr learned my addr 172.21.15.103:0/88428783 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217dd51640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2178100a80 0x7f21781984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217e552640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2178106a80 0x7f217819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 -- 172.21.15.103:0/88428783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2178106a80 msgr2=0x7f217819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2178106a80 0x7f217819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 -- 172.21.15.103:0/88428783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2178100a80 msgr2=0x7f21781984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2178100a80 0x7f21781984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 -- 172.21.15.103:0/88428783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217819d480 con 0x7f2178101420 2024-09-19T07:42:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217e552640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2178106a80 0x7f217819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217d550640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178101420 0x7f2178198a10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2168009230 tx=0x7f2168004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f216effd640 1 -- 172.21.15.103:0/88428783 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216803f030 con 0x7f2178101420 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f216effd640 1 -- 172.21.15.103:0/88428783 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2168003ad0 con 0x7f2178101420 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f216effd640 1 -- 172.21.15.103:0/88428783 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168035070 con 0x7f2178101420 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217819d710 con 0x7f2178101420 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.986+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21781a5050 con 0x7f2178101420 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f216effd640 1 -- 172.21.15.103:0/88428783 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2168042030 con 0x7f2178101420 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 Processor -- start 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 -- start start 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe9b40 0x561b04fe8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04fe8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b04f4e2f0 con 0x561b04fe9b40 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561b04ef5cf0 con 0x561b04fe85d0 2024-09-19T07:42:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f218238e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b04f4a2b0 con 0x561b04ff3370 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f217e552640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe9b40 0x561b04fe8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f217e552640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe9b40 0x561b04fe8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39270/0 (socket says 172.21.15.103:39270) 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f217e552640 1 -- 172.21.15.103:0/1043542084 learned_addr learned my addr 172.21.15.103:0/1043542084 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f217dd51640 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04fe8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.989+0000 7f216effd640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f21640a33b0 0x7f21640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217d550640 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217e552640 1 -- 172.21.15.103:0/1043542084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 msgr2=0x561b04ff5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217e552640 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217e552640 1 -- 172.21.15.103:0/1043542084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 msgr2=0x561b04fe8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217e552640 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04fe8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217e552640 1 -- 172.21.15.103:0/1043542084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b04eda3a0 con 0x561b04fe9b40 2024-09-19T07:42:59.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217d550640 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:59.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217dd51640 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04fe8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:59.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217e552640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f21640a33b0 0x7f21640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f216effd640 1 -- 172.21.15.103:0/88428783 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f21680f0200 con 0x7f2178101420 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f217e552640 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe9b40 0x561b04fe8090 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2170009d50 tx=0x7f217000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f7de035c43b6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f2163fff640 1 -- 172.21.15.103:0/1043542084 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170020070 con 0x561b04fe9b40 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f2163fff640 1 -- 172.21.15.103:0/1043542084 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f217001ac10 con 0x561b04fe9b40 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.990+0000 7f2163fff640 1 -- 172.21.15.103:0/1043542084 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170019390 con 0x561b04fe9b40 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 -- 172.21.15.103:0/1043542084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe9b40 msgr2=0x561b04fe8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f217e552640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f21640a33b0 0x7f21640a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f217001d000 tx=0x7f2170019ac0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe9b40 0x561b04fe8090 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2170009d50 tx=0x7f217000e6c0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 -- 172.21.15.103:0/1043542084 shutdown_connections 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04fe8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 --2- 172.21.15.103:0/1043542084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe9b40 0x561b04fe8090 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 -- 172.21.15.103:0/1043542084 >> 172.21.15.103:0/1043542084 conn(0x561b04fb1310 msgr2=0x561b04fe4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 -- 172.21.15.103:0/1043542084 shutdown_connections 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 -- 172.21.15.103:0/1043542084 wait complete. 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 -- start start 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.991+0000 7f218238e740 1 Processor -- start 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe85d0 0x561b04ff2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b04ff6070 con 0x561b04fe85d0 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561b04f09060 con 0x561b04fe9b40 2024-09-19T07:42:59.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b04ff61f0 con 0x561b04ff3370 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 10 client.0 _ll_register_callbacks cb 0x561b04c5c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe85d0 0x561b04ff2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe85d0 0x561b04ff2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39286/0 (socket says 172.21.15.103:39286) 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 -- 172.21.15.103:0/2204224285 learned_addr learned my addr 172.21.15.103:0/2204224285 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217e552640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 -- 172.21.15.103:0/2204224285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 msgr2=0x561b04ff04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 -- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 msgr2=0x561b04feff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217d550640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 -- 172.21.15.103:0/2204224285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b04cbf3c0 con 0x561b04fe85d0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217e552640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f217dd51640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe85d0 0x561b04ff2cb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f217400ab90 tx=0x7f2174007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda37380555dea8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f21617fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f21617fa640 1 -- 172.21.15.103:0/2204224285 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174016610 con 0x561b04fe85d0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f21617fa640 1 -- 172.21.15.103:0/2204224285 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f21740167b0 con 0x561b04fe85d0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.992+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f217001ac10 con 0x561b04fe85d0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f21617fa640 1 -- 172.21.15.103:0/2204224285 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f21740169b0 con 0x561b04fe85d0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 10 client.16527 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561b05001a80 con 0x561b04fe85d0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f21617fa640 1 -- 172.21.15.103:0/2204224285 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f217001ac10 con 0x561b04fe85d0 2024-09-19T07:42:59.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f21617fa640 1 -- 172.21.15.103:0/2204224285 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f217401fb90 con 0x561b04fe85d0 2024-09-19T07:42:59.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 10 client.16527 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:59.026 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:42:59.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 10 client.16527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:42:59.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 20 client.16527 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:59.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b05002f40 0x561b050053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 4 client.16527 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b04f5b860 con 0x561b05002f40 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.993+0000 7f217d550640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b05002f40 0x561b050053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.994+0000 7f21617fa640 10 client.16527 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.994+0000 7f217d550640 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b05002f40 0x561b050053e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c82cb7803ed6e8d1 server_cookie=10675f19238b9ab4 in_seq=0 out_seq=0 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.994+0000 7f21617fa640 1 -- 172.21.15.103:0/2204224285 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b04f5b860 con 0x561b05002f40 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.994+0000 7f21617fa640 10 client.16527 handle_command_reply: tid=0 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 client.16527 shutdown 2024-09-19T07:42:59.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe85d0 msgr2=0x561b04ff2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe85d0 0x561b04ff2cb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f217400ab90 tx=0x7f2174007680 comp rx=0 tx=0).stop 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 shutdown_connections 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b05002f40 0x561b050053e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04ff3370 0x561b04ff04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f21617fa640 0 client.16527 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 --2- 172.21.15.103:0/2204224285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04fe85d0 0x561b04ff2cb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 >> 172.21.15.103:0/2204224285 conn(0x561b04fb1310 msgr2=0x561b04fea930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 shutdown_connections 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 1 -- 172.21.15.103:0/2204224285 wait complete. 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:58.995+0000 7f218238e740 20 client.16527 trim_cache size 0 max 16384 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 Processor -- start 2024-09-19T07:42:59.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 -- start start 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe85d0 0x561b04fff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b04feb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21781ad810 con 0x561b04ff3370 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21781af3b0 con 0x561b04fe9b40 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f218238e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21781adee0 con 0x561b04fe85d0 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b04feb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b04feb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39300/0 (socket says 172.21.15.103:39300) 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 -- 172.21.15.103:0/1173237112 learned_addr learned my addr 172.21.15.103:0/1173237112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217d550640 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217dd51640 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe85d0 0x561b04fff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 -- 172.21.15.103:0/1173237112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe85d0 msgr2=0x561b04fff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe85d0 0x561b04fff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 -- 172.21.15.103:0/1173237112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 msgr2=0x561b04feb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217e552640 1 -- 172.21.15.103:0/1173237112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b04febba0 con 0x561b04ff3370 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.008+0000 7f217dd51640 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe85d0 0x561b04fff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f217e552640 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b04feb660 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2170009f60 tx=0x7f2170021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23eff1022bea4d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f21637fe640 1 -- 172.21.15.103:0/1173237112 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217002a040 con 0x561b04ff3370 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f21637fe640 1 -- 172.21.15.103:0/1173237112 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2170033070 con 0x561b04ff3370 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f21637fe640 1 -- 172.21.15.103:0/1173237112 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170020040 con 0x561b04ff3370 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 -- 172.21.15.103:0/1173237112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 msgr2=0x561b04feb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b04feb660 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2170009f60 tx=0x7f2170021000 comp rx=0 tx=0).stop 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 -- 172.21.15.103:0/1173237112 shutdown_connections 2024-09-19T07:42:59.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b04feb660 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe9b40 0x561b04feb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 --2- 172.21.15.103:0/1173237112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe85d0 0x561b04fff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 -- 172.21.15.103:0/1173237112 >> 172.21.15.103:0/1173237112 conn(0x561b04fb1310 msgr2=0x561b04fee830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 -- 172.21.15.103:0/1173237112 shutdown_connections 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.009+0000 7f218238e740 1 -- 172.21.15.103:0/1173237112 wait complete. 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 -- start start 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 Processor -- start 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04ff69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe9b40 0x561b04ff6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b0506bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21781aece0 con 0x561b04ff3370 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21781ae5b0 con 0x561b04fe85d0 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21781ab7f0 con 0x561b04fe9b40 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f218238e740 10 client.0 _ll_register_callbacks cb 0x7f2174009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f217e552640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b0506bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f217e552640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b0506bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39312/0 (socket says 172.21.15.103:39312) 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f217dd51640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04ff69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.010+0000 7f217e552640 1 -- 172.21.15.103:0/1052907701 learned_addr learned my addr 172.21.15.103:0/1052907701 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217d550640 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe9b40 0x561b04ff6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217e552640 1 -- 172.21.15.103:0/1052907701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe9b40 msgr2=0x561b04ff6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217e552640 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe9b40 0x561b04ff6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217e552640 1 -- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 msgr2=0x561b04ff69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217e552640 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04ff69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217e552640 1 -- 172.21.15.103:0/1052907701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0506c300 con 0x561b04ff3370 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217dd51640 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04ff69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f217e552640 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b0506bcc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2170013fd0 tx=0x7f2170000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a1cf5125df23cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f21617fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f21617fa640 1 -- 172.21.15.103:0/1052907701 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217003d020 con 0x561b04ff3370 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f21617fa640 1 -- 172.21.15.103:0/1052907701 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f21700123f0 con 0x561b04ff3370 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2170033070 con 0x561b04ff3370 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f21617fa640 1 -- 172.21.15.103:0/1052907701 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2170048050 con 0x561b04ff3370 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f218238e740 10 client.16533 fetch_fsmap learned FSMap version 36 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561b0506d5e0 con 0x561b04ff3370 2024-09-19T07:42:59.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.011+0000 7f21617fa640 1 -- 172.21.15.103:0/1052907701 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2170033070 con 0x561b04ff3370 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f21617fa640 1 -- 172.21.15.103:0/1052907701 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f217002d030 con 0x561b04ff3370 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f218238e740 10 client.16533 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f218238e740 10 client.16533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f218238e740 20 client.16533 populate_metadata read hostname 'smithi103' 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f218238e740 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b0506db50 0x561b0506ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f218238e740 4 client.16533 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b04f55d70 con 0x561b0506db50 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f217dd51640 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b0506db50 0x561b0506ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f21617fa640 10 client.16533 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f217dd51640 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b0506db50 0x561b0506ff40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=836d5f887582c950 server_cookie=7c184f0bcf7bc501 in_seq=0 out_seq=0 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f21617fa640 1 -- 172.21.15.103:0/1052907701 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b04f55d70 con 0x561b0506db50 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f21617fa640 10 client.16533 handle_command_reply: tid=0 2024-09-19T07:42:59.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.012+0000 7f218238e740 1 client.16533 shutdown 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 msgr2=0x561b0506bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b0506bcc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2170013fd0 tx=0x7f2170000bc0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 shutdown_connections 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x561b0506db50 0x561b0506ff40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b04ff3370 0x561b0506bcc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b04fe9b40 0x561b04ff6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f21617fa640 0 client.16533 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 --2- 172.21.15.103:0/1052907701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b04fe85d0 0x561b04ff69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 >> 172.21.15.103:0/1052907701 conn(0x561b04fb1310 msgr2=0x561b04ff17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 shutdown_connections 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 1 -- 172.21.15.103:0/1052907701 wait complete. 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f218238e740 20 client.16533 trim_cache size 0 max 16384 2024-09-19T07:42:59.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f21640a33b0 msgr2=0x7f21640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f217ffb5640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f21640a33b0 0x7f21640a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f217001d000 tx=0x7f2170019ac0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178101420 msgr2=0x7f2178198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f217ffb5640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178101420 0x7f2178198a10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2168009230 tx=0x7f2168004270 comp rx=0 tx=0).stop 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 shutdown_connections 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.013+0000 7f217ffb5640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f21640a33b0 0x7f21640a5870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.014+0000 7f217ffb5640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2178106a80 0x7f217819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.014+0000 7f217ffb5640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2178101420 0x7f2178198a10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.014+0000 7f217ffb5640 1 --2- 172.21.15.103:0/88428783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2178100a80 0x7f21781984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.014+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 >> 172.21.15.103:0/88428783 conn(0x7f21780fc7a0 msgr2=0x7f217806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.014+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 shutdown_connections 2024-09-19T07:42:59.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:42:59.014+0000 7f217ffb5640 1 -- 172.21.15.103:0/88428783 wait complete. 2024-09-19T07:42:59.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:42:59.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:42:59 smithi174 ceph-mon[35370]: pgmap v153: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-09-19T07:42:59.935 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:42:59.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:42:59 smithi103 ceph-mon[31201]: pgmap v153: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-09-19T07:43:00.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:42:59 smithi134 ceph-mon[35414]: pgmap v153: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-09-19T07:43:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 -- 172.21.15.103:0/3880570007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880106f10 msgr2=0x7f988010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 --2- 172.21.15.103:0/3880570007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880106f10 0x7f988010ea90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f987802a410 comp rx=0 tx=0).stop 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 -- 172.21.15.103:0/3880570007 shutdown_connections 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 --2- 172.21.15.103:0/3880570007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880106f10 0x7f988010ea90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 --2- 172.21.15.103:0/3880570007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880102790 0x7f98801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 --2- 172.21.15.103:0/3880570007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9880101de0 0x7f98801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 -- 172.21.15.103:0/3880570007 >> 172.21.15.103:0/3880570007 conn(0x7f98800fd8c0 msgr2=0x7f98800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 -- 172.21.15.103:0/3880570007 shutdown_connections 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 -- 172.21.15.103:0/3880570007 wait complete. 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 Processor -- start 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.396+0000 7f9885d59640 1 -- start start 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f9885d59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880101de0 0x7f98800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f9885d59640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9880102790 0x7f9880074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f9885d59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880106f10 0x7f9880074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f9885d59640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9880111f70 con 0x7f9880101de0 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f9885d59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9880111df0 con 0x7f9880106f10 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f9885d59640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98801120f0 con 0x7f9880102790 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880101de0 0x7f98800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880101de0 0x7f98800744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39332/0 (socket says 172.21.15.103:39332) 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 -- 172.21.15.103:0/3469497058 learned_addr learned my addr 172.21.15.103:0/3469497058 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987effd640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9880102790 0x7f9880074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987ffff640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880106f10 0x7f9880074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 -- 172.21.15.103:0/3469497058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9880102790 msgr2=0x7f9880074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9880102790 0x7f9880074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 -- 172.21.15.103:0/3469497058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880106f10 msgr2=0x7f9880074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880106f10 0x7f9880074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 -- 172.21.15.103:0/3469497058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98801a68b0 con 0x7f9880101de0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987f7fe640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880101de0 0x7f98800744f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f987000ea80 tx=0x7f987000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987cff9640 1 -- 172.21.15.103:0/3469497058 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987000ce40 con 0x7f9880101de0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98801a6ba0 con 0x7f9880101de0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987cff9640 1 -- 172.21.15.103:0/3469497058 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9870005020 con 0x7f9880101de0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.397+0000 7f987cff9640 1 -- 172.21.15.103:0/3469497058 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870011760 con 0x7f9880101de0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.398+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98801a70e0 con 0x7f9880101de0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.400+0000 7f9888132740 1 Processor -- start 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.400+0000 7f987cff9640 1 -- 172.21.15.103:0/3469497058 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f98700051d0 con 0x7f9880101de0 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.400+0000 7f9888132740 1 -- start start 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f9888132740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f9888132740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f9888132740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f9888132740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556f4fb7a2f0 con 0x556f4fc1f370 2024-09-19T07:43:00.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f9888132740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f4fb21cf0 con 0x556f4fc145d0 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f9888132740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f4fb762b0 con 0x556f4fc15b40 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc21760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39348/0 (socket says 172.21.15.103:39348) 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 -- 172.21.15.103:0/2354292761 learned_addr learned my addr 172.21.15.103:0/2354292761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987f7fe640 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc14a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987ffff640 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 -- 172.21.15.103:0/2354292761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 msgr2=0x556f4fc14090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc14090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 -- 172.21.15.103:0/2354292761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 msgr2=0x556f4fc14a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc14a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 -- 172.21.15.103:0/2354292761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4fb063a0 con 0x556f4fc1f370 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987f7fe640 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:00.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987ffff640 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc14090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987cff9640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f98680a33b0 0x7f98680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987effd640 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc21760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9874009c10 tx=0x7f9874011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a144e850e4b3d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987ffff640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f98680a33b0 0x7f98680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.401+0000 7f987cff9640 1 -- 172.21.15.103:0/3469497058 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f98700d0280 con 0x7f9880101de0 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9865ffb640 1 -- 172.21.15.103:0/2354292761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874004230 con 0x556f4fc1f370 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9865ffb640 1 -- 172.21.15.103:0/2354292761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98740043d0 con 0x556f4fc1f370 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9865ffb640 1 -- 172.21.15.103:0/2354292761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874004e00 con 0x556f4fc1f370 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 -- 172.21.15.103:0/2354292761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 msgr2=0x556f4fc21760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f987ffff640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f98680a33b0 0x7f98680a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9878006010 tx=0x7f9878009560 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc21760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9874009c10 tx=0x7f9874011810 comp rx=0 tx=0).stop 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 -- 172.21.15.103:0/2354292761 shutdown_connections 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc21760 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 --2- 172.21.15.103:0/2354292761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc14090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 -- 172.21.15.103:0/2354292761 >> 172.21.15.103:0/2354292761 conn(0x556f4fbdd310 msgr2=0x556f4fc10c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 -- 172.21.15.103:0/2354292761 shutdown_connections 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.402+0000 7f9888132740 1 -- 172.21.15.103:0/2354292761 wait complete. 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 -- start start 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 Processor -- start 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc145d0 0x556f4fc1eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc15b40 0x556f4fc1bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc1f370 0x556f4fc1c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556f4fc22070 con 0x556f4fc15b40 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f4fb35060 con 0x556f4fc1f370 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f4fc221f0 con 0x556f4fc145d0 2024-09-19T07:43:00.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f9888132740 10 client.0 _ll_register_callbacks cb 0x556f4f8888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f987effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc15b40 0x556f4fc1bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f987effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc15b40 0x556f4fc1bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39352/0 (socket says 172.21.15.103:39352) 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.403+0000 7f987effd640 1 -- 172.21.15.103:0/1992010350 learned_addr learned my addr 172.21.15.103:0/1992010350 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987f7fe640 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc145d0 0x556f4fc1eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987ffff640 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc1f370 0x556f4fc1c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987effd640 1 -- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc145d0 msgr2=0x556f4fc1eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987effd640 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc145d0 0x556f4fc1eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987effd640 1 -- 172.21.15.103:0/1992010350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc1f370 msgr2=0x556f4fc1c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987effd640 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc1f370 0x556f4fc1c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987effd640 1 -- 172.21.15.103:0/1992010350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4f8eb3c0 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f987effd640 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc15b40 0x556f4fc1bf50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9874000c00 tx=0x7f9874019c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4502189008dab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f985effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f985effd640 1 -- 172.21.15.103:0/1992010350 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874028020 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f985effd640 1 -- 172.21.15.103:0/1992010350 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9874024620 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98740043d0 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f985effd640 1 -- 172.21.15.103:0/1992010350 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9874033050 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f9888132740 10 client.16545 fetch_fsmap learned FSMap version 36 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556f4fc2da90 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.404+0000 7f985effd640 1 -- 172.21.15.103:0/1992010350 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98740043d0 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f985effd640 1 -- 172.21.15.103:0/1992010350 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f9874018070 con 0x556f4fc15b40 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f9888132740 10 client.16545 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:43:00.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f9888132740 10 client.16545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f9888132740 20 client.16545 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f9888132740 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc2ede0 0x556f4fc311d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f9888132740 4 client.16545 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f4fb87860 con 0x556f4fc2ede0 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f987f7fe640 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc2ede0 0x556f4fc311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f987f7fe640 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc2ede0 0x556f4fc311d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90a726d9d93528f0 server_cookie=6e7b2e387843ff67 in_seq=0 out_seq=0 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.405+0000 7f985effd640 10 client.16545 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f985effd640 1 -- 172.21.15.103:0/1992010350 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f4fb87860 con 0x556f4fc2ede0 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f985effd640 10 client.16545 handle_command_reply: tid=0 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 client.16545 shutdown 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc15b40 msgr2=0x556f4fc1bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc15b40 0x556f4fc1bf50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9874000c00 tx=0x7f9874019c30 comp rx=0 tx=0).stop 2024-09-19T07:43:00.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 shutdown_connections 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc2ede0 0x556f4fc311d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc1f370 0x556f4fc1c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc15b40 0x556f4fc1bf50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.406+0000 7f9888132740 1 --2- 172.21.15.103:0/1992010350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc145d0 0x556f4fc1eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.407+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 >> 172.21.15.103:0/1992010350 conn(0x556f4fbdd310 msgr2=0x556f4fc16960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.407+0000 7f985effd640 0 client.16545 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.407+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 shutdown_connections 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.407+0000 7f9888132740 1 -- 172.21.15.103:0/1992010350 wait complete. 2024-09-19T07:43:00.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.407+0000 7f9888132740 20 client.16545 trim_cache size 0 max 16384 2024-09-19T07:43:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 Processor -- start 2024-09-19T07:43:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 -- start start 2024-09-19T07:43:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc2b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc20ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98801afa50 con 0x556f4fc1f370 2024-09-19T07:43:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98801b15f0 con 0x556f4fc145d0 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f9888132740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98801b0120 con 0x556f4fc15b40 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.420+0000 7f987ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc20ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc20ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39358/0 (socket says 172.21.15.103:39358) 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 -- 172.21.15.103:0/3852497564 learned_addr learned my addr 172.21.15.103:0/3852497564 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987effd640 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987f7fe640 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc2b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 -- 172.21.15.103:0/3852497564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 msgr2=0x556f4fc20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 -- 172.21.15.103:0/3852497564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 msgr2=0x556f4fc2b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc2b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 -- 172.21.15.103:0/3852497564 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4fc11960 con 0x556f4fc1f370 2024-09-19T07:43:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f987ffff640 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc20ca0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f987803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ce0223eea667fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f98657fa640 1 -- 172.21.15.103:0/3852497564 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878035040 con 0x556f4fc1f370 2024-09-19T07:43:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f98657fa640 1 -- 172.21.15.103:0/3852497564 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9878004400 con 0x556f4fc1f370 2024-09-19T07:43:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.421+0000 7f98657fa640 1 -- 172.21.15.103:0/3852497564 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987803ca70 con 0x556f4fc1f370 2024-09-19T07:43:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 -- 172.21.15.103:0/3852497564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 msgr2=0x556f4fc20ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc20ca0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f987803e000 comp rx=0 tx=0).stop 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 -- 172.21.15.103:0/3852497564 shutdown_connections 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc20ca0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 --2- 172.21.15.103:0/3852497564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc2b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 -- 172.21.15.103:0/3852497564 >> 172.21.15.103:0/3852497564 conn(0x556f4fbdd310 msgr2=0x556f4fc1a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 -- 172.21.15.103:0/3852497564 shutdown_connections 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 -- 172.21.15.103:0/3852497564 wait complete. 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.422+0000 7f9888132740 1 -- start start 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 1 Processor -- start 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc27930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc19710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98801b0f20 con 0x556f4fc1f370 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98801b07f0 con 0x556f4fc145d0 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98801ada30 con 0x556f4fc15b40 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f9888132740 10 client.0 _ll_register_callbacks cb 0x7f9874009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc19710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc19710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39372/0 (socket says 172.21.15.103:39372) 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 -- 172.21.15.103:0/170465866 learned_addr learned my addr 172.21.15.103:0/170465866 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987f7fe640 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc27930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 -- 172.21.15.103:0/170465866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 msgr2=0x556f4fc191d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 -- 172.21.15.103:0/170465866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 msgr2=0x556f4fc27930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:00.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc27930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 -- 172.21.15.103:0/170465866 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4fc19c80 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f987ffff640 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc19710 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9878031ad0 tx=0x7f9878003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09a053e7d3a248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.423+0000 7f985effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f985effd640 1 -- 172.21.15.103:0/170465866 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878061060 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f985effd640 1 -- 172.21.15.103:0/170465866 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9878004200 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9878004400 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f985effd640 1 -- 172.21.15.103:0/170465866 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9878003c20 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 10 client.16551 fetch_fsmap learned FSMap version 36 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556f4fc36e30 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f985effd640 1 -- 172.21.15.103:0/170465866 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9878004400 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f985effd640 1 -- 172.21.15.103:0/170465866 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f987805c7a0 con 0x556f4fc1f370 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 10 client.16551 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:43:00.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 10 client.16551 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 20 client.16551 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc99e60 0x556f4fc9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 4 client.16551 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.424+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f4fb81d70 con 0x556f4fc99e60 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f987f7fe640 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc99e60 0x556f4fc9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f985effd640 10 client.16551 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f987f7fe640 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc99e60 0x556f4fc9c280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e0867af1b7e3eb2 server_cookie=6921bff9a8ffdeb4 in_seq=0 out_seq=0 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f985effd640 1 -- 172.21.15.103:0/170465866 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f4fb81d70 con 0x556f4fc99e60 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f985effd640 10 client.16551 handle_command_reply: tid=0 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 client.16551 shutdown 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 msgr2=0x556f4fc19710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc19710 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9878031ad0 tx=0x7f9878003820 comp rx=0 tx=0).stop 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 shutdown_connections 2024-09-19T07:43:00.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x556f4fc99e60 0x556f4fc9c280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f4fc1f370 0x556f4fc19710 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f4fc15b40 0x556f4fc191d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 --2- 172.21.15.103:0/170465866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f4fc145d0 0x556f4fc27930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 >> 172.21.15.103:0/170465866 conn(0x556f4fbdd310 msgr2=0x556f4fc1d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f985effd640 0 client.16551 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 shutdown_connections 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 1 -- 172.21.15.103:0/170465866 wait complete. 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.425+0000 7f9888132740 20 client.16551 trim_cache size 0 max 16384 2024-09-19T07:43:00.432 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:43:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f98680a33b0 msgr2=0x7f98680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f98680a33b0 0x7f98680a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9878006010 tx=0x7f9878009560 comp rx=0 tx=0).stop 2024-09-19T07:43:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880101de0 msgr2=0x7f98800744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880101de0 0x7f98800744f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f987000ea80 tx=0x7f987000ef40 comp rx=0 tx=0).stop 2024-09-19T07:43:00.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 shutdown_connections 2024-09-19T07:43:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f98680a33b0 0x7f98680a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880106f10 0x7f9880074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9880102790 0x7f9880074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 --2- 172.21.15.103:0/3469497058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9880101de0 0x7f98800744f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 >> 172.21.15.103:0/3469497058 conn(0x7f98800fd8c0 msgr2=0x7f98800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 shutdown_connections 2024-09-19T07:43:00.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:00.426+0000 7f9885d59640 1 -- 172.21.15.103:0/3469497058 wait complete. 2024-09-19T07:43:00.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 265.0 s, 367.1 tps, lat 38.148 ms stddev 43.367 2024-09-19T07:43:01.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:43:01.562 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:01.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:01 smithi174 ceph-mon[35370]: pgmap v154: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2024-09-19T07:43:01.928 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:01 smithi103 ceph-mon[31201]: pgmap v154: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2024-09-19T07:43:02.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/1608990830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c106f00 msgr2=0x7fd69c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/1608990830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c106f00 0x7fd69c10ea80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd694009ca0 tx=0x7fd69402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/1608990830 shutdown_connections 2024-09-19T07:43:02.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/1608990830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c106f00 0x7fd69c10ea80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/1608990830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd69c102780 0x7fd69c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/1608990830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd69c101dd0 0x7fd69c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/1608990830 >> 172.21.15.103:0/1608990830 conn(0x7fd69c0fd8b0 msgr2=0x7fd69c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/1608990830 shutdown_connections 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.008+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/1608990830 wait complete. 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 Processor -- start 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 -- start start 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd69c101dd0 0x7fd69c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c102780 0x7fd69c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd69c106f00 0x7fd69c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c111f80 con 0x7fd69c102780 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c111e00 con 0x7fd69c106f00 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a3fc2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c112100 con 0x7fd69c101dd0 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a155d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c102780 0x7fd69c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a155d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c102780 0x7fd69c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39396/0 (socket says 172.21.15.103:39396) 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.009+0000 7fd6a155d640 1 -- 172.21.15.103:0/379941623 learned_addr learned my addr 172.21.15.103:0/379941623 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a255f640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd69c106f00 0x7fd69c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd69c101dd0 0x7fd69c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a155d640 1 -- 172.21.15.103:0/379941623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd69c101dd0 msgr2=0x7fd69c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a155d640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd69c101dd0 0x7fd69c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a155d640 1 -- 172.21.15.103:0/379941623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd69c106f00 msgr2=0x7fd69c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a155d640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd69c106f00 0x7fd69c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a155d640 1 -- 172.21.15.103:0/379941623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c19d450 con 0x7fd69c102780 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a155d640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c102780 0x7fd69c1989e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c920 tx=0x7fd69800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd692ffd640 1 -- 172.21.15.103:0/379941623 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698007c70 con 0x7fd69c102780 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69c19d740 con 0x7fd69c102780 2024-09-19T07:43:02.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd692ffd640 1 -- 172.21.15.103:0/379941623 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd698007e10 con 0x7fd69c102780 2024-09-19T07:43:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd692ffd640 1 -- 172.21.15.103:0/379941623 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698018890 con 0x7fd69c102780 2024-09-19T07:43:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.010+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69c1a5020 con 0x7fd69c102780 2024-09-19T07:43:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.012+0000 7fd692ffd640 1 -- 172.21.15.103:0/379941623 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd698018a70 con 0x7fd69c102780 2024-09-19T07:43:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 Processor -- start 2024-09-19T07:43:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 -- start start 2024-09-19T07:43:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda833e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833e5d0 0x55fda833ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8349370 0x55fda834b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fda82a42f0 con 0x55fda833fb40 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda824bcf0 con 0x55fda8349370 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a639b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fda82a02b0 con 0x55fda833e5d0 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd692ffd640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6880a33b0 0x7fd6880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a255f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda833e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a1d5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833e5d0 0x55fda833ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a155d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8349370 0x55fda834b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a255f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda833e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39408/0 (socket says 172.21.15.103:39408) 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a255f640 1 -- 172.21.15.103:0/3879835116 learned_addr learned my addr 172.21.15.103:0/3879835116 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.013+0000 7fd6a255f640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6880a33b0 0x7fd6880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd692ffd640 1 -- 172.21.15.103:0/379941623 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd6980cff10 con 0x7fd69c102780 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a1d5e640 1 -- 172.21.15.103:0/3879835116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8349370 msgr2=0x55fda834b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8349370 0x55fda834b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a1d5e640 1 -- 172.21.15.103:0/3879835116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 msgr2=0x55fda833e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda833e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a1d5e640 1 -- 172.21.15.103:0/3879835116 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda82303a0 con 0x55fda833e5d0 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a255f640 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda833e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833e5d0 0x55fda833ea10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00b760 tx=0x7fd68c00bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8228bb8a77f7cab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd6a255f640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6880a33b0 0x7fd6880a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd694031f80 tx=0x7fd69402a8c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd687fff640 1 -- 172.21.15.103:0/3879835116 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00d950 con 0x55fda833e5d0 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd687fff640 1 -- 172.21.15.103:0/3879835116 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68c019070 con 0x55fda833e5d0 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.014+0000 7fd687fff640 1 -- 172.21.15.103:0/3879835116 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c014e00 con 0x55fda833e5d0 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 -- 172.21.15.103:0/3879835116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833e5d0 msgr2=0x55fda833ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833e5d0 0x55fda833ea10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00b760 tx=0x7fd68c00bc20 comp rx=0 tx=0).stop 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 -- 172.21.15.103:0/3879835116 shutdown_connections 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8349370 0x55fda834b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833e5d0 0x55fda833ea10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 --2- 172.21.15.103:0/3879835116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda833e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 -- 172.21.15.103:0/3879835116 >> 172.21.15.103:0/3879835116 conn(0x55fda8307310 msgr2=0x55fda833ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 -- 172.21.15.103:0/3879835116 shutdown_connections 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 -- 172.21.15.103:0/3879835116 wait complete. 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 -- start start 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.015+0000 7fd6a639b740 1 Processor -- start 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8348ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda8345f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda8349370 0x55fda83464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a639b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fda834c070 con 0x55fda833fb40 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a639b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda825f060 con 0x55fda833e5d0 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a639b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fda834c1f0 con 0x55fda8349370 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a639b740 10 client.0 _ll_register_callbacks cb 0x55fda7fb28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda8345f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda8345f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39420/0 (socket says 172.21.15.103:39420) 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 -- 172.21.15.103:0/1287517088 learned_addr learned my addr 172.21.15.103:0/1287517088 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8348ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a255f640 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda8349370 0x55fda83464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 -- 172.21.15.103:0/1287517088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda8349370 msgr2=0x55fda83464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda8349370 0x55fda83464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 -- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 msgr2=0x55fda8348ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8348ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 -- 172.21.15.103:0/1287517088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda80153c0 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6a155d640 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda8345f90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd698098660 tx=0x7fd69801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a2e05251e017a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6857fa640 1 -- 172.21.15.103:0/1287517088 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69800de20 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.016+0000 7fd6857fa640 1 -- 172.21.15.103:0/1287517088 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd698098870 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68c019070 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6857fa640 1 -- 172.21.15.103:0/1287517088 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd69809c200 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 10 client.16563 fetch_fsmap learned FSMap version 36 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fda8357ae0 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6857fa640 1 -- 172.21.15.103:0/1287517088 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68c019070 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6857fa640 1 -- 172.21.15.103:0/1287517088 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd69809c460 con 0x55fda833fb40 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 10 client.16563 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 20 client.16563 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda8358fa0 0x55fda835b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 4 client.16563 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fda82b1860 con 0x55fda8358fa0 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.017+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda8358fa0 0x55fda835b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.018+0000 7fd6857fa640 10 client.16563 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.018+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda8358fa0 0x55fda835b3d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c41a08f00432a8 server_cookie=7d58e7c0ba4ed88 in_seq=0 out_seq=0 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6857fa640 1 -- 172.21.15.103:0/1287517088 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fda82b1860 con 0x55fda8358fa0 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6857fa640 10 client.16563 handle_command_reply: tid=0 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 client.16563 shutdown 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 msgr2=0x55fda8345f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda8345f90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd698098660 tx=0x7fd69801c000 comp rx=0 tx=0).stop 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 shutdown_connections 2024-09-19T07:43:02.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda8358fa0 0x55fda835b3d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda8349370 0x55fda83464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6857fa640 0 client.16563 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda833fb40 0x55fda8345f90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1287517088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8348ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 >> 172.21.15.103:0/1287517088 conn(0x55fda8307310 msgr2=0x55fda8340960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 shutdown_connections 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 1 -- 172.21.15.103:0/1287517088 wait complete. 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.019+0000 7fd6a639b740 20 client.16563 trim_cache size 0 max 16384 2024-09-19T07:43:02.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 Processor -- start 2024-09-19T07:43:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 -- start start 2024-09-19T07:43:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8355310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda8341120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8341660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1ad7d0 con 0x55fda8349370 2024-09-19T07:43:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1af370 con 0x55fda833e5d0 2024-09-19T07:43:02.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a639b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1adea0 con 0x55fda833fb40 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a255f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8341660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a255f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8341660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39432/0 (socket says 172.21.15.103:39432) 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a255f640 1 -- 172.21.15.103:0/1189271567 learned_addr learned my addr 172.21.15.103:0/1189271567 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8355310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a255f640 1 -- 172.21.15.103:0/1189271567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 msgr2=0x55fda8341120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a155d640 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda8341120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.032+0000 7fd6a255f640 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda8341120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6a255f640 1 -- 172.21.15.103:0/1189271567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 msgr2=0x55fda8355310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6a255f640 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8355310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6a255f640 1 -- 172.21.15.103:0/1189271567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda833bdd0 con 0x55fda8349370 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6a255f640 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8341660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd694009ca0 tx=0x7fd694009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebf7cff2051a405 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6877fe640 1 -- 172.21.15.103:0/1189271567 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694050070 con 0x55fda8349370 2024-09-19T07:43:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6877fe640 1 -- 172.21.15.103:0/1189271567 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd694036040 con 0x55fda8349370 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6877fe640 1 -- 172.21.15.103:0/1189271567 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69404c770 con 0x55fda8349370 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6a639b740 1 -- 172.21.15.103:0/1189271567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 msgr2=0x55fda8341660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.033+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8341660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd694009ca0 tx=0x7fd694009360 comp rx=0 tx=0).stop 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 -- 172.21.15.103:0/1189271567 shutdown_connections 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8341660 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda8341120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1189271567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8355310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 -- 172.21.15.103:0/1189271567 >> 172.21.15.103:0/1189271567 conn(0x55fda8307310 msgr2=0x55fda8345080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 -- 172.21.15.103:0/1189271567 shutdown_connections 2024-09-19T07:43:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 -- 172.21.15.103:0/1189271567 wait complete. 2024-09-19T07:43:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 -- start start 2024-09-19T07:43:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 Processor -- start 2024-09-19T07:43:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8342fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda83434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8343a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1aeca0 con 0x55fda8349370 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.034+0000 7fd6a639b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1ae570 con 0x55fda833e5d0 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a639b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1ab7b0 con 0x55fda833fb40 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a639b740 10 client.0 _ll_register_callbacks cb 0x7fd69800a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8343a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8343a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39434/0 (socket says 172.21.15.103:39434) 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 -- 172.21.15.103:0/1680192140 learned_addr learned my addr 172.21.15.103:0/1680192140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:02.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a155d640 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda83434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8342fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 -- 172.21.15.103:0/1680192140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 msgr2=0x55fda83434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda83434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 -- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 msgr2=0x55fda8342fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8342fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 -- 172.21.15.103:0/1680192140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda83c2c20 con 0x55fda8349370 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a255f640 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8343a30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd6940307f0 tx=0x7fd694030820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcc54ef408a91c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6857fa640 1 -- 172.21.15.103:0/1680192140 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694050040 con 0x55fda8349370 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6857fa640 1 -- 172.21.15.103:0/1680192140 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd694030a40 con 0x55fda8349370 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.035+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd694036040 con 0x55fda8349370 2024-09-19T07:43:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6857fa640 1 -- 172.21.15.103:0/1680192140 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd694030c40 con 0x55fda8349370 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 10 client.16569 fetch_fsmap learned FSMap version 36 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fda83c3f00 con 0x55fda8349370 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6857fa640 1 -- 172.21.15.103:0/1680192140 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd694036040 con 0x55fda8349370 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6857fa640 1 -- 172.21.15.103:0/1680192140 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd694049050 con 0x55fda8349370 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 10 client.16569 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 20 client.16569 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda83c4610 0x55fda83c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 4 client.16569 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fda82abd70 con 0x55fda83c4610 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.036+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda83c4610 0x55fda83c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a1d5e640 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda83c4610 0x55fda83c6a30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3884857a269d022f server_cookie=f0ac564441f9c64f in_seq=0 out_seq=0 2024-09-19T07:43:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6857fa640 10 client.16569 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6857fa640 1 -- 172.21.15.103:0/1680192140 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fda82abd70 con 0x55fda83c4610 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6857fa640 10 client.16569 handle_command_reply: tid=0 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 client.16569 shutdown 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 msgr2=0x55fda8343a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8343a30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd6940307f0 tx=0x7fd694030820 comp rx=0 tx=0).stop 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 shutdown_connections 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fda83c4610 0x55fda83c6a30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fda8349370 0x55fda8343a30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fda833fb40 0x55fda83434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 --2- 172.21.15.103:0/1680192140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda833e5d0 0x55fda8342fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6857fa640 0 client.16569 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 >> 172.21.15.103:0/1680192140 conn(0x55fda8307310 msgr2=0x55fda83477f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 shutdown_connections 2024-09-19T07:43:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 1 -- 172.21.15.103:0/1680192140 wait complete. 2024-09-19T07:43:02.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.037+0000 7fd6a639b740 20 client.16569 trim_cache size 0 max 16384 2024-09-19T07:43:02.057 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:43:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6880a33b0 msgr2=0x7fd6880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6880a33b0 0x7fd6880a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd694031f80 tx=0x7fd69402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c102780 msgr2=0x7fd69c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c102780 0x7fd69c1989e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c920 tx=0x7fd69800cde0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 shutdown_connections 2024-09-19T07:43:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6880a33b0 0x7fd6880a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd69c106f00 0x7fd69c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd69c102780 0x7fd69c1989e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 --2- 172.21.15.103:0/379941623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd69c101dd0 0x7fd69c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 >> 172.21.15.103:0/379941623 conn(0x7fd69c0fd8b0 msgr2=0x7fd69c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 shutdown_connections 2024-09-19T07:43:02.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:02.038+0000 7fd6a3fc2640 1 -- 172.21.15.103:0/379941623 wait complete. 2024-09-19T07:43:02.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:01 smithi134 ceph-mon[35414]: pgmap v154: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 16 MiB/s wr, 1.30k op/s 2024-09-19T07:43:03.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:03 smithi174 ceph-mon[35370]: pgmap v155: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-19T07:43:03.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:03 smithi103 ceph-mon[31201]: pgmap v155: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-19T07:43:04.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:03 smithi134 ceph-mon[35414]: pgmap v155: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-19T07:43:04.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:43:04.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:43:04.362 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 -- 172.21.15.103:0/2930325142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac101ba0 msgr2=0x7f56ac10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 --2- 172.21.15.103:0/2930325142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac101ba0 0x7f56ac10d8f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009ca0 tx=0x7f56a402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 -- 172.21.15.103:0/2930325142 shutdown_connections 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 --2- 172.21.15.103:0/2930325142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac101ba0 0x7f56ac10d8f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 --2- 172.21.15.103:0/2930325142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac101220 0x7f56ac101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 --2- 172.21.15.103:0/2930325142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ac105a70 0x7f56ac105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 -- 172.21.15.103:0/2930325142 >> 172.21.15.103:0/2930325142 conn(0x7f56ac0fbee0 msgr2=0x7f56ac0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 -- 172.21.15.103:0/2930325142 shutdown_connections 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 -- 172.21.15.103:0/2930325142 wait complete. 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.795+0000 7f56b348d640 1 Processor -- start 2024-09-19T07:43:04.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b348d640 1 -- start start 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b348d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac101220 0x7f56ac10e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b348d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ac101ba0 0x7f56ac10e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b348d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac105a70 0x7f56ac10ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b348d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1120b0 con 0x7f56ac105a70 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b348d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac111f30 con 0x7f56ac101220 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b348d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac112230 con 0x7f56ac101ba0 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac105a70 0x7f56ac10ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac105a70 0x7f56ac10ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39450/0 (socket says 172.21.15.103:39450) 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 -- 172.21.15.103:0/1697157472 learned_addr learned my addr 172.21.15.103:0/1697157472 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b0a28640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ac101ba0 0x7f56ac10e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1229640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac101220 0x7f56ac10e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 -- 172.21.15.103:0/1697157472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ac101ba0 msgr2=0x7f56ac10e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ac101ba0 0x7f56ac10e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 -- 172.21.15.103:0/1697157472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac101220 msgr2=0x7f56ac10e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac101220 0x7f56ac10e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 -- 172.21.15.103:0/1697157472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac1a6970 con 0x7f56ac105a70 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1229640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac101220 0x7f56ac10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.796+0000 7f56b1a2a640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac105a70 0x7f56ac10ee80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006010 tx=0x7f56a4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.797+0000 7f569e7fc640 1 -- 172.21.15.103:0/1697157472 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4031650 con 0x7f56ac105a70 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.797+0000 7f569e7fc640 1 -- 172.21.15.103:0/1697157472 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56a40317f0 con 0x7f56ac105a70 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.797+0000 7f569e7fc640 1 -- 172.21.15.103:0/1697157472 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a40309c0 con 0x7f56ac105a70 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.797+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac1a6c00 con 0x7f56ac105a70 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.797+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac1a7140 con 0x7f56ac105a70 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.799+0000 7f569e7fc640 1 -- 172.21.15.103:0/1697157472 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f56a4030b60 con 0x7f56ac105a70 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.799+0000 7f56b5866740 1 Processor -- start 2024-09-19T07:43:04.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.799+0000 7f56b5866740 1 -- start start 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b5866740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d5b60 0x557a586d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b5866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b5866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586e3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b5866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a5863b280 con 0x557a586e0d40 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b5866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a585e2b10 con 0x557a586d6480 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b5866740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a58637240 con 0x557a586d5b60 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586e3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39464/0 (socket says 172.21.15.103:39464) 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f569e7fc640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f56980a33b0 0x7f56980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 -- 172.21.15.103:0/2589216692 learned_addr learned my addr 172.21.15.103:0/2589216692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d5b60 0x557a586d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b1a2a640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f56980a33b0 0x7f56980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b1229640 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 -- 172.21.15.103:0/2589216692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d5b60 msgr2=0x557a586d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d5b60 0x557a586d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 -- 172.21.15.103:0/2589216692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 msgr2=0x557a586e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 -- 172.21.15.103:0/2589216692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a585ef1f0 con 0x557a586e0d40 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f569e7fc640 1 -- 172.21.15.103:0/1697157472 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f56a40f10f0 con 0x7f56ac105a70 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b0a28640 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586e3130 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009f50 tx=0x7f56a80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2558cae83b8008ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56977fe640 1 -- 172.21.15.103:0/2589216692 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8017470 con 0x557a586e0d40 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.800+0000 7f56b1a2a640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f56980a33b0 0x7f56980a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56a40b98f0 tx=0x7f56a408f330 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56977fe640 1 -- 172.21.15.103:0/2589216692 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56a801c070 con 0x557a586e0d40 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56977fe640 1 -- 172.21.15.103:0/2589216692 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8015a60 con 0x557a586e0d40 2024-09-19T07:43:04.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 -- 172.21.15.103:0/2589216692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 msgr2=0x557a586e3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586e3130 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009f50 tx=0x7f56a80095c0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 -- 172.21.15.103:0/2589216692 shutdown_connections 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586e3130 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 --2- 172.21.15.103:0/2589216692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d5b60 0x557a586d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 -- 172.21.15.103:0/2589216692 >> 172.21.15.103:0/2589216692 conn(0x557a586d20f0 msgr2=0x557a586d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 -- 172.21.15.103:0/2589216692 shutdown_connections 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.801+0000 7f56b5866740 1 -- 172.21.15.103:0/2589216692 wait complete. 2024-09-19T07:43:04.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 -- start start 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 Processor -- start 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 0x557a586de700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557a586e3910 con 0x557a586d5b60 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a585f5f10 con 0x557a586d6480 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a586e3a90 con 0x557a586e0d40 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b5866740 10 client.0 _ll_register_callbacks cb 0x557a585bea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586ddc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39472/0 (socket says 172.21.15.103:39472) 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 -- 172.21.15.103:0/601740962 learned_addr learned my addr 172.21.15.103:0/601740962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:04.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b1229640 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 -- 172.21.15.103:0/601740962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 msgr2=0x557a586de700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 0x557a586de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 -- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 msgr2=0x557a586de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586de1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b0a28640 1 -- 172.21.15.103:0/601740962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a583ac6a0 con 0x557a586d5b60 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.802+0000 7f56b1229640 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b0a28640 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586ddc80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f56a8001eb0 tx=0x7f56a8009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0300416002fc44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f5694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f5694ff9640 1 -- 172.21.15.103:0/601740962 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8017670 con 0x557a586d5b60 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f5694ff9640 1 -- 172.21.15.103:0/601740962 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f56a8017810 con 0x557a586d5b60 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56a801c070 con 0x557a586d5b60 2024-09-19T07:43:04.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f5694ff9640 1 -- 172.21.15.103:0/601740962 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f56a8017a10 con 0x557a586d5b60 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 10 client.16581 fetch_fsmap learned FSMap version 36 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557a586ef5a0 con 0x557a586d5b60 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f5694ff9640 1 -- 172.21.15.103:0/601740962 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56a801c070 con 0x557a586d5b60 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f5694ff9640 1 -- 172.21.15.103:0/601740962 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f56a8029400 con 0x557a586d5b60 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 10 client.16581 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 20 client.16581 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a586f0950 0x557a586f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 4 client.16581 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.803+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a586487f0 con 0x557a586f0950 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.804+0000 7f56b1229640 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a586f0950 0x557a586f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.804+0000 7f5694ff9640 10 client.16581 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.804+0000 7f56b1229640 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a586f0950 0x557a586f2d40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b91dde1dcb69053 server_cookie=1595c06f5ec30686 in_seq=0 out_seq=0 2024-09-19T07:43:04.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f5694ff9640 1 -- 172.21.15.103:0/601740962 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a586487f0 con 0x557a586f0950 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f5694ff9640 10 client.16581 handle_command_reply: tid=0 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 client.16581 shutdown 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 msgr2=0x557a586ddc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586ddc80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f56a8001eb0 tx=0x7f56a8009280 comp rx=0 tx=0).stop 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 shutdown_connections 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a586f0950 0x557a586f2d40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 0x557a586de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f5694ff9640 0 client.16581 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 --2- 172.21.15.103:0/601740962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586ddc80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 >> 172.21.15.103:0/601740962 conn(0x557a586d20f0 msgr2=0x557a586d7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 shutdown_connections 2024-09-19T07:43:04.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 1 -- 172.21.15.103:0/601740962 wait complete. 2024-09-19T07:43:04.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.805+0000 7f56b5866740 20 client.16581 trim_cache size 0 max 16384 2024-09-19T07:43:04.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.822+0000 7f56b5866740 1 Processor -- start 2024-09-19T07:43:04.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.822+0000 7f56b5866740 1 -- start start 2024-09-19T07:43:04.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b5866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d5b60 0x557a586dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b5866740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d6480 0x557a586f8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b5866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b5866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1afa40 con 0x557a586e0d40 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b5866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b15e0 con 0x557a586d5b60 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b5866740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b0110 con 0x557a586d6480 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39480/0 (socket says 172.21.15.103:39480) 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 -- 172.21.15.103:0/359866645 learned_addr learned my addr 172.21.15.103:0/359866645 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d5b60 0x557a586dcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b1229640 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d6480 0x557a586f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 -- 172.21.15.103:0/359866645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d6480 msgr2=0x557a586f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d6480 0x557a586f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 -- 172.21.15.103:0/359866645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d5b60 msgr2=0x557a586dcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d5b60 0x557a586dcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 -- 172.21.15.103:0/359866645 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a586e3a70 con 0x557a586e0d40 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f56b0a28640 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586f8880 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f56a80156d0 tx=0x7f56a8006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a95b029886ae7e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f5696ffd640 1 -- 172.21.15.103:0/359866645 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8036040 con 0x557a586e0d40 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.823+0000 7f5696ffd640 1 -- 172.21.15.103:0/359866645 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56a801c070 con 0x557a586e0d40 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f5696ffd640 1 -- 172.21.15.103:0/359866645 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8021430 con 0x557a586e0d40 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 -- 172.21.15.103:0/359866645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 msgr2=0x557a586f8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586f8880 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f56a80156d0 tx=0x7f56a8006e40 comp rx=0 tx=0).stop 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 -- 172.21.15.103:0/359866645 shutdown_connections 2024-09-19T07:43:04.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586e0d40 0x557a586f8880 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586d6480 0x557a586f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 --2- 172.21.15.103:0/359866645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d5b60 0x557a586dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 -- 172.21.15.103:0/359866645 >> 172.21.15.103:0/359866645 conn(0x557a586d20f0 msgr2=0x557a586dc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 -- 172.21.15.103:0/359866645 shutdown_connections 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.824+0000 7f56b5866740 1 -- 172.21.15.103:0/359866645 wait complete. 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 -- start start 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 Processor -- start 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 0x557a586e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b0f10 con 0x557a586d5b60 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1b07e0 con 0x557a586d6480 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac1ada20 con 0x557a586e0d40 2024-09-19T07:43:04.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b5866740 10 client.0 _ll_register_callbacks cb 0x7f56a8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b1229640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b0a28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586e8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39492/0 (socket says 172.21.15.103:39492) 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b0a28640 1 -- 172.21.15.103:0/1474070193 learned_addr learned my addr 172.21.15.103:0/1474070193 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.825+0000 7f56b0a28640 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 0x557a586e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b0a28640 1 -- 172.21.15.103:0/1474070193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 msgr2=0x557a586e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b0a28640 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 0x557a586e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b0a28640 1 -- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 msgr2=0x557a586e9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b0a28640 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b0a28640 1 -- 172.21.15.103:0/1474070193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a586e65d0 con 0x557a586d5b60 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b1229640 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:04.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b0a28640 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586e8fc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f56a8004810 tx=0x7f56a80294a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c8b3df8e11bd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f5694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f5694ff9640 1 -- 172.21.15.103:0/1474070193 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8036040 con 0x557a586d5b60 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f5694ff9640 1 -- 172.21.15.103:0/1474070193 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f56a8029c40 con 0x557a586d5b60 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56a801c070 con 0x557a586d5b60 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f5694ff9640 1 -- 172.21.15.103:0/1474070193 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f56a8045050 con 0x557a586d5b60 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b5866740 10 client.16587 fetch_fsmap learned FSMap version 36 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557a5875bf40 con 0x557a586d5b60 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.826+0000 7f5694ff9640 1 -- 172.21.15.103:0/1474070193 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56a801c070 con 0x557a586d5b60 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f5694ff9640 1 -- 172.21.15.103:0/1474070193 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f56a803c070 con 0x557a586d5b60 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b5866740 10 client.16587 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b5866740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:04.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b5866740 20 client.16587 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:04.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b5866740 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a5875c500 0x557a5875e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:04.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b5866740 4 client.16587 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:43:04.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a58642d00 con 0x557a5875c500 2024-09-19T07:43:04.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b1229640 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a5875c500 0x557a5875e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:04.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f5694ff9640 10 client.16587 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:04.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.827+0000 7f56b1229640 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a5875c500 0x557a5875e8f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b9430ee0df19c1 server_cookie=899aa87e6aefc8c2 in_seq=0 out_seq=0 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.828+0000 7f5694ff9640 1 -- 172.21.15.103:0/1474070193 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a58642d00 con 0x557a5875c500 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f5694ff9640 10 client.16587 handle_command_reply: tid=0 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 client.16587 shutdown 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 msgr2=0x557a586e8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586e8fc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f56a8004810 tx=0x7f56a80294a0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 shutdown_connections 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557a5875c500 0x557a5875e8f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a586e0d40 0x557a586e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a586d6480 0x557a586e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f5694ff9640 0 client.16587 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 --2- 172.21.15.103:0/1474070193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557a586d5b60 0x557a586e8fc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 >> 172.21.15.103:0/1474070193 conn(0x557a586d20f0 msgr2=0x557a586de460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 shutdown_connections 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 1 -- 172.21.15.103:0/1474070193 wait complete. 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.829+0000 7f56b5866740 20 client.16587 trim_cache size 0 max 16384 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:43:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "99d686a3-8676-4c5f-ae68-c15bf435908d", 2024-09-19T07:43:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:43:04.837 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f56980a33b0 msgr2=0x7f56980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f56980a33b0 0x7f56980a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f56a40b98f0 tx=0x7f56a408f330 comp rx=0 tx=0).stop 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac105a70 msgr2=0x7f56ac10ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac105a70 0x7f56ac10ee80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006010 tx=0x7f56a4003f10 comp rx=0 tx=0).stop 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 shutdown_connections 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f56980a33b0 0x7f56980a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac105a70 0x7f56ac10ee80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ac101ba0 0x7f56ac10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 --2- 172.21.15.103:0/1697157472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56ac101220 0x7f56ac10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 >> 172.21.15.103:0/1697157472 conn(0x7f56ac0fbee0 msgr2=0x7f56ac0fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:04.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 shutdown_connections 2024-09-19T07:43:04.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:04.830+0000 7f56b348d640 1 -- 172.21.15.103:0/1697157472 wait complete. 2024-09-19T07:43:05.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 270.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:43:05.913 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: pgmap v156: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-09-19T07:43:05.913 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:43:05.913 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.913 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:05.913 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.914 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:05 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: pgmap v156: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:05.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:05.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:05.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:05 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:05.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:43:06.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: pgmap v156: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-09-19T07:43:06.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:06.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:06.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:06.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:05 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:06.214 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:06.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 -- 172.21.15.103:0/2636741821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c10c170 msgr2=0x7fb50c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2636741821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c10c170 0x7fb50c10e610 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb504009ca0 tx=0x7fb50402a950 comp rx=0 tx=0).stop 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 -- 172.21.15.103:0/2636741821 shutdown_connections 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2636741821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c10c170 0x7fb50c10e610 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2636741821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c103660 0x7fb50c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2636741821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c102d40 0x7fb50c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 -- 172.21.15.103:0/2636741821 >> 172.21.15.103:0/2636741821 conn(0x7fb50c0fc7a0 msgr2=0x7fb50c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 -- 172.21.15.103:0/2636741821 shutdown_connections 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.706+0000 7fb513dc3640 1 -- 172.21.15.103:0/2636741821 wait complete. 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 Processor -- start 2024-09-19T07:43:06.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 -- start start 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c102d40 0x7fb50c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c103660 0x7fb50c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c10c170 0x7fb50c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c111fb0 con 0x7fb50c103660 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c111e30 con 0x7fb50c10c170 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb513dc3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c112130 con 0x7fb50c102d40 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c103660 0x7fb50c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c103660 0x7fb50c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39508/0 (socket says 172.21.15.103:39508) 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 -- 172.21.15.103:0/2177401834 learned_addr learned my addr 172.21.15.103:0/2177401834 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb512360640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c10c170 0x7fb50c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb511b5f640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c102d40 0x7fb50c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 -- 172.21.15.103:0/2177401834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c102d40 msgr2=0x7fb50c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c102d40 0x7fb50c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 -- 172.21.15.103:0/2177401834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c10c170 msgr2=0x7fb50c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c10c170 0x7fb50c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 -- 172.21.15.103:0/2177401834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c19d380 con 0x7fb50c103660 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb512360640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c10c170 0x7fb50c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.707+0000 7fb51135e640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c103660 0x7fb50c198910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c970 tx=0x7fb50800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.708+0000 7fb502ffd640 1 -- 172.21.15.103:0/2177401834 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508007bb0 con 0x7fb50c103660 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.708+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c19d670 con 0x7fb50c103660 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.708+0000 7fb502ffd640 1 -- 172.21.15.103:0/2177401834 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb508007d50 con 0x7fb50c103660 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.708+0000 7fb502ffd640 1 -- 172.21.15.103:0/2177401834 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508018820 con 0x7fb50c103660 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.708+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c1a4f50 con 0x7fb50c103660 2024-09-19T07:43:06.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.710+0000 7fb502ffd640 1 -- 172.21.15.103:0/2177401834 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb5080189c0 con 0x7fb50c103660 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.710+0000 7fb51619c740 1 Processor -- start 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.710+0000 7fb51619c740 1 -- start start 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb51619c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c973880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb51619c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c973dc0 0x55964c97e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb51619c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c97e680 0x55964c980a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb51619c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55964c8d92e0 con 0x55964c9734a0 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb51619c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55964c880ce0 con 0x55964c97e680 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb51619c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55964c8d52a0 con 0x55964c973dc0 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c973880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c973880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39524/0 (socket says 172.21.15.103:39524) 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 -- 172.21.15.103:0/3611863973 learned_addr learned my addr 172.21.15.103:0/3611863973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb511b5f640 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c973dc0 0x55964c97e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb51135e640 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c97e680 0x55964c980a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb502ffd640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb4f80a33b0 0x7fb4f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 -- 172.21.15.103:0/3611863973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c973dc0 msgr2=0x55964c97e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c973dc0 0x55964c97e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 -- 172.21.15.103:0/3611863973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c97e680 msgr2=0x55964c980a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c97e680 0x55964c980a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 -- 172.21.15.103:0/3611863973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964c865440 con 0x55964c9734a0 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb511b5f640 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c973dc0 0x55964c97e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:06.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb502ffd640 1 -- 172.21.15.103:0/2177401834 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb5080cfe80 con 0x7fb50c103660 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb4f80a33b0 0x7fb4f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.711+0000 7fb512360640 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c973880 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb50402fa90 tx=0x7fb50402fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cdb48a66d976f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb4f7fff640 1 -- 172.21.15.103:0/3611863973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504041040 con 0x55964c9734a0 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb4f7fff640 1 -- 172.21.15.103:0/3611863973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb504004520 con 0x55964c9734a0 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb4f7fff640 1 -- 172.21.15.103:0/3611863973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504034040 con 0x55964c9734a0 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb512360640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb4f80a33b0 0x7fb4f80a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb50404c000 tx=0x7fb50404d000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb51619c740 1 -- 172.21.15.103:0/3611863973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 msgr2=0x55964c973880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb51619c740 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c973880 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb50402fa90 tx=0x7fb50402fac0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb51619c740 1 -- 172.21.15.103:0/3611863973 shutdown_connections 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb51619c740 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c97e680 0x55964c980a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb51619c740 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c973dc0 0x55964c97e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb51619c740 1 --2- 172.21.15.103:0/3611863973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c973880 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.712+0000 7fb51619c740 1 -- 172.21.15.103:0/3611863973 >> 172.21.15.103:0/3611863973 conn(0x55964c96fa30 msgr2=0x55964c96fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 -- 172.21.15.103:0/3611863973 shutdown_connections 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 -- 172.21.15.103:0/3611863973 wait complete. 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 -- start start 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 Processor -- start 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c97df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c97b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c97b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55964c981250 con 0x55964c9734a0 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55964c894050 con 0x55964c973dc0 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55964c9813d0 con 0x55964c97e680 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb51619c740 10 client.0 _ll_register_callbacks cb 0x55964c8d4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:06.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb511b5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c97df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb511b5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c97df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39536/0 (socket says 172.21.15.103:39536) 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.713+0000 7fb511b5f640 1 -- 172.21.15.103:0/1619332680 learned_addr learned my addr 172.21.15.103:0/1619332680 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb51135e640 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c97b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb511b5f640 1 -- 172.21.15.103:0/1619332680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 msgr2=0x55964c97b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb511b5f640 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c97b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb511b5f640 1 -- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 msgr2=0x55964c97b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb511b5f640 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c97b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb511b5f640 1 -- 172.21.15.103:0/1619332680 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964c64a3c0 con 0x55964c9734a0 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb511b5f640 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c97df70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d470 tx=0x7fb4fc00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c579a7f2562567e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb4f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb4f57fa640 1 -- 172.21.15.103:0/1619332680 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc01c060 con 0x55964c9734a0 2024-09-19T07:43:06.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb4f57fa640 1 -- 172.21.15.103:0/1619332680 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb4fc015750 con 0x55964c9734a0 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb504004520 con 0x55964c9734a0 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb4f57fa640 1 -- 172.21.15.103:0/1619332680 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb4fc015950 con 0x55964c9734a0 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb51619c740 10 client.16599 fetch_fsmap learned FSMap version 37 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55964c98cc20 con 0x55964c9734a0 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.714+0000 7fb4f57fa640 1 -- 172.21.15.103:0/1619332680 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb504004520 con 0x55964c9734a0 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb4f57fa640 1 -- 172.21.15.103:0/1619332680 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb4fc01f3a0 con 0x55964c9734a0 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51619c740 10 client.16599 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51619c740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51619c740 20 client.16599 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51619c740 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c98e0e0 0x55964c990570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51619c740 4 client.16599 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:06.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55964c8e6850 con 0x55964c98e0e0 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51135e640 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c98e0e0 0x55964c990570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb4f57fa640 10 client.16599 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.715+0000 7fb51135e640 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c98e0e0 0x55964c990570 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da9346708bc37a0 server_cookie=aa37a17d2bba60ec in_seq=0 out_seq=0 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb4f57fa640 1 -- 172.21.15.103:0/1619332680 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55964c8e6850 con 0x55964c98e0e0 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb4f57fa640 10 client.16599 handle_command_reply: tid=0 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 client.16599 shutdown 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 msgr2=0x55964c97df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c97df70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d470 tx=0x7fb4fc00d930 comp rx=0 tx=0).stop 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 shutdown_connections 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c98e0e0 0x55964c990570 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c97b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c97b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb4f57fa640 0 client.16599 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 --2- 172.21.15.103:0/1619332680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c97df70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 >> 172.21.15.103:0/1619332680 conn(0x55964c96fa30 msgr2=0x55964c96eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 shutdown_connections 2024-09-19T07:43:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 1 -- 172.21.15.103:0/1619332680 wait complete. 2024-09-19T07:43:06.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.716+0000 7fb51619c740 20 client.16599 trim_cache size 0 max 16384 2024-09-19T07:43:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 Processor -- start 2024-09-19T07:43:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 -- start start 2024-09-19T07:43:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c98a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c9767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c995e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1ad7e0 con 0x55964c9734a0 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1af320 con 0x55964c973dc0 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51619c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1adeb0 con 0x55964c97e680 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c98a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c98a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39544/0 (socket says 172.21.15.103:39544) 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 -- 172.21.15.103:0/2411859048 learned_addr learned my addr 172.21.15.103:0/2411859048 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51135e640 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c9767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 -- 172.21.15.103:0/2411859048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 msgr2=0x55964c995e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb512360640 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c995e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c995e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 -- 172.21.15.103:0/2411859048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 msgr2=0x55964c9767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c9767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb511b5f640 1 -- 172.21.15.103:0/2411859048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964c9732d0 con 0x55964c9734a0 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb51135e640 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c9767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.734+0000 7fb512360640 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c995e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.735+0000 7fb511b5f640 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c98a4b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc000c00 tx=0x7fb4fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b73dae7589d18c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.735+0000 7fb4f77fe640 1 -- 172.21.15.103:0/2411859048 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc02e070 con 0x55964c9734a0 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.735+0000 7fb4f77fe640 1 -- 172.21.15.103:0/2411859048 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc01f5a0 con 0x55964c9734a0 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.735+0000 7fb4f77fe640 1 -- 172.21.15.103:0/2411859048 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0296e0 con 0x55964c9734a0 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- 172.21.15.103:0/2411859048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 msgr2=0x55964c98a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c98a4b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc000c00 tx=0x7fb4fc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- 172.21.15.103:0/2411859048 shutdown_connections 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c995e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c973dc0 0x55964c9767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 --2- 172.21.15.103:0/2411859048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c9734a0 0x55964c98a4b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- 172.21.15.103:0/2411859048 >> 172.21.15.103:0/2411859048 conn(0x55964c96fa30 msgr2=0x55964c9760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- 172.21.15.103:0/2411859048 shutdown_connections 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- 172.21.15.103:0/2411859048 wait complete. 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- start start 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 Processor -- start 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c9734a0 0x55964c986d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c973dc0 0x55964c982080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c9825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1aec50 con 0x55964c973dc0 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1ae580 con 0x55964c9734a0 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.736+0000 7fb51619c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1ab7c0 con 0x55964c97e680 2024-09-19T07:43:06.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51619c740 10 client.0 _ll_register_callbacks cb 0x7fb4fc00b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c973dc0 0x55964c982080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c973dc0 0x55964c982080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39558/0 (socket says 172.21.15.103:39558) 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 -- 172.21.15.103:0/3166618406 learned_addr learned my addr 172.21.15.103:0/3166618406 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb512360640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c9825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb511b5f640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c9734a0 0x55964c986d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 -- 172.21.15.103:0/3166618406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 msgr2=0x55964c9825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c9825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 -- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c9734a0 msgr2=0x55964c986d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c9734a0 0x55964c986d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 -- 172.21.15.103:0/3166618406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964c982df0 con 0x55964c973dc0 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb512360640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c9825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:06.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51135e640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c973dc0 0x55964c982080 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb50809c100 tx=0x7fb5080bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb296b236bcf83a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb4f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb4f57fa640 1 -- 172.21.15.103:0/3166618406 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508018cb0 con 0x55964c973dc0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb4f57fa640 1 -- 172.21.15.103:0/3166618406 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fb5080cd860 con 0x55964c973dc0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc01f5a0 con 0x55964c973dc0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb4f57fa640 1 -- 172.21.15.103:0/3166618406 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb50809c870 con 0x55964c973dc0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51619c740 10 client.16605 fetch_fsmap learned FSMap version 37 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.737+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55964c9f8de0 con 0x55964c973dc0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb4f57fa640 1 -- 172.21.15.103:0/3166618406 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc01f5a0 con 0x55964c973dc0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb4f57fa640 1 -- 172.21.15.103:0/3166618406 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb5080987e0 con 0x55964c973dc0 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb51619c740 10 client.16605 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb51619c740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb51619c740 20 client.16605 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:06.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb51619c740 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c9f94b0 0x55964c9fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb51619c740 4 client.16605 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55964c97baf0 con 0x55964c9f94b0 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.738+0000 7fb511b5f640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c9f94b0 0x55964c9fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb4f57fa640 10 client.16605 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb511b5f640 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c9f94b0 0x55964c9fb8d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ea09f758c34fd5 server_cookie=3d7f8cdd0bf4d2e9 in_seq=0 out_seq=0 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb4f57fa640 1 -- 172.21.15.103:0/3166618406 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55964c97baf0 con 0x55964c9f94b0 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb4f57fa640 10 client.16605 handle_command_reply: tid=0 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 client.16605 shutdown 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c973dc0 msgr2=0x55964c982080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c973dc0 0x55964c982080 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb50809c100 tx=0x7fb5080bc040 comp rx=0 tx=0).stop 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 shutdown_connections 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55964c9f94b0 0x55964c9fb8d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55964c97e680 0x55964c9825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb4f57fa640 0 client.16605 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55964c973dc0 0x55964c982080 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 --2- 172.21.15.103:0/3166618406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55964c9734a0 0x55964c986d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 >> 172.21.15.103:0/3166618406 conn(0x55964c96fa30 msgr2=0x55964c96e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 shutdown_connections 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 1 -- 172.21.15.103:0/3166618406 wait complete. 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.739+0000 7fb51619c740 20 client.16605 trim_cache size 0 max 16384 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:43:06.756 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout: "99d686a3-8676-4c5f-ae68-c15bf435908d": { 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "99d686a3-8676-4c5f-ae68-c15bf435908d", 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:06.757 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb4f80a33b0 msgr2=0x7fb4f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb4f80a33b0 0x7fb4f80a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb50404c000 tx=0x7fb50404d000 comp rx=0 tx=0).stop 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c103660 msgr2=0x7fb50c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c103660 0x7fb50c198910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c970 tx=0x7fb50800ce30 comp rx=0 tx=0).stop 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 shutdown_connections 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb4f80a33b0 0x7fb4f80a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb50c10c170 0x7fb50c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50c103660 0x7fb50c198910 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 --2- 172.21.15.103:0/2177401834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c102d40 0x7fb50c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:06.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 >> 172.21.15.103:0/2177401834 conn(0x7fb50c0fc7a0 msgr2=0x7fb50c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:06.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 shutdown_connections 2024-09-19T07:43:06.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:06.740+0000 7fb513dc3640 1 -- 172.21.15.103:0/2177401834 wait complete. 2024-09-19T07:43:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:06 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:43:06.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:06 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:43:07.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:06 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:43:07.931 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:07 smithi103 ceph-mon[31201]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-09-19T07:43:07.958 INFO:tasks.cephfs.filesystem:scrub status for tag:99d686a3-8676-4c5f-ae68-c15bf435908d - {'path': '/', 'tag': '99d686a3-8676-4c5f-ae68-c15bf435908d', 'options': 'recursive,force'} 2024-09-19T07:43:08.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:07 smithi134 ceph-mon[35414]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-09-19T07:43:08.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:07 smithi174 ceph-mon[35370]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-09-19T07:43:09.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:09 smithi103 ceph-mon[31201]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-09-19T07:43:10.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:09 smithi134 ceph-mon[35414]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-09-19T07:43:10.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:09 smithi174 ceph-mon[35370]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-09-19T07:43:10.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 275.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:43:11.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:11 smithi103 ceph-mon[31201]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-09-19T07:43:12.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:11 smithi134 ceph-mon[35414]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-09-19T07:43:12.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:11 smithi174 ceph-mon[35370]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.3 MiB/s wr, 347 op/s 2024-09-19T07:43:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:12 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:43:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:12 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:43:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:12 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:43:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:12 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:43:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:12 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:43:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:12 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:43:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:12 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:43:13.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:12 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:43:13.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:12 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:43:13.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:12 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:43:13.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:12 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:43:13.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:12 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:43:14.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:14 smithi134 ceph-mon[35414]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.3 MiB/s wr, 229 op/s 2024-09-19T07:43:14.789 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:43:14.789 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:43:14.789 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:43:14.789 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:43:14.790 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:14 smithi103 ceph-mon[31201]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.3 MiB/s wr, 229 op/s 2024-09-19T07:43:14.790 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:43:14.790 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:43:14.790 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:43:14.791 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:43:14.791 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:14 smithi174 ceph-mon[35370]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.3 MiB/s wr, 229 op/s 2024-09-19T07:43:14.791 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:43:14.791 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:43:14.791 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:43:14.791 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:43:15.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 280.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:43:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:15 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:43:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:15 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:43:15.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:15 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:43:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:16 smithi134 ceph-mon[35414]: pgmap v161: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-09-19T07:43:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:16 smithi174 ceph-mon[35370]: pgmap v161: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-09-19T07:43:16.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:16 smithi103 ceph-mon[31201]: pgmap v161: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-09-19T07:43:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:18 smithi134 ceph-mon[35414]: pgmap v162: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 MiB/s wr, 215 op/s 2024-09-19T07:43:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:18 smithi174 ceph-mon[35370]: pgmap v162: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 MiB/s wr, 215 op/s 2024-09-19T07:43:18.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:18 smithi103 ceph-mon[31201]: pgmap v162: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 MiB/s wr, 215 op/s 2024-09-19T07:43:20.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 285.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:43:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:20 smithi134 ceph-mon[35414]: pgmap v163: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 MiB/s wr, 215 op/s 2024-09-19T07:43:20.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:20 smithi174 ceph-mon[35370]: pgmap v163: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 MiB/s wr, 215 op/s 2024-09-19T07:43:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:20 smithi103 ceph-mon[31201]: pgmap v163: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.3 MiB/s wr, 215 op/s 2024-09-19T07:43:21.293 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:43:21.552 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:22.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.118+0000 7f535d2bb640 1 -- 172.21.15.103:0/3110016863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358106f00 msgr2=0x7f535810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.118+0000 7f535d2bb640 1 --2- 172.21.15.103:0/3110016863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358106f00 0x7f535810ea80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5350009ca0 tx=0x7f535002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.118+0000 7f535d2bb640 1 -- 172.21.15.103:0/3110016863 shutdown_connections 2024-09-19T07:43:22.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.118+0000 7f535d2bb640 1 --2- 172.21.15.103:0/3110016863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358106f00 0x7f535810ea80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.118+0000 7f535d2bb640 1 --2- 172.21.15.103:0/3110016863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5358102780 0x7f53581067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.119+0000 7f535d2bb640 1 --2- 172.21.15.103:0/3110016863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5358101dd0 0x7f53581021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.119+0000 7f535d2bb640 1 -- 172.21.15.103:0/3110016863 >> 172.21.15.103:0/3110016863 conn(0x7f53580fd8b0 msgr2=0x7f53580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.119+0000 7f535d2bb640 1 -- 172.21.15.103:0/3110016863 shutdown_connections 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.119+0000 7f535d2bb640 1 -- 172.21.15.103:0/3110016863 wait complete. 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.119+0000 7f535d2bb640 1 Processor -- start 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.119+0000 7f535d2bb640 1 -- start start 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f535d2bb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5358101dd0 0x7f53580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f535d2bb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358102780 0x7f5358074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f535d2bb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5358106f00 0x7f5358074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f535d2bb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5358111f60 con 0x7f5358102780 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f535d2bb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5358111de0 con 0x7f5358101dd0 2024-09-19T07:43:22.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f535d2bb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53581120e0 con 0x7f5358106f00 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358102780 0x7f5358074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358102780 0x7f5358074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37912/0 (socket says 172.21.15.103:37912) 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 -- 172.21.15.103:0/1046775814 learned_addr learned my addr 172.21.15.103:0/1046775814 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f5356ffd640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5358101dd0 0x7f53580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f53577fe640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5358106f00 0x7f5358074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 -- 172.21.15.103:0/1046775814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5358106f00 msgr2=0x7f5358074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5358106f00 0x7f5358074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 -- 172.21.15.103:0/1046775814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5358101dd0 msgr2=0x7f53580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5358101dd0 0x7f53580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 -- 172.21.15.103:0/1046775814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53581a6850 con 0x7f5358102780 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f534ffff640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358102780 0x7f5358074a30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f534000c890 tx=0x7f534000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.120+0000 7f5354ff9640 1 -- 172.21.15.103:0/1046775814 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340004220 con 0x7f5358102780 2024-09-19T07:43:22.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.121+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53581a6b40 con 0x7f5358102780 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.121+0000 7f5354ff9640 1 -- 172.21.15.103:0/1046775814 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53400043c0 con 0x7f5358102780 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.121+0000 7f5354ff9640 1 -- 172.21.15.103:0/1046775814 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340004a00 con 0x7f5358102780 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.121+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53581a70d0 con 0x7f5358102780 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.122+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53581021b0 con 0x7f5358102780 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.123+0000 7f5354ff9640 1 -- 172.21.15.103:0/1046775814 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f53400079b0 con 0x7f5358102780 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.124+0000 7f5354ff9640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f53440a33b0 0x7f53440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.124+0000 7f5356ffd640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f53440a33b0 0x7f53440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.124+0000 7f5354ff9640 1 -- 172.21.15.103:0/1046775814 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f53400cfd60 con 0x7f5358102780 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.125+0000 7f5356ffd640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f53440a33b0 0x7f53440a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5348009bd0 tx=0x7f5348005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:22.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.127+0000 7f5354ff9640 1 -- 172.21.15.103:0/1046775814 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5340014030 con 0x7f5358102780 2024-09-19T07:43:22.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.341+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f535810a0e0 con 0x7f5358102780 2024-09-19T07:43:22.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.342+0000 7f5354ff9640 1 -- 172.21.15.103:0/1046775814 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11991 (secure 0 0 0) 0x7f534009c750 con 0x7f5358102780 2024-09-19T07:43:22.343 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:43:22.343 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":37,"btime":"2024-09-19T07:43:05:652659+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:43:04.960804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:43:22.344 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 37 2024-09-19T07:43:22.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f53440a33b0 msgr2=0x7f53440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f53440a33b0 0x7f53440a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5348009bd0 tx=0x7f5348005f90 comp rx=0 tx=0).stop 2024-09-19T07:43:22.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358102780 msgr2=0x7f5358074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:22.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358102780 0x7f5358074a30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f534000c890 tx=0x7f534000cd50 comp rx=0 tx=0).stop 2024-09-19T07:43:22.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 shutdown_connections 2024-09-19T07:43:22.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f53440a33b0 0x7f53440a5870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5358106f00 0x7f5358074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5358102780 0x7f5358074a30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 --2- 172.21.15.103:0/1046775814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5358101dd0 0x7f53580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:22.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 >> 172.21.15.103:0/1046775814 conn(0x7f53580fd8b0 msgr2=0x7f53580febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:22.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 shutdown_connections 2024-09-19T07:43:22.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:22.344+0000 7f535d2bb640 1 -- 172.21.15.103:0/1046775814 wait complete. 2024-09-19T07:43:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:22 smithi103 ceph-mon[31201]: pgmap v164: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-19T07:43:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:22 smithi134 ceph-mon[35414]: pgmap v164: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-19T07:43:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:22 smithi174 ceph-mon[35370]: pgmap v164: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-09-19T07:43:23.128 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 94fd66e0 --release --await' 2024-09-19T07:43:23.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 94fd66e0 --release --await 2024-09-19T07:43:23.381 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:23 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1046775814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:43:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:23 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1046775814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:43:23.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:23 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1046775814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 -- 172.21.15.103:0/1594915256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 msgr2=0x7fd95010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 --2- 172.21.15.103:0/1594915256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 0x7fd95010da40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd940009900 tx=0x7fd94002a480 comp rx=0 tx=0).stop 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 -- 172.21.15.103:0/1594915256 shutdown_connections 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 --2- 172.21.15.103:0/1594915256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 0x7fd95010da40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 --2- 172.21.15.103:0/1594915256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950102570 0x7fd9501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 --2- 172.21.15.103:0/1594915256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950100e20 0x7fd950101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 -- 172.21.15.103:0/1594915256 >> 172.21.15.103:0/1594915256 conn(0x7fd9500fc7d0 msgr2=0x7fd9500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 -- 172.21.15.103:0/1594915256 shutdown_connections 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.866+0000 7fd95788d640 1 -- 172.21.15.103:0/1594915256 wait complete. 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 Processor -- start 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 -- start start 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950100e20 0x7fd9500fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950102570 0x7fd9500fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 0x7fd9500fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd950110f80 con 0x7fd950102ef0 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd950110e00 con 0x7fd950100e20 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd95788d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd950111100 con 0x7fd950102570 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd955629640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950100e20 0x7fd9500fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd955629640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950100e20 0x7fd9500fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:35970/0 (socket says 172.21.15.103:35970) 2024-09-19T07:43:23.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd955629640 1 -- 172.21.15.103:0/703517112 learned_addr learned my addr 172.21.15.103:0/703517112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd954e28640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950102570 0x7fd9500fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.867+0000 7fd955e2a640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 0x7fd9500fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd955629640 1 -- 172.21.15.103:0/703517112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950102570 msgr2=0x7fd9500fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd955629640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950102570 0x7fd9500fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd955629640 1 -- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 msgr2=0x7fd9500fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd955629640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 0x7fd9500fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd955629640 1 -- 172.21.15.103:0/703517112 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501aae80 con 0x7fd950100e20 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd955e2a640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 0x7fd9500fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd955629640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950100e20 0x7fd9500fcc80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ea80 tx=0x7fd94800ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd94e7fc640 1 -- 172.21.15.103:0/703517112 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94800ce40 con 0x7fd950100e20 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9501ab0b0 con 0x7fd950100e20 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd94e7fc640 1 -- 172.21.15.103:0/703517112 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd948005020 con 0x7fd950100e20 2024-09-19T07:43:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501ab5a0 con 0x7fd950100e20 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.868+0000 7fd94e7fc640 1 -- 172.21.15.103:0/703517112 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94801f060 con 0x7fd950100e20 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.871+0000 7fd959c66740 1 Processor -- start 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.871+0000 7fd959c66740 1 -- start start 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.871+0000 7fd959c66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.871+0000 7fd959c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.871+0000 7fd94e7fc640 1 -- 172.21.15.103:0/703517112 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd9480051d0 con 0x7fd950100e20 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.871+0000 7fd959c66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd959c66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc422620 con 0x55f4dc4bd060 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd959c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc3c9e50 con 0x55f4dc4bd980 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd959c66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc41e5e0 con 0x55f4dc4c8240 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4bd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37948/0 (socket says 172.21.15.103:37948) 2024-09-19T07:43:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 -- 172.21.15.103:0/554759811 learned_addr learned my addr 172.21.15.103:0/554759811 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955629640 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd954e28640 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 -- 172.21.15.103:0/554759811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 msgr2=0x55f4dc4ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 -- 172.21.15.103:0/554759811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 msgr2=0x55f4dc4c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 -- 172.21.15.103:0/554759811 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dc3bad40 con 0x55f4dc4bd060 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955629640 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd955e2a640 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4bd440 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd940009900 tx=0x7fd9400043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c15e60bf30eddf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd9377fe640 1 -- 172.21.15.103:0/554759811 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940035030 con 0x55f4dc4bd060 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd9377fe640 1 -- 172.21.15.103:0/554759811 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940040070 con 0x55f4dc4bd060 2024-09-19T07:43:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.872+0000 7fd9377fe640 1 -- 172.21.15.103:0/554759811 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940031e10 con 0x55f4dc4bd060 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd94e7fc640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd9380a33b0 0x7fd9380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd955629640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd9380a33b0 0x7fd9380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd94e7fc640 1 -- 172.21.15.103:0/703517112 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd9480d0350 con 0x7fd950100e20 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 -- 172.21.15.103:0/554759811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 msgr2=0x55f4dc4bd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4bd440 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd940009900 tx=0x7fd9400043f0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd955629640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd9380a33b0 0x7fd9380a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55f4dc4ba450 tx=0x7fd9480142d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 -- 172.21.15.103:0/554759811 shutdown_connections 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 --2- 172.21.15.103:0/554759811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4bd440 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 -- 172.21.15.103:0/554759811 >> 172.21.15.103:0/554759811 conn(0x55f4dc4b95f0 msgr2=0x55f4dc4b99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 -- 172.21.15.103:0/554759811 shutdown_connections 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.873+0000 7fd959c66740 1 -- 172.21.15.103:0/554759811 wait complete. 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 -- start start 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 Processor -- start 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc4cae10 con 0x55f4dc4bd060 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc3aaf10 con 0x55f4dc4bd980 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc4caf90 con 0x55f4dc4c8240 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd959c66740 10 client.0 _ll_register_callbacks cb 0x55f4dc3cedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:23.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37952/0 (socket says 172.21.15.103:37952) 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd955e2a640 1 -- 172.21.15.103:0/579672279 learned_addr learned my addr 172.21.15.103:0/579672279 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd954e28640 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.874+0000 7fd955e2a640 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd955e2a640 1 -- 172.21.15.103:0/579672279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 msgr2=0x55f4dc4c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd955e2a640 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd955e2a640 1 -- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 msgr2=0x55f4dc4c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd955e2a640 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd955e2a640 1 -- 172.21.15.103:0/579672279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dc131580 con 0x55f4dc4bd060 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd955e2a640 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4c5160 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd9400314e0 tx=0x7fd940031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95a34bc9d13757c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd934ff9640 1 -- 172.21.15.103:0/579672279 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940053060 con 0x55f4dc4bd060 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd934ff9640 1 -- 172.21.15.103:0/579672279 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd940003ac0 con 0x55f4dc4bd060 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd940040070 con 0x55f4dc4bd060 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd934ff9640 1 -- 172.21.15.103:0/579672279 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd940049070 con 0x55f4dc4bd060 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 10 client.16623 fetch_fsmap learned FSMap version 37 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f4dc1fb960 con 0x55f4dc4bd060 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd934ff9640 1 -- 172.21.15.103:0/579672279 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940040070 con 0x55f4dc4bd060 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd934ff9640 1 -- 172.21.15.103:0/579672279 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd94003c440 con 0x55f4dc4bd060 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 10 client.16623 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 10 client.16623 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 20 client.16623 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc4d7cb0 0x55f4dc4da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 4 client.16623 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.875+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4dc42fb90 con 0x55f4dc4d7cb0 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.876+0000 7fd954e28640 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc4d7cb0 0x55f4dc4da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.876+0000 7fd934ff9640 10 client.16623 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.876+0000 7fd954e28640 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc4d7cb0 0x55f4dc4da0a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417d4c430c3f9e24 server_cookie=d9d2bc4b4d48803f in_seq=0 out_seq=0 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd934ff9640 1 -- 172.21.15.103:0/579672279 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4dc42fb90 con 0x55f4dc4d7cb0 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd934ff9640 10 client.16623 handle_command_reply: tid=0 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 client.16623 shutdown 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 msgr2=0x55f4dc4c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4c5160 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd9400314e0 tx=0x7fd940031910 comp rx=0 tx=0).stop 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 shutdown_connections 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc4d7cb0 0x55f4dc4da0a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd934ff9640 0 client.16623 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:23.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 --2- 172.21.15.103:0/579672279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4c5160 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 >> 172.21.15.103:0/579672279 conn(0x55f4dc4b95f0 msgr2=0x55f4dc4b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 shutdown_connections 2024-09-19T07:43:23.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 1 -- 172.21.15.103:0/579672279 wait complete. 2024-09-19T07:43:23.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.877+0000 7fd959c66740 20 client.16623 trim_cache size 0 max 16384 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 Processor -- start 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 -- start start 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501b3e20 con 0x55f4dc4bd060 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501b5960 con 0x55f4dc4bd980 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd959c66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501b44f0 con 0x55f4dc4c8240 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4d3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37966/0 (socket says 172.21.15.103:37966) 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd955e2a640 1 -- 172.21.15.103:0/3500642197 learned_addr learned my addr 172.21.15.103:0/3500642197 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.894+0000 7fd954e28640 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd955e2a640 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd955e2a640 1 -- 172.21.15.103:0/3500642197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 msgr2=0x55f4dc4c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd955e2a640 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd955e2a640 1 -- 172.21.15.103:0/3500642197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 msgr2=0x55f4dc4c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd955e2a640 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd955e2a640 1 -- 172.21.15.103:0/3500642197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dc4bd7b0 con 0x55f4dc4bd060 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd955e2a640 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4d3950 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd940009850 tx=0x7fd94002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4f5387c4ad7124 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd936ffd640 1 -- 172.21.15.103:0/3500642197 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940053070 con 0x55f4dc4bd060 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd936ffd640 1 -- 172.21.15.103:0/3500642197 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940040040 con 0x55f4dc4bd060 2024-09-19T07:43:23.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.895+0000 7fd936ffd640 1 -- 172.21.15.103:0/3500642197 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940035050 con 0x55f4dc4bd060 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 -- 172.21.15.103:0/3500642197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 msgr2=0x55f4dc4d3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4d3950 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd940009850 tx=0x7fd94002a480 comp rx=0 tx=0).stop 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 -- 172.21.15.103:0/3500642197 shutdown_connections 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 --2- 172.21.15.103:0/3500642197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4d3950 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 -- 172.21.15.103:0/3500642197 >> 172.21.15.103:0/3500642197 conn(0x55f4dc4b95f0 msgr2=0x55f4dc4d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 -- 172.21.15.103:0/3500642197 shutdown_connections 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 -- 172.21.15.103:0/3500642197 wait complete. 2024-09-19T07:43:23.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 -- start start 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.896+0000 7fd959c66740 1 Processor -- start 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4db9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4dbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501b5290 con 0x55f4dc4bd060 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501b4bc0 con 0x55f4dc4bd980 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501b1e00 con 0x55f4dc4c8240 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 10 client.0 _ll_register_callbacks cb 0x7fd9400317c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4db9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4db9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37978/0 (socket says 172.21.15.103:37978) 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 -- 172.21.15.103:0/233799564 learned_addr learned my addr 172.21.15.103:0/233799564 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd954e28640 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4dbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 -- 172.21.15.103:0/233799564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 msgr2=0x55f4dc4dcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4dcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 -- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 msgr2=0x55f4dc4dbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4dbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 -- 172.21.15.103:0/233799564 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dc4dd580 con 0x55f4dc4bd060 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd955e2a640 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4db9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd9400314e0 tx=0x7fd94003c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4fa6aa5d4fc96d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd934ff9640 1 -- 172.21.15.103:0/233799564 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940066030 con 0x55f4dc4bd060 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd934ff9640 1 -- 172.21.15.103:0/233799564 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fd9400038c0 con 0x55f4dc4bd060 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.897+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd940040040 con 0x55f4dc4bd060 2024-09-19T07:43:23.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd934ff9640 1 -- 172.21.15.103:0/233799564 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd940035070 con 0x55f4dc4bd060 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 10 client.16629 fetch_fsmap learned FSMap version 37 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f4dc5431d0 con 0x55f4dc4bd060 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd934ff9640 1 -- 172.21.15.103:0/233799564 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940040040 con 0x55f4dc4bd060 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd934ff9640 1 -- 172.21.15.103:0/233799564 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd940045b70 con 0x55f4dc4bd060 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 10 client.16629 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 10 client.16629 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 20 client.16629 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc543780 0x55f4dc545b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 4 client.16629 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "94fd66e0", "release": true, "await": true}] 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "94fd66e0", "release": true, "await": true}) -- 0x55f4dc42a0a0 con 0x55f4dc543780 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.898+0000 7fd954e28640 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc543780 0x55f4dc545b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:23.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.899+0000 7fd934ff9640 10 client.16629 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:23.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.899+0000 7fd954e28640 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc543780 0x55f4dc545b70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b9c7c364ee35e7 server_cookie=ac49d4e2954a2ee3 in_seq=0 out_seq=0 2024-09-19T07:43:23.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd934ff9640 1 -- 172.21.15.103:0/233799564 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4dc42a0a0 con 0x55f4dc543780 2024-09-19T07:43:23.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd934ff9640 10 client.16629 handle_command_reply: tid=0 2024-09-19T07:43:23.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 client.16629 shutdown 2024-09-19T07:43:23.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 msgr2=0x55f4dc4db9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4db9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd9400314e0 tx=0x7fd94003c440 comp rx=0 tx=0).stop 2024-09-19T07:43:23.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 shutdown_connections 2024-09-19T07:43:23.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f4dc543780 0x55f4dc545b70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4dc4c8240 0x55f4dc4dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4dc4bd980 0x55f4dc4dbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 --2- 172.21.15.103:0/233799564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4dc4bd060 0x55f4dc4db9c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.920+0000 7fd934ff9640 0 client.16629 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.919+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 >> 172.21.15.103:0/233799564 conn(0x55f4dc4b95f0 msgr2=0x55f4dc4c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.920+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 shutdown_connections 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.920+0000 7fd959c66740 1 -- 172.21.15.103:0/233799564 wait complete. 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.920+0000 7fd959c66740 20 client.16629 trim_cache size 0 max 16384 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 37, 2024-09-19T07:43:23.922 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 24, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "94fd66e0": { 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 24, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.8, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:43:23.923 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:23.924 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd9380a33b0 msgr2=0x7fd9380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd9380a33b0 0x7fd9380a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55f4dc4ba450 tx=0x7fd9480142d0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950100e20 msgr2=0x7fd9500fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950100e20 0x7fd9500fcc80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ea80 tx=0x7fd94800ef40 comp rx=0 tx=0).stop 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 shutdown_connections 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd9380a33b0 0x7fd9380a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd950102ef0 0x7fd9500fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950102570 0x7fd9500fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 --2- 172.21.15.103:0/703517112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd950100e20 0x7fd9500fcc80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.921+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 >> 172.21.15.103:0/703517112 conn(0x7fd9500fc7d0 msgr2=0x7fd95006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.922+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 shutdown_connections 2024-09-19T07:43:23.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:23.922+0000 7fd95788d640 1 -- 172.21.15.103:0/703517112 wait complete. 2024-09-19T07:43:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:24 smithi134 ceph-mon[35414]: pgmap v165: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-19T07:43:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:24 smithi174 ceph-mon[35370]: pgmap v165: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-19T07:43:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:24 smithi103 ceph-mon[31201]: pgmap v165: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-19T07:43:24.783 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '94fd66e0', total seconds elapsed: 33.3 2024-09-19T07:43:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 290.0 s, 559.6 tps, lat 1048.606 ms stddev 5158.414 2024-09-19T07:43:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:26 smithi134 ceph-mon[35414]: pgmap v166: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 195 KiB/s wr, 41 op/s 2024-09-19T07:43:26.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:26 smithi174 ceph-mon[35370]: pgmap v166: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 195 KiB/s wr, 41 op/s 2024-09-19T07:43:26.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:26 smithi103 ceph-mon[31201]: pgmap v166: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 195 KiB/s wr, 41 op/s 2024-09-19T07:43:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:28 smithi134 ceph-mon[35414]: pgmap v167: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T07:43:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:28 smithi174 ceph-mon[35370]: pgmap v167: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T07:43:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:28 smithi103 ceph-mon[31201]: pgmap v167: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T07:43:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 295.0 s, 3163.6 tps, lat 31.644 ms stddev 34.280 2024-09-19T07:43:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:30 smithi134 ceph-mon[35414]: pgmap v168: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T07:43:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:30 smithi174 ceph-mon[35370]: pgmap v168: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T07:43:30.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:30 smithi103 ceph-mon[31201]: pgmap v168: 97 pgs: 97 active+clean; 2.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 190 KiB/s wr, 35 op/s 2024-09-19T07:43:32.668 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:32 smithi174 ceph-mon[35370]: pgmap v169: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-09-19T07:43:32.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:32 smithi103 ceph-mon[31201]: pgmap v169: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-09-19T07:43:32.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:32 smithi134 ceph-mon[35414]: pgmap v169: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-09-19T07:43:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:34 smithi174 ceph-mon[35370]: pgmap v170: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-19T07:43:34.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:34 smithi103 ceph-mon[31201]: pgmap v170: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-19T07:43:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:34 smithi134 ceph-mon[35414]: pgmap v170: 97 pgs: 97 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-19T07:43:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 300.0 s, 3787.3 tps, lat 26.460 ms stddev 31.289 2024-09-19T07:43:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:36 smithi174 ceph-mon[35370]: pgmap v171: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 1.37k op/s 2024-09-19T07:43:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:36 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:43:36.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:36 smithi103 ceph-mon[31201]: pgmap v171: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 1.37k op/s 2024-09-19T07:43:36.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:36 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:43:36.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:36 smithi134 ceph-mon[35414]: pgmap v171: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 1.37k op/s 2024-09-19T07:43:36.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:36 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:43:37.959 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:43:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:38.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:38 smithi103 ceph-mon[31201]: pgmap v172: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.34k op/s 2024-09-19T07:43:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:38 smithi174 ceph-mon[35370]: pgmap v172: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.34k op/s 2024-09-19T07:43:38.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:38 smithi134 ceph-mon[35414]: pgmap v172: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.34k op/s 2024-09-19T07:43:38.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.957+0000 7f345aefb640 1 -- 172.21.15.103:0/3664764769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3454109830 msgr2=0x7f3454109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:38.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.957+0000 7f345aefb640 1 --2- 172.21.15.103:0/3664764769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3454109830 0x7f3454109c90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f344001a910 tx=0x7f344003b450 comp rx=0 tx=0).stop 2024-09-19T07:43:38.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.958+0000 7f345aefb640 1 -- 172.21.15.103:0/3664764769 shutdown_connections 2024-09-19T07:43:38.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.958+0000 7f345aefb640 1 --2- 172.21.15.103:0/3664764769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f345411b140 0x7f345411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.958+0000 7f345aefb640 1 --2- 172.21.15.103:0/3664764769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3454109830 0x7f3454109c90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.958+0000 7f345aefb640 1 --2- 172.21.15.103:0/3664764769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 0x7f3454110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.958+0000 7f345aefb640 1 -- 172.21.15.103:0/3664764769 >> 172.21.15.103:0/3664764769 conn(0x7f345406e140 msgr2=0x7f345406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:38.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.959+0000 7f345aefb640 1 -- 172.21.15.103:0/3664764769 shutdown_connections 2024-09-19T07:43:38.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.959+0000 7f345aefb640 1 -- 172.21.15.103:0/3664764769 wait complete. 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.960+0000 7f345aefb640 1 Processor -- start 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.960+0000 7f345aefb640 1 -- start start 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f345aefb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 0x7f345411a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f345aefb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3454109830 0x7f34541168f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f345aefb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f345411b140 0x7f3454116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f345aefb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3454113320 con 0x7f3454109830 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f345aefb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34541131a0 con 0x7f3454109450 2024-09-19T07:43:38.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f345aefb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34541134a0 con 0x7f345411b140 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3453fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3454109830 0x7f34541168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 0x7f345411a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3459498640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f345411b140 0x7f3454116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 0x7f345411a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58566/0 (socket says 172.21.15.103:58566) 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 -- 172.21.15.103:0/3779601857 learned_addr learned my addr 172.21.15.103:0/3779601857 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3453fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3454109830 0x7f34541168f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52656/0 (socket says 172.21.15.103:52656) 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 -- 172.21.15.103:0/3779601857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f345411b140 msgr2=0x7f3454116e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f345411b140 0x7f3454116e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 -- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3454109830 msgr2=0x7f34541168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3454109830 0x7f34541168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 -- 172.21.15.103:0/3779601857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3454117560 con 0x7f3454109450 2024-09-19T07:43:38.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3459498640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f345411b140 0x7f3454116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:38.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.961+0000 7f3458c97640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 0x7f345411a850 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f344800c890 tx=0x7f344800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:38.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.962+0000 7f3451ffb640 1 -- 172.21.15.103:0/3779601857 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448004220 con 0x7f3454109450 2024-09-19T07:43:38.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.962+0000 7f3451ffb640 1 -- 172.21.15.103:0/3779601857 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34480043c0 con 0x7f3454109450 2024-09-19T07:43:38.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.962+0000 7f3451ffb640 1 -- 172.21.15.103:0/3779601857 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34480049e0 con 0x7f3454109450 2024-09-19T07:43:38.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.962+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34541ab170 con 0x7f3454109450 2024-09-19T07:43:38.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.962+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34541ab5c0 con 0x7f3454109450 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 Processor -- start 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 -- start start 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98529880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 0x55ca98534140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca98536a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca9848f2e0 con 0x55ca98529dc0 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca98436ce0 con 0x55ca985294a0 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f345d2d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca9848b2a0 con 0x55ca98534680 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f3459498640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98529880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.965+0000 7f3459498640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98529880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58570/0 (socket says 172.21.15.103:58570) 2024-09-19T07:43:38.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3459498640 1 -- 172.21.15.103:0/2486706859 learned_addr learned my addr 172.21.15.103:0/2486706859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3453fff640 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca98536a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3459498640 1 -- 172.21.15.103:0/2486706859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 msgr2=0x55ca98536a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3459498640 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca98536a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3459498640 1 -- 172.21.15.103:0/2486706859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 msgr2=0x55ca98534140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3459498640 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 0x55ca98534140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3459498640 1 -- 172.21.15.103:0/2486706859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9841b440 con 0x55ca985294a0 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3459498640 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98529880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f344c00f5c0 tx=0x7f344c00fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348248d4d8eec5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:38.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f3451ffb640 1 -- 172.21.15.103:0/3779601857 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f34480079b0 con 0x7f3454109450 2024-09-19T07:43:38.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f343effd640 1 -- 172.21.15.103:0/2486706859 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c018020 con 0x55ca985294a0 2024-09-19T07:43:38.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f343effd640 1 -- 172.21.15.103:0/2486706859 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c014b60 con 0x55ca985294a0 2024-09-19T07:43:38.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.966+0000 7f343effd640 1 -- 172.21.15.103:0/2486706859 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c00dcc0 con 0x55ca985294a0 2024-09-19T07:43:38.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.967+0000 7f345d2d4740 1 -- 172.21.15.103:0/2486706859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 msgr2=0x55ca98529880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.967+0000 7f345d2d4740 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98529880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f344c00f5c0 tx=0x7f344c00fa80 comp rx=0 tx=0).stop 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.968+0000 7f345d2d4740 1 -- 172.21.15.103:0/2486706859 shutdown_connections 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.968+0000 7f345d2d4740 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca98536a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.968+0000 7f345d2d4740 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 0x55ca98534140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.968+0000 7f345d2d4740 1 --2- 172.21.15.103:0/2486706859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98529880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.968+0000 7f345d2d4740 1 -- 172.21.15.103:0/2486706859 >> 172.21.15.103:0/2486706859 conn(0x55ca98525a30 msgr2=0x55ca98525e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.968+0000 7f3451ffb640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34440a3500 0x7f34440a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.968+0000 7f3451ffb640 1 -- 172.21.15.103:0/3779601857 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f34480cfc30 con 0x7f3454109450 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.969+0000 7f3453fff640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34440a3500 0x7f34440a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.969+0000 7f345d2d4740 1 -- 172.21.15.103:0/2486706859 shutdown_connections 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.969+0000 7f345d2d4740 1 -- 172.21.15.103:0/2486706859 wait complete. 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.969+0000 7f345d2d4740 1 -- start start 2024-09-19T07:43:38.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 1 Processor -- start 2024-09-19T07:43:38.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98534050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98531300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca985318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f3453fff640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34440a3500 0x7f34440a59c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3440006010 tx=0x7f344001a630 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:38.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca98537250 con 0x55ca98534680 2024-09-19T07:43:38.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca9844a050 con 0x55ca985294a0 2024-09-19T07:43:38.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca985373d0 con 0x55ca98529dc0 2024-09-19T07:43:38.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f345d2d4740 10 client.0 _ll_register_callbacks cb 0x55ca9848a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:38.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f3458c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98534050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f3458c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98534050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58586/0 (socket says 172.21.15.103:58586) 2024-09-19T07:43:38.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f3458c97640 1 -- 172.21.15.103:0/284494553 learned_addr learned my addr 172.21.15.103:0/284494553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:38.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f3453fff640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98531300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.970+0000 7f3459498640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca985318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3459498640 1 -- 172.21.15.103:0/284494553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 msgr2=0x55ca98531300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:38.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3459498640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98531300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3459498640 1 -- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 msgr2=0x55ca98534050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3459498640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98534050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3459498640 1 -- 172.21.15.103:0/284494553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca982003c0 con 0x55ca98534680 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3458c97640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98534050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3459498640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca985318f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f344c004a80 tx=0x7f344c015f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce74872b4ee298b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.971+0000 7f3433fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.972+0000 7f3433fff640 1 -- 172.21.15.103:0/284494553 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c018040 con 0x55ca98534680 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.972+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f344c014b60 con 0x55ca98534680 2024-09-19T07:43:38.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.972+0000 7f3433fff640 1 -- 172.21.15.103:0/284494553 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f344c00da00 con 0x55ca98534680 2024-09-19T07:43:38.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.972+0000 7f3433fff640 1 -- 172.21.15.103:0/284494553 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f344c00dc00 con 0x55ca98534680 2024-09-19T07:43:38.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.972+0000 7f345d2d4740 10 client.16635 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:38.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.972+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ca98542cb0 con 0x55ca98534680 2024-09-19T07:43:38.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f3433fff640 1 -- 172.21.15.103:0/284494553 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c014b60 con 0x55ca98534680 2024-09-19T07:43:38.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f3433fff640 1 -- 172.21.15.103:0/284494553 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f344c02d400 con 0x55ca98534680 2024-09-19T07:43:38.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f345d2d4740 10 client.16635 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:38.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f345d2d4740 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f345d2d4740 20 client.16635 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f345d2d4740 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca98544110 0x55ca98546500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f345d2d4740 4 client.16635 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca9849c850 con 0x55ca98544110 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.973+0000 7f3458c97640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca98544110 0x55ca98546500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.974+0000 7f3433fff640 10 client.16635 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.974+0000 7f3458c97640 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca98544110 0x55ca98546500 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c3ddf1e3325f9a server_cookie=88946f416fd90513 in_seq=0 out_seq=0 2024-09-19T07:43:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f3433fff640 1 -- 172.21.15.103:0/284494553 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca9849c850 con 0x55ca98544110 2024-09-19T07:43:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f3433fff640 10 client.16635 handle_command_reply: tid=0 2024-09-19T07:43:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 client.16635 shutdown 2024-09-19T07:43:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 msgr2=0x55ca985318f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca985318f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f344c004a80 tx=0x7f344c015f50 comp rx=0 tx=0).stop 2024-09-19T07:43:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 shutdown_connections 2024-09-19T07:43:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca98544110 0x55ca98546500 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca985318f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98531300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f3433fff640 0 client.16635 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:38.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 --2- 172.21.15.103:0/284494553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98534050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:38.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.975+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 >> 172.21.15.103:0/284494553 conn(0x55ca98525a30 msgr2=0x55ca9852b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:38.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.976+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 shutdown_connections 2024-09-19T07:43:38.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.976+0000 7f345d2d4740 1 -- 172.21.15.103:0/284494553 wait complete. 2024-09-19T07:43:38.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:38.976+0000 7f345d2d4740 20 client.16635 trim_cache size 0 max 16384 2024-09-19T07:43:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 Processor -- start 2024-09-19T07:43:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 -- start start 2024-09-19T07:43:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98530bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98533270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca98549530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b3fb0 con 0x55ca98534680 2024-09-19T07:43:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b5af0 con 0x55ca985294a0 2024-09-19T07:43:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f345d2d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b4680 con 0x55ca98529dc0 2024-09-19T07:43:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f3459498640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca98549530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f3459498640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca98549530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52680/0 (socket says 172.21.15.103:52680) 2024-09-19T07:43:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f3458c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98530bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f3459498640 1 -- 172.21.15.103:0/3921828571 learned_addr learned my addr 172.21.15.103:0/3921828571 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.001+0000 7f3453fff640 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98533270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f3458c97640 1 -- 172.21.15.103:0/3921828571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 msgr2=0x55ca98533270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f3458c97640 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98533270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f3458c97640 1 -- 172.21.15.103:0/3921828571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 msgr2=0x55ca98549530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f3458c97640 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca98549530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f3458c97640 1 -- 172.21.15.103:0/3921828571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca985302b0 con 0x55ca985294a0 2024-09-19T07:43:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f3458c97640 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98530bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3448005fd0 tx=0x7f3448005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2ec8114be82db0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f343e7fc640 1 -- 172.21.15.103:0/3921828571 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344801d040 con 0x55ca985294a0 2024-09-19T07:43:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f343e7fc640 1 -- 172.21.15.103:0/3921828571 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ca9849c850 con 0x55ca985294a0 2024-09-19T07:43:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.002+0000 7f343e7fc640 1 -- 172.21.15.103:0/3921828571 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34480a46a0 con 0x55ca985294a0 2024-09-19T07:43:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.003+0000 7f345d2d4740 1 -- 172.21.15.103:0/3921828571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 msgr2=0x55ca98530bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.003+0000 7f345d2d4740 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98530bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3448005fd0 tx=0x7f3448005300 comp rx=0 tx=0).stop 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.004+0000 7f345d2d4740 1 -- 172.21.15.103:0/3921828571 shutdown_connections 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.004+0000 7f345d2d4740 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98534680 0x55ca98549530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.004+0000 7f345d2d4740 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98529dc0 0x55ca98533270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.004+0000 7f345d2d4740 1 --2- 172.21.15.103:0/3921828571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca98530bc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.004+0000 7f345d2d4740 1 -- 172.21.15.103:0/3921828571 >> 172.21.15.103:0/3921828571 conn(0x55ca98525a30 msgr2=0x55ca98532b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.005+0000 7f345d2d4740 1 -- 172.21.15.103:0/3921828571 shutdown_connections 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.005+0000 7f345d2d4740 1 -- 172.21.15.103:0/3921828571 wait complete. 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.005+0000 7f345d2d4740 1 -- start start 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.005+0000 7f345d2d4740 1 Processor -- start 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca985a7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 0x55ca985a7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f345d2d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca985490d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f345d2d4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b5420 con 0x55ca98529dc0 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f345d2d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b4d50 con 0x55ca985294a0 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f345d2d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b1f90 con 0x55ca98534680 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f345d2d4740 10 client.0 _ll_register_callbacks cb 0x7f344c00f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca985a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca985a7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58602/0 (socket says 172.21.15.103:58602) 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 -- 172.21.15.103:0/1395997328 learned_addr learned my addr 172.21.15.103:0/1395997328 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3453fff640 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 0x55ca985a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3459498640 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca985490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 -- 172.21.15.103:0/1395997328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 msgr2=0x55ca985490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:39.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca985490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 -- 172.21.15.103:0/1395997328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 msgr2=0x55ca985a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 0x55ca985a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 -- 172.21.15.103:0/1395997328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca98529bf0 con 0x55ca985294a0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3458c97640 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca985a7310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34480a4600 tx=0x7f34480a4ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21cb4a15cbad10ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.006+0000 7f3433fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.007+0000 7f3433fff640 1 -- 172.21.15.103:0/1395997328 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344807f060 con 0x55ca985294a0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.007+0000 7f3433fff640 1 -- 172.21.15.103:0/1395997328 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f34480cd5b0 con 0x55ca985294a0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.007+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca9849c850 con 0x55ca985294a0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.008+0000 7f3433fff640 1 -- 172.21.15.103:0/1395997328 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f344801d040 con 0x55ca985294a0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.008+0000 7f345d2d4740 10 client.26126 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.008+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ca985ae940 con 0x55ca985294a0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.008+0000 7f3433fff640 1 -- 172.21.15.103:0/1395997328 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34480cd7b0 con 0x55ca985294a0 2024-09-19T07:43:39.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f3433fff640 1 -- 172.21.15.103:0/1395997328 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f344807acc0 con 0x55ca985294a0 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f345d2d4740 10 client.26126 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f345d2d4740 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f345d2d4740 20 client.26126 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f345d2d4740 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca985af000 0x55ca985b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f345d2d4740 4 client.26126 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca9849c850 con 0x55ca985af000 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f3453fff640 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca985af000 0x55ca985b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f3453fff640 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca985af000 0x55ca985b13f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62537ab223ec3a8e server_cookie=921afd930b91e6e7 in_seq=0 out_seq=0 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.009+0000 7f3433fff640 10 client.26126 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.010+0000 7f3433fff640 1 -- 172.21.15.103:0/1395997328 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca9849c850 con 0x55ca985af000 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.010+0000 7f3433fff640 10 client.26126 handle_command_reply: tid=0 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.010+0000 7f345d2d4740 1 client.26126 shutdown 2024-09-19T07:43:39.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 msgr2=0x55ca985a7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca985a7310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34480a4600 tx=0x7f34480a4ba0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 shutdown_connections 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ca985af000 0x55ca985b13f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca98534680 0x55ca985490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca98529dc0 0x55ca985a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 --2- 172.21.15.103:0/1395997328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca985294a0 0x55ca985a7310 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 >> 172.21.15.103:0/1395997328 conn(0x55ca98525a30 msgr2=0x55ca98533380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f3433fff640 0 client.26126 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 shutdown_connections 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 1 -- 172.21.15.103:0/1395997328 wait complete. 2024-09-19T07:43:39.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.011+0000 7f345d2d4740 20 client.26126 trim_cache size 0 max 16384 2024-09-19T07:43:39.018 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:43:39.018 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:43:39.018 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:43:39.018 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:43:39.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34440a3500 msgr2=0x7f34440a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34440a3500 0x7f34440a59c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3440006010 tx=0x7f344001a630 comp rx=0 tx=0).stop 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 msgr2=0x7f345411a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 0x7f345411a850 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f344800c890 tx=0x7f344800cd50 comp rx=0 tx=0).stop 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 shutdown_connections 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34440a3500 0x7f34440a59c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f345411b140 0x7f3454116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3454109830 0x7f34541168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 --2- 172.21.15.103:0/3779601857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3454109450 0x7f345411a850 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 >> 172.21.15.103:0/3779601857 conn(0x7f345406e140 msgr2=0x7f3454072390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 shutdown_connections 2024-09-19T07:43:39.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:39.012+0000 7f345aefb640 1 -- 172.21.15.103:0/3779601857 wait complete. 2024-09-19T07:43:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:39 smithi174 ceph-mon[35370]: pgmap v173: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.34k op/s 2024-09-19T07:43:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:39 smithi103 ceph-mon[31201]: pgmap v173: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.34k op/s 2024-09-19T07:43:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:39 smithi134 ceph-mon[35414]: pgmap v173: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 1.34k op/s 2024-09-19T07:43:39.979 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:43:39.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:43:40.392 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 305.0 s, 3968.6 tps, lat 25.163 ms stddev 29.635 2024-09-19T07:43:40.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/2082826674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8109e00 msgr2=0x7fc0d810ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:40.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/2082826674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8109e00 0x7fc0d810ebe0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009860 tx=0x7fc0c002a340 comp rx=0 tx=0).stop 2024-09-19T07:43:40.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/2082826674 shutdown_connections 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/2082826674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d8069ab0 0x7fc0d810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/2082826674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8109e00 0x7fc0d810ebe0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/2082826674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 0x7fc0d8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/2082826674 >> 172.21.15.103:0/2082826674 conn(0x7fc0d806e2d0 msgr2=0x7fc0d806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/2082826674 shutdown_connections 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.960+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/2082826674 wait complete. 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.961+0000 7fc0dd7a6640 1 Processor -- start 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.961+0000 7fc0dd7a6640 1 -- start start 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0dd7a6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8069ab0 0x7fc0d81a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0dd7a6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 0x7fc0d81a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:40.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0dd7a6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d8109e00 0x7fc0d81a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0dd7a6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d8119650 con 0x7fc0d8069ab0 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0dd7a6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d81194d0 con 0x7fc0d8109450 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0dd7a6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d81197d0 con 0x7fc0d8109e00 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 0x7fc0d81a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 0x7fc0d81a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58620/0 (socket says 172.21.15.103:58620) 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 -- 172.21.15.103:0/1410070970 learned_addr learned my addr 172.21.15.103:0/1410070970 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0d6ffd640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8069ab0 0x7fc0d81a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0d77fe640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d8109e00 0x7fc0d81a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 -- 172.21.15.103:0/1410070970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d8109e00 msgr2=0x7fc0d81a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d8109e00 0x7fc0d81a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 -- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8069ab0 msgr2=0x7fc0d81a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:40.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8069ab0 0x7fc0d81a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:40.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.962+0000 7fc0cffff640 1 -- 172.21.15.103:0/1410070970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d81a5d30 con 0x7fc0d8109450 2024-09-19T07:43:40.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.963+0000 7fc0cffff640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 0x7fc0d81a12c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0035040 tx=0x7fc0c0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:40.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.963+0000 7fc0d4ff9640 1 -- 172.21.15.103:0/1410070970 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0037040 con 0x7fc0d8109450 2024-09-19T07:43:40.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.963+0000 7fc0d4ff9640 1 -- 172.21.15.103:0/1410070970 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c0040070 con 0x7fc0d8109450 2024-09-19T07:43:40.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.963+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d81a5fc0 con 0x7fc0d8109450 2024-09-19T07:43:40.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.963+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d80707f0 con 0x7fc0d8109450 2024-09-19T07:43:40.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.963+0000 7fc0d4ff9640 1 -- 172.21.15.103:0/1410070970 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00039b0 con 0x7fc0d8109450 2024-09-19T07:43:40.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.965+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b8005290 con 0x7fc0d8109450 2024-09-19T07:43:40.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.966+0000 7fc0d4ff9640 1 -- 172.21.15.103:0/1410070970 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc0c0003b50 con 0x7fc0d8109450 2024-09-19T07:43:40.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.967+0000 7fc0d4ff9640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc0c40a3430 0x7fc0c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:40.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.967+0000 7fc0d6ffd640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc0c40a3430 0x7fc0c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:40.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.967+0000 7fc0d4ff9640 1 -- 172.21.15.103:0/1410070970 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fc0c00ef9f0 con 0x7fc0d8109450 2024-09-19T07:43:40.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.967+0000 7fc0d6ffd640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc0c40a3430 0x7fc0c40a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8005fd0 tx=0x7fc0c8009260 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:40.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:40.971+0000 7fc0d4ff9640 1 -- 172.21.15.103:0/1410070970 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc0c00bc4f0 con 0x7fc0d8109450 2024-09-19T07:43:41.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.317+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b8005850 con 0x7fc0d8109450 2024-09-19T07:43:41.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.318+0000 7fc0d4ff9640 1 -- 172.21.15.103:0/1410070970 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11992 (secure 0 0 0) 0x7fc0c00bbdf0 con 0x7fc0d8109450 2024-09-19T07:43:41.319 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:43:41.319 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":38,"btime":"2024-09-19T07:43:35:333431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:43:34.852717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:43:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 38 2024-09-19T07:43:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.321+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc0c40a3430 msgr2=0x7fc0c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.321+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc0c40a3430 0x7fc0c40a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8005fd0 tx=0x7fc0c8009260 comp rx=0 tx=0).stop 2024-09-19T07:43:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.321+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 msgr2=0x7fc0d81a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.321+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 0x7fc0d81a12c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0035040 tx=0x7fc0c0033d90 comp rx=0 tx=0).stop 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.322+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 shutdown_connections 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.323+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc0c40a3430 0x7fc0c40a58f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.323+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d8109e00 0x7fc0d81a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.323+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d8109450 0x7fc0d81a12c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.323+0000 7fc0dd7a6640 1 --2- 172.21.15.103:0/1410070970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d8069ab0 0x7fc0d81a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.323+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 >> 172.21.15.103:0/1410070970 conn(0x7fc0d806e2d0 msgr2=0x7fc0d810c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.323+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 shutdown_connections 2024-09-19T07:43:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:41.323+0000 7fc0dd7a6640 1 -- 172.21.15.103:0/1410070970 wait complete. 2024-09-19T07:43:41.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:41 smithi103 ceph-mon[31201]: pgmap v174: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 MiB/s wr, 1.80k op/s 2024-09-19T07:43:41.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:41 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1410070970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:43:41.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:41 smithi134 ceph-mon[35414]: pgmap v174: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 MiB/s wr, 1.80k op/s 2024-09-19T07:43:41.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:41 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1410070970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:43:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:41 smithi174 ceph-mon[35370]: pgmap v174: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 MiB/s wr, 1.80k op/s 2024-09-19T07:43:41.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:41 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1410070970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:43:42.146 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:43:42.530 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:43.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.066+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/2520168833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 msgr2=0x7fbb58113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.066+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/2520168833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 0x7fbb58113370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009860 tx=0x7fbb4802a340 comp rx=0 tx=0).stop 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.066+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/2520168833 shutdown_connections 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.066+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/2520168833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb58113940 0x7fbb58115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.066+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/2520168833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 0x7fbb58113370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.066+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/2520168833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb5810a560 0x7fbb5810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.066+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/2520168833 >> 172.21.15.103:0/2520168833 conn(0x7fbb5806e140 msgr2=0x7fbb5806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.067+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/2520168833 shutdown_connections 2024-09-19T07:43:43.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.067+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/2520168833 wait complete. 2024-09-19T07:43:43.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.067+0000 7fbb5ff2f640 1 Processor -- start 2024-09-19T07:43:43.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.068+0000 7fbb5ff2f640 1 -- start start 2024-09-19T07:43:43.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.068+0000 7fbb5ff2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb5810a560 0x7fbb581a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.068+0000 7fbb5ff2f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 0x7fbb5819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.068+0000 7fbb5ff2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb58113940 0x7fbb5819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.068+0000 7fbb5ff2f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58119640 con 0x7fbb5810a560 2024-09-19T07:43:43.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.068+0000 7fbb5ff2f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581194c0 con 0x7fbb5810ae80 2024-09-19T07:43:43.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.068+0000 7fbb5ff2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581197c0 con 0x7fbb58113940 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.069+0000 7fbb5e4cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb58113940 0x7fbb5819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.069+0000 7fbb5e4cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb58113940 0x7fbb5819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:58724/0 (socket says 172.21.15.103:58724) 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.069+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/3673842136 learned_addr learned my addr 172.21.15.103:0/3673842136 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.069+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 0x7fbb5819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/3673842136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 msgr2=0x7fbb5819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 0x7fbb5819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb5810a560 0x7fbb581a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb5810a560 msgr2=0x7fbb581a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb5810a560 0x7fbb581a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/3673842136 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5819f590 con 0x7fbb58113940 2024-09-19T07:43:43.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 0x7fbb5819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:43.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb5810a560 0x7fbb581a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:43.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.070+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb58113940 0x7fbb5819f050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e940 tx=0x7fbb5000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.071+0000 7fbb4effd640 1 -- 172.21.15.103:0/3673842136 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000cd80 con 0x7fbb58113940 2024-09-19T07:43:43.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.071+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5819f880 con 0x7fbb58113940 2024-09-19T07:43:43.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.071+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb581bc570 con 0x7fbb58113940 2024-09-19T07:43:43.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.071+0000 7fbb4effd640 1 -- 172.21.15.103:0/3673842136 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb500035f0 con 0x7fbb58113940 2024-09-19T07:43:43.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.071+0000 7fbb4effd640 1 -- 172.21.15.103:0/3673842136 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50010690 con 0x7fbb58113940 2024-09-19T07:43:43.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.074+0000 7fbb4effd640 1 -- 172.21.15.103:0/3673842136 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbb50010830 con 0x7fbb58113940 2024-09-19T07:43:43.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.075+0000 7fbb62308740 1 Processor -- start 2024-09-19T07:43:43.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.075+0000 7fbb62308740 1 -- start start 2024-09-19T07:43:43.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb4effd640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbb440a3430 0x7fbb440a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb62308740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353019090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb62308740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353019a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb62308740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x557353026760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb62308740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557352f7f2f0 con 0x557353024370 2024-09-19T07:43:43.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb62308740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557352f26cf0 con 0x5573530195d0 2024-09-19T07:43:43.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb62308740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557352f7b2b0 con 0x55735301ab40 2024-09-19T07:43:43.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb5dccb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353019090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb5dccb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353019090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:58734/0 (socket says 172.21.15.103:58734) 2024-09-19T07:43:43.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb5dccb640 1 -- 172.21.15.103:0/1617658341 learned_addr learned my addr 172.21.15.103:0/1617658341 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:43.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.076+0000 7fbb4effd640 1 -- 172.21.15.103:0/3673842136 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fbb500cfe30 con 0x7fbb58113940 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.077+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbb440a3430 0x7fbb440a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.077+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353019a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.077+0000 7fbb5dccb640 1 -- 172.21.15.103:0/1617658341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 msgr2=0x557353019a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.077+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353019a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.077+0000 7fbb5dccb640 1 -- 172.21.15.103:0/1617658341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 msgr2=0x557353026760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.077+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x557353026760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x557353026760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb5dccb640 1 -- 172.21.15.103:0/1617658341 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557352f0b3a0 con 0x55735301ab40 2024-09-19T07:43:43.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353019a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x557353026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbb440a3430 0x7fbb440a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb4802a930 tx=0x7fbb4802c040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353019090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb54008f10 tx=0x7fbb5400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5e7ecec23819d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb43fff640 1 -- 172.21.15.103:0/1617658341 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400f830 con 0x55735301ab40 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb43fff640 1 -- 172.21.15.103:0/1617658341 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb54018480 con 0x55735301ab40 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.078+0000 7fbb43fff640 1 -- 172.21.15.103:0/1617658341 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400e5f0 con 0x55735301ab40 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.079+0000 7fbb62308740 1 -- 172.21.15.103:0/1617658341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 msgr2=0x557353019090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.079+0000 7fbb62308740 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353019090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbb54008f10 tx=0x7fbb5400d710 comp rx=0 tx=0).stop 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.080+0000 7fbb62308740 1 -- 172.21.15.103:0/1617658341 shutdown_connections 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.080+0000 7fbb62308740 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x557353026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.080+0000 7fbb62308740 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353019a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.080+0000 7fbb62308740 1 --2- 172.21.15.103:0/1617658341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353019090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.080+0000 7fbb62308740 1 -- 172.21.15.103:0/1617658341 >> 172.21.15.103:0/1617658341 conn(0x557352fe2310 msgr2=0x557353015c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.081+0000 7fbb62308740 1 -- 172.21.15.103:0/1617658341 shutdown_connections 2024-09-19T07:43:43.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.081+0000 7fbb62308740 1 -- 172.21.15.103:0/1617658341 wait complete. 2024-09-19T07:43:43.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.082+0000 7fbb62308740 1 -- start start 2024-09-19T07:43:43.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.082+0000 7fbb62308740 1 Processor -- start 2024-09-19T07:43:43.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.082+0000 7fbb62308740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573530195d0 0x557353023d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.082+0000 7fbb62308740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55735301ab40 0x557353020fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.082+0000 7fbb62308740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x5573530215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb5dccb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573530195d0 0x557353023d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb5dccb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573530195d0 0x557353023d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:58736/0 (socket says 172.21.15.103:58736) 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb5e4cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x5573530215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb5dccb640 1 -- 172.21.15.103:0/423446429 learned_addr learned my addr 172.21.15.103:0/423446429 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557353027070 con 0x557353024370 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55735301ab40 0x557353020fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557352f3a060 con 0x55735301ab40 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573530271f0 con 0x5573530195d0 2024-09-19T07:43:43.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.083+0000 7fbb62308740 10 client.0 _ll_register_callbacks cb 0x557352c8d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.084+0000 7fbb5dccb640 1 -- 172.21.15.103:0/423446429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55735301ab40 msgr2=0x557353020fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.084+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55735301ab40 0x557353020fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.084+0000 7fbb5dccb640 1 -- 172.21.15.103:0/423446429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 msgr2=0x5573530215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.084+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x5573530215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.084+0000 7fbb5dccb640 1 -- 172.21.15.103:0/423446429 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557352cf03c0 con 0x5573530195d0 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.084+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55735301ab40 0x557353020fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.084+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573530195d0 0x557353023d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb54018ec0 tx=0x7fbb54004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21ad4f396ba3b751 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb417fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb417fa640 1 -- 172.21.15.103:0/423446429 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54027060 con 0x5573530195d0 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb417fa640 1 -- 172.21.15.103:0/423446429 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fbb5400f630 con 0x5573530195d0 2024-09-19T07:43:43.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb54018480 con 0x5573530195d0 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb417fa640 1 -- 172.21.15.103:0/423446429 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbb54008b20 con 0x5573530195d0 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb62308740 10 client.26062 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557353032aa0 con 0x5573530195d0 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.085+0000 7fbb417fa640 1 -- 172.21.15.103:0/423446429 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb54018480 con 0x5573530195d0 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb417fa640 1 -- 172.21.15.103:0/423446429 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fbb54005450 con 0x5573530195d0 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb62308740 10 client.26062 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb62308740 10 client.26062 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb62308740 20 client.26062 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb62308740 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530362f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb62308740 4 client.26062 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:43.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557352f8c860 con 0x557353033f00 2024-09-19T07:43:43.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530362f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e49f6702a7cdcade server_cookie=1462246a9d015cd8 in_seq=0 out_seq=0 2024-09-19T07:43:43.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.086+0000 7fbb417fa640 10 client.26062 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:43:43.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.087+0000 7fbb417fa640 1 -- 172.21.15.103:0/423446429 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557352f8c860 con 0x557353033f00 2024-09-19T07:43:43.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.087+0000 7fbb417fa640 10 client.26062 handle_command_reply: tid=0 2024-09-19T07:43:43.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.087+0000 7fbb62308740 1 client.26062 shutdown 2024-09-19T07:43:43.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.087+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573530195d0 msgr2=0x557353023d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.087+0000 7fbb62308740 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573530195d0 0x557353023d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb54018ec0 tx=0x7fbb54004c70 comp rx=0 tx=0).stop 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.088+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 shutdown_connections 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.088+0000 7fbb62308740 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530362f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.088+0000 7fbb417fa640 0 client.26062 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.088+0000 7fbb62308740 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x5573530215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.088+0000 7fbb62308740 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55735301ab40 0x557353020fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.088+0000 7fbb62308740 1 --2- 172.21.15.103:0/423446429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573530195d0 0x557353023d30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.088+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 >> 172.21.15.103:0/423446429 conn(0x557352fe2310 msgr2=0x55735301b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.089+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 shutdown_connections 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.089+0000 7fbb62308740 1 -- 172.21.15.103:0/423446429 wait complete. 2024-09-19T07:43:43.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.089+0000 7fbb62308740 20 client.26062 trim_cache size 0 max 16384 2024-09-19T07:43:43.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 Processor -- start 2024-09-19T07:43:43.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 -- start start 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353014c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55735301ab40 0x55735301c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557353024370 0x55735301c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c00e740 con 0x55735301ab40 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c010280 con 0x5573530195d0 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.110+0000 7fbb62308740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c00ee10 con 0x557353024370 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5dccb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353014c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55735301ab40 0x55735301c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55735301ab40 0x55735301c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52744/0 (socket says 172.21.15.103:52744) 2024-09-19T07:43:43.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 -- 172.21.15.103:0/1896763496 learned_addr learned my addr 172.21.15.103:0/1896763496 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557353024370 0x55735301c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 -- 172.21.15.103:0/1896763496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557353024370 msgr2=0x55735301c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557353024370 0x55735301c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 -- 172.21.15.103:0/1896763496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 msgr2=0x557353014c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353014c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 -- 172.21.15.103:0/1896763496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735301ff90 con 0x55735301ab40 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55735301ab40 0x55735301c120 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb48031000 tx=0x7fbb480058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96846b02c50bddce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb437fe640 1 -- 172.21.15.103:0/1896763496 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4805d070 con 0x55735301ab40 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb437fe640 1 -- 172.21.15.103:0/1896763496 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557352f8c860 con 0x55735301ab40 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.111+0000 7fbb437fe640 1 -- 172.21.15.103:0/1896763496 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48037040 con 0x55735301ab40 2024-09-19T07:43:43.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 -- 172.21.15.103:0/1896763496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55735301ab40 msgr2=0x55735301c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55735301ab40 0x55735301c120 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb48031000 tx=0x7fbb480058d0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 -- 172.21.15.103:0/1896763496 shutdown_connections 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557353024370 0x55735301c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55735301ab40 0x55735301c120 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 --2- 172.21.15.103:0/1896763496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x557353014c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 -- 172.21.15.103:0/1896763496 >> 172.21.15.103:0/1896763496 conn(0x557352fe2310 msgr2=0x557353014500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 -- 172.21.15.103:0/1896763496 shutdown_connections 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.112+0000 7fbb62308740 1 -- 172.21.15.103:0/1896763496 wait complete. 2024-09-19T07:43:43.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 -- start start 2024-09-19T07:43:43.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 Processor -- start 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x5573530285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353028b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x55735309d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c00fbb0 con 0x557353024370 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c00f4e0 con 0x5573530195d0 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.113+0000 7fbb62308740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c00c720 con 0x55735301ab40 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.114+0000 7fbb62308740 10 client.0 _ll_register_callbacks cb 0x7fbb54008d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.114+0000 7fbb5e4cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x55735309d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.114+0000 7fbb5e4cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x55735309d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52746/0 (socket says 172.21.15.103:52746) 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.114+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/2273750164 learned_addr learned my addr 172.21.15.103:0/2273750164 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:43.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.114+0000 7fbb5d4ca640 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353028b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.114+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x5573530285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 msgr2=0x557353028b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353028b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/2273750164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 msgr2=0x5573530285d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x5573530285d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb5e4cc640 1 -- 172.21.15.103:0/2273750164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557353029410 con 0x557353024370 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb5e4cc640 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x55735309d950 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb5009d240 tx=0x7fbb5009cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ca93830c0d009f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb417fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.115+0000 7fbb417fa640 1 -- 172.21.15.103:0/2273750164 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb500a3980 con 0x557353024370 2024-09-19T07:43:43.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.116+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557352f8c860 con 0x557353024370 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.116+0000 7fbb417fa640 1 -- 172.21.15.103:0/2273750164 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fbb500bd0b0 con 0x557353024370 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.116+0000 7fbb417fa640 1 -- 172.21.15.103:0/2273750164 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbb500a3c00 con 0x557353024370 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.116+0000 7fbb62308740 10 client.16659 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.116+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55735309efc0 con 0x557353024370 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb417fa640 1 -- 172.21.15.103:0/2273750164 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557352f8c860 con 0x557353024370 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb417fa640 1 -- 172.21.15.103:0/2273750164 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7fbb5009c700 con 0x557353024370 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb62308740 10 client.16659 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb62308740 10 client.16659 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb62308740 20 client.16659 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb62308740 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb62308740 4 client.16659 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:43:43.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.117+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557352f86d70 con 0x557353033f00 2024-09-19T07:43:43.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.118+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:43.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.118+0000 7fbb417fa640 10 client.16659 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:43:43.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb5dccb640 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530a16f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f99004f6d6534bb server_cookie=6f8c9dd6b36c1e2b in_seq=0 out_seq=0 2024-09-19T07:43:43.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb417fa640 1 -- 172.21.15.103:0/2273750164 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557352f86d70 con 0x557353033f00 2024-09-19T07:43:43.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb417fa640 10 client.16659 handle_command_reply: tid=0 2024-09-19T07:43:43.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 client.16659 shutdown 2024-09-19T07:43:43.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 msgr2=0x55735309d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x55735309d950 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb5009d240 tx=0x7fbb5009cd00 comp rx=0 tx=0).stop 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 shutdown_connections 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x557353033f00 0x5573530a16f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557353024370 0x55735309d950 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55735301ab40 0x557353028b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 --2- 172.21.15.103:0/2273750164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573530195d0 0x5573530285d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 >> 172.21.15.103:0/2273750164 conn(0x557352fe2310 msgr2=0x557353014d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.119+0000 7fbb417fa640 0 client.16659 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.120+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 shutdown_connections 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.120+0000 7fbb62308740 1 -- 172.21.15.103:0/2273750164 wait complete. 2024-09-19T07:43:43.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.120+0000 7fbb62308740 20 client.16659 trim_cache size 0 max 16384 2024-09-19T07:43:43.124 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:43:43.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbb440a3430 msgr2=0x7fbb440a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbb440a3430 0x7fbb440a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb4802a930 tx=0x7fbb4802c040 comp rx=0 tx=0).stop 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb58113940 msgr2=0x7fbb5819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb58113940 0x7fbb5819f050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000e940 tx=0x7fbb5000ee00 comp rx=0 tx=0).stop 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 shutdown_connections 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbb440a3430 0x7fbb440a58f0 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb4802a930 tx=0x7fbb4802c040 comp rx=0 tx=0).stop 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb58113940 0x7fbb5819f050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.122+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5810ae80 0x7fbb5819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.123+0000 7fbb5ff2f640 1 --2- 172.21.15.103:0/3673842136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb5810a560 0x7fbb581a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.123+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 >> 172.21.15.103:0/3673842136 conn(0x7fbb5806e140 msgr2=0x7fbb58109390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.123+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 shutdown_connections 2024-09-19T07:43:43.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:43.123+0000 7fbb5ff2f640 1 -- 172.21.15.103:0/3673842136 wait complete. 2024-09-19T07:43:43.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:43 smithi103 ceph-mon[31201]: pgmap v175: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 MiB/s wr, 890 op/s 2024-09-19T07:43:43.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:43 smithi134 ceph-mon[35414]: pgmap v175: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 MiB/s wr, 890 op/s 2024-09-19T07:43:43.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:43 smithi174 ceph-mon[35370]: pgmap v175: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 MiB/s wr, 890 op/s 2024-09-19T07:43:44.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:43:44.631 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.164+0000 7f88c8af2640 1 -- 172.21.15.103:0/1172985488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c410c170 msgr2=0x7f88c410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.164+0000 7f88c8af2640 1 --2- 172.21.15.103:0/1172985488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c410c170 0x7f88c410e610 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009ca0 tx=0x7f88bc02a950 comp rx=0 tx=0).stop 2024-09-19T07:43:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.165+0000 7f88c8af2640 1 -- 172.21.15.103:0/1172985488 shutdown_connections 2024-09-19T07:43:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.165+0000 7f88c8af2640 1 --2- 172.21.15.103:0/1172985488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c410c170 0x7f88c410e610 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.165+0000 7f88c8af2640 1 --2- 172.21.15.103:0/1172985488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88c4103660 0x7f88c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.165+0000 7f88c8af2640 1 --2- 172.21.15.103:0/1172985488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 0x7f88c4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.165+0000 7f88c8af2640 1 -- 172.21.15.103:0/1172985488 >> 172.21.15.103:0/1172985488 conn(0x7f88c40fc7a0 msgr2=0x7f88c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.165+0000 7f88c8af2640 1 -- 172.21.15.103:0/1172985488 shutdown_connections 2024-09-19T07:43:45.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.165+0000 7f88c8af2640 1 -- 172.21.15.103:0/1172985488 wait complete. 2024-09-19T07:43:45.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.166+0000 7f88c8af2640 1 Processor -- start 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.166+0000 7f88c8af2640 1 -- start start 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.166+0000 7f88c8af2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 0x7f88c41a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.166+0000 7f88c8af2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c4103660 0x7f88c41a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.166+0000 7f88c8af2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88c410c170 0x7f88c410ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c8af2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4111f80 con 0x7f88c4103660 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c8af2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4111e00 con 0x7f88c4102d40 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c8af2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4112100 con 0x7f88c410c170 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c4103660 0x7f88c41a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 0x7f88c41a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c4103660 0x7f88c41a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52778/0 (socket says 172.21.15.103:52778) 2024-09-19T07:43:45.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c1d9b640 1 -- 172.21.15.103:0/646609530 learned_addr learned my addr 172.21.15.103:0/646609530 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 0x7f88c41a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58720/0 (socket says 172.21.15.103:58720) 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c2d9d640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88c410c170 0x7f88c410ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 -- 172.21.15.103:0/646609530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88c410c170 msgr2=0x7f88c410ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88c410c170 0x7f88c410ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 -- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c4103660 msgr2=0x7f88c41a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c4103660 0x7f88c41a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 -- 172.21.15.103:0/646609530 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c410b3e0 con 0x7f88c4102d40 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c2d9d640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88c410c170 0x7f88c410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:45.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.167+0000 7f88c259c640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 0x7f88c41a4690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88b400e9f0 tx=0x7f88b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.169+0000 7f88b37fe640 1 -- 172.21.15.103:0/646609530 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b400ccc0 con 0x7f88c4102d40 2024-09-19T07:43:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.169+0000 7f88b37fe640 1 -- 172.21.15.103:0/646609530 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4010910 con 0x7f88c4102d40 2024-09-19T07:43:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.169+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c410b6d0 con 0x7f88c4102d40 2024-09-19T07:43:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.169+0000 7f88b37fe640 1 -- 172.21.15.103:0/646609530 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4018960 con 0x7f88c4102d40 2024-09-19T07:43:45.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.169+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c41ab4f0 con 0x7f88c4102d40 2024-09-19T07:43:45.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.172+0000 7f88b37fe640 1 -- 172.21.15.103:0/646609530 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f88b4010ac0 con 0x7f88c4102d40 2024-09-19T07:43:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.173+0000 7f88b37fe640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f88ac0a3580 0x7f88ac0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.173+0000 7f88caecb740 1 Processor -- start 2024-09-19T07:43:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.173+0000 7f88b37fe640 1 -- 172.21.15.103:0/646609530 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f88b40d0080 con 0x7f88c4102d40 2024-09-19T07:43:45.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.173+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f88ac0a3580 0x7f88ac0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.174+0000 7f88caecb740 1 -- start start 2024-09-19T07:43:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.174+0000 7f88caecb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe9323b40 0x555fe9322090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.174+0000 7f88c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe9323b40 0x555fe9322090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.174+0000 7f88c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe9323b40 0x555fe9322090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52794/0 (socket says 172.21.15.103:52794) 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.174+0000 7f88caecb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe93225d0 0x555fe9322a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f88ac0a3580 0x7f88ac0a5a40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88c410eaf0 tx=0x7f88b8005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c259c640 1 -- 172.21.15.103:0/1722071524 learned_addr learned my addr 172.21.15.103:0/1722071524 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88caecb740 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fe92882f0 con 0x555fe9323b40 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555fe922fcf0 con 0x555fe932d370 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fe92842b0 con 0x555fe93225d0 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe93225d0 0x555fe9322a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c2d9d640 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c2d9d640 1 -- 172.21.15.103:0/1722071524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe93225d0 msgr2=0x555fe9322a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c2d9d640 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe93225d0 0x555fe9322a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c2d9d640 1 -- 172.21.15.103:0/1722071524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe9323b40 msgr2=0x555fe9322090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c2d9d640 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe9323b40 0x555fe9322090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c2d9d640 1 -- 172.21.15.103:0/1722071524 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe92143a0 con 0x555fe932d370 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.175+0000 7f88c259c640 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe9323b40 0x555fe9322090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.176+0000 7f88c2d9d640 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932f760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88bc006010 tx=0x7f88bc009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ad78a752e2efd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.176+0000 7f88b0ff9640 1 -- 172.21.15.103:0/1722071524 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc034040 con 0x555fe932d370 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.176+0000 7f88b0ff9640 1 -- 172.21.15.103:0/1722071524 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88bc003e50 con 0x555fe932d370 2024-09-19T07:43:45.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.176+0000 7f88b0ff9640 1 -- 172.21.15.103:0/1722071524 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc004280 con 0x555fe932d370 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 msgr2=0x555fe932f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932f760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88bc006010 tx=0x7f88bc009a20 comp rx=0 tx=0).stop 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 shutdown_connections 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932f760 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe93225d0 0x555fe9322a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 --2- 172.21.15.103:0/1722071524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe9323b40 0x555fe9322090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 >> 172.21.15.103:0/1722071524 conn(0x555fe92eb310 msgr2=0x555fe931ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 shutdown_connections 2024-09-19T07:43:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.177+0000 7f88caecb740 1 -- 172.21.15.103:0/1722071524 wait complete. 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.178+0000 7f88caecb740 1 -- start start 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.178+0000 7f88caecb740 1 Processor -- start 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.178+0000 7f88caecb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe932cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.178+0000 7f88caecb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88caecb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88caecb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fe9330010 con 0x555fe93225d0 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88caecb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555fe9243060 con 0x555fe932d370 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88caecb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fe9330190 con 0x555fe9323b40 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88caecb740 10 client.0 _ll_register_callbacks cb 0x555fe8f968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe932cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe932cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52798/0 (socket says 172.21.15.103:52798) 2024-09-19T07:43:45.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 -- 172.21.15.103:0/2483430374 learned_addr learned my addr 172.21.15.103:0/2483430374 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c2d9d640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 -- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 msgr2=0x555fe9329fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 -- 172.21.15.103:0/2483430374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 msgr2=0x555fe932a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 -- 172.21.15.103:0/2483430374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe8ff93c0 con 0x555fe93225d0 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88c259c640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe932cd00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f88b40cd6f0 tx=0x7f88b409bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1fccc066fb236d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88a9ffb640 1 -- 172.21.15.103:0/2483430374 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b40cdc10 con 0x555fe93225d0 2024-09-19T07:43:45.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88a9ffb640 1 -- 172.21.15.103:0/2483430374 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f88b40b52b0 con 0x555fe93225d0 2024-09-19T07:43:45.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.179+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88bc003e50 con 0x555fe93225d0 2024-09-19T07:43:45.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.180+0000 7f88a9ffb640 1 -- 172.21.15.103:0/2483430374 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f88b40a4f80 con 0x555fe93225d0 2024-09-19T07:43:45.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.180+0000 7f88caecb740 10 client.16671 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:45.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.180+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555fe933ba70 con 0x555fe93225d0 2024-09-19T07:43:45.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88a9ffb640 1 -- 172.21.15.103:0/2483430374 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88bc003e50 con 0x555fe93225d0 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88a9ffb640 1 -- 172.21.15.103:0/2483430374 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f88b40bc1d0 con 0x555fe93225d0 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88caecb740 10 client.16671 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88caecb740 10 client.16671 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88caecb740 20 client.16671 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88caecb740 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe933cf80 0x555fe933f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88caecb740 4 client.16671 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.181+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fe9295860 con 0x555fe933cf80 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.182+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe933cf80 0x555fe933f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.182+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe933cf80 0x555fe933f3e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=121432268f8086d3 server_cookie=9e3d64b5284a6ad3 in_seq=0 out_seq=0 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.182+0000 7f88a9ffb640 10 client.16671 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88a9ffb640 1 -- 172.21.15.103:0/2483430374 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fe9295860 con 0x555fe933cf80 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88a9ffb640 10 client.16671 handle_command_reply: tid=0 2024-09-19T07:43:45.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 client.16671 shutdown 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 msgr2=0x555fe932cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe932cd00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f88b40cd6f0 tx=0x7f88b409bd30 comp rx=0 tx=0).stop 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 shutdown_connections 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe933cf80 0x555fe933f3e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe932a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88a9ffb640 0 client.16671 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 --2- 172.21.15.103:0/2483430374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe932cd00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 >> 172.21.15.103:0/2483430374 conn(0x555fe92eb310 msgr2=0x555fe9324930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 shutdown_connections 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 1 -- 172.21.15.103:0/2483430374 wait complete. 2024-09-19T07:43:45.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.184+0000 7f88caecb740 20 client.16671 trim_cache size 0 max 16384 2024-09-19T07:43:45.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 Processor -- start 2024-09-19T07:43:45.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 -- start start 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe93392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe9344cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b3e80 con 0x555fe93225d0 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b59c0 con 0x555fe932d370 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88caecb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b4550 con 0x555fe9323b40 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88c2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe9344cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88c2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe9344cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58750/0 (socket says 172.21.15.103:58750) 2024-09-19T07:43:45.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88c2d9d640 1 -- 172.21.15.103:0/3496932102 learned_addr learned my addr 172.21.15.103:0/3496932102 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.210+0000 7f88c259c640 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe93392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88c259c640 1 -- 172.21.15.103:0/3496932102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 msgr2=0x555fe9329870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88c259c640 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88c259c640 1 -- 172.21.15.103:0/3496932102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 msgr2=0x555fe9344cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88c259c640 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe9344cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88c259c640 1 -- 172.21.15.103:0/3496932102 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe9323760 con 0x555fe93225d0 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88c259c640 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe93392e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f88b4098ce0 tx=0x7f88b409bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40be7ce03122102 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88abfff640 1 -- 172.21.15.103:0/3496932102 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b40cdc10 con 0x555fe93225d0 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88abfff640 1 -- 172.21.15.103:0/3496932102 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b40b5260 con 0x555fe93225d0 2024-09-19T07:43:45.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.211+0000 7f88abfff640 1 -- 172.21.15.103:0/3496932102 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b409be50 con 0x555fe93225d0 2024-09-19T07:43:45.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 -- 172.21.15.103:0/3496932102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 msgr2=0x555fe93392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe93392e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f88b4098ce0 tx=0x7f88b409bda0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 -- 172.21.15.103:0/3496932102 shutdown_connections 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe932d370 0x555fe9344cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9329870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 --2- 172.21.15.103:0/3496932102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe93225d0 0x555fe93392e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 -- 172.21.15.103:0/3496932102 >> 172.21.15.103:0/3496932102 conn(0x555fe92eb310 msgr2=0x555fe9328880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 -- 172.21.15.103:0/3496932102 shutdown_connections 2024-09-19T07:43:45.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.212+0000 7f88caecb740 1 -- 172.21.15.103:0/3496932102 wait complete. 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.213+0000 7f88caecb740 1 -- start start 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.213+0000 7f88caecb740 1 Processor -- start 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.213+0000 7f88caecb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe93225d0 0x555fe93359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.213+0000 7f88caecb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9330d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.213+0000 7f88caecb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe932d370 0x555fe9331260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.213+0000 7f88c259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe93225d0 0x555fe93359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.213+0000 7f88c259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe93225d0 0x555fe93359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58758/0 (socket says 172.21.15.103:58758) 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9330d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9330d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:58784/0 (socket says 172.21.15.103:58784) 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88caecb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b52f0 con 0x555fe932d370 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88caecb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b4c20 con 0x555fe93225d0 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88caecb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b1e60 con 0x555fe9323b40 2024-09-19T07:43:45.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c259c640 1 -- 172.21.15.103:0/724547973 learned_addr learned my addr 172.21.15.103:0/724547973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88caecb740 10 client.0 _ll_register_callbacks cb 0x7f88b40baf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c259c640 1 -- 172.21.15.103:0/724547973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 msgr2=0x555fe9330d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c259c640 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9330d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c259c640 1 -- 172.21.15.103:0/724547973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe932d370 msgr2=0x555fe9331260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c259c640 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe932d370 0x555fe9331260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c259c640 1 -- 172.21.15.103:0/724547973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe93318a0 con 0x555fe93225d0 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.214+0000 7f88c259c640 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe93225d0 0x555fe93359f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f88b409c2a0 tx=0x7f88b40985b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578d53b720b2384d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88a9ffb640 1 -- 172.21.15.103:0/724547973 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b409be50 con 0x555fe93225d0 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88a9ffb640 1 -- 172.21.15.103:0/724547973 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f88b40a0380 con 0x555fe93225d0 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b40b5260 con 0x555fe93225d0 2024-09-19T07:43:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88a9ffb640 1 -- 172.21.15.103:0/724547973 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f88b40cdc10 con 0x555fe93225d0 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88caecb740 10 client.26168 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555fe93318a0 con 0x555fe93225d0 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88a9ffb640 1 -- 172.21.15.103:0/724547973 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4018c70 con 0x555fe93225d0 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.215+0000 7f88a9ffb640 1 -- 172.21.15.103:0/724547973 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f88b4098140 con 0x555fe93225d0 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.216+0000 7f88caecb740 10 client.26168 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.216+0000 7f88caecb740 10 client.26168 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.216+0000 7f88caecb740 20 client.26168 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.216+0000 7f88caecb740 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe93a7d20 0x555fe93aa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.216+0000 7f88caecb740 4 client.26168 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.216+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f88b40b5260 con 0x555fe93a7d20 2024-09-19T07:43:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.216+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe93a7d20 0x555fe93aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.217+0000 7f88c1d9b640 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe93a7d20 0x555fe93aa110 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bbc30a219493219e server_cookie=3de478d04dde608e in_seq=0 out_seq=0 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.217+0000 7f88a9ffb640 10 client.26168 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88a9ffb640 1 -- 172.21.15.103:0/724547973 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f88b40b5260 con 0x555fe93a7d20 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88a9ffb640 10 client.26168 handle_command_reply: tid=0 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 client.26168 shutdown 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe93225d0 msgr2=0x555fe93359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe93225d0 0x555fe93359f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f88b409c2a0 tx=0x7f88b40985b0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 shutdown_connections 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x555fe93a7d20 0x555fe93aa110 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fe932d370 0x555fe9331260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88a9ffb640 0 client.26168 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fe9323b40 0x555fe9330d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 --2- 172.21.15.103:0/724547973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fe93225d0 0x555fe93359f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 >> 172.21.15.103:0/724547973 conn(0x555fe92eb310 msgr2=0x555fe9328c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 shutdown_connections 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 1 -- 172.21.15.103:0/724547973 wait complete. 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.218+0000 7f88caecb740 20 client.26168 trim_cache size 0 max 16384 2024-09-19T07:43:45.292 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:43:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f88ac0a3580 msgr2=0x7f88ac0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f88ac0a3580 0x7f88ac0a5a40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88c410eaf0 tx=0x7f88b8005f90 comp rx=0 tx=0).stop 2024-09-19T07:43:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 msgr2=0x7f88c41a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 0x7f88c41a4690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88b400e9f0 tx=0x7f88b400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 shutdown_connections 2024-09-19T07:43:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f88ac0a3580 0x7f88ac0a5a40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88c410c170 0x7f88c410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88c4103660 0x7f88c41a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 --2- 172.21.15.103:0/646609530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c4102d40 0x7f88c41a4690 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 >> 172.21.15.103:0/646609530 conn(0x7f88c40fc7a0 msgr2=0x7f88c410d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 shutdown_connections 2024-09-19T07:43:45.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:45.219+0000 7f88c8af2640 1 -- 172.21.15.103:0/646609530 wait complete. 2024-09-19T07:43:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 310.0 s, 3467.1 tps, lat 28.835 ms stddev 34.389 2024-09-19T07:43:45.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:45 smithi103 ceph-mon[31201]: pgmap v176: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 MiB/s wr, 1.23k op/s 2024-09-19T07:43:45.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:45 smithi134 ceph-mon[35414]: pgmap v176: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 MiB/s wr, 1.23k op/s 2024-09-19T07:43:45.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:45 smithi174 ceph-mon[35370]: pgmap v176: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 MiB/s wr, 1.23k op/s 2024-09-19T07:43:46.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:43:46.460 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:47.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 -- 172.21.15.103:0/2114306854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c104980 msgr2=0x7ff93c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 --2- 172.21.15.103:0/2114306854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c104980 0x7ff93c1089d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff930009860 tx=0x7ff93002a360 comp rx=0 tx=0).stop 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 -- 172.21.15.103:0/2114306854 shutdown_connections 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 --2- 172.21.15.103:0/2114306854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff93c109100 0x7ff93c110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 --2- 172.21.15.103:0/2114306854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c104980 0x7ff93c1089d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 --2- 172.21.15.103:0/2114306854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c103fd0 0x7ff93c1043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 -- 172.21.15.103:0/2114306854 >> 172.21.15.103:0/2114306854 conn(0x7ff93c0ffa30 msgr2=0x7ff93c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 -- 172.21.15.103:0/2114306854 shutdown_connections 2024-09-19T07:43:47.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.056+0000 7ff940ed5640 1 -- 172.21.15.103:0/2114306854 wait complete. 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.057+0000 7ff940ed5640 1 Processor -- start 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff940ed5640 1 -- start start 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff940ed5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff93c103fd0 0x7ff93c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff940ed5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c104980 0x7ff93c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff940ed5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c109100 0x7ff93c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff940ed5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c114120 con 0x7ff93c103fd0 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff940ed5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c113fa0 con 0x7ff93c104980 2024-09-19T07:43:47.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff940ed5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1142a0 con 0x7ff93c109100 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff93c103fd0 0x7ff93c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff939d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c104980 0x7ff93c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff93c103fd0 0x7ff93c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52834/0 (socket says 172.21.15.103:52834) 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff939d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c104980 0x7ff93c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58770/0 (socket says 172.21.15.103:58770) 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff93a59c640 1 -- 172.21.15.103:0/792466813 learned_addr learned my addr 172.21.15.103:0/792466813 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.058+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c109100 0x7ff93c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff939d9b640 1 -- 172.21.15.103:0/792466813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c109100 msgr2=0x7ff93c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff939d9b640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c109100 0x7ff93c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff939d9b640 1 -- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff93c103fd0 msgr2=0x7ff93c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff939d9b640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff93c103fd0 0x7ff93c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff939d9b640 1 -- 172.21.15.103:0/792466813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c19f5e0 con 0x7ff93c104980 2024-09-19T07:43:47.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c109100 0x7ff93c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:47.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff939d9b640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c104980 0x7ff93c19ab70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff93002a880 tx=0x7ff930033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff92b7fe640 1 -- 172.21.15.103:0/792466813 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93003f020 con 0x7ff93c104980 2024-09-19T07:43:47.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c19f870 con 0x7ff93c104980 2024-09-19T07:43:47.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff92b7fe640 1 -- 172.21.15.103:0/792466813 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff930030560 con 0x7ff93c104980 2024-09-19T07:43:47.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.059+0000 7ff92b7fe640 1 -- 172.21.15.103:0/792466813 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930037040 con 0x7ff93c104980 2024-09-19T07:43:47.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.060+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c076630 con 0x7ff93c104980 2024-09-19T07:43:47.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.064+0000 7ff92b7fe640 1 -- 172.21.15.103:0/792466813 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff930042020 con 0x7ff93c104980 2024-09-19T07:43:47.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.064+0000 7ff9432ae740 1 Processor -- start 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.064+0000 7ff9432ae740 1 -- start start 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.064+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.064+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff9432ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df7f9cb2f0 con 0x55df7fa70370 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff9432ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df7f972cf0 con 0x55df7fa655d0 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff9432ae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df7f9c72b0 con 0x55df7fa66b40 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa65a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58776/0 (socket says 172.21.15.103:58776) 2024-09-19T07:43:47.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 -- 172.21.15.103:0/3067950316 learned_addr learned my addr 172.21.15.103:0/3067950316 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 -- 172.21.15.103:0/3067950316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 msgr2=0x55df7fa65090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa65090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 -- 172.21.15.103:0/3067950316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 msgr2=0x55df7fa72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 -- 172.21.15.103:0/3067950316 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7f9573a0 con 0x55df7fa655d0 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff93a59c640 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa65a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009de0 tx=0x7ff92c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e166eb48d229d5cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.065+0000 7ff92b7fe640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9240a3500 0x7ff9240a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.066+0000 7ff928ff9640 1 -- 172.21.15.103:0/3067950316 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c01e070 con 0x55df7fa655d0 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.066+0000 7ff928ff9640 1 -- 172.21.15.103:0/3067950316 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c019b10 con 0x55df7fa655d0 2024-09-19T07:43:47.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.066+0000 7ff928ff9640 1 -- 172.21.15.103:0/3067950316 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00eb60 con 0x55df7fa655d0 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.066+0000 7ff92b7fe640 1 -- 172.21.15.103:0/792466813 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff9300efb00 con 0x7ff93c104980 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.067+0000 7ff9432ae740 1 -- 172.21.15.103:0/3067950316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 msgr2=0x55df7fa65a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.067+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa65a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009de0 tx=0x7ff92c011810 comp rx=0 tx=0).stop 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.067+0000 7ff9432ae740 1 -- 172.21.15.103:0/3067950316 shutdown_connections 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.067+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.067+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa65a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.067+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3067950316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa65090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.067+0000 7ff9432ae740 1 -- 172.21.15.103:0/3067950316 >> 172.21.15.103:0/3067950316 conn(0x55df7fa2e310 msgr2=0x55df7fa61c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.068+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9240a3500 0x7ff9240a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.068+0000 7ff9432ae740 1 -- 172.21.15.103:0/3067950316 shutdown_connections 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.068+0000 7ff9432ae740 1 -- 172.21.15.103:0/3067950316 wait complete. 2024-09-19T07:43:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.068+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9240a3500 0x7ff9240a59c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff934005fd0 tx=0x7ff934009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.068+0000 7ff9432ae740 1 -- start start 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 1 Processor -- start 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa6fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa66b40 0x55df7fa6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa70370 0x55df7fa6d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa6fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa6fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58788/0 (socket says 172.21.15.103:58788) 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df7fa73070 con 0x55df7fa66b40 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff93a59c640 1 -- 172.21.15.103:0/3720983664 learned_addr learned my addr 172.21.15.103:0/3720983664 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55df7f986060 con 0x55df7fa655d0 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df7fa731f0 con 0x55df7fa70370 2024-09-19T07:43:47.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff9432ae740 10 client.0 _ll_register_callbacks cb 0x55df7f6d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa70370 0x55df7fa6d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.069+0000 7ff939d9b640 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa66b40 0x55df7fa6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.070+0000 7ff93ad9d640 1 -- 172.21.15.103:0/3720983664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 msgr2=0x55df7fa6fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.070+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa6fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.070+0000 7ff93ad9d640 1 -- 172.21.15.103:0/3720983664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa66b40 msgr2=0x55df7fa6d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.070+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa66b40 0x55df7fa6d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.070+0000 7ff93ad9d640 1 -- 172.21.15.103:0/3720983664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7f73c3c0 con 0x55df7fa70370 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.070+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa70370 0x55df7fa6d860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff934003f20 tx=0x7ff934004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f281edae9645419 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.070+0000 7ff921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.071+0000 7ff921ffb640 1 -- 172.21.15.103:0/3720983664 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9340045d0 con 0x55df7fa70370 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.071+0000 7ff921ffb640 1 -- 172.21.15.103:0/3720983664 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff934004770 con 0x55df7fa70370 2024-09-19T07:43:47.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.071+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92c019b10 con 0x55df7fa70370 2024-09-19T07:43:47.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.072+0000 7ff921ffb640 1 -- 172.21.15.103:0/3720983664 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff934029440 con 0x55df7fa70370 2024-09-19T07:43:47.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.072+0000 7ff9432ae740 10 client.26098 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:47.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.072+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55df7fa7ec70 con 0x55df7fa70370 2024-09-19T07:43:47.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.072+0000 7ff921ffb640 1 -- 172.21.15.103:0/3720983664 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c019b10 con 0x55df7fa70370 2024-09-19T07:43:47.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff921ffb640 1 -- 172.21.15.103:0/3720983664 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7ff934027070 con 0x55df7fa70370 2024-09-19T07:43:47.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff9432ae740 10 client.26098 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff9432ae740 10 client.26098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff9432ae740 20 client.26098 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7fa800d0 0x55df7fa824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff9432ae740 4 client.26098 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df7f9d8860 con 0x55df7fa800d0 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.073+0000 7ff93a59c640 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7fa800d0 0x55df7fa824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.074+0000 7ff93a59c640 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7fa800d0 0x55df7fa824c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4a4a1fc767d8d6 server_cookie=96da8ee5724f251f in_seq=0 out_seq=0 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.075+0000 7ff921ffb640 10 client.26098 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.075+0000 7ff921ffb640 1 -- 172.21.15.103:0/3720983664 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df7f9d8860 con 0x55df7fa800d0 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.075+0000 7ff921ffb640 10 client.26098 handle_command_reply: tid=0 2024-09-19T07:43:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.075+0000 7ff9432ae740 1 client.26098 shutdown 2024-09-19T07:43:47.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.076+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa70370 msgr2=0x55df7fa6d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.076+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa70370 0x55df7fa6d860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff934003f20 tx=0x7ff934004ab0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.077+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 shutdown_connections 2024-09-19T07:43:47.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.077+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7fa800d0 0x55df7fa824c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.077+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa70370 0x55df7fa6d860 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.077+0000 7ff921ffb640 0 client.26098 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.077+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa66b40 0x55df7fa6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.077+0000 7ff9432ae740 1 --2- 172.21.15.103:0/3720983664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa6fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.077+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 >> 172.21.15.103:0/3720983664 conn(0x55df7fa2e310 msgr2=0x55df7fa67960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.078+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 shutdown_connections 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.078+0000 7ff9432ae740 1 -- 172.21.15.103:0/3720983664 wait complete. 2024-09-19T07:43:47.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.078+0000 7ff9432ae740 20 client.26098 trim_cache size 0 max 16384 2024-09-19T07:43:47.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.100+0000 7ff9432ae740 1 Processor -- start 2024-09-19T07:43:47.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.100+0000 7ff9432ae740 1 -- start start 2024-09-19T07:43:47.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.100+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa655d0 0x55df7fa61d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.100+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa66b40 0x55df7fa6f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff9432ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1b3ef0 con 0x55df7fa70370 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff9432ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1b5a30 con 0x55df7fa66b40 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff9432ae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1b45c0 con 0x55df7fa655d0 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa66b40 0x55df7fa6f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa66b40 0x55df7fa6f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58798/0 (socket says 172.21.15.103:58798) 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 -- 172.21.15.103:0/492826250 learned_addr learned my addr 172.21.15.103:0/492826250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff93ad9d640 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff93a59c640 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa655d0 0x55df7fa61d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 -- 172.21.15.103:0/492826250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa655d0 msgr2=0x55df7fa61d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa655d0 0x55df7fa61d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 -- 172.21.15.103:0/492826250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 msgr2=0x55df7fa68120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa68120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 -- 172.21.15.103:0/492826250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7fa62960 con 0x55df7fa66b40 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff93a59c640 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa655d0 0x55df7fa61d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.101+0000 7ff939d9b640 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa66b40 0x55df7fa6f040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff9300bbc00 tx=0x7ff93002c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=426d752d369b30be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.102+0000 7ff923fff640 1 -- 172.21.15.103:0/492826250 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9300bba40 con 0x55df7fa66b40 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.102+0000 7ff923fff640 1 -- 172.21.15.103:0/492826250 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff930009470 con 0x55df7fa66b40 2024-09-19T07:43:47.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.102+0000 7ff923fff640 1 -- 172.21.15.103:0/492826250 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9300bb860 con 0x55df7fa66b40 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.102+0000 7ff9432ae740 1 -- 172.21.15.103:0/492826250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa66b40 msgr2=0x55df7fa6f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.102+0000 7ff9432ae740 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa66b40 0x55df7fa6f040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff9300bbc00 tx=0x7ff93002c040 comp rx=0 tx=0).stop 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 -- 172.21.15.103:0/492826250 shutdown_connections 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa66b40 0x55df7fa6f040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 --2- 172.21.15.103:0/492826250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa655d0 0x55df7fa61d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 -- 172.21.15.103:0/492826250 >> 172.21.15.103:0/492826250 conn(0x55df7fa2e310 msgr2=0x55df7fa6e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 -- 172.21.15.103:0/492826250 shutdown_connections 2024-09-19T07:43:47.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 -- 172.21.15.103:0/492826250 wait complete. 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 -- start start 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.103+0000 7ff9432ae740 1 Processor -- start 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa74880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff9432ae740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa74dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff9432ae740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1b5360 con 0x55df7fa70370 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff9432ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1b4c90 con 0x55df7fa655d0 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff9432ae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1b1ef0 con 0x55df7fa66b40 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa74340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:58804/0 (socket says 172.21.15.103:58804) 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff93a59c640 1 -- 172.21.15.103:0/1946069774 learned_addr learned my addr 172.21.15.103:0/1946069774 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:47.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff93ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa74dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff9432ae740 10 client.0 _ll_register_callbacks cb 0x7ff934012d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.104+0000 7ff939d9b640 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa74880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff939d9b640 1 -- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 msgr2=0x55df7fa74340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff939d9b640 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa74340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff939d9b640 1 -- 172.21.15.103:0/1946069774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 msgr2=0x55df7fa74dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff939d9b640 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa74dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff939d9b640 1 -- 172.21.15.103:0/1946069774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7fae9e20 con 0x55df7fa66b40 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff93a59c640 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff939d9b640 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa74880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55df7fa7d100 tx=0x7ff9300050e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf31305a0ff9f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff921ffb640 1 -- 172.21.15.103:0/1946069774 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9300d4070 con 0x55df7fa66b40 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff921ffb640 1 -- 172.21.15.103:0/1946069774 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7ff9300c46e0 con 0x55df7fa66b40 2024-09-19T07:43:47.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff930009470 con 0x55df7fa66b40 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff921ffb640 1 -- 172.21.15.103:0/1946069774 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff9300bb860 con 0x55df7fa66b40 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff9432ae740 10 client.26104 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.105+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55df7faeb080 con 0x55df7fa66b40 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff921ffb640 1 -- 172.21.15.103:0/1946069774 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9300c4960 con 0x55df7fa66b40 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff921ffb640 1 -- 172.21.15.103:0/1946069774 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7ff93003a030 con 0x55df7fa66b40 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff9432ae740 10 client.26104 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff9432ae740 10 client.26104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff9432ae740 20 client.26104 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff9432ae740 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7faeb730 0x55df7faedb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff9432ae740 4 client.26104 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff930009470 con 0x55df7faeb730 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff93a59c640 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7faeb730 0x55df7faedb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:47.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.106+0000 7ff921ffb640 10 client.26104 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:43:47.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.107+0000 7ff93a59c640 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7faeb730 0x55df7faedb50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c12e402682359546 server_cookie=b1d312d8e9ab4152 in_seq=0 out_seq=0 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff921ffb640 1 -- 172.21.15.103:0/1946069774 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff930009470 con 0x55df7faeb730 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff921ffb640 10 client.26104 handle_command_reply: tid=0 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 client.26104 shutdown 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 msgr2=0x55df7fa74880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa74880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55df7fa7d100 tx=0x7ff9300050e0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 shutdown_connections 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55df7faeb730 0x55df7faedb50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df7fa70370 0x55df7fa74dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df7fa66b40 0x55df7fa74880 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 --2- 172.21.15.103:0/1946069774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df7fa655d0 0x55df7fa74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 >> 172.21.15.103:0/1946069774 conn(0x55df7fa2e310 msgr2=0x55df7fa6f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.109+0000 7ff921ffb640 0 client.26104 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:43:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.110+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 shutdown_connections 2024-09-19T07:43:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.110+0000 7ff9432ae740 1 -- 172.21.15.103:0/1946069774 wait complete. 2024-09-19T07:43:47.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.110+0000 7ff9432ae740 20 client.26104 trim_cache size 0 max 16384 2024-09-19T07:43:47.114 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9240a3500 msgr2=0x7ff9240a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9240a3500 0x7ff9240a59c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff934005fd0 tx=0x7ff934009500 comp rx=0 tx=0).stop 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c104980 msgr2=0x7ff93c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c104980 0x7ff93c19ab70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff93002a880 tx=0x7ff930033ae0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 shutdown_connections 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9240a3500 0x7ff9240a59c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff93c109100 0x7ff93c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff93c104980 0x7ff93c19ab70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 --2- 172.21.15.103:0/792466813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff93c103fd0 0x7ff93c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:47.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 >> 172.21.15.103:0/792466813 conn(0x7ff93c0ffa30 msgr2=0x7ff93c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:47.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 shutdown_connections 2024-09-19T07:43:47.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:47.111+0000 7ff940ed5640 1 -- 172.21.15.103:0/792466813 wait complete. 2024-09-19T07:43:47.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:47 smithi103 ceph-mon[31201]: pgmap v177: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 791 op/s 2024-09-19T07:43:47.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:47 smithi134 ceph-mon[35414]: pgmap v177: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 791 op/s 2024-09-19T07:43:47.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:47 smithi174 ceph-mon[35370]: pgmap v177: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 791 op/s 2024-09-19T07:43:47.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:43:48.478 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:49.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:49 smithi103 ceph-mon[31201]: pgmap v178: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 791 op/s 2024-09-19T07:43:49.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.787+0000 7f4b22f34640 1 -- 172.21.15.103:0/561592653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 msgr2=0x7f4b1c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.787+0000 7f4b22f34640 1 --2- 172.21.15.103:0/561592653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 0x7f4b1c10da40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009be0 tx=0x7f4b1402c6d0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.787+0000 7f4b22f34640 1 -- 172.21.15.103:0/561592653 shutdown_connections 2024-09-19T07:43:49.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 --2- 172.21.15.103:0/561592653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 0x7f4b1c10da40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 --2- 172.21.15.103:0/561592653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c102570 0x7f4b1c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 --2- 172.21.15.103:0/561592653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b1c100e20 0x7f4b1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 -- 172.21.15.103:0/561592653 >> 172.21.15.103:0/561592653 conn(0x7f4b1c0fc7d0 msgr2=0x7f4b1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 -- 172.21.15.103:0/561592653 shutdown_connections 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 -- 172.21.15.103:0/561592653 wait complete. 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 Processor -- start 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.788+0000 7f4b22f34640 1 -- start start 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b22f34640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c100e20 0x7f4b1c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b22f34640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b1c102570 0x7f4b1c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b22f34640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 0x7f4b1c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b22f34640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c114290 con 0x7f4b1c102ef0 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b22f34640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c114110 con 0x7f4b1c100e20 2024-09-19T07:43:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b22f34640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c114410 con 0x7f4b1c102570 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b1c102570 0x7f4b1c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c100e20 0x7f4b1c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c100e20 0x7f4b1c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57026/0 (socket says 172.21.15.103:57026) 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b1c102570 0x7f4b1c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43956/0 (socket says 172.21.15.103:43956) 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 -- 172.21.15.103:0/2053800270 learned_addr learned my addr 172.21.15.103:0/2053800270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b214d1640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 0x7f4b1c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 -- 172.21.15.103:0/2053800270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b1c102570 msgr2=0x7f4b1c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b1c102570 0x7f4b1c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 -- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 msgr2=0x7f4b1c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 0x7f4b1c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 -- 172.21.15.103:0/2053800270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c19f6c0 con 0x7f4b1c100e20 2024-09-19T07:43:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b214d1640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 0x7f4b1c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.789+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c100e20 0x7f4b1c19a710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e9c0 tx=0x7f4b1000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.790+0000 7f4b19ffb640 1 -- 172.21.15.103:0/2053800270 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000ccc0 con 0x7f4b1c100e20 2024-09-19T07:43:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.790+0000 7f4b19ffb640 1 -- 172.21.15.103:0/2053800270 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b10005500 con 0x7f4b1c100e20 2024-09-19T07:43:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.790+0000 7f4b19ffb640 1 -- 172.21.15.103:0/2053800270 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10010430 con 0x7f4b1c100e20 2024-09-19T07:43:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.790+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1c19f9b0 con 0x7f4b1c100e20 2024-09-19T07:43:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.790+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1c1a7290 con 0x7f4b1c100e20 2024-09-19T07:43:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 Processor -- start 2024-09-19T07:43:49.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 -- start start 2024-09-19T07:43:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7fb40 0x5638b3a7e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a7ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a93f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5638b39e42f0 con 0x5638b3a91b80 2024-09-19T07:43:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5638b398bcf0 con 0x5638b3a7e5d0 2024-09-19T07:43:49.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.794+0000 7f4b2530d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638b39e02b0 con 0x5638b3a7fb40 2024-09-19T07:43:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.795+0000 7f4b214d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7fb40 0x5638b3a7e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.795+0000 7f4b214d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7fb40 0x5638b3a7e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43962/0 (socket says 172.21.15.103:43962) 2024-09-19T07:43:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.795+0000 7f4b214d1640 1 -- 172.21.15.103:0/1121276016 learned_addr learned my addr 172.21.15.103:0/1121276016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:49.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.795+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a7ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.796+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a93f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.796+0000 7f4b19ffb640 1 -- 172.21.15.103:0/2053800270 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4b100046c0 con 0x7f4b1c100e20 2024-09-19T07:43:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.797+0000 7f4b20cd0640 1 -- 172.21.15.103:0/1121276016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7fb40 msgr2=0x5638b3a7e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.797+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7fb40 0x5638b3a7e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.797+0000 7f4b20cd0640 1 -- 172.21.15.103:0/1121276016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 msgr2=0x5638b3a93f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.797+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a93f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.797+0000 7f4b20cd0640 1 -- 172.21.15.103:0/1121276016 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b39703a0 con 0x5638b3a7e5d0 2024-09-19T07:43:49.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.797+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a7ea10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b100cd840 tx=0x7f4b100d3f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb30e7d642c8e6c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.797+0000 7f4b19ffb640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4b0c0a34b0 0x7f4b0c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.798+0000 7f4b19ffb640 1 -- 172.21.15.103:0/2053800270 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4b100d0140 con 0x7f4b1c100e20 2024-09-19T07:43:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.798+0000 7f4b214d1640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4b0c0a34b0 0x7f4b0c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.798+0000 7f4b06ffd640 1 -- 172.21.15.103:0/1121276016 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100e3070 con 0x5638b3a7e5d0 2024-09-19T07:43:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.798+0000 7f4b06ffd640 1 -- 172.21.15.103:0/1121276016 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b1009cad0 con 0x5638b3a7e5d0 2024-09-19T07:43:49.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.799+0000 7f4b06ffd640 1 -- 172.21.15.103:0/1121276016 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1009bd30 con 0x5638b3a7e5d0 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.799+0000 7f4b2530d740 1 -- 172.21.15.103:0/1121276016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 msgr2=0x5638b3a7ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.799+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a7ea10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b100cd840 tx=0x7f4b100d3f90 comp rx=0 tx=0).stop 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b214d1640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4b0c0a34b0 0x7f4b0c0a5970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b14005f00 tx=0x7f4b1400d380 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b2530d740 1 -- 172.21.15.103:0/1121276016 shutdown_connections 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a93f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a7ea10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1121276016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7fb40 0x5638b3a7e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b2530d740 1 -- 172.21.15.103:0/1121276016 >> 172.21.15.103:0/1121276016 conn(0x5638b3a47310 msgr2=0x5638b3a7ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b2530d740 1 -- 172.21.15.103:0/1121276016 shutdown_connections 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.800+0000 7f4b2530d740 1 -- 172.21.15.103:0/1121276016 wait complete. 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.801+0000 7f4b2530d740 1 -- start start 2024-09-19T07:43:49.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.801+0000 7f4b2530d740 1 Processor -- start 2024-09-19T07:43:49.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7fb40 0x5638b3a8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a8f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7fb40 0x5638b3a8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7fb40 0x5638b3a8e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57038/0 (socket says 172.21.15.103:57038) 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 -- 172.21.15.103:0/3352136738 learned_addr learned my addr 172.21.15.103:0/3352136738 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5638b3a94920 con 0x5638b3a91b80 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5638b399f060 con 0x5638b3a7fb40 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638b3a94aa0 con 0x5638b3a7e5d0 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b214d1640 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a8f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 10 client.0 _ll_register_callbacks cb 0x5638b36f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 -- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 msgr2=0x5638b3a915c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a915c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 -- 172.21.15.103:0/3352136738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 msgr2=0x5638b3a8f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a8f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b1bfff640 1 -- 172.21.15.103:0/3352136738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b37553c0 con 0x5638b3a7fb40 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.802+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1009cad0 con 0x5638b3a7fb40 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.803+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7fb40 0x5638b3a8e870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b08012720 tx=0x7f4b08012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19969cdf77d01301 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.803+0000 7f4afbfff640 10 client.26198 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.803+0000 7f4afbfff640 1 -- 172.21.15.103:0/3352136738 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800f920 con 0x5638b3a7fb40 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.803+0000 7f4afbfff640 1 -- 172.21.15.103:0/3352136738 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4b0800fac0 con 0x5638b3a7fb40 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.803+0000 7f4afbfff640 1 -- 172.21.15.103:0/3352136738 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4b0800fd00 con 0x5638b3a7fb40 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.803+0000 7f4b2530d740 10 client.26198 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.803+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=38}) -- 0x5638b3aa03f0 con 0x5638b3a7fb40 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.804+0000 7f4afbfff640 1 -- 172.21.15.103:0/3352136738 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f4b08026470 con 0x5638b3a7fb40 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.804+0000 7f4b2530d740 10 client.26198 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4b2530d740 10 client.26198 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:43:49.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4b2530d740 20 client.26198 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:49.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4b2530d740 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3aa3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4b2530d740 4 client.26198 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:49.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638b39f1860 con 0x5638b3aa17a0 2024-09-19T07:43:49.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3aa3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4afbfff640 10 client.26198 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:49.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.805+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3aa3bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15842f6cdc7ad43b server_cookie=d72d38cad725b8e3 in_seq=0 out_seq=0 2024-09-19T07:43:49.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.806+0000 7f4afbfff640 1 -- 172.21.15.103:0/3352136738 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638b39f1860 con 0x5638b3aa17a0 2024-09-19T07:43:49.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.806+0000 7f4afbfff640 10 client.26198 handle_command_reply: tid=0 2024-09-19T07:43:49.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.806+0000 7f4b2530d740 1 client.26198 shutdown 2024-09-19T07:43:49.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7fb40 msgr2=0x5638b3a8e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7fb40 0x5638b3a8e870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b08012720 tx=0x7f4b08012be0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 shutdown_connections 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3aa3bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a91b80 0x5638b3a8f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4afbfff640 0 client.26198 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7fb40 0x5638b3a8e870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 --2- 172.21.15.103:0/3352136738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a915c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 >> 172.21.15.103:0/3352136738 conn(0x5638b3a47310 msgr2=0x5638b3a80960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 shutdown_connections 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 1 -- 172.21.15.103:0/3352136738 wait complete. 2024-09-19T07:43:49.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.807+0000 7f4b2530d740 20 client.26198 trim_cache size 0 max 16384 2024-09-19T07:43:49.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:49 smithi134 ceph-mon[35414]: pgmap v178: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 791 op/s 2024-09-19T07:43:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.830+0000 7f4b2530d740 1 Processor -- start 2024-09-19T07:43:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.830+0000 7f4b2530d740 1 -- start start 2024-09-19T07:43:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3aa94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a91b80 0x5638b3aa99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b2530d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1ab7c0 con 0x5638b3a7fb40 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b2530d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1ad300 con 0x5638b3a7e5d0 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b2530d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1abe90 con 0x5638b3a91b80 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b214d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a91b80 0x5638b3aa99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a907e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57040/0 (socket says 172.21.15.103:57040) 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 -- 172.21.15.103:0/4157262913 learned_addr learned my addr 172.21.15.103:0/4157262913 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3aa94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 -- 172.21.15.103:0/4157262913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a91b80 msgr2=0x5638b3aa99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a91b80 0x5638b3aa99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 -- 172.21.15.103:0/4157262913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 msgr2=0x5638b3aa94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3aa94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b20cd0640 1 -- 172.21.15.103:0/4157262913 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b3a7aed0 con 0x5638b3a7e5d0 2024-09-19T07:43:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.831+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3aa94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.832+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a907e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b1001f000 tx=0x7f4b10098040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2642701c0e9366a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.832+0000 7f4b067fc640 1 -- 172.21.15.103:0/4157262913 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100d3c60 con 0x5638b3a7e5d0 2024-09-19T07:43:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.832+0000 7f4b067fc640 1 -- 172.21.15.103:0/4157262913 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5638b39f1860 con 0x5638b3a7e5d0 2024-09-19T07:43:49.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.832+0000 7f4b067fc640 1 -- 172.21.15.103:0/4157262913 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100056b0 con 0x5638b3a7e5d0 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 -- 172.21.15.103:0/4157262913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 msgr2=0x5638b3a907e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a907e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b1001f000 tx=0x7f4b10098040 comp rx=0 tx=0).stop 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 -- 172.21.15.103:0/4157262913 shutdown_connections 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a91b80 0x5638b3aa99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3aa94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 --2- 172.21.15.103:0/4157262913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a7e5d0 0x5638b3a907e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 -- 172.21.15.103:0/4157262913 >> 172.21.15.103:0/4157262913 conn(0x5638b3a47310 msgr2=0x5638b3a900d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 -- 172.21.15.103:0/4157262913 shutdown_connections 2024-09-19T07:43:49.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 -- 172.21.15.103:0/4157262913 wait complete. 2024-09-19T07:43:49.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 -- start start 2024-09-19T07:43:49.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.834+0000 7f4b2530d740 1 Processor -- start 2024-09-19T07:43:49.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a95dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3a96300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b2530d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a91b80 0x5638b3a96880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b2530d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1acc30 con 0x5638b3a7fb40 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b2530d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1ac560 con 0x5638b3a91b80 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b2530d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1c1a97c0 con 0x5638b3a7e5d0 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b2530d740 10 client.0 _ll_register_callbacks cb 0x7f4b08009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b20cd0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a95dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b214d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a91b80 0x5638b3a96880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3a96300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b214d1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a91b80 0x5638b3a96880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57056/0 (socket says 172.21.15.103:57056) 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b214d1640 1 -- 172.21.15.103:0/1583000898 learned_addr learned my addr 172.21.15.103:0/1583000898 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:49.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.835+0000 7f4b1bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3a96300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51684/0 (socket says 172.21.15.103:51684) 2024-09-19T07:43:49.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.836+0000 7f4b1bfff640 1 -- 172.21.15.103:0/1583000898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 msgr2=0x5638b3a95dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.836+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a95dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.836+0000 7f4b1bfff640 1 -- 172.21.15.103:0/1583000898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a91b80 msgr2=0x5638b3a96880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:49.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.836+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a91b80 0x5638b3a96880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.836+0000 7f4b1bfff640 1 -- 172.21.15.103:0/1583000898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b3b0b760 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.836+0000 7f4b1bfff640 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3a96300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x5638b3a76730 tx=0x7f4b080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442b7f0af696a6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.836+0000 7f4afbfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.837+0000 7f4afbfff640 1 -- 172.21.15.103:0/1583000898 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800fea0 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.837+0000 7f4afbfff640 1 -- 172.21.15.103:0/1583000898 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f4b0800f920 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.837+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638b39f1860 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.837+0000 7f4afbfff640 1 -- 172.21.15.103:0/1583000898 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4b0800fb20 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.837+0000 7f4b2530d740 10 client.16701 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.837+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5638b3a81310 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.837+0000 7f4afbfff640 1 -- 172.21.15.103:0/1583000898 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b1009cad0 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4afbfff640 1 -- 172.21.15.103:0/1583000898 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f4b0803f020 con 0x5638b3a7fb40 2024-09-19T07:43:49.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b2530d740 10 client.16701 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b2530d740 10 client.16701 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b2530d740 20 client.16701 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3a83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b2530d740 4 client.16701 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638b39f1860 con 0x5638b3aa17a0 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3a83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4b20cd0640 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3a83a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3483f4bcb48f50f server_cookie=c6c7981f988d73fe in_seq=0 out_seq=0 2024-09-19T07:43:49.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.838+0000 7f4afbfff640 10 client.16701 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:49.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.839+0000 7f4afbfff640 1 -- 172.21.15.103:0/1583000898 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638b39f1860 con 0x5638b3aa17a0 2024-09-19T07:43:49.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.839+0000 7f4afbfff640 10 client.16701 handle_command_reply: tid=0 2024-09-19T07:43:49.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.839+0000 7f4b2530d740 1 client.16701 shutdown 2024-09-19T07:43:49.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 msgr2=0x5638b3a96300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3a96300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x5638b3a76730 tx=0x7f4b080046c0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 shutdown_connections 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5638b3aa17a0 0x5638b3a83a40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638b3a91b80 0x5638b3a96880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638b3a7fb40 0x5638b3a96300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 --2- 172.21.15.103:0/1583000898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638b3a7e5d0 0x5638b3a95dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 >> 172.21.15.103:0/1583000898 conn(0x5638b3a47310 msgr2=0x5638b3a8cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.840+0000 7f4afbfff640 0 client.16701 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.841+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 shutdown_connections 2024-09-19T07:43:49.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.841+0000 7f4b2530d740 1 -- 172.21.15.103:0/1583000898 wait complete. 2024-09-19T07:43:49.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.843+0000 7f4b2530d740 20 client.16701 trim_cache size 0 max 16384 2024-09-19T07:43:49.845 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:43:49.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.844+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4b0c0a34b0 msgr2=0x7f4b0c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.844+0000 7f4b22f34640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4b0c0a34b0 0x7f4b0c0a5970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b14005f00 tx=0x7f4b1400d380 comp rx=0 tx=0).stop 2024-09-19T07:43:49.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.844+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c100e20 msgr2=0x7f4b1c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:49.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.844+0000 7f4b22f34640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c100e20 0x7f4b1c19a710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e9c0 tx=0x7f4b1000ee80 comp rx=0 tx=0).stop 2024-09-19T07:43:49.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 shutdown_connections 2024-09-19T07:43:49.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4b0c0a34b0 0x7f4b0c0a5970 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b1c102ef0 0x7f4b1c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b1c102570 0x7f4b1c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 --2- 172.21.15.103:0/2053800270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b1c100e20 0x7f4b1c19a710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:49.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 >> 172.21.15.103:0/2053800270 conn(0x7f4b1c0fc7d0 msgr2=0x7f4b1c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:49.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 shutdown_connections 2024-09-19T07:43:49.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:49.845+0000 7f4b22f34640 1 -- 172.21.15.103:0/2053800270 wait complete. 2024-09-19T07:43:49.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:49 smithi174 ceph-mon[35370]: pgmap v178: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 791 op/s 2024-09-19T07:43:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 315.0 s, 3408.0 tps, lat 29.388 ms stddev 52.999 2024-09-19T07:43:51.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:43:51.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:51 smithi103 ceph-mon[31201]: pgmap v179: 97 pgs: 97 active+clean; 3.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 69 MiB/s wr, 2.23k op/s 2024-09-19T07:43:51.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:51 smithi134 ceph-mon[35414]: pgmap v179: 97 pgs: 97 active+clean; 3.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 69 MiB/s wr, 2.23k op/s 2024-09-19T07:43:51.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:51 smithi174 ceph-mon[35370]: pgmap v179: 97 pgs: 97 active+clean; 3.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 69 MiB/s wr, 2.23k op/s 2024-09-19T07:43:52.060 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:52.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.719+0000 7f0d69ffb640 1 -- 172.21.15.103:0/380481858 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c030940 con 0x7f0d6c106a80 2024-09-19T07:43:52.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.720+0000 7f0d7168a640 1 -- 172.21.15.103:0/380481858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 msgr2=0x7f0d6c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.720+0000 7f0d7168a640 1 --2- 172.21.15.103:0/380481858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c106e60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009890 tx=0x7f0d5c02a370 comp rx=0 tx=0).stop 2024-09-19T07:43:52.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.720+0000 7f0d7168a640 1 -- 172.21.15.103:0/380481858 shutdown_connections 2024-09-19T07:43:52.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.720+0000 7f0d7168a640 1 --2- 172.21.15.103:0/380481858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6c101420 0x7f0d6c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.720+0000 7f0d7168a640 1 --2- 172.21.15.103:0/380481858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d6c100a80 0x7f0d6c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.720+0000 7f0d7168a640 1 --2- 172.21.15.103:0/380481858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c106e60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.720+0000 7f0d7168a640 1 -- 172.21.15.103:0/380481858 >> 172.21.15.103:0/380481858 conn(0x7f0d6c0fc7a0 msgr2=0x7f0d6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.726+0000 7f0d7168a640 1 -- 172.21.15.103:0/380481858 shutdown_connections 2024-09-19T07:43:52.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.727+0000 7f0d7168a640 1 -- 172.21.15.103:0/380481858 wait complete. 2024-09-19T07:43:52.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.728+0000 7f0d7168a640 1 Processor -- start 2024-09-19T07:43:52.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.729+0000 7f0d7168a640 1 -- start start 2024-09-19T07:43:52.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.729+0000 7f0d7168a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6c100a80 0x7f0d6c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.729+0000 7f0d7168a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d6c101420 0x7f0d6c06eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.729+0000 7f0d7168a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.729+0000 7f0d7168a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c06d4b0 con 0x7f0d6c106a80 2024-09-19T07:43:52.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.729+0000 7f0d7168a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c06d330 con 0x7f0d6c101420 2024-09-19T07:43:52.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.729+0000 7f0d7168a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6c06d630 con 0x7f0d6c100a80 2024-09-19T07:43:52.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.730+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.730+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c06f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51704/0 (socket says 172.21.15.103:51704) 2024-09-19T07:43:52.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.730+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2320540106 learned_addr learned my addr 172.21.15.103:0/2320540106 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:52.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.731+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2320540106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6c100a80 msgr2=0x7f0d6c073a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:43:52.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.731+0000 7f0d6b7fe640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6c100a80 0x7f0d6c073a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.731+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2320540106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d6c101420 msgr2=0x7f0d6c06eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.731+0000 7f0d6b7fe640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d6c101420 0x7f0d6c06eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.731+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2320540106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6c06f5e0 con 0x7f0d6c106a80 2024-09-19T07:43:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.731+0000 7f0d6b7fe640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c06f020 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0d6401fd30 tx=0x7f0d6401fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.733+0000 7f0d53fff640 1 -- 172.21.15.103:0/2320540106 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6401dbd0 con 0x7f0d6c106a80 2024-09-19T07:43:52.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.734+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6c06f8d0 con 0x7f0d6c106a80 2024-09-19T07:43:52.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.734+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6c10e860 con 0x7f0d6c106a80 2024-09-19T07:43:52.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.734+0000 7f0d53fff640 1 -- 172.21.15.103:0/2320540106 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d64003e50 con 0x7f0d6c106a80 2024-09-19T07:43:52.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.735+0000 7f0d53fff640 1 -- 172.21.15.103:0/2320540106 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64028440 con 0x7f0d6c106a80 2024-09-19T07:43:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.737+0000 7f0d73a63740 1 Processor -- start 2024-09-19T07:43:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.737+0000 7f0d73a63740 1 -- start start 2024-09-19T07:43:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.738+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e59b40 0x55e9c7e58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.738+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 0x55e9c7e58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.738+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e63370 0x55e9c7e65760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.738+0000 7f0d73a63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e9c7dbe2f0 con 0x55e9c7e63370 2024-09-19T07:43:52.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.738+0000 7f0d73a63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e9c7d65cf0 con 0x55e9c7e59b40 2024-09-19T07:43:52.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.738+0000 7f0d73a63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9c7dba2b0 con 0x55e9c7e585d0 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e59b40 0x55e9c7e58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e59b40 0x55e9c7e58090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57086/0 (socket says 172.21.15.103:57086) 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/4086656859 learned_addr learned my addr 172.21.15.103:0/4086656859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e63370 0x55e9c7e65760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/4086656859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 msgr2=0x55e9c7e58a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 0x55e9c7e58a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/4086656859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e63370 msgr2=0x55e9c7e65760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e63370 0x55e9c7e65760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.739+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/4086656859 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c7d4a3a0 con 0x55e9c7e59b40 2024-09-19T07:43:52.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.740+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e59b40 0x55e9c7e58090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000cc80 tx=0x7f0d6000c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903a868ca3c72999 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.741+0000 7f0d517fa640 1 -- 172.21.15.103:0/4086656859 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60010e40 con 0x55e9c7e59b40 2024-09-19T07:43:52.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.741+0000 7f0d517fa640 1 -- 172.21.15.103:0/4086656859 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6000cd70 con 0x55e9c7e59b40 2024-09-19T07:43:52.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.741+0000 7f0d517fa640 1 -- 172.21.15.103:0/4086656859 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60012480 con 0x55e9c7e59b40 2024-09-19T07:43:52.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.742+0000 7f0d73a63740 1 -- 172.21.15.103:0/4086656859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e59b40 msgr2=0x55e9c7e58090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.742+0000 7f0d73a63740 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e59b40 0x55e9c7e58090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000cc80 tx=0x7f0d6000c270 comp rx=0 tx=0).stop 2024-09-19T07:43:52.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.746+0000 7f0d73a63740 1 -- 172.21.15.103:0/4086656859 shutdown_connections 2024-09-19T07:43:52.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.746+0000 7f0d73a63740 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e63370 0x55e9c7e65760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.746+0000 7f0d73a63740 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 0x55e9c7e58a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.746+0000 7f0d73a63740 1 --2- 172.21.15.103:0/4086656859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e59b40 0x55e9c7e58090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.746+0000 7f0d73a63740 1 -- 172.21.15.103:0/4086656859 >> 172.21.15.103:0/4086656859 conn(0x55e9c7e21310 msgr2=0x55e9c7e54c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.746+0000 7f0d53fff640 1 -- 172.21.15.103:0/2320540106 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0d64004020 con 0x7f0d6c106a80 2024-09-19T07:43:52.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.747+0000 7f0d53fff640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d540a3580 0x7f0d540a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.748+0000 7f0d53fff640 1 -- 172.21.15.103:0/2320540106 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d640e0840 con 0x7f0d6c106a80 2024-09-19T07:43:52.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.750+0000 7f0d73a63740 1 -- 172.21.15.103:0/4086656859 shutdown_connections 2024-09-19T07:43:52.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.751+0000 7f0d6affd640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d540a3580 0x7f0d540a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.751+0000 7f0d73a63740 1 -- 172.21.15.103:0/4086656859 wait complete. 2024-09-19T07:43:52.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.751+0000 7f0d73a63740 1 -- start start 2024-09-19T07:43:52.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.751+0000 7f0d73a63740 1 Processor -- start 2024-09-19T07:43:52.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.752+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e585d0 0x55e9c7e62c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.752+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7e5ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.752+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e63370 0x55e9c7e60480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.752+0000 7f0d73a63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e9c7e66070 con 0x55e9c7e59b40 2024-09-19T07:43:52.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.752+0000 7f0d73a63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e9c7d79060 con 0x55e9c7e585d0 2024-09-19T07:43:52.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.752+0000 7f0d73a63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9c7e661f0 con 0x55e9c7e63370 2024-09-19T07:43:52.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.752+0000 7f0d73a63740 10 client.0 _ll_register_callbacks cb 0x55e9c7acc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:52.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e63370 0x55e9c7e60480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e63370 0x55e9c7e60480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:44022/0 (socket says 172.21.15.103:44022) 2024-09-19T07:43:52.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2387154731 learned_addr learned my addr 172.21.15.103:0/2387154731 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:52.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7e5ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2387154731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e585d0 msgr2=0x55e9c7e62c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:43:52.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e585d0 0x55e9c7e62c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2387154731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 msgr2=0x55e9c7e5ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:52.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7e5ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.753+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2387154731 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c7b2f3c0 con 0x55e9c7e63370 2024-09-19T07:43:52.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.754+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7e5ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:52.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.755+0000 7f0d6b7fe640 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e63370 0x55e9c7e60480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d640a9040 tx=0x7f0d640ad420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f62d43fe52e4b1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.759+0000 7f0d4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:43:52.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.759+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2387154731 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d640de3c0 con 0x55e9c7e63370 2024-09-19T07:43:52.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.759+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2387154731 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d640b53c0 con 0x55e9c7e63370 2024-09-19T07:43:52.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.759+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6000cd70 con 0x55e9c7e63370 2024-09-19T07:43:52.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.760+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2387154731 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0d640ca050 con 0x55e9c7e63370 2024-09-19T07:43:52.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.760+0000 7f0d73a63740 10 client.26128 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:52.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.760+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e9c7e71b20 con 0x55e9c7e63370 2024-09-19T07:43:52.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.761+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2387154731 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6000cd70 con 0x55e9c7e63370 2024-09-19T07:43:52.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.761+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2387154731 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f0d640a8930 con 0x55e9c7e63370 2024-09-19T07:43:52.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.762+0000 7f0d73a63740 10 client.26128 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:52.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.762+0000 7f0d73a63740 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:52.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.762+0000 7f0d73a63740 20 client.26128 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:52.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.762+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7e734a0 0x55e9c7e75920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.762+0000 7f0d73a63740 4 client.26128 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:52.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.762+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9c7dcb860 con 0x55e9c7e734a0 2024-09-19T07:43:52.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.765+0000 7f0d6affd640 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7e734a0 0x55e9c7e75920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.765+0000 7f0d6affd640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d540a3580 0x7f0d540a5a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0095f0 tx=0x7f0d5c030400 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.766+0000 7f0d6affd640 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7e734a0 0x55e9c7e75920 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ec783565e0802a server_cookie=a03d78a1c68234bd in_seq=0 out_seq=0 2024-09-19T07:43:52.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.767+0000 7f0d4eff5640 10 client.26128 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:52.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.767+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2387154731 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9c7dcb860 con 0x55e9c7e734a0 2024-09-19T07:43:52.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.767+0000 7f0d4eff5640 10 client.26128 handle_command_reply: tid=0 2024-09-19T07:43:52.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.767+0000 7f0d73a63740 1 client.26128 shutdown 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e63370 msgr2=0x55e9c7e60480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e63370 0x55e9c7e60480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d640a9040 tx=0x7f0d640ad420 comp rx=0 tx=0).stop 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 shutdown_connections 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7e734a0 0x55e9c7e75920 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e63370 0x55e9c7e60480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7e5ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2387154731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e585d0 0x55e9c7e62c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 >> 172.21.15.103:0/2387154731 conn(0x55e9c7e21310 msgr2=0x55e9c7e5a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d4eff5640 0 client.26128 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:52.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.771+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 shutdown_connections 2024-09-19T07:43:52.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.772+0000 7f0d73a63740 1 -- 172.21.15.103:0/2387154731 wait complete. 2024-09-19T07:43:52.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.776+0000 7f0d73a63740 20 client.26128 trim_cache size 0 max 16384 2024-09-19T07:43:52.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 Processor -- start 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 -- start start 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e585d0 0x55e9c7e6f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e59b40 0x55e9c7e54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e5b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5800e860 con 0x55e9c7e585d0 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580103a0 con 0x55e9c7e63370 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d73a63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5800ef30 con 0x55e9c7e59b40 2024-09-19T07:43:52.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e5b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.800+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e5b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57100/0 (socket says 172.21.15.103:57100) 2024-09-19T07:43:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.801+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/719394846 learned_addr learned my addr 172.21.15.103:0/719394846 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.801+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e59b40 0x55e9c7e54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.801+0000 7f0d6affd640 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e585d0 0x55e9c7e6f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/719394846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 msgr2=0x55e9c7e5b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e5b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/719394846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e585d0 msgr2=0x55e9c7e6f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e585d0 0x55e9c7e6f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/719394846 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c7e75fa0 con 0x55e9c7e59b40 2024-09-19T07:43:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e59b40 0x55e9c7e54d70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d60005890 tx=0x7f0d6000c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b44472eb22bb956 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d50ff9640 1 -- 172.21.15.103:0/719394846 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60027070 con 0x55e9c7e59b40 2024-09-19T07:43:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.802+0000 7f0d50ff9640 1 -- 172.21.15.103:0/719394846 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6000cd70 con 0x55e9c7e59b40 2024-09-19T07:43:52.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.803+0000 7f0d50ff9640 1 -- 172.21.15.103:0/719394846 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60012480 con 0x55e9c7e59b40 2024-09-19T07:43:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.804+0000 7f0d73a63740 1 -- 172.21.15.103:0/719394846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e59b40 msgr2=0x55e9c7e54d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.804+0000 7f0d73a63740 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e59b40 0x55e9c7e54d70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d60005890 tx=0x7f0d6000c560 comp rx=0 tx=0).stop 2024-09-19T07:43:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.804+0000 7f0d73a63740 1 -- 172.21.15.103:0/719394846 shutdown_connections 2024-09-19T07:43:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.804+0000 7f0d73a63740 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e5b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.804+0000 7f0d73a63740 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e59b40 0x55e9c7e54d70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.805+0000 7f0d73a63740 1 --2- 172.21.15.103:0/719394846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e585d0 0x55e9c7e6f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.805+0000 7f0d73a63740 1 -- 172.21.15.103:0/719394846 >> 172.21.15.103:0/719394846 conn(0x55e9c7e21310 msgr2=0x55e9c7e61fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.805+0000 7f0d73a63740 1 -- 172.21.15.103:0/719394846 shutdown_connections 2024-09-19T07:43:52.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.806+0000 7f0d73a63740 1 -- 172.21.15.103:0/719394846 wait complete. 2024-09-19T07:43:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.806+0000 7f0d73a63740 1 -- start start 2024-09-19T07:43:52.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.806+0000 7f0d73a63740 1 Processor -- start 2024-09-19T07:43:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 0x55e9c7ed6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7ed6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d73a63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e66ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d73a63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5800fcd0 con 0x55e9c7e59b40 2024-09-19T07:43:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d73a63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5800f600 con 0x55e9c7e63370 2024-09-19T07:43:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d73a63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5800c840 con 0x55e9c7e585d0 2024-09-19T07:43:52.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d73a63740 10 client.0 _ll_register_callbacks cb 0x7f0d640a8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e66ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e66ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57116/0 (socket says 172.21.15.103:57116) 2024-09-19T07:43:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.807+0000 7f0d6b7fe640 1 -- 172.21.15.103:0/2427645108 learned_addr learned my addr 172.21.15.103:0/2427645108 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.808+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7ed6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.808+0000 7f0d6affd640 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 0x55e9c7ed6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.808+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/2427645108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 msgr2=0x55e9c7ed6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.808+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 0x55e9c7ed6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.808+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/2427645108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 msgr2=0x55e9c7e66ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.808+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e66ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.808+0000 7f0d6a7fc640 1 -- 172.21.15.103:0/2427645108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c7e672b0 con 0x55e9c7e59b40 2024-09-19T07:43:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.809+0000 7f0d6a7fc640 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7ed6aa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55e9c7e627f0 tx=0x7f0d600044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e9bc14dad193d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:52.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.809+0000 7f0d4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:43:52.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.809+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2427645108 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60027030 con 0x55e9c7e59b40 2024-09-19T07:43:52.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.809+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2427645108 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d60009050 con 0x55e9c7e59b40 2024-09-19T07:43:52.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.809+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6000cd70 con 0x55e9c7e59b40 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.810+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2427645108 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0d60003ea0 con 0x55e9c7e59b40 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.810+0000 7f0d73a63740 10 client.16716 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.810+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e9c7eddd00 con 0x55e9c7e59b40 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2427645108 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6000cd70 con 0x55e9c7e59b40 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2427645108 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f0d6000f040 con 0x55e9c7e59b40 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d73a63740 10 client.16716 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d73a63740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d73a63740 20 client.16716 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7ede040 0x55e9c7ee0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d73a63740 4 client.16716 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9c7dc5d70 con 0x55e9c7ede040 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.811+0000 7f0d6affd640 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7ede040 0x55e9c7ee0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:52.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.812+0000 7f0d6affd640 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7ede040 0x55e9c7ee0430 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b778843e3484fef4 server_cookie=688d0107e04b17cd in_seq=0 out_seq=0 2024-09-19T07:43:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.812+0000 7f0d4eff5640 10 client.16716 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.812+0000 7f0d4eff5640 1 -- 172.21.15.103:0/2427645108 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9c7dc5d70 con 0x55e9c7ede040 2024-09-19T07:43:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.812+0000 7f0d4eff5640 10 client.16716 handle_command_reply: tid=0 2024-09-19T07:43:52.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.812+0000 7f0d73a63740 1 client.16716 shutdown 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.814+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 msgr2=0x55e9c7ed6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.814+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7ed6aa0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55e9c7e627f0 tx=0x7f0d600044f0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 shutdown_connections 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e9c7ede040 0x55e9c7ee0430 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9c7e63370 0x55e9c7e66ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9c7e59b40 0x55e9c7ed6aa0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 --2- 172.21.15.103:0/2427645108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9c7e585d0 0x55e9c7ed6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 >> 172.21.15.103:0/2427645108 conn(0x55e9c7e21310 msgr2=0x55e9c7e5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d4eff5640 0 client.16716 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:52.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 shutdown_connections 2024-09-19T07:43:52.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 1 -- 172.21.15.103:0/2427645108 wait complete. 2024-09-19T07:43:52.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.815+0000 7f0d73a63740 20 client.16716 trim_cache size 0 max 16384 2024-09-19T07:43:52.817 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:43:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.816+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d540a3580 msgr2=0x7f0d540a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d540a3580 0x7f0d540a5a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0095f0 tx=0x7f0d5c030400 comp rx=0 tx=0).stop 2024-09-19T07:43:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 msgr2=0x7f0d6c06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c06f020 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0d6401fd30 tx=0x7f0d6401fe20 comp rx=0 tx=0).stop 2024-09-19T07:43:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 shutdown_connections 2024-09-19T07:43:52.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d540a3580 0x7f0d540a5a40 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c0095f0 tx=0x7f0d5c030400 comp rx=0 tx=0).stop 2024-09-19T07:43:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d6c106a80 0x7f0d6c06f020 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d6c101420 0x7f0d6c06eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 --2- 172.21.15.103:0/2320540106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d6c100a80 0x7f0d6c073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:52.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.817+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 >> 172.21.15.103:0/2320540106 conn(0x7f0d6c0fc7a0 msgr2=0x7f0d6c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.819+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 shutdown_connections 2024-09-19T07:43:52.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:52.819+0000 7f0d7168a640 1 -- 172.21.15.103:0/2320540106 wait complete. 2024-09-19T07:43:53.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:53 smithi103 ceph-mon[31201]: pgmap v180: 97 pgs: 97 active+clean; 3.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 49 MiB/s wr, 1.77k op/s 2024-09-19T07:43:53.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:53 smithi134 ceph-mon[35414]: pgmap v180: 97 pgs: 97 active+clean; 3.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 49 MiB/s wr, 1.77k op/s 2024-09-19T07:43:53.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:53 smithi174 ceph-mon[35370]: pgmap v180: 97 pgs: 97 active+clean; 3.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 49 MiB/s wr, 1.77k op/s 2024-09-19T07:43:55.082 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:43:55.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:43:55.467 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:55 smithi103 ceph-mon[31201]: pgmap v181: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 56 MiB/s wr, 2.41k op/s 2024-09-19T07:43:55.468 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 320.0 s, 3293.4 tps, lat 30.216 ms stddev 41.087 2024-09-19T07:43:55.483 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:55.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:55 smithi134 ceph-mon[35414]: pgmap v181: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 56 MiB/s wr, 2.41k op/s 2024-09-19T07:43:55.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:55 smithi174 ceph-mon[35370]: pgmap v181: 97 pgs: 97 active+clean; 3.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 56 MiB/s wr, 2.41k op/s 2024-09-19T07:43:56.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 -- 172.21.15.103:0/2071412669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074eb0 msgr2=0x7f8354112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 --2- 172.21.15.103:0/2071412669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074eb0 0x7f8354112640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f833c009900 tx=0x7f833c02a490 comp rx=0 tx=0).stop 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 -- 172.21.15.103:0/2071412669 shutdown_connections 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 --2- 172.21.15.103:0/2071412669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 0x7f8354114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 --2- 172.21.15.103:0/2071412669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074eb0 0x7f8354112640 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 --2- 172.21.15.103:0/2071412669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354074590 0x7f8354074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 -- 172.21.15.103:0/2071412669 >> 172.21.15.103:0/2071412669 conn(0x7f83540fd8b0 msgr2=0x7f83540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 -- 172.21.15.103:0/2071412669 shutdown_connections 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.040+0000 7f835952d640 1 -- 172.21.15.103:0/2071412669 wait complete. 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.041+0000 7f835952d640 1 Processor -- start 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.041+0000 7f835952d640 1 -- start start 2024-09-19T07:43:56.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f835952d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074590 0x7f83541a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f835952d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354074eb0 0x7f83541a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f835952d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 0x7f835419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f835952d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8354118530 con 0x7f8354074590 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f835952d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83541183b0 con 0x7f8354112b80 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f83537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 0x7f835419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f83537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 0x7f835419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57140/0 (socket says 172.21.15.103:57140) 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f83537fe640 1 -- 172.21.15.103:0/3393333842 learned_addr learned my addr 172.21.15.103:0/3393333842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f835952d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83541186b0 con 0x7f8354074eb0 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f8352ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074590 0x7f83541a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.042+0000 7f834bfff640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354074eb0 0x7f83541a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.043+0000 7f83537fe640 1 -- 172.21.15.103:0/3393333842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354074eb0 msgr2=0x7f83541a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.043+0000 7f83537fe640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354074eb0 0x7f83541a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.043+0000 7f83537fe640 1 -- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074590 msgr2=0x7f83541a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.043+0000 7f83537fe640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074590 0x7f83541a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.043+0000 7f83537fe640 1 -- 172.21.15.103:0/3393333842 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835419eff0 con 0x7f8354112b80 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.043+0000 7f83537fe640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 0x7f835419e850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f834c00e9d0 tx=0x7f834c00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.044+0000 7f8350ff9640 1 -- 172.21.15.103:0/3393333842 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c004d20 con 0x7f8354112b80 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.044+0000 7f8350ff9640 1 -- 172.21.15.103:0/3393333842 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c005020 con 0x7f8354112b80 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.044+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835419f2e0 con 0x7f8354112b80 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.044+0000 7f8350ff9640 1 -- 172.21.15.103:0/3393333842 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c0116d0 con 0x7f8354112b80 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.044+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83541ab530 con 0x7f8354112b80 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.047+0000 7f8350ff9640 1 -- 172.21.15.103:0/3393333842 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f834c005560 con 0x7f8354112b80 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.048+0000 7f8350ff9640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a3500 0x7f83400a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.048+0000 7f835b906740 1 Processor -- start 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.048+0000 7f835b906740 1 -- start start 2024-09-19T07:43:56.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.048+0000 7f8350ff9640 1 -- 172.21.15.103:0/3393333842 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f834c0cfce0 con 0x7f8354112b80 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.048+0000 7f835b906740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bcaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.048+0000 7f835b906740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcb480 0x557dd5bd5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.048+0000 7f835b906740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f835b906740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dd5b30280 con 0x557dd5bcab60 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f835b906740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557dd5ad7b10 con 0x557dd5bd5d40 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f835b906740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557dd5b2c240 con 0x557dd5bcb480 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:57146/0 (socket says 172.21.15.103:57146) 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 -- 172.21.15.103:0/3420811134 learned_addr learned my addr 172.21.15.103:0/3420811134 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 -- 172.21.15.103:0/3420811134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcb480 msgr2=0x557dd5bd5800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 --2- 172.21.15.103:0/3420811134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcb480 0x557dd5bd5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 -- 172.21.15.103:0/3420811134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 msgr2=0x557dd5bcaf40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:43:56.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 --2- 172.21.15.103:0/3420811134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bcaf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.049+0000 7f83537fe640 1 -- 172.21.15.103:0/3420811134 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd5ae41f0 con 0x557dd5bd5d40 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.050+0000 7f83537fe640 1 --2- 172.21.15.103:0/3420811134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd8130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f834c098430 tx=0x7f834c004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3042b65b1699fc1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.050+0000 7f8352ffd640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a3500 0x7f83400a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.050+0000 7f83497fa640 1 -- 172.21.15.103:0/3420811134 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c01f040 con 0x557dd5bd5d40 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.050+0000 7f83497fa640 1 -- 172.21.15.103:0/3420811134 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c09c6d0 con 0x557dd5bd5d40 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.050+0000 7f83497fa640 1 -- 172.21.15.103:0/3420811134 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c098640 con 0x557dd5bd5d40 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 -- 172.21.15.103:0/3420811134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 msgr2=0x557dd5bd8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 --2- 172.21.15.103:0/3420811134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd8130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f834c098430 tx=0x7f834c004fb0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 -- 172.21.15.103:0/3420811134 shutdown_connections 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 --2- 172.21.15.103:0/3420811134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd8130 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 --2- 172.21.15.103:0/3420811134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcb480 0x557dd5bd5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 --2- 172.21.15.103:0/3420811134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bcaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 -- 172.21.15.103:0/3420811134 >> 172.21.15.103:0/3420811134 conn(0x557dd5bc70f0 msgr2=0x557dd5bc74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f8352ffd640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a3500 0x7f83400a59c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f834400ab80 tx=0x7f834400a2d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.051+0000 7f835b906740 1 -- 172.21.15.103:0/3420811134 shutdown_connections 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.052+0000 7f835b906740 1 -- 172.21.15.103:0/3420811134 wait complete. 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.052+0000 7f835b906740 1 -- start start 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.052+0000 7f835b906740 1 Processor -- start 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f835b906740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcab60 0x557dd5bd2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f835b906740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcb480 0x557dd5bd3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f835b906740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f835b906740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dd5bd8910 con 0x557dd5bcb480 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f835b906740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557dd5aeaf10 con 0x557dd5bd5d40 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f835b906740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557dd5bd8a90 con 0x557dd5bcab60 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f835b906740 10 client.0 _ll_register_callbacks cb 0x557dd5ab3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:56.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcab60 0x557dd5bd2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcab60 0x557dd5bd2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:44080/0 (socket says 172.21.15.103:44080) 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 -- 172.21.15.103:0/3881815108 learned_addr learned my addr 172.21.15.103:0/3881815108 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f834bfff640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcb480 0x557dd5bd3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f83537fe640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 -- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 msgr2=0x557dd5bd3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 -- 172.21.15.103:0/3881815108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcb480 msgr2=0x557dd5bd3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcb480 0x557dd5bd3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f8352ffd640 1 -- 172.21.15.103:0/3881815108 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd58a16a0 con 0x557dd5bcab60 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.053+0000 7f834bfff640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcb480 0x557dd5bd3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.054+0000 7f8352ffd640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcab60 0x557dd5bd2c00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8344010a90 tx=0x7f8344004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777f92bc52f8a2cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.054+0000 7f833a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.054+0000 7f833a7fc640 1 -- 172.21.15.103:0/3881815108 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344016070 con 0x557dd5bcab60 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.054+0000 7f833a7fc640 1 -- 172.21.15.103:0/3881815108 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f834401bd40 con 0x557dd5bcab60 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.054+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834c09c6d0 con 0x557dd5bcab60 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.055+0000 7f833a7fc640 1 -- 172.21.15.103:0/3881815108 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8344027480 con 0x557dd5bcab60 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.055+0000 7f835b906740 10 client.26140 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.055+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557dd5be4550 con 0x557dd5bcab60 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f833a7fc640 1 -- 172.21.15.103:0/3881815108 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c09c6d0 con 0x557dd5bcab60 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f833a7fc640 1 -- 172.21.15.103:0/3881815108 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f83440308f0 con 0x557dd5bcab60 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f835b906740 10 client.26140 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f835b906740 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f835b906740 20 client.26140 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f835b906740 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5be5960 0x557dd5be7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f835b906740 4 client.26140 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:56.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dd5b3d7f0 con 0x557dd5be5960 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.056+0000 7f834bfff640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5be5960 0x557dd5be7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.057+0000 7f834bfff640 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5be5960 0x557dd5be7d50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2ecacacd4cdd37 server_cookie=580f0d3e2822307e in_seq=0 out_seq=0 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.057+0000 7f833a7fc640 10 client.26140 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.058+0000 7f833a7fc640 1 -- 172.21.15.103:0/3881815108 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dd5b3d7f0 con 0x557dd5be5960 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.058+0000 7f833a7fc640 10 client.26140 handle_command_reply: tid=0 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.058+0000 7f835b906740 1 client.26140 shutdown 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcab60 msgr2=0x557dd5bd2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcab60 0x557dd5bd2c00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8344010a90 tx=0x7f8344004730 comp rx=0 tx=0).stop 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 shutdown_connections 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5be5960 0x557dd5be7d50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bd5d40 0x557dd5bd3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcb480 0x557dd5bd3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 --2- 172.21.15.103:0/3881815108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bcab60 0x557dd5bd2c00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 >> 172.21.15.103:0/3881815108 conn(0x557dd5bc70f0 msgr2=0x557dd5bcce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f833a7fc640 0 client.26140 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 shutdown_connections 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 1 -- 172.21.15.103:0/3881815108 wait complete. 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.059+0000 7f835b906740 20 client.26140 trim_cache size 0 max 16384 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.092+0000 7f835b906740 1 Processor -- start 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.093+0000 7f835b906740 1 -- start start 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.093+0000 7f835b906740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bd4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.093+0000 7f835b906740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.093+0000 7f835b906740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5beda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.093+0000 7f835b906740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b3fa0 con 0x557dd5bcab60 2024-09-19T07:43:56.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.093+0000 7f835b906740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b5ae0 con 0x557dd5bcb480 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.093+0000 7f835b906740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b4670 con 0x557dd5bd5d40 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.094+0000 7f8352ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bd4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.094+0000 7f8352ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bd4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51784/0 (socket says 172.21.15.103:51784) 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.094+0000 7f8352ffd640 1 -- 172.21.15.103:0/1773466047 learned_addr learned my addr 172.21.15.103:0/1773466047 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.094+0000 7f83537fe640 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5beda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.095+0000 7f83537fe640 1 -- 172.21.15.103:0/1773466047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 msgr2=0x557dd5bed530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.095+0000 7f83537fe640 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.095+0000 7f83537fe640 1 -- 172.21.15.103:0/1773466047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 msgr2=0x557dd5bd4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.095+0000 7f83537fe640 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bd4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.095+0000 7f83537fe640 1 -- 172.21.15.103:0/1773466047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd5bd54d0 con 0x557dd5bd5d40 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.096+0000 7f83537fe640 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5beda70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f834c09c880 tx=0x7f834c0054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=830d210d99b7c85f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.096+0000 7f8348ff9640 1 -- 172.21.15.103:0/1773466047 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c01f070 con 0x557dd5bd5d40 2024-09-19T07:43:56.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.097+0000 7f8348ff9640 1 -- 172.21.15.103:0/1773466047 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c09bfd0 con 0x557dd5bd5d40 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.097+0000 7f8348ff9640 1 -- 172.21.15.103:0/1773466047 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c0cdb90 con 0x557dd5bd5d40 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.098+0000 7f835b906740 1 -- 172.21.15.103:0/1773466047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 msgr2=0x557dd5beda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.098+0000 7f835b906740 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5beda70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f834c09c880 tx=0x7f834c0054d0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.098+0000 7f835b906740 1 -- 172.21.15.103:0/1773466047 shutdown_connections 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.098+0000 7f835b906740 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5beda70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.098+0000 7f835b906740 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.098+0000 7f835b906740 1 --2- 172.21.15.103:0/1773466047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bd4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.098+0000 7f835b906740 1 -- 172.21.15.103:0/1773466047 >> 172.21.15.103:0/1773466047 conn(0x557dd5bc70f0 msgr2=0x557dd5bd4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.101+0000 7f835b906740 1 -- 172.21.15.103:0/1773466047 shutdown_connections 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.102+0000 7f835b906740 1 -- 172.21.15.103:0/1773466047 wait complete. 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.103+0000 7f835b906740 1 -- start start 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.103+0000 7f835b906740 1 Processor -- start 2024-09-19T07:43:56.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.103+0000 7f835b906740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.104+0000 7f835b906740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bde220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.104+0000 7f835b906740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5bdacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.104+0000 7f835b906740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b5410 con 0x557dd5bcab60 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.104+0000 7f835b906740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b4d40 con 0x557dd5bcb480 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.104+0000 7f835b906740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b1f80 con 0x557dd5bd5d40 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.104+0000 7f835b906740 10 client.0 _ll_register_callbacks cb 0x7f834400a850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f83537fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5bdacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f83537fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5bdacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:44104/0 (socket says 172.21.15.103:44104) 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f83537fe640 1 -- 172.21.15.103:0/3090091078 learned_addr learned my addr 172.21.15.103:0/3090091078 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f834bfff640 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bde220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f834bfff640 1 -- 172.21.15.103:0/3090091078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 msgr2=0x557dd5bdacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f834bfff640 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5bdacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f834bfff640 1 -- 172.21.15.103:0/3090091078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 msgr2=0x557dd5bddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f834bfff640 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.105+0000 7f834bfff640 1 -- 172.21.15.103:0/3090091078 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd5bde8e0 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.107+0000 7f834bfff640 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bde220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f833c005940 tx=0x7f833c031990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddf86af948900ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.108+0000 7f833a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.108+0000 7f833a7fc640 1 -- 172.21.15.103:0/3090091078 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c035040 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.108+0000 7f833a7fc640 1 -- 172.21.15.103:0/3090091078 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f833c004250 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.108+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834c09bfd0 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.109+0000 7f833a7fc640 1 -- 172.21.15.103:0/3090091078 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f833c051050 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.110+0000 7f835b906740 10 client.26228 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.110+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557dd5c50b40 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f833a7fc640 1 -- 172.21.15.103:0/3090091078 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834c09bfd0 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f833a7fc640 1 -- 172.21.15.103:0/3090091078 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f833c0433e0 con 0x557dd5bcb480 2024-09-19T07:43:56.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f835b906740 10 client.26228 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f835b906740 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f835b906740 20 client.26228 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f835b906740 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5c512a0 0x557dd5c536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f835b906740 4 client.26228 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.112+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557dd5b37d00 con 0x557dd5c512a0 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.113+0000 7f8352ffd640 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5c512a0 0x557dd5c536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.120+0000 7f8352ffd640 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5c512a0 0x557dd5c536c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628fdfb59d12384f server_cookie=1460894cca2a6dd0 in_seq=0 out_seq=0 2024-09-19T07:43:56.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.120+0000 7f833a7fc640 10 client.26228 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:56.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.122+0000 7f833a7fc640 1 -- 172.21.15.103:0/3090091078 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557dd5b37d00 con 0x557dd5c512a0 2024-09-19T07:43:56.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.123+0000 7f833a7fc640 10 client.26228 handle_command_reply: tid=0 2024-09-19T07:43:56.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.123+0000 7f835b906740 1 client.26228 shutdown 2024-09-19T07:43:56.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 msgr2=0x557dd5bde220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bde220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f833c005940 tx=0x7f833c031990 comp rx=0 tx=0).stop 2024-09-19T07:43:56.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 shutdown_connections 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557dd5c512a0 0x557dd5c536c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557dd5bd5d40 0x557dd5bdacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557dd5bcb480 0x557dd5bde220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 --2- 172.21.15.103:0/3090091078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd5bcab60 0x557dd5bddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 >> 172.21.15.103:0/3090091078 conn(0x557dd5bc70f0 msgr2=0x557dd5bc6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f833a7fc640 0 client.26228 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 shutdown_connections 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.126+0000 7f835b906740 1 -- 172.21.15.103:0/3090091078 wait complete. 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.127+0000 7f835b906740 20 client.26228 trim_cache size 0 max 16384 2024-09-19T07:43:56.140 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "0ca0830c-33c6-4aee-aae9-56590866a199", 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a3500 msgr2=0x7f83400a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a3500 0x7f83400a59c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f834400ab80 tx=0x7f834400a2d0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 msgr2=0x7f835419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 0x7f835419e850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f834c00e9d0 tx=0x7f834c00edd0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 shutdown_connections 2024-09-19T07:43:56.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a3500 0x7f83400a59c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8354112b80 0x7f835419e850 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354074eb0 0x7f83541a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 --2- 172.21.15.103:0/3393333842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354074590 0x7f83541a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:56.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 >> 172.21.15.103:0/3393333842 conn(0x7f83540fd8b0 msgr2=0x7f83540ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:56.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 shutdown_connections 2024-09-19T07:43:56.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:56.128+0000 7f835952d640 1 -- 172.21.15.103:0/3393333842 wait complete. 2024-09-19T07:43:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:56 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:56 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:56.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:56 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:56.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:56 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:56.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:56 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:56.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:56 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:43:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:43:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:57 smithi134 ceph-mon[35414]: pgmap v182: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 46 MiB/s wr, 2.30k op/s 2024-09-19T07:43:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:43:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:57.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:43:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:57 smithi174 ceph-mon[35370]: pgmap v182: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 46 MiB/s wr, 2.30k op/s 2024-09-19T07:43:57.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:43:57.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:57.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:57.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:43:57.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:43:57.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:57 smithi103 ceph-mon[31201]: pgmap v182: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 46 MiB/s wr, 2.30k op/s 2024-09-19T07:43:58.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:43:58.917 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:43:59.653 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:43:59 smithi103 ceph-mon[31201]: pgmap v183: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 46 MiB/s wr, 2.30k op/s 2024-09-19T07:43:59.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.662+0000 7f9e6c02a640 1 -- 172.21.15.103:0/662172655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64109ba0 msgr2=0x7f9e64114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.662+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/662172655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64109ba0 0x7f9e64114570 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402a440 comp rx=0 tx=0).stop 2024-09-19T07:43:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.663+0000 7f9e6c02a640 1 -- 172.21.15.103:0/662172655 shutdown_connections 2024-09-19T07:43:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.663+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/662172655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e64114ab0 0x7f9e64116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.663+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/662172655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64109ba0 0x7f9e64114570 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.663+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/662172655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e641091f0 0x7f9e641095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.663+0000 7f9e6c02a640 1 -- 172.21.15.103:0/662172655 >> 172.21.15.103:0/662172655 conn(0x7f9e6406e2d0 msgr2=0x7f9e6406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.670+0000 7f9e6c02a640 1 -- 172.21.15.103:0/662172655 shutdown_connections 2024-09-19T07:43:59.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.671+0000 7f9e6c02a640 1 -- 172.21.15.103:0/662172655 wait complete. 2024-09-19T07:43:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.672+0000 7f9e6c02a640 1 Processor -- start 2024-09-19T07:43:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.672+0000 7f9e6c02a640 1 -- start start 2024-09-19T07:43:59.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.674+0000 7f9e6c02a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e641091f0 0x7f9e641a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.674+0000 7f9e6c02a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e64109ba0 0x7f9e641a3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.674+0000 7f9e6c02a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e64114ab0 0x7f9e641a7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.674+0000 7f9e6c02a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6411a820 con 0x7f9e641091f0 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.674+0000 7f9e6c02a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6411a6a0 con 0x7f9e64109ba0 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.674+0000 7f9e6c02a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6411a9a0 con 0x7f9e64114ab0 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e695c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e64109ba0 0x7f9e641a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e695c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e64109ba0 0x7f9e641a3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:51406/0 (socket says 172.21.15.103:51406) 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e695c5640 1 -- 172.21.15.103:0/1613922166 learned_addr learned my addr 172.21.15.103:0/1613922166 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e64114ab0 0x7f9e641a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e641091f0 0x7f9e641a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/1613922166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e64109ba0 msgr2=0x7f9e641a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e64109ba0 0x7f9e641a3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e641091f0 msgr2=0x7f9e641a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e641091f0 0x7f9e641a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.675+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/1613922166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e641a7e10 con 0x7f9e64114ab0 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.676+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e64114ab0 0x7f9e641a7690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c005fd0 tx=0x7f9e5c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.676+0000 7f9e5affd640 1 -- 172.21.15.103:0/1613922166 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00cdf0 con 0x7f9e64114ab0 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.676+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e641a8100 con 0x7f9e64114ab0 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.676+0000 7f9e5affd640 1 -- 172.21.15.103:0/1613922166 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5c005020 con 0x7f9e64114ab0 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.676+0000 7f9e5affd640 1 -- 172.21.15.103:0/1613922166 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c010690 con 0x7f9e64114ab0 2024-09-19T07:43:59.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.676+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6410c510 con 0x7f9e64114ab0 2024-09-19T07:43:59.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.680+0000 7f9e5affd640 1 -- 172.21.15.103:0/1613922166 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9e5c0054d0 con 0x7f9e64114ab0 2024-09-19T07:43:59.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.681+0000 7f9e5affd640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9e500a3480 0x7f9e500a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.682+0000 7f9e5affd640 1 -- 172.21.15.103:0/1613922166 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9e5c0d0370 con 0x7f9e64114ab0 2024-09-19T07:43:59.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.682+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9e500a3480 0x7f9e500a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.682+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9e500a3480 0x7f9e500a5940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000bbe0 tx=0x7f9e6000b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.683+0000 7f9e6e403740 1 Processor -- start 2024-09-19T07:43:59.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.683+0000 7f9e6e403740 1 -- start start 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.683+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda45880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.683+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda50140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda50680 0x5645eda52a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6e403740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5645ed9ab2e0 con 0x5645eda50680 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6e403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5645ed952ce0 con 0x5645eda454a0 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6e403740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645ed9a72a0 con 0x5645eda45dc0 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda50680 0x5645eda52a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda50680 0x5645eda52a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32774/0 (socket says 172.21.15.103:32774) 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/1787566436 learned_addr learned my addr 172.21.15.103:0/1787566436 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e695c5640 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda50140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda45880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/1787566436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 msgr2=0x5645eda50140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda50140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/1787566436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 msgr2=0x5645eda45880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda45880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/1787566436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ed937440 con 0x5645eda50680 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda45880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.684+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda50680 0x5645eda52a70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0a4da0 tx=0x7f9e5c09ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba5fca82e4602b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.685+0000 7f9e4ffff640 1 -- 172.21.15.103:0/1787566436 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0a52d0 con 0x5645eda50680 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.685+0000 7f9e4ffff640 1 -- 172.21.15.103:0/1787566436 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5c0051d0 con 0x5645eda50680 2024-09-19T07:43:59.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.685+0000 7f9e4ffff640 1 -- 172.21.15.103:0/1787566436 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0a50a0 con 0x5645eda50680 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.686+0000 7f9e6e403740 1 -- 172.21.15.103:0/1787566436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda50680 msgr2=0x5645eda52a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.686+0000 7f9e6e403740 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda50680 0x5645eda52a70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0a4da0 tx=0x7f9e5c09ce20 comp rx=0 tx=0).stop 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.686+0000 7f9e6e403740 1 -- 172.21.15.103:0/1787566436 shutdown_connections 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.686+0000 7f9e6e403740 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda50680 0x5645eda52a70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.686+0000 7f9e6e403740 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda50140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.686+0000 7f9e6e403740 1 --2- 172.21.15.103:0/1787566436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda45880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.686+0000 7f9e6e403740 1 -- 172.21.15.103:0/1787566436 >> 172.21.15.103:0/1787566436 conn(0x5645eda41a30 msgr2=0x5645eda41e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.687+0000 7f9e6e403740 1 -- 172.21.15.103:0/1787566436 shutdown_connections 2024-09-19T07:43:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.687+0000 7f9e6e403740 1 -- 172.21.15.103:0/1787566436 wait complete. 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.687+0000 7f9e6e403740 1 -- start start 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.687+0000 7f9e6e403740 1 Processor -- start 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda4ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda50680 0x5645eda4d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6e403740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5645eda53250 con 0x5645eda45dc0 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6e403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5645ed966050 con 0x5645eda454a0 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6e403740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645eda533d0 con 0x5645eda50680 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e69dc6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda4ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e69dc6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda4ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:51422/0 (socket says 172.21.15.103:51422) 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6e403740 10 client.0 _ll_register_callbacks cb 0x5645ed9a6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e69dc6640 1 -- 172.21.15.103:0/7585055 learned_addr learned my addr 172.21.15.103:0/7585055 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:59.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e695c5640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda50680 0x5645eda4d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e69dc6640 1 -- 172.21.15.103:0/7585055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda50680 msgr2=0x5645eda4d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.688+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda50680 0x5645eda4d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e69dc6640 1 -- 172.21.15.103:0/7585055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 msgr2=0x5645eda4d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e69dc6640 1 -- 172.21.15.103:0/7585055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ed71c3c0 con 0x5645eda454a0 2024-09-19T07:43:59.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e695c5640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda4ff80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e600116e0 tx=0x7f9e60004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7592989b8079704b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/7585055 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6001da00 con 0x5645eda454a0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/7585055 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9e6001dba0 con 0x5645eda454a0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e5c0051d0 con 0x5645eda454a0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/7585055 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9e6001dda0 con 0x5645eda454a0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e6e403740 10 client.26237 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.689+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5645eda5eb90 con 0x5645eda454a0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.690+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/7585055 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6002f930 con 0x5645eda454a0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.690+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/7585055 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f9e6001e7f0 con 0x5645eda454a0 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.691+0000 7f9e6e403740 10 client.26237 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.691+0000 7f9e6e403740 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:59.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.691+0000 7f9e6e403740 20 client.26237 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.691+0000 7f9e6e403740 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda60050 0x5645eda62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.691+0000 7f9e6e403740 4 client.26237 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:43:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.691+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e5c0051d0 con 0x5645eda60050 2024-09-19T07:43:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.692+0000 7f9e695c5640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda60050 0x5645eda62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.692+0000 7f9e695c5640 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda60050 0x5645eda62460 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82924fe29d2e5073 server_cookie=e7e0992cf5520dfe in_seq=0 out_seq=0 2024-09-19T07:43:59.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.692+0000 7f9e4d7fa640 10 client.26237 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:59.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/7585055 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9e5c0051d0 con 0x5645eda60050 2024-09-19T07:43:59.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e4d7fa640 10 client.26237 handle_command_reply: tid=0 2024-09-19T07:43:59.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 client.26237 shutdown 2024-09-19T07:43:59.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 msgr2=0x5645eda4ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda4ff80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e600116e0 tx=0x7f9e60004730 comp rx=0 tx=0).stop 2024-09-19T07:43:59.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 shutdown_connections 2024-09-19T07:43:59.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda60050 0x5645eda62460 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e4d7fa640 0 client.26237 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda50680 0x5645eda4d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 --2- 172.21.15.103:0/7585055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda454a0 0x5645eda4ff80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 >> 172.21.15.103:0/7585055 conn(0x5645eda41a30 msgr2=0x5645eda47710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 shutdown_connections 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 1 -- 172.21.15.103:0/7585055 wait complete. 2024-09-19T07:43:59.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.694+0000 7f9e6e403740 20 client.26237 trim_cache size 0 max 16384 2024-09-19T07:43:59.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.719+0000 7f9e6e403740 1 Processor -- start 2024-09-19T07:43:59.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.719+0000 7f9e6e403740 1 -- start start 2024-09-19T07:43:59.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda454a0 0x5645eda4d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda487a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6e403740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641bc7e0 con 0x5645eda454a0 2024-09-19T07:43:59.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6e403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641be320 con 0x5645eda41a30 2024-09-19T07:43:59.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6e403740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641bceb0 con 0x5645eda45dc0 2024-09-19T07:43:59.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e695c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda454a0 0x5645eda4d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e695c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda454a0 0x5645eda4d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:32790/0 (socket says 172.21.15.103:32790) 2024-09-19T07:43:59.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e695c5640 1 -- 172.21.15.103:0/748757836 learned_addr learned my addr 172.21.15.103:0/748757836 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:59.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/748757836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 msgr2=0x5645eda5c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda5c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/748757836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda454a0 msgr2=0x5645eda4d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda454a0 0x5645eda4d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.720+0000 7f9e6a5c7640 1 -- 172.21.15.103:0/748757836 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645eda4fba0 con 0x5645eda45dc0 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.721+0000 7f9e6a5c7640 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda487a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5645eda46360 tx=0x7f9e5c0a0b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b0670a01b99357 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.721+0000 7f9e4f7fe640 1 -- 172.21.15.103:0/748757836 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0b9070 con 0x5645eda45dc0 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.721+0000 7f9e4f7fe640 1 -- 172.21.15.103:0/748757836 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5c0051d0 con 0x5645eda45dc0 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.721+0000 7f9e4f7fe640 1 -- 172.21.15.103:0/748757836 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0a4e40 con 0x5645eda45dc0 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 -- 172.21.15.103:0/748757836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 msgr2=0x5645eda487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda487a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5645eda46360 tx=0x7f9e5c0a0b90 comp rx=0 tx=0).stop 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 -- 172.21.15.103:0/748757836 shutdown_connections 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda45dc0 0x5645eda487a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda454a0 0x5645eda4d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 --2- 172.21.15.103:0/748757836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda5c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 -- 172.21.15.103:0/748757836 >> 172.21.15.103:0/748757836 conn(0x5645eda4d8d0 msgr2=0x5645eda48090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 -- 172.21.15.103:0/748757836 shutdown_connections 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.722+0000 7f9e6e403740 1 -- 172.21.15.103:0/748757836 wait complete. 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 -- start start 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 Processor -- start 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda4a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda454a0 0x5645eda4a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641bdc50 con 0x5645eda45dc0 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641bd580 con 0x5645eda41a30 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.723+0000 7f9e6e403740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e641ba7e0 con 0x5645eda454a0 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e6e403740 10 client.0 _ll_register_callbacks cb 0x7f9e60010eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e6a5c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda4a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e695c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda454a0 0x5645eda4a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda4a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:51444/0 (socket says 172.21.15.103:51444) 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 -- 172.21.15.103:0/871073504 learned_addr learned my addr 172.21.15.103:0/871073504 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 -- 172.21.15.103:0/871073504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda454a0 msgr2=0x5645eda4a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda454a0 0x5645eda4a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 -- 172.21.15.103:0/871073504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 msgr2=0x5645eda4ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:43:59.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 -- 172.21.15.103:0/871073504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645edac9dd0 con 0x5645eda41a30 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e695c5640 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda454a0 0x5645eda4a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e69dc6640 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda4a300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e6001ec10 tx=0x7f9e6001e650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22534d978cc99acd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.724+0000 7f9e4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.725+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/871073504 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6003c070 con 0x5645eda41a30 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.725+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/871073504 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f9e60005240 con 0x5645eda41a30 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.725+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e5c0051d0 con 0x5645eda41a30 2024-09-19T07:43:59.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.725+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/871073504 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9e60048050 con 0x5645eda41a30 2024-09-19T07:43:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 10 client.26243 fetch_fsmap learned FSMap version 38 2024-09-19T07:43:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5645edacb0c0 con 0x5645eda41a30 2024-09-19T07:43:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/871073504 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5c0051d0 con 0x5645eda41a30 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/871073504 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 38) ==== 8366+0+0 (secure 0 0 0) 0x7f9e60038460 con 0x5645eda41a30 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 10 client.26243 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 20 client.26243 populate_metadata read hostname 'smithi103' 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda50680 0x5645edacd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 4 client.26243 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.726+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645ed9b8850 con 0x5645eda50680 2024-09-19T07:43:59.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.727+0000 7f9e695c5640 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda50680 0x5645edacd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e695c5640 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda50680 0x5645edacd880 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cfa82d9515fe138 server_cookie=169e78ef21b19af0 in_seq=0 out_seq=0 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e4d7fa640 10 client.26243 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e4d7fa640 1 -- 172.21.15.103:0/871073504 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5645ed9b8850 con 0x5645eda50680 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e4d7fa640 10 client.26243 handle_command_reply: tid=0 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 client.26243 shutdown 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 msgr2=0x5645eda4a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda4a300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9e6001ec10 tx=0x7f9e6001e650 comp rx=0 tx=0).stop 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 shutdown_connections 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5645eda50680 0x5645edacd880 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645eda45dc0 0x5645eda4ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645eda454a0 0x5645eda4a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 --2- 172.21.15.103:0/871073504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645eda41a30 0x5645eda4a300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e4d7fa640 0 client.26243 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 >> 172.21.15.103:0/871073504 conn(0x5645eda4d8d0 msgr2=0x5645eda488b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 shutdown_connections 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.728+0000 7f9e6e403740 1 -- 172.21.15.103:0/871073504 wait complete. 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.729+0000 7f9e6e403740 20 client.26243 trim_cache size 0 max 16384 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:43:59.734 INFO:teuthology.orchestra.run.smithi103.stdout: "0ca0830c-33c6-4aee-aae9-56590866a199": { 2024-09-19T07:43:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:43:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "0ca0830c-33c6-4aee-aae9-56590866a199", 2024-09-19T07:43:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:43:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:59.735 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:43:59.735 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:43:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.729+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9e500a3480 msgr2=0x7f9e500a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.729+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9e500a3480 0x7f9e500a5940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000bbe0 tx=0x7f9e6000b390 comp rx=0 tx=0).stop 2024-09-19T07:43:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.729+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e64114ab0 msgr2=0x7f9e641a7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:43:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.729+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e64114ab0 0x7f9e641a7690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c005fd0 tx=0x7f9e5c00ef10 comp rx=0 tx=0).stop 2024-09-19T07:43:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 shutdown_connections 2024-09-19T07:43:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9e500a3480 0x7f9e500a5940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e64114ab0 0x7f9e641a7690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e64109ba0 0x7f9e641a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 --2- 172.21.15.103:0/1613922166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e641091f0 0x7f9e641a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:43:59.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 >> 172.21.15.103:0/1613922166 conn(0x7f9e6406e2d0 msgr2=0x7f9e6410a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:43:59.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 shutdown_connections 2024-09-19T07:43:59.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:43:59.730+0000 7f9e6c02a640 1 -- 172.21.15.103:0/1613922166 wait complete. 2024-09-19T07:43:59.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:43:59 smithi134 ceph-mon[35414]: pgmap v183: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 46 MiB/s wr, 2.30k op/s 2024-09-19T07:43:59.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:43:59 smithi174 ceph-mon[35370]: pgmap v183: 97 pgs: 97 active+clean; 3.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 46 MiB/s wr, 2.30k op/s 2024-09-19T07:44:00.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 325.0 s, 3151.8 tps, lat 28.027 ms stddev 31.867 2024-09-19T07:44:00.528 INFO:tasks.cephfs.filesystem:scrub status for tag:0ca0830c-33c6-4aee-aae9-56590866a199 - {'path': '/', 'tag': '0ca0830c-33c6-4aee-aae9-56590866a199', 'options': 'recursive,force'} 2024-09-19T07:44:01.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:01 smithi134 ceph-mon[35414]: pgmap v184: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 52 MiB/s wr, 2.69k op/s 2024-09-19T07:44:01.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:01 smithi174 ceph-mon[35370]: pgmap v184: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 52 MiB/s wr, 2.69k op/s 2024-09-19T07:44:01.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:01 smithi103 ceph-mon[31201]: pgmap v184: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 52 MiB/s wr, 2.69k op/s 2024-09-19T07:44:03.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:03 smithi134 ceph-mon[35414]: pgmap v185: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 17 MiB/s wr, 1.25k op/s 2024-09-19T07:44:03.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:03 smithi174 ceph-mon[35370]: pgmap v185: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 17 MiB/s wr, 1.25k op/s 2024-09-19T07:44:03.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:03 smithi103 ceph-mon[31201]: pgmap v185: 97 pgs: 97 active+clean; 3.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 17 MiB/s wr, 1.25k op/s 2024-09-19T07:44:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 330.0 s, 432.5 tps, lat 259.265 ms stddev 1065.826 2024-09-19T07:44:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:05 smithi134 ceph-mon[35414]: pgmap v186: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 26 MiB/s wr, 2.12k op/s 2024-09-19T07:44:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:05 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:44:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:05 smithi174 ceph-mon[35370]: pgmap v186: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 26 MiB/s wr, 2.12k op/s 2024-09-19T07:44:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:05 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:44:05.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:05 smithi103 ceph-mon[31201]: pgmap v186: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 26 MiB/s wr, 2.12k op/s 2024-09-19T07:44:05.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:05 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:44:06.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:06 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:06.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:06 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:06 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:06 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:06.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:06 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:06.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:06 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:07.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:07 smithi134 ceph-mon[35414]: pgmap v187: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-09-19T07:44:07.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:07 smithi174 ceph-mon[35370]: pgmap v187: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-09-19T07:44:07.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:07 smithi103 ceph-mon[31201]: pgmap v187: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 22 MiB/s wr, 1.75k op/s 2024-09-19T07:44:09.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:09 smithi134 ceph-mon[35414]: pgmap v188: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-09-19T07:44:09.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:09 smithi174 ceph-mon[35370]: pgmap v188: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-09-19T07:44:09.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:09 smithi103 ceph-mon[31201]: pgmap v188: 97 pgs: 97 active+clean; 3.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-09-19T07:44:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 335.0 s, 3666.0 tps, lat 27.267 ms stddev 29.392 2024-09-19T07:44:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:11 smithi134 ceph-mon[35414]: pgmap v189: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.92k op/s 2024-09-19T07:44:11.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:11 smithi174 ceph-mon[35370]: pgmap v189: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.92k op/s 2024-09-19T07:44:11.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:11 smithi103 ceph-mon[31201]: pgmap v189: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 24 MiB/s wr, 1.92k op/s 2024-09-19T07:44:12.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:12 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:44:12.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:12 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:44:12.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:12 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:44:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:13 smithi134 ceph-mon[35414]: pgmap v190: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-09-19T07:44:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:13 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:44:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:13 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:44:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:13 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:44:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:13 smithi174 ceph-mon[35370]: pgmap v190: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-09-19T07:44:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:13 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:44:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:13 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:44:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:13 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:44:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:13 smithi103 ceph-mon[31201]: pgmap v190: 97 pgs: 97 active+clean; 3.7 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 18 MiB/s wr, 1.53k op/s 2024-09-19T07:44:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:13 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:44:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:13 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:44:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:13 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:44:14.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:14.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:14.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:14.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:14.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:14 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:14.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:14.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:14.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:14.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:14.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:14 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:14.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:14.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:14.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:14.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:14.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:14 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 340.0 s, 4090.4 tps, lat 24.442 ms stddev 26.396 2024-09-19T07:44:15.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:15 smithi134 ceph-mon[35414]: pgmap v191: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-09-19T07:44:15.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:15 smithi174 ceph-mon[35370]: pgmap v191: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-09-19T07:44:15.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:15 smithi103 ceph-mon[31201]: pgmap v191: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-09-19T07:44:16.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:16 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:44:16.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:16 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:44:16.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:16 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:44:17.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:17 smithi134 ceph-mon[35414]: pgmap v192: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 989 op/s 2024-09-19T07:44:17.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:17 smithi174 ceph-mon[35370]: pgmap v192: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 989 op/s 2024-09-19T07:44:17.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:17 smithi103 ceph-mon[31201]: pgmap v192: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 989 op/s 2024-09-19T07:44:19.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:19 smithi134 ceph-mon[35414]: pgmap v193: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.4 MiB/s wr, 718 op/s 2024-09-19T07:44:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:19 smithi174 ceph-mon[35370]: pgmap v193: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.4 MiB/s wr, 718 op/s 2024-09-19T07:44:19.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:19 smithi103 ceph-mon[31201]: pgmap v193: 97 pgs: 97 active+clean; 3.7 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.4 MiB/s wr, 718 op/s 2024-09-19T07:44:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 345.0 s, 3836.2 tps, lat 26.061 ms stddev 27.072 2024-09-19T07:44:21.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:21 smithi134 ceph-mon[35414]: pgmap v194: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-19T07:44:21.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:21 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:44:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:21 smithi174 ceph-mon[35370]: pgmap v194: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-19T07:44:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:21 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:44:21.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:21 smithi103 ceph-mon[31201]: pgmap v194: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-19T07:44:21.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:21 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:44:23.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:23 smithi134 ceph-mon[35414]: pgmap v195: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 655 op/s 2024-09-19T07:44:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:23 smithi174 ceph-mon[35370]: pgmap v195: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 655 op/s 2024-09-19T07:44:23.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:23 smithi103 ceph-mon[31201]: pgmap v195: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 MiB/s wr, 655 op/s 2024-09-19T07:44:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 350.0 s, 3695.2 tps, lat 27.043 ms stddev 28.328 2024-09-19T07:44:25.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:25 smithi134 ceph-mon[35414]: pgmap v196: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 932 op/s 2024-09-19T07:44:25.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:25 smithi174 ceph-mon[35370]: pgmap v196: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 932 op/s 2024-09-19T07:44:25.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:25 smithi103 ceph-mon[31201]: pgmap v196: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 932 op/s 2024-09-19T07:44:27.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:27 smithi134 ceph-mon[35414]: pgmap v197: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 769 op/s 2024-09-19T07:44:27.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:27 smithi174 ceph-mon[35370]: pgmap v197: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 769 op/s 2024-09-19T07:44:27.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:27 smithi103 ceph-mon[31201]: pgmap v197: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.6 MiB/s wr, 769 op/s 2024-09-19T07:44:29.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:29 smithi134 ceph-mon[35414]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 664 op/s 2024-09-19T07:44:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:29 smithi174 ceph-mon[35370]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 664 op/s 2024-09-19T07:44:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:29 smithi103 ceph-mon[31201]: pgmap v198: 97 pgs: 97 active+clean; 3.8 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.9 MiB/s wr, 664 op/s 2024-09-19T07:44:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 355.0 s, 3040.8 tps, lat 32.872 ms stddev 36.458 2024-09-19T07:44:30.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:44:30.889 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:31.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 -- 172.21.15.103:0/438962862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad58110190 msgr2=0x7fad5806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 --2- 172.21.15.103:0/438962862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad58110190 0x7fad5806e4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009900 tx=0x7fad4c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 -- 172.21.15.103:0/438962862 shutdown_connections 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 --2- 172.21.15.103:0/438962862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad5811b140 0x7fad5811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 --2- 172.21.15.103:0/438962862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5806ea30 0x7fad5806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 --2- 172.21.15.103:0/438962862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad58110190 0x7fad5806e4f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 -- 172.21.15.103:0/438962862 >> 172.21.15.103:0/438962862 conn(0x7fad58073180 msgr2=0x7fad58073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.476+0000 7fad5d963640 1 -- 172.21.15.103:0/438962862 shutdown_connections 2024-09-19T07:44:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.477+0000 7fad5d963640 1 -- 172.21.15.103:0/438962862 wait complete. 2024-09-19T07:44:31.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.478+0000 7fad5d963640 1 Processor -- start 2024-09-19T07:44:31.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.478+0000 7fad5d963640 1 -- start start 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5d963640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5806ea30 0x7fad581a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5d963640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad58110190 0x7fad581a7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5d963640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5811b140 0x7fad581ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5d963640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad58120be0 con 0x7fad5806ea30 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5d963640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad58120a60 con 0x7fad5811b140 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5d963640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad58120d60 con 0x7fad58110190 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad57fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad58110190 0x7fad581a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad57fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad58110190 0x7fad581a7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56130/0 (socket says 172.21.15.103:56130) 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad57fff640 1 -- 172.21.15.103:0/2088891726 learned_addr learned my addr 172.21.15.103:0/2088891726 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5d162640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5811b140 0x7fad581ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.479+0000 7fad5c961640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5806ea30 0x7fad581a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad57fff640 1 -- 172.21.15.103:0/2088891726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5811b140 msgr2=0x7fad581ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad57fff640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5811b140 0x7fad581ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad57fff640 1 -- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5806ea30 msgr2=0x7fad581a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad57fff640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5806ea30 0x7fad581a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad57fff640 1 -- 172.21.15.103:0/2088891726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad581ac0b0 con 0x7fad58110190 2024-09-19T07:44:31.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad57fff640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad58110190 0x7fad581a7620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad5000cce0 tx=0x7fad5000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:31.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad55ffb640 1 -- 172.21.15.103:0/2088891726 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50004b20 con 0x7fad58110190 2024-09-19T07:44:31.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.480+0000 7fad55ffb640 1 -- 172.21.15.103:0/2088891726 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50004cc0 con 0x7fad58110190 2024-09-19T07:44:31.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.482+0000 7fad55ffb640 1 -- 172.21.15.103:0/2088891726 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5000f7d0 con 0x7fad58110190 2024-09-19T07:44:31.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.482+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad581ac3a0 con 0x7fad58110190 2024-09-19T07:44:31.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.482+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad58110920 con 0x7fad58110190 2024-09-19T07:44:31.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.487+0000 7fad60f9e740 1 Processor -- start 2024-09-19T07:44:31.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.487+0000 7fad60f9e740 1 -- start start 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.487+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab6f670 0x556c0ab6fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.487+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab7a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 0x556c0ab7cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad60f9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c0aad52e0 con 0x556c0ab7a850 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad60f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c0aa7cce0 con 0x556c0ab6f670 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad60f9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c0aad12a0 con 0x556c0ab6ff90 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab6f670 0x556c0ab6fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab6f670 0x556c0ab6fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:36270/0 (socket says 172.21.15.103:36270) 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 -- 172.21.15.103:0/1439100484 learned_addr learned my addr 172.21.15.103:0/1439100484 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5c961640 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab7a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 -- 172.21.15.103:0/1439100484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 msgr2=0x556c0ab7a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab7a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 -- 172.21.15.103:0/1439100484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 msgr2=0x556c0ab7cc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 0x556c0ab7cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 -- 172.21.15.103:0/1439100484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0aa61440 con 0x556c0ab6f670 2024-09-19T07:44:31.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.488+0000 7fad5d162640 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab6f670 0x556c0ab6fa50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad48009b20 tx=0x7fad480113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7709dd8775382204 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.489+0000 7fad3effd640 1 -- 172.21.15.103:0/1439100484 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48011e10 con 0x556c0ab6f670 2024-09-19T07:44:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.489+0000 7fad3effd640 1 -- 172.21.15.103:0/1439100484 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad48012930 con 0x556c0ab6f670 2024-09-19T07:44:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.489+0000 7fad3effd640 1 -- 172.21.15.103:0/1439100484 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4800fae0 con 0x556c0ab6f670 2024-09-19T07:44:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.489+0000 7fad55ffb640 1 -- 172.21.15.103:0/2088891726 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fad5000f970 con 0x7fad58110190 2024-09-19T07:44:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad60f9e740 1 -- 172.21.15.103:0/1439100484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab6f670 msgr2=0x556c0ab6fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab6f670 0x556c0ab6fa50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad48009b20 tx=0x7fad480113e0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad55ffb640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fad400a33e0 0x7fad400a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad60f9e740 1 -- 172.21.15.103:0/1439100484 shutdown_connections 2024-09-19T07:44:31.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 0x556c0ab7cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab7a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1439100484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab6f670 0x556c0ab6fa50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad60f9e740 1 -- 172.21.15.103:0/1439100484 >> 172.21.15.103:0/1439100484 conn(0x556c0ab6bc00 msgr2=0x556c0ab6bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:31.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.490+0000 7fad55ffb640 1 -- 172.21.15.103:0/2088891726 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fad500d0050 con 0x7fad58110190 2024-09-19T07:44:31.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.492+0000 7fad5d162640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fad400a33e0 0x7fad400a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.492+0000 7fad60f9e740 1 -- 172.21.15.103:0/1439100484 shutdown_connections 2024-09-19T07:44:31.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.492+0000 7fad60f9e740 1 -- 172.21.15.103:0/1439100484 wait complete. 2024-09-19T07:44:31.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad5d162640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fad400a33e0 0x7fad400a58a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad4800f560 tx=0x7fad48000bc0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:31.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 -- start start 2024-09-19T07:44:31.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 Processor -- start 2024-09-19T07:44:31.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6f670 0x556c0ab7a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab77420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab7a850 0x556c0ab77960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c0ab7d420 con 0x556c0ab6ff90 2024-09-19T07:44:31.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c0aa90050 con 0x556c0ab7a850 2024-09-19T07:44:31.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c0ab7d5a0 con 0x556c0ab6f670 2024-09-19T07:44:31.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad60f9e740 10 client.0 _ll_register_callbacks cb 0x556c0aad0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:31.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad5c961640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6f670 0x556c0ab7a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad5d162640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab7a850 0x556c0ab77960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad5c961640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6f670 0x556c0ab7a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56148/0 (socket says 172.21.15.103:56148) 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad5c961640 1 -- 172.21.15.103:0/2803268793 learned_addr learned my addr 172.21.15.103:0/2803268793 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.493+0000 7fad57fff640 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab77420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad5c961640 1 -- 172.21.15.103:0/2803268793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab7a850 msgr2=0x556c0ab77960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad5c961640 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab7a850 0x556c0ab77960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad5c961640 1 -- 172.21.15.103:0/2803268793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 msgr2=0x556c0ab77420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad5c961640 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab77420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad5c961640 1 -- 172.21.15.103:0/2803268793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0a8463c0 con 0x556c0ab6f670 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad5c961640 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6f670 0x556c0ab7a170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009230 tx=0x7fad4c004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba007a7d02d42b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad37fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:31.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad37fff640 1 -- 172.21.15.103:0/2803268793 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c035040 con 0x556c0ab6f670 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad37fff640 1 -- 172.21.15.103:0/2803268793 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fad4c005590 con 0x556c0ab6f670 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad48012930 con 0x556c0ab6f670 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad37fff640 1 -- 172.21.15.103:0/2803268793 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fad4c0046c0 con 0x556c0ab6f670 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad60f9e740 10 client.26176 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.494+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556c0ab88da0 con 0x556c0ab6f670 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad37fff640 1 -- 172.21.15.103:0/2803268793 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad48012930 con 0x556c0ab6f670 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad37fff640 1 -- 172.21.15.103:0/2803268793 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fad4c003bd0 con 0x556c0ab6f670 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad60f9e740 10 client.26176 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad60f9e740 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad60f9e740 20 client.26176 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0ab8a1b0 0x556c0ab8c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad60f9e740 4 client.26176 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c0aae2850 con 0x556c0ab8a1b0 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.495+0000 7fad57fff640 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0ab8a1b0 0x556c0ab8c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.496+0000 7fad37fff640 10 client.26176 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.496+0000 7fad57fff640 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0ab8a1b0 0x556c0ab8c5a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d305d618fc42172c server_cookie=39e13cd1fd3bc95f in_seq=0 out_seq=0 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.496+0000 7fad37fff640 1 -- 172.21.15.103:0/2803268793 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c0aae2850 con 0x556c0ab8a1b0 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad37fff640 10 client.26176 handle_command_reply: tid=0 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 client.26176 shutdown 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6f670 msgr2=0x556c0ab7a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6f670 0x556c0ab7a170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009230 tx=0x7fad4c004580 comp rx=0 tx=0).stop 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 shutdown_connections 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0ab8a1b0 0x556c0ab8c5a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab7a850 0x556c0ab77960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab77420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2803268793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6f670 0x556c0ab7a170 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009230 tx=0x7fad4c004580 comp rx=0 tx=0).stop 2024-09-19T07:44:31.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 >> 172.21.15.103:0/2803268793 conn(0x556c0ab6bc00 msgr2=0x556c0ab718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:31.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.497+0000 7fad37fff640 0 client.26176 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:31.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.498+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 shutdown_connections 2024-09-19T07:44:31.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.498+0000 7fad60f9e740 1 -- 172.21.15.103:0/2803268793 wait complete. 2024-09-19T07:44:31.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.498+0000 7fad60f9e740 20 client.26176 trim_cache size 0 max 16384 2024-09-19T07:44:31.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 Processor -- start 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 -- start start 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab91e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 0x556c0ab92390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab74ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad581c4c70 con 0x556c0ab7a850 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad581c6690 con 0x556c0ab74680 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad5c961640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab91e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad57fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 0x556c0ab92390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad5c961640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab91e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56154/0 (socket says 172.21.15.103:56154) 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad5c961640 1 -- 172.21.15.103:0/2052076094 learned_addr learned my addr 172.21.15.103:0/2052076094 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad60f9e740 1 -- 172.21.15.103:0/2052076094 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad581c5220 con 0x556c0ab6ff90 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.523+0000 7fad5d162640 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab74ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad5c961640 1 -- 172.21.15.103:0/2052076094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 msgr2=0x556c0ab74ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad5c961640 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab74ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad5c961640 1 -- 172.21.15.103:0/2052076094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 msgr2=0x556c0ab92390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad5c961640 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 0x556c0ab92390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad5c961640 1 -- 172.21.15.103:0/2052076094 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0ab7a4e0 con 0x556c0ab6ff90 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad5d162640 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab74ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad5c961640 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab91e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0092a0 tx=0x7fad4c0319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c719f80dd798a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad3e7fc640 1 -- 172.21.15.103:0/2052076094 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c035030 con 0x556c0ab6ff90 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad3e7fc640 1 -- 172.21.15.103:0/2052076094 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4c041070 con 0x556c0ab6ff90 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.524+0000 7fad3e7fc640 1 -- 172.21.15.103:0/2052076094 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0035f0 con 0x556c0ab6ff90 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 -- 172.21.15.103:0/2052076094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 msgr2=0x556c0ab91e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab91e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0092a0 tx=0x7fad4c0319f0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 -- 172.21.15.103:0/2052076094 shutdown_connections 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab74ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab7a850 0x556c0ab92390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 --2- 172.21.15.103:0/2052076094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab6ff90 0x556c0ab91e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 -- 172.21.15.103:0/2052076094 >> 172.21.15.103:0/2052076094 conn(0x556c0ab6bc00 msgr2=0x556c0ab794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:31.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 -- 172.21.15.103:0/2052076094 shutdown_connections 2024-09-19T07:44:31.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 -- 172.21.15.103:0/2052076094 wait complete. 2024-09-19T07:44:31.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.525+0000 7fad60f9e740 1 -- start start 2024-09-19T07:44:31.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.526+0000 7fad60f9e740 1 Processor -- start 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab80e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad60f9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab7a850 0x556c0abede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad60f9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad581c5fc0 con 0x556c0ab6ff90 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad60f9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad581c58f0 con 0x556c0ab74680 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad60f9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad58116ff0 con 0x556c0ab7a850 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab7a850 0x556c0abede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab7a850 0x556c0abede70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56158/0 (socket says 172.21.15.103:56158) 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 -- 172.21.15.103:0/1951588752 learned_addr learned my addr 172.21.15.103:0/1951588752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:31.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad57fff640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5c961640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab80e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad60f9e740 10 client.0 _ll_register_callbacks cb 0x7fad4c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 -- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 msgr2=0x556c0ab813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 -- 172.21.15.103:0/1951588752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 msgr2=0x556c0ab80e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab80e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5d162640 1 -- 172.21.15.103:0/1951588752 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0abee4b0 con 0x556c0ab7a850 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.527+0000 7fad5c961640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab80e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.528+0000 7fad5d162640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab7a850 0x556c0abede70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad4800f630 tx=0x7fad4800f5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1c1aa2b5c293a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.528+0000 7fad37fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:31.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.528+0000 7fad37fff640 1 -- 172.21.15.103:0/1951588752 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48037060 con 0x556c0ab7a850 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.528+0000 7fad37fff640 1 -- 172.21.15.103:0/1951588752 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fad4800fe00 con 0x556c0ab7a850 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.528+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad4c041070 con 0x556c0ab7a850 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad37fff640 1 -- 172.21.15.103:0/1951588752 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fad48004600 con 0x556c0ab7a850 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 10 client.26182 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556c0abf5850 con 0x556c0ab7a850 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad37fff640 1 -- 172.21.15.103:0/1951588752 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4c041070 con 0x556c0ab7a850 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad37fff640 1 -- 172.21.15.103:0/1951588752 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fad480268d0 con 0x556c0ab7a850 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 10 client.26182 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 10 client.26182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:31.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 20 client.26182 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0abf5c40 0x556c0abf8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 4 client.26182 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c0aadcd60 con 0x556c0abf5c40 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.529+0000 7fad5c961640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0abf5c40 0x556c0abf8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad37fff640 10 client.26182 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad5c961640 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0abf5c40 0x556c0abf8030 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7870594cce2dfb0b server_cookie=6b5b4ea441a45dc5 in_seq=0 out_seq=0 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad37fff640 1 -- 172.21.15.103:0/1951588752 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c0aadcd60 con 0x556c0abf5c40 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad37fff640 10 client.26182 handle_command_reply: tid=0 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 client.26182 shutdown 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab7a850 msgr2=0x556c0abede70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab7a850 0x556c0abede70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad4800f630 tx=0x7fad4800f5c0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 shutdown_connections 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c0abf5c40 0x556c0abf8030 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c0ab7a850 0x556c0abede70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c0ab74680 0x556c0ab813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 --2- 172.21.15.103:0/1951588752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c0ab6ff90 0x556c0ab80e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 >> 172.21.15.103:0/1951588752 conn(0x556c0ab6bc00 msgr2=0x556c0ab794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad37fff640 0 client.26182 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 shutdown_connections 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 1 -- 172.21.15.103:0/1951588752 wait complete. 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.530+0000 7fad60f9e740 20 client.26182 trim_cache size 0 max 16384 2024-09-19T07:44:31.539 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:44:31.540 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:44:31.540 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:44:31.540 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.531+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fad400a33e0 msgr2=0x7fad400a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.531+0000 7fad5d963640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fad400a33e0 0x7fad400a58a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad4800f560 tx=0x7fad48000bc0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad58110190 msgr2=0x7fad581a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad58110190 0x7fad581a7620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad5000cce0 tx=0x7fad5000a620 comp rx=0 tx=0).stop 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 shutdown_connections 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fad400a33e0 0x7fad400a58a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5811b140 0x7fad581ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad58110190 0x7fad581a7620 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 --2- 172.21.15.103:0/2088891726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad5806ea30 0x7fad581a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:31.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 >> 172.21.15.103:0/2088891726 conn(0x7fad58073180 msgr2=0x7fad58119310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:31.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.532+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 shutdown_connections 2024-09-19T07:44:31.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:31.533+0000 7fad5d963640 1 -- 172.21.15.103:0/2088891726 wait complete. 2024-09-19T07:44:32.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:31 smithi174 ceph-mon[35370]: pgmap v199: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-19T07:44:32.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:31 smithi103 ceph-mon[31201]: pgmap v199: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-19T07:44:32.302 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:44:32.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:44:32.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:31 smithi134 ceph-mon[35414]: pgmap v199: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 MiB/s wr, 994 op/s 2024-09-19T07:44:32.663 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:33.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.231+0000 7f249bfe9640 1 -- 172.21.15.103:0/2961344951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 msgr2=0x7f2494115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.231+0000 7f249bfe9640 1 --2- 172.21.15.103:0/2961344951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 0x7f2494115d30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2484009900 tx=0x7f248402a410 comp rx=0 tx=0).stop 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.231+0000 7f249bfe9640 1 -- 172.21.15.103:0/2961344951 shutdown_connections 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.231+0000 7f249bfe9640 1 --2- 172.21.15.103:0/2961344951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 0x7f2494115d30 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.231+0000 7f249bfe9640 1 --2- 172.21.15.103:0/2961344951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410ae80 0x7f2494113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.231+0000 7f249bfe9640 1 --2- 172.21.15.103:0/2961344951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249410a560 0x7f249410a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.231+0000 7f249bfe9640 1 -- 172.21.15.103:0/2961344951 >> 172.21.15.103:0/2961344951 conn(0x7f249406e140 msgr2=0x7f249406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.232+0000 7f249bfe9640 1 -- 172.21.15.103:0/2961344951 shutdown_connections 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.232+0000 7f249bfe9640 1 -- 172.21.15.103:0/2961344951 wait complete. 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.232+0000 7f249bfe9640 1 Processor -- start 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.232+0000 7f249bfe9640 1 -- start start 2024-09-19T07:44:33.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f249bfe9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410a560 0x7f24941a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f249bfe9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249410ae80 0x7f24941a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f249bfe9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 0x7f24941a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f249bfe9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2494119650 con 0x7f2494113940 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f249bfe9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24941194d0 con 0x7f249410a560 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f249bfe9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24941197d0 con 0x7f249410ae80 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410a560 0x7f24941a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410a560 0x7f24941a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:36308/0 (socket says 172.21.15.103:36308) 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 -- 172.21.15.103:0/638571339 learned_addr learned my addr 172.21.15.103:0/638571339 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499584640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249410ae80 0x7f24941a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f249a586640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 0x7f24941a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 -- 172.21.15.103:0/638571339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249410ae80 msgr2=0x7f24941a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249410ae80 0x7f24941a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 -- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 msgr2=0x7f24941a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:33.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 0x7f24941a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 -- 172.21.15.103:0/638571339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24941acea0 con 0x7f249410a560 2024-09-19T07:44:33.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.233+0000 7f2499d85640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410a560 0x7f24941a1aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f248c00eb90 tx=0x7f248c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:33.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.234+0000 7f248affd640 1 -- 172.21.15.103:0/638571339 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c004d20 con 0x7f249410a560 2024-09-19T07:44:33.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.234+0000 7f248affd640 1 -- 172.21.15.103:0/638571339 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f248c00ee90 con 0x7f249410a560 2024-09-19T07:44:33.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.234+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941ad190 con 0x7f249410a560 2024-09-19T07:44:33.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.234+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941ad670 con 0x7f249410a560 2024-09-19T07:44:33.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.235+0000 7f248affd640 1 -- 172.21.15.103:0/638571339 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c010830 con 0x7f249410a560 2024-09-19T07:44:33.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.236+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2478005290 con 0x7f249410a560 2024-09-19T07:44:33.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.237+0000 7f248affd640 1 -- 172.21.15.103:0/638571339 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f248c005020 con 0x7f249410a560 2024-09-19T07:44:33.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.237+0000 7f248affd640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24800a3500 0x7f24800a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:33.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.238+0000 7f248affd640 1 -- 172.21.15.103:0/638571339 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f248c0d0120 con 0x7f249410a560 2024-09-19T07:44:33.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.238+0000 7f2499584640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24800a3500 0x7f24800a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:33.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.239+0000 7f2499584640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24800a3500 0x7f24800a59c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f24940731f0 tx=0x7f2490005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:33.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.242+0000 7f248affd640 1 -- 172.21.15.103:0/638571339 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f248c09cb10 con 0x7f249410a560 2024-09-19T07:44:33.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.584+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2478005580 con 0x7f249410a560 2024-09-19T07:44:33.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.587+0000 7f248affd640 1 -- 172.21.15.103:0/638571339 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11996 (secure 0 0 0) 0x7f248c09c410 con 0x7f249410a560 2024-09-19T07:44:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:44:33.588 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":40,"btime":"2024-09-19T07:44:20:535666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:44:19.854970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:44:33.588 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 40 2024-09-19T07:44:33.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.593+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24800a3500 msgr2=0x7f24800a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:33.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.593+0000 7f249bfe9640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24800a3500 0x7f24800a59c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f24940731f0 tx=0x7f2490005f90 comp rx=0 tx=0).stop 2024-09-19T07:44:33.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.593+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410a560 msgr2=0x7f24941a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:33.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.593+0000 7f249bfe9640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410a560 0x7f24941a1aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f248c00eb90 tx=0x7f248c00c6a0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 shutdown_connections 2024-09-19T07:44:33.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24800a3500 0x7f24800a59c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2494113940 0x7f24941a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249410ae80 0x7f24941a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 --2- 172.21.15.103:0/638571339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f249410a560 0x7f24941a1aa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:33.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 >> 172.21.15.103:0/638571339 conn(0x7f249406e140 msgr2=0x7f2494071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:33.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 shutdown_connections 2024-09-19T07:44:33.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:33.597+0000 7f249bfe9640 1 -- 172.21.15.103:0/638571339 wait complete. 2024-09-19T07:44:34.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:33 smithi174 ceph-mon[35370]: pgmap v200: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-19T07:44:34.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:33 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/638571339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:44:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:33 smithi103 ceph-mon[31201]: pgmap v200: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-19T07:44:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:33 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/638571339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:44:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:33 smithi134 ceph-mon[35414]: pgmap v200: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-09-19T07:44:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:33 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/638571339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:44:34.391 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:44:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:35.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 -- 172.21.15.103:0/525770112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28100e20 msgr2=0x7f0d28101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 --2- 172.21.15.103:0/525770112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28100e20 0x7f0d28101200 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009860 tx=0x7f0d2402a3d0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 -- 172.21.15.103:0/525770112 shutdown_connections 2024-09-19T07:44:35.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 --2- 172.21.15.103:0/525770112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d28102ef0 0x7f0d2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 --2- 172.21.15.103:0/525770112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d28102570 0x7f0d281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 --2- 172.21.15.103:0/525770112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28100e20 0x7f0d28101200 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 -- 172.21.15.103:0/525770112 >> 172.21.15.103:0/525770112 conn(0x7f0d280fc7d0 msgr2=0x7f0d280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:35.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 -- 172.21.15.103:0/525770112 shutdown_connections 2024-09-19T07:44:35.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.377+0000 7f0d2f796640 1 -- 172.21.15.103:0/525770112 wait complete. 2024-09-19T07:44:35.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.378+0000 7f0d2f796640 1 Processor -- start 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.378+0000 7f0d2f796640 1 -- start start 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.378+0000 7f0d2f796640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d28100e20 0x7f0d281962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.378+0000 7f0d2f796640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d28102570 0x7f0d281967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.378+0000 7f0d2f796640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28102ef0 0x7f0d2819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2f796640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2810fe80 con 0x7f0d28102ef0 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2f796640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2810fd00 con 0x7f0d28100e20 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2cd31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d28102570 0x7f0d281967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2cd31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d28102570 0x7f0d281967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56180/0 (socket says 172.21.15.103:56180) 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2cd31640 1 -- 172.21.15.103:0/118991033 learned_addr learned my addr 172.21.15.103:0/118991033 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d28110000 con 0x7f0d28102570 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28102ef0 0x7f0d2819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.379+0000 7f0d2d532640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d28100e20 0x7f0d281962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.380+0000 7f0d2d532640 1 -- 172.21.15.103:0/118991033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d28102570 msgr2=0x7f0d281967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.380+0000 7f0d2d532640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d28102570 0x7f0d281967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.380+0000 7f0d2d532640 1 -- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28102ef0 msgr2=0x7f0d2819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.380+0000 7f0d2d532640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28102ef0 0x7f0d2819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.380+0000 7f0d2d532640 1 -- 172.21.15.103:0/118991033 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2819b250 con 0x7f0d28100e20 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.381+0000 7f0d2d532640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d28100e20 0x7f0d281962a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009830 tx=0x7f0d240055e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.381+0000 7f0d1e7fc640 1 -- 172.21.15.103:0/118991033 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2402f300 con 0x7f0d28100e20 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.381+0000 7f0d1e7fc640 1 -- 172.21.15.103:0/118991033 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d24030e80 con 0x7f0d28100e20 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.381+0000 7f0d1e7fc640 1 -- 172.21.15.103:0/118991033 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004d20 con 0x7f0d28100e20 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.381+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2819b4e0 con 0x7f0d28100e20 2024-09-19T07:44:35.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.382+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2819b8b0 con 0x7f0d28100e20 2024-09-19T07:44:35.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.385+0000 7f0d1e7fc640 1 -- 172.21.15.103:0/118991033 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0d24039040 con 0x7f0d28100e20 2024-09-19T07:44:35.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.386+0000 7f0d31b6f740 1 Processor -- start 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d31b6f740 1 -- start start 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de01b40 0x55ca5de00090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de005d0 0x55ca5de00a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca5dd662f0 con 0x55ca5de01b40 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca5dd0dcf0 con 0x55ca5de005d0 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca5dd622b0 con 0x55ca5de0b370 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d1e7fc640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d140a7990 0x7f0d140a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.387+0000 7f0d2cd31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de0d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56186/0 (socket says 172.21.15.103:56186) 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 -- 172.21.15.103:0/1030790424 learned_addr learned my addr 172.21.15.103:0/1030790424 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2d532640 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de005d0 0x55ca5de00a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de01b40 0x55ca5de00090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 -- 172.21.15.103:0/1030790424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de005d0 msgr2=0x55ca5de00a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d140a7990 0x7f0d140a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de005d0 0x55ca5de00a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 -- 172.21.15.103:0/1030790424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de01b40 msgr2=0x55ca5de00090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de01b40 0x55ca5de00090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 -- 172.21.15.103:0/1030790424 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5dcf23a0 con 0x55ca5de0b370 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d1e7fc640 1 -- 172.21.15.103:0/118991033 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d240f0b30 con 0x7f0d28100e20 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de0d760 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009d50 tx=0x7f0d18011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b23f0133c3fcad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:35.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.388+0000 7f0d137fe640 1 -- 172.21.15.103:0/1030790424 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1801e070 con 0x55ca5de0b370 2024-09-19T07:44:35.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.389+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d140a7990 0x7f0d140a9e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009fd0 tx=0x7f0d20004ab0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:35.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.389+0000 7f0d137fe640 1 -- 172.21.15.103:0/1030790424 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d18019b10 con 0x55ca5de0b370 2024-09-19T07:44:35.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.389+0000 7f0d137fe640 1 -- 172.21.15.103:0/1030790424 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800eb70 con 0x55ca5de0b370 2024-09-19T07:44:35.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1030790424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 msgr2=0x55ca5de0d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de0d760 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009d50 tx=0x7f0d18011810 comp rx=0 tx=0).stop 2024-09-19T07:44:35.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1030790424 shutdown_connections 2024-09-19T07:44:35.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de0d760 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de005d0 0x55ca5de00a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1030790424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de01b40 0x55ca5de00090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1030790424 >> 172.21.15.103:0/1030790424 conn(0x55ca5ddc9310 msgr2=0x55ca5ddfcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:35.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1030790424 shutdown_connections 2024-09-19T07:44:35.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.390+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1030790424 wait complete. 2024-09-19T07:44:35.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.391+0000 7f0d31b6f740 1 -- start start 2024-09-19T07:44:35.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.391+0000 7f0d31b6f740 1 Processor -- start 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.392+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de07f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de08480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d2d532640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d2d532640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de0ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56188/0 (socket says 172.21.15.103:56188) 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d2d532640 1 -- 172.21.15.103:0/1902717098 learned_addr learned my addr 172.21.15.103:0/1902717098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca5de0e0c0 con 0x55ca5de0b370 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca5dd21060 con 0x55ca5de01b40 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de07f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca5de0e240 con 0x55ca5de005d0 2024-09-19T07:44:35.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de08480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.393+0000 7f0d31b6f740 10 client.0 _ll_register_callbacks cb 0x55ca5da748c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:35.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.394+0000 7f0d2d532640 1 -- 172.21.15.103:0/1902717098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 msgr2=0x55ca5de07f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.394+0000 7f0d2d532640 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de07f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.394+0000 7f0d2d532640 1 -- 172.21.15.103:0/1902717098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 msgr2=0x55ca5de08480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.394+0000 7f0d2d532640 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de08480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.394+0000 7f0d2d532640 1 -- 172.21.15.103:0/1902717098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5dad73c0 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.395+0000 7f0d2d532640 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de0ac90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0d24005940 tx=0x7f0d24004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4202d46287b41e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.395+0000 7f0d10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.395+0000 7f0d10ff9640 1 -- 172.21.15.103:0/1902717098 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d240bc900 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d10ff9640 1 -- 172.21.15.103:0/1902717098 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d240c57a0 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d18019b10 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d10ff9640 1 -- 172.21.15.103:0/1902717098 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0d24005c90 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 10 client.26200 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ca5de0e170 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d10ff9640 1 -- 172.21.15.103:0/1902717098 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d18019b10 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d10ff9640 1 -- 172.21.15.103:0/1902717098 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f0d2404c030 con 0x55ca5de005d0 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 10 client.26200 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:35.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 10 client.26200 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:44:35.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 20 client.26200 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:35.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de1d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 4 client.26200 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:35.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.396+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca5dd73860 con 0x55ca5de1af60 2024-09-19T07:44:35.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.397+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de1d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.397+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de1d370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc708a6acf603689 server_cookie=d2a820f96d7c9d31 in_seq=0 out_seq=0 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.397+0000 7f0d10ff9640 10 client.26200 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.398+0000 7f0d10ff9640 1 -- 172.21.15.103:0/1902717098 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca5dd73860 con 0x55ca5de1af60 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.398+0000 7f0d10ff9640 10 client.26200 handle_command_reply: tid=0 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.398+0000 7f0d31b6f740 1 client.26200 shutdown 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 msgr2=0x55ca5de0ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de0ac90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0d24005940 tx=0x7f0d24004f80 comp rx=0 tx=0).stop 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 shutdown_connections 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de1d370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de08480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de07f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/1902717098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de0ac90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d10ff9640 0 client.26200 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:44:35.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 >> 172.21.15.103:0/1902717098 conn(0x55ca5ddc9310 msgr2=0x55ca5de02930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:35.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.399+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 shutdown_connections 2024-09-19T07:44:35.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.400+0000 7f0d31b6f740 1 -- 172.21.15.103:0/1902717098 wait complete. 2024-09-19T07:44:35.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.400+0000 7f0d31b6f740 20 client.26200 trim_cache size 0 max 16384 2024-09-19T07:44:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 Processor -- start 2024-09-19T07:44:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 -- start start 2024-09-19T07:44:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de173b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de23180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281ab510 con 0x55ca5de0b370 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281ad050 con 0x55ca5de01b40 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281abbe0 con 0x55ca5de005d0 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de23180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de23180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51444/0 (socket says 172.21.15.103:51444) 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 -- 172.21.15.103:0/2843773852 learned_addr learned my addr 172.21.15.103:0/2843773852 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2d532640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de173b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 -- 172.21.15.103:0/2843773852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 msgr2=0x55ca5de173b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de173b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 -- 172.21.15.103:0/2843773852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 msgr2=0x55ca5de22c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de22c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 -- 172.21.15.103:0/2843773852 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca5de0a8b0 con 0x55ca5de0b370 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2d532640 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.418+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de23180 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000f8e0 tx=0x7f0d20018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc129a0de25d6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.419+0000 7f0d12ffd640 1 -- 172.21.15.103:0/2843773852 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2000fe60 con 0x55ca5de0b370 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.419+0000 7f0d12ffd640 1 -- 172.21.15.103:0/2843773852 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2002a480 con 0x55ca5de0b370 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.419+0000 7f0d12ffd640 1 -- 172.21.15.103:0/2843773852 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d20010670 con 0x55ca5de0b370 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.419+0000 7f0d31b6f740 1 -- 172.21.15.103:0/2843773852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 msgr2=0x55ca5de23180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.419+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de23180 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000f8e0 tx=0x7f0d20018000 comp rx=0 tx=0).stop 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 -- 172.21.15.103:0/2843773852 shutdown_connections 2024-09-19T07:44:35.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de0b370 0x55ca5de23180 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de22c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/2843773852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de005d0 0x55ca5de173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 -- 172.21.15.103:0/2843773852 >> 172.21.15.103:0/2843773852 conn(0x55ca5ddc9310 msgr2=0x55ca5de07f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 -- 172.21.15.103:0/2843773852 shutdown_connections 2024-09-19T07:44:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 -- 172.21.15.103:0/2843773852 wait complete. 2024-09-19T07:44:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 -- start start 2024-09-19T07:44:35.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.420+0000 7f0d31b6f740 1 Processor -- start 2024-09-19T07:44:35.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de005d0 0x55ca5de0faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de0ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d31b6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de84c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281ac980 con 0x55ca5de005d0 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281ac2b0 con 0x55ca5de01b40 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d31b6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d281a94f0 con 0x55ca5de0b370 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d31b6f740 10 client.0 _ll_register_callbacks cb 0x7f0d240f4090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d2dd33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de84c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d2dd33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de84c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56218/0 (socket says 172.21.15.103:56218) 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d2dd33640 1 -- 172.21.15.103:0/861212846 learned_addr learned my addr 172.21.15.103:0/861212846 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.421+0000 7f0d2cd31640 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de0ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d2dd33640 1 -- 172.21.15.103:0/861212846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 msgr2=0x55ca5de0ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de0ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d2dd33640 1 -- 172.21.15.103:0/861212846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de005d0 msgr2=0x55ca5de0faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de005d0 0x55ca5de0faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d2dd33640 1 -- 172.21.15.103:0/861212846 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca5de85250 con 0x55ca5de0b370 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d2dd33640 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de84c10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d200120a0 tx=0x7f0d200124c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65435dfebec8ead server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.422+0000 7f0d10ff9640 1 -- 172.21.15.103:0/861212846 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2000fe60 con 0x55ca5de0b370 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.423+0000 7f0d10ff9640 1 -- 172.21.15.103:0/861212846 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f0d20020bc0 con 0x55ca5de0b370 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.423+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d2002a480 con 0x55ca5de0b370 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d10ff9640 1 -- 172.21.15.103:0/861212846 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0d20038050 con 0x55ca5de0b370 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d31b6f740 10 client.26206 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ca5de8ed40 con 0x55ca5de0b370 2024-09-19T07:44:35.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d10ff9640 1 -- 172.21.15.103:0/861212846 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d2002a480 con 0x55ca5de0b370 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d10ff9640 1 -- 172.21.15.103:0/861212846 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f0d20010cb0 con 0x55ca5de0b370 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d31b6f740 10 client.26206 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d31b6f740 10 client.26206 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d31b6f740 20 client.26206 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.425+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de91390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d31b6f740 4 client.26206 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca5dd6dd70 con 0x55ca5de1af60 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d2d532640 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de91390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d2d532640 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de91390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4e68cb394a4a8226 server_cookie=2bc8d74f6e3c9bb1 in_seq=0 out_seq=0 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d10ff9640 10 client.26206 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d10ff9640 1 -- 172.21.15.103:0/861212846 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca5dd6dd70 con 0x55ca5de1af60 2024-09-19T07:44:35.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d10ff9640 10 client.26206 handle_command_reply: tid=0 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.426+0000 7f0d31b6f740 1 client.26206 shutdown 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 msgr2=0x55ca5de84c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de84c10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d200120a0 tx=0x7f0d200124c0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 shutdown_connections 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55ca5de1af60 0x55ca5de91390 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca5de0b370 0x55ca5de84c10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d10ff9640 0 client.26206 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca5de01b40 0x55ca5de0ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 --2- 172.21.15.103:0/861212846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca5de005d0 0x55ca5de0faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 >> 172.21.15.103:0/861212846 conn(0x55ca5ddc9310 msgr2=0x55ca5de02110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 shutdown_connections 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 1 -- 172.21.15.103:0/861212846 wait complete. 2024-09-19T07:44:35.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.427+0000 7f0d31b6f740 20 client.26206 trim_cache size 0 max 16384 2024-09-19T07:44:35.433 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:44:35.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d140a7990 msgr2=0x7f0d140a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d140a7990 0x7f0d140a9e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009fd0 tx=0x7f0d20004ab0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d28100e20 msgr2=0x7f0d281962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:35.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d28100e20 0x7f0d281962a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009830 tx=0x7f0d240055e0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 shutdown_connections 2024-09-19T07:44:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0d140a7990 0x7f0d140a9e50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d28102ef0 0x7f0d2819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d28102570 0x7f0d281967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 --2- 172.21.15.103:0/118991033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d28100e20 0x7f0d281962a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.428+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 >> 172.21.15.103:0/118991033 conn(0x7f0d280fc7d0 msgr2=0x7f0d280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.429+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 shutdown_connections 2024-09-19T07:44:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:35.429+0000 7f0d2f796640 1 -- 172.21.15.103:0/118991033 wait complete. 2024-09-19T07:44:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 360.0 s, 3237.0 tps, lat 30.910 ms stddev 34.001 2024-09-19T07:44:36.155 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:35 smithi103 ceph-mon[31201]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 884 op/s 2024-09-19T07:44:36.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:44:36.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:35 smithi174 ceph-mon[35370]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 884 op/s 2024-09-19T07:44:36.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:35 smithi134 ceph-mon[35414]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 884 op/s 2024-09-19T07:44:36.516 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:37.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.118+0000 7fe7a0283640 1 -- 172.21.15.103:0/3255216454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 msgr2=0x7fe798101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.118+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3255216454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 0x7fe798101200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe794009900 tx=0x7fe79402a4a0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.119+0000 7fe7a0283640 1 -- 172.21.15.103:0/3255216454 shutdown_connections 2024-09-19T07:44:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.119+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3255216454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 0x7fe79810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.119+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3255216454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe798102570 0x7fe7981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.119+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3255216454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 0x7fe798101200 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.119+0000 7fe7a0283640 1 -- 172.21.15.103:0/3255216454 >> 172.21.15.103:0/3255216454 conn(0x7fe7980fc7d0 msgr2=0x7fe7980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.119+0000 7fe7a0283640 1 -- 172.21.15.103:0/3255216454 shutdown_connections 2024-09-19T07:44:37.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.119+0000 7fe7a0283640 1 -- 172.21.15.103:0/3255216454 wait complete. 2024-09-19T07:44:37.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 Processor -- start 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 -- start start 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 0x7fe798073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe798102570 0x7fe79806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 0x7fe79806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe798110ff0 con 0x7fe798100e20 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe798110e70 con 0x7fe798102570 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe7a0283640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe798111170 con 0x7fe798102ef0 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe79e01f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 0x7fe798073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe79e820640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 0x7fe79806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe79e820640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 0x7fe79806f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56250/0 (socket says 172.21.15.103:56250) 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.120+0000 7fe79e820640 1 -- 172.21.15.103:0/3505502021 learned_addr learned my addr 172.21.15.103:0/3505502021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:37.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe79d81e640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe798102570 0x7fe79806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe79e820640 1 -- 172.21.15.103:0/3505502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe798102570 msgr2=0x7fe79806ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe79e820640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe798102570 0x7fe79806ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe79e820640 1 -- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 msgr2=0x7fe798073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe79e820640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 0x7fe798073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe79e820640 1 -- 172.21.15.103:0/3505502021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79806f750 con 0x7fe798102ef0 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe79e820640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 0x7fe79806f110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b970 tx=0x7fe79000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe78f7fe640 1 -- 172.21.15.103:0/3505502021 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790019070 con 0x7fe798102ef0 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe78f7fe640 1 -- 172.21.15.103:0/3505502021 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe79000cb40 con 0x7fe798102ef0 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7981ad500 con 0x7fe798102ef0 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe78f7fe640 1 -- 172.21.15.103:0/3505502021 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790007650 con 0x7fe798102ef0 2024-09-19T07:44:37.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.121+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981ad8a0 con 0x7fe798102ef0 2024-09-19T07:44:37.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.125+0000 7fe7a265c740 1 Processor -- start 2024-09-19T07:44:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.125+0000 7fe78f7fe640 1 -- 172.21.15.103:0/3505502021 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe79001c070 con 0x7fe798102ef0 2024-09-19T07:44:37.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.125+0000 7fe7a265c740 1 -- start start 2024-09-19T07:44:37.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3ace090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3acea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3add780 0x55a0f3adfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe7a265c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a0f3a342f0 con 0x55a0f3add780 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe7a265c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a0f39dbcf0 con 0x55a0f3ace5d0 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe7a265c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0f3a302b0 con 0x55a0f3acfb40 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3ace090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3ace090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56258/0 (socket says 172.21.15.103:56258) 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 -- 172.21.15.103:0/430952045 learned_addr learned my addr 172.21.15.103:0/430952045 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3add780 0x55a0f3adfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79e01f640 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3acea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe78f7fe640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe7840a33b0 0x7fe7840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 -- 172.21.15.103:0/430952045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 msgr2=0x55a0f3acea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3acea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 -- 172.21.15.103:0/430952045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3add780 msgr2=0x55a0f3adfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3add780 0x55a0f3adfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 -- 172.21.15.103:0/430952045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f39c03a0 con 0x55a0f3acfb40 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe78f7fe640 1 -- 172.21.15.103:0/3505502021 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe7900cf1f0 con 0x7fe798102ef0 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.126+0000 7fe79d81e640 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3ace090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe788009fd0 tx=0x7fe788009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95bd37631c0cd5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.127+0000 7fe79e820640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe7840a33b0 0x7fe7840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.127+0000 7fe78cdea640 1 -- 172.21.15.103:0/430952045 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78800f790 con 0x55a0f3acfb40 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.127+0000 7fe78cdea640 1 -- 172.21.15.103:0/430952045 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78800fe10 con 0x55a0f3acfb40 2024-09-19T07:44:37.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.127+0000 7fe78cdea640 1 -- 172.21.15.103:0/430952045 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78801fe30 con 0x55a0f3acfb40 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.128+0000 7fe7a265c740 1 -- 172.21.15.103:0/430952045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 msgr2=0x55a0f3ace090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.128+0000 7fe7a265c740 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3ace090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe788009fd0 tx=0x7fe788009370 comp rx=0 tx=0).stop 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe79e820640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe7840a33b0 0x7fe7840a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7900a3d30 tx=0x7fe790004b50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe7a265c740 1 -- 172.21.15.103:0/430952045 shutdown_connections 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe7a265c740 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3add780 0x55a0f3adfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe7a265c740 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3acea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe7a265c740 1 --2- 172.21.15.103:0/430952045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3ace090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe7a265c740 1 -- 172.21.15.103:0/430952045 >> 172.21.15.103:0/430952045 conn(0x55a0f3a97310 msgr2=0x55a0f3acac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe7a265c740 1 -- 172.21.15.103:0/430952045 shutdown_connections 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.129+0000 7fe7a265c740 1 -- 172.21.15.103:0/430952045 wait complete. 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 -- start start 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 Processor -- start 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3add1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ada460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3add780 0x55a0f3adabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a0f3ae0480 con 0x55a0f3acfb40 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a0f39ef060 con 0x55a0f3ace5d0 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0f3ae0600 con 0x55a0f3add780 2024-09-19T07:44:37.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe7a265c740 10 client.0 _ll_register_callbacks cb 0x55a0f37428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:37.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe79d81e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3add780 0x55a0f3adabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe79d81e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3add780 0x55a0f3adabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:56270/0 (socket says 172.21.15.103:56270) 2024-09-19T07:44:37.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe79d81e640 1 -- 172.21.15.103:0/2698915016 learned_addr learned my addr 172.21.15.103:0/2698915016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:37.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe79d81e640 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3add1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.130+0000 7fe79e01f640 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ada460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe79d81e640 1 -- 172.21.15.103:0/2698915016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 msgr2=0x55a0f3add1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:37.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe79d81e640 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3add1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe79d81e640 1 -- 172.21.15.103:0/2698915016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 msgr2=0x55a0f3ada460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe79d81e640 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ada460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe79d81e640 1 -- 172.21.15.103:0/2698915016 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f37a53c0 con 0x55a0f3add780 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe79d81e640 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3add780 0x55a0f3adabf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe78801fb10 tx=0x7fe78801fa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db2135e356354ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe781ffb640 1 -- 172.21.15.103:0/2698915016 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788004230 con 0x55a0f3add780 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.131+0000 7fe781ffb640 1 -- 172.21.15.103:0/2698915016 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe7880043d0 con 0x55a0f3add780 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.132+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78800fe10 con 0x55a0f3add780 2024-09-19T07:44:37.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.132+0000 7fe781ffb640 1 -- 172.21.15.103:0/2698915016 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe78800f790 con 0x55a0f3add780 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.132+0000 7fe7a265c740 10 client.26215 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.132+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a0f3ad1310 con 0x55a0f3add780 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.132+0000 7fe781ffb640 1 -- 172.21.15.103:0/2698915016 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78800fe10 con 0x55a0f3add780 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe781ffb640 1 -- 172.21.15.103:0/2698915016 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fe78801d020 con 0x55a0f3add780 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe7a265c740 10 client.26215 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe7a265c740 10 client.26215 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe7a265c740 20 client.26215 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3ad26c0 0x55a0f3ad4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe7a265c740 4 client.26215 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0f3a41860 con 0x55a0f3ad26c0 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe79e01f640 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3ad26c0 0x55a0f3ad4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.133+0000 7fe781ffb640 10 client.26215 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.134+0000 7fe79e01f640 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3ad26c0 0x55a0f3ad4ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd8bbf3c0d44b0bb server_cookie=d45dd664e7d91285 in_seq=0 out_seq=0 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.135+0000 7fe781ffb640 1 -- 172.21.15.103:0/2698915016 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0f3a41860 con 0x55a0f3ad26c0 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.135+0000 7fe781ffb640 10 client.26215 handle_command_reply: tid=0 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.135+0000 7fe7a265c740 1 client.26215 shutdown 2024-09-19T07:44:37.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.135+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3add780 msgr2=0x55a0f3adabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.135+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3add780 0x55a0f3adabf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe78801fb10 tx=0x7fe78801fa10 comp rx=0 tx=0).stop 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 shutdown_connections 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3ad26c0 0x55a0f3ad4ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe781ffb640 0 client.26215 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3add780 0x55a0f3adabf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ada460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2698915016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3add1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 >> 172.21.15.103:0/2698915016 conn(0x55a0f3a97310 msgr2=0x55a0f3ad0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 shutdown_connections 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 1 -- 172.21.15.103:0/2698915016 wait complete. 2024-09-19T07:44:37.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.136+0000 7fe7a265c740 20 client.26215 trim_cache size 0 max 16384 2024-09-19T07:44:37.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 Processor -- start 2024-09-19T07:44:37.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 -- start start 2024-09-19T07:44:37.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3ad76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ad7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3ad26c0 0x55a0f3ad5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b3ed0 con 0x55a0f3acfb40 2024-09-19T07:44:37.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b5a10 con 0x55a0f3ace5d0 2024-09-19T07:44:37.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.158+0000 7fe7a265c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b45a0 con 0x55a0f3ad26c0 2024-09-19T07:44:37.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ad7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ad7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51500/0 (socket says 172.21.15.103:51500) 2024-09-19T07:44:37.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 -- 172.21.15.103:0/2628736630 learned_addr learned my addr 172.21.15.103:0/2628736630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:37.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79d81e640 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3ad26c0 0x55a0f3ad5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79d81e640 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3ad76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 -- 172.21.15.103:0/2628736630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3ad26c0 msgr2=0x55a0f3ad5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3ad26c0 0x55a0f3ad5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 -- 172.21.15.103:0/2628736630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 msgr2=0x55a0f3ad76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3ad76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79e01f640 1 -- 172.21.15.103:0/2628736630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f3acf3a0 con 0x55a0f3acfb40 2024-09-19T07:44:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.159+0000 7fe79d81e640 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3ad26c0 0x55a0f3ad5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:37.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.160+0000 7fe79e01f640 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ad7c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe79402a9c0 tx=0x7fe794043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac16a136530c3af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.160+0000 7fe783fff640 1 -- 172.21.15.103:0/2628736630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794035040 con 0x55a0f3acfb40 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.160+0000 7fe783fff640 1 -- 172.21.15.103:0/2628736630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a0f3a41860 con 0x55a0f3acfb40 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.160+0000 7fe783fff640 1 -- 172.21.15.103:0/2628736630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794004e60 con 0x55a0f3acfb40 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.161+0000 7fe7a265c740 1 -- 172.21.15.103:0/2628736630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 msgr2=0x55a0f3ad7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.161+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ad7c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe79402a9c0 tx=0x7fe794043040 comp rx=0 tx=0).stop 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.161+0000 7fe7a265c740 1 -- 172.21.15.103:0/2628736630 shutdown_connections 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.161+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3ad26c0 0x55a0f3ad5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.161+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3acfb40 0x55a0f3ad7c20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.161+0000 7fe7a265c740 1 --2- 172.21.15.103:0/2628736630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ace5d0 0x55a0f3ad76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.161+0000 7fe7a265c740 1 -- 172.21.15.103:0/2628736630 >> 172.21.15.103:0/2628736630 conn(0x55a0f3a97310 msgr2=0x55a0f3ad9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.162+0000 7fe7a265c740 1 -- 172.21.15.103:0/2628736630 shutdown_connections 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.162+0000 7fe7a265c740 1 -- 172.21.15.103:0/2628736630 wait complete. 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.162+0000 7fe7a265c740 1 -- start start 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 1 Processor -- start 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3ace5d0 0x55a0f3aee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3aeece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ad26c0 0x55a0f3b4af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b5340 con 0x55a0f3ace5d0 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b4c70 con 0x55a0f3ad26c0 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7981b1eb0 con 0x55a0f3acfb40 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe7a265c740 10 client.0 _ll_register_callbacks cb 0x7fe788009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:37.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe79d81e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ad26c0 0x55a0f3b4af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe79d81e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ad26c0 0x55a0f3b4af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:36430/0 (socket says 172.21.15.103:36430) 2024-09-19T07:44:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.163+0000 7fe79d81e640 1 -- 172.21.15.103:0/1548298415 learned_addr learned my addr 172.21.15.103:0/1548298415 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.164+0000 7fe79d81e640 1 -- 172.21.15.103:0/1548298415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 msgr2=0x55a0f3aeece0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:44:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.164+0000 7fe79d81e640 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3aeece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.164+0000 7fe79d81e640 1 -- 172.21.15.103:0/1548298415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3ace5d0 msgr2=0x55a0f3aee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.164+0000 7fe79d81e640 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3ace5d0 0x55a0f3aee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.164+0000 7fe79d81e640 1 -- 172.21.15.103:0/1548298415 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f3aca510 con 0x55a0f3ad26c0 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.164+0000 7fe79d81e640 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ad26c0 0x55a0f3b4af20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe78803e440 tx=0x7fe788009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b5709d5714f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.164+0000 7fe781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.165+0000 7fe781ffb640 1 -- 172.21.15.103:0/1548298415 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7880041d0 con 0x55a0f3ad26c0 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.165+0000 7fe781ffb640 1 -- 172.21.15.103:0/1548298415 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe788004370 con 0x55a0f3ad26c0 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.165+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0f3a41860 con 0x55a0f3ad26c0 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.165+0000 7fe781ffb640 1 -- 172.21.15.103:0/1548298415 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe788058050 con 0x55a0f3ad26c0 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.165+0000 7fe7a265c740 10 client.26303 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:37.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.165+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a0f3b53930 con 0x55a0f3ad26c0 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe781ffb640 1 -- 172.21.15.103:0/1548298415 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a0f3a41860 con 0x55a0f3ad26c0 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe781ffb640 1 -- 172.21.15.103:0/1548298415 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fe78801d070 con 0x55a0f3ad26c0 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe7a265c740 10 client.26303 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe7a265c740 10 client.26303 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe7a265c740 20 client.26303 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe7a265c740 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3add780 0x55a0f3b56060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe7a265c740 4 client.26303 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.166+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0f3a3bd70 con 0x55a0f3add780 2024-09-19T07:44:37.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.169+0000 7fe79e01f640 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3add780 0x55a0f3b56060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.177+0000 7fe79e01f640 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3add780 0x55a0f3b56060 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c65b623999b12d13 server_cookie=f5d715922039c082 in_seq=0 out_seq=0 2024-09-19T07:44:37.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.178+0000 7fe781ffb640 10 client.26303 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:44:37.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.179+0000 7fe781ffb640 1 -- 172.21.15.103:0/1548298415 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0f3a3bd70 con 0x55a0f3add780 2024-09-19T07:44:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.179+0000 7fe781ffb640 10 client.26303 handle_command_reply: tid=0 2024-09-19T07:44:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.179+0000 7fe7a265c740 1 client.26303 shutdown 2024-09-19T07:44:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.179+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ad26c0 msgr2=0x55a0f3b4af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.179+0000 7fe7a265c740 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ad26c0 0x55a0f3b4af20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe78803e440 tx=0x7fe788009da0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 shutdown_connections 2024-09-19T07:44:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe7a265c740 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a0f3add780 0x55a0f3b56060 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe7a265c740 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a0f3ad26c0 0x55a0f3b4af20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe7a265c740 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0f3acfb40 0x55a0f3aeece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe7a265c740 1 --2- 172.21.15.103:0/1548298415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0f3ace5d0 0x55a0f3aee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 >> 172.21.15.103:0/1548298415 conn(0x55a0f3a97310 msgr2=0x55a0f3ac94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe781ffb640 0 client.26303 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:44:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.182+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 shutdown_connections 2024-09-19T07:44:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.183+0000 7fe7a265c740 1 -- 172.21.15.103:0/1548298415 wait complete. 2024-09-19T07:44:37.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.183+0000 7fe7a265c740 20 client.26303 trim_cache size 0 max 16384 2024-09-19T07:44:37.188 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:44:37.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.188+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe7840a33b0 msgr2=0x7fe7840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.188+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe7840a33b0 0x7fe7840a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7900a3d30 tx=0x7fe790004b50 comp rx=0 tx=0).stop 2024-09-19T07:44:37.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.188+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 msgr2=0x7fe79806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:37.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.188+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 0x7fe79806f110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b970 tx=0x7fe79000be30 comp rx=0 tx=0).stop 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 shutdown_connections 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe7840a33b0 0x7fe7840a5870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe798102ef0 0x7fe79806f110 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe798102570 0x7fe79806ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 --2- 172.21.15.103:0/3505502021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe798100e20 0x7fe798073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 >> 172.21.15.103:0/3505502021 conn(0x7fe7980fc7d0 msgr2=0x7fe7981068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 shutdown_connections 2024-09-19T07:44:37.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:37.189+0000 7fe7a0283640 1 -- 172.21.15.103:0/3505502021 wait complete. 2024-09-19T07:44:38.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:44:38.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:37 smithi174 ceph-mon[35370]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 693 op/s 2024-09-19T07:44:38.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:37 smithi103 ceph-mon[31201]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 693 op/s 2024-09-19T07:44:38.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:37 smithi134 ceph-mon[35414]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 693 op/s 2024-09-19T07:44:38.407 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:38.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 -- 172.21.15.103:0/670636990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c109e00 msgr2=0x7f660c10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 --2- 172.21.15.103:0/670636990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c109e00 0x7f660c10e350 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009900 tx=0x7f65f402a410 comp rx=0 tx=0).stop 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 -- 172.21.15.103:0/670636990 shutdown_connections 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 --2- 172.21.15.103:0/670636990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f660c10ea80 0x7f660c1154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 --2- 172.21.15.103:0/670636990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c109e00 0x7f660c10e350 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 --2- 172.21.15.103:0/670636990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f660c109450 0x7f660c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 -- 172.21.15.103:0/670636990 >> 172.21.15.103:0/670636990 conn(0x7f660c06e2d0 msgr2=0x7f660c06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.984+0000 7f661289a640 1 -- 172.21.15.103:0/670636990 shutdown_connections 2024-09-19T07:44:38.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.985+0000 7f661289a640 1 -- 172.21.15.103:0/670636990 wait complete. 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.985+0000 7f661289a640 1 Processor -- start 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.985+0000 7f661289a640 1 -- start start 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f661289a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f660c109450 0x7f660c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f661289a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f660c109e00 0x7f660c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f661289a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c10ea80 0x7f660c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f661289a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1185d0 con 0x7f660c10ea80 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f661289a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f660c118450 con 0x7f660c109450 2024-09-19T07:44:38.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f661289a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f660c118750 con 0x7f660c109e00 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c10ea80 0x7f660c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c10ea80 0x7f660c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53748/0 (socket says 172.21.15.103:53748) 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 -- 172.21.15.103:0/2992833364 learned_addr learned my addr 172.21.15.103:0/2992833364 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f660bfff640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f660c109450 0x7f660c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f660b7fe640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f660c109e00 0x7f660c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 -- 172.21.15.103:0/2992833364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f660c109e00 msgr2=0x7f660c1a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f660c109e00 0x7f660c1a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 -- 172.21.15.103:0/2992833364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f660c109450 msgr2=0x7f660c1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f660c109450 0x7f660c1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 -- 172.21.15.103:0/2992833364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c19efe0 con 0x7f660c10ea80 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f6610e37640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c10ea80 0x7f660c19e8b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f660400eb10 tx=0x7f660400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:38.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.986+0000 7f66097fa640 1 -- 172.21.15.103:0/2992833364 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604004b20 con 0x7f660c10ea80 2024-09-19T07:44:38.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.987+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c19f2d0 con 0x7f660c10ea80 2024-09-19T07:44:38.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.987+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c1ab5c0 con 0x7f660c10ea80 2024-09-19T07:44:38.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.987+0000 7f66097fa640 1 -- 172.21.15.103:0/2992833364 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6604004cc0 con 0x7f660c10ea80 2024-09-19T07:44:38.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.987+0000 7f66097fa640 1 -- 172.21.15.103:0/2992833364 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604010760 con 0x7f660c10ea80 2024-09-19T07:44:38.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.990+0000 7f66097fa640 1 -- 172.21.15.103:0/2992833364 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6604005020 con 0x7f660c10ea80 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 Processor -- start 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 -- start start 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b7ab40 0x55a648b79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b795d0 0x55a648b79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a648adf2f0 con 0x55a648b795d0 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a648a86cf0 con 0x55a648b7ab40 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6614c73740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a648adb2b0 con 0x55a648b84370 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f66097fa640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f65f80a7880 0x7f65f80a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f660bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b795d0 0x55a648b79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f660b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:38.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f660bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b795d0 0x55a648b79a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53756/0 (socket says 172.21.15.103:53756) 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f660b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b86760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:57218/0 (socket says 172.21.15.103:57218) 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f660bfff640 1 -- 172.21.15.103:0/4006387579 learned_addr learned my addr 172.21.15.103:0/4006387579 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f660b7fe640 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b7ab40 0x55a648b79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.991+0000 7f6610e37640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f65f80a7880 0x7f65f80a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f66097fa640 1 -- 172.21.15.103:0/2992833364 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f66040d0250 con 0x7f660c10ea80 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f660b7fe640 1 -- 172.21.15.103:0/4006387579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b7ab40 msgr2=0x55a648b79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f660b7fe640 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b7ab40 0x55a648b79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f660b7fe640 1 -- 172.21.15.103:0/4006387579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b795d0 msgr2=0x55a648b79a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f660b7fe640 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b795d0 0x55a648b79a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f660b7fe640 1 -- 172.21.15.103:0/4006387579 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a648a6b3a0 con 0x55a648b84370 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f660bfff640 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b795d0 0x55a648b79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:38.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f660b7fe640 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b86760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009850 tx=0x7f65f4005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee6c264742dd2c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:38.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f6610e37640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f65f80a7880 0x7f65f80a9d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66040cda60 tx=0x7f660400efd0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:38.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f66027fc640 1 -- 172.21.15.103:0/4006387579 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4035030 con 0x55a648b84370 2024-09-19T07:44:38.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f66027fc640 1 -- 172.21.15.103:0/4006387579 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f40046c0 con 0x55a648b84370 2024-09-19T07:44:38.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.992+0000 7f66027fc640 1 -- 172.21.15.103:0/4006387579 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4003850 con 0x55a648b84370 2024-09-19T07:44:38.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.995+0000 7f6614c73740 1 -- 172.21.15.103:0/4006387579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 msgr2=0x55a648b86760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:38.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.995+0000 7f6614c73740 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b86760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009850 tx=0x7f65f4005710 comp rx=0 tx=0).stop 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.995+0000 7f6614c73740 1 -- 172.21.15.103:0/4006387579 shutdown_connections 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.995+0000 7f6614c73740 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b86760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.995+0000 7f6614c73740 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b795d0 0x55a648b79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.995+0000 7f6614c73740 1 --2- 172.21.15.103:0/4006387579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b7ab40 0x55a648b79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.995+0000 7f6614c73740 1 -- 172.21.15.103:0/4006387579 >> 172.21.15.103:0/4006387579 conn(0x55a648b42310 msgr2=0x55a648b75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.996+0000 7f6614c73740 1 -- 172.21.15.103:0/4006387579 shutdown_connections 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.996+0000 7f6614c73740 1 -- 172.21.15.103:0/4006387579 wait complete. 2024-09-19T07:44:38.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.996+0000 7f6614c73740 1 -- start start 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.996+0000 7f6614c73740 1 Processor -- start 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f6614c73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f6614c73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b81060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f6614c73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f6614c73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a648b87010 con 0x55a648b7ab40 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f6614c73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a648a9a060 con 0x55a648b795d0 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f6614c73740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a648b87190 con 0x55a648b84370 2024-09-19T07:44:39.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b81060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f6614c73740 10 client.0 _ll_register_callbacks cb 0x55a6487ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b81060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53762/0 (socket says 172.21.15.103:53762) 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 -- 172.21.15.103:0/909741861 learned_addr learned my addr 172.21.15.103:0/909741861 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660b7fe640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660b7fe640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 -- 172.21.15.103:0/909741861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 msgr2=0x55a648b817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 -- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b795d0 msgr2=0x55a648b83db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 -- 172.21.15.103:0/909741861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6488503c0 con 0x55a648b7ab40 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660b7fe640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:39.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f660bfff640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b81060 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009550 tx=0x7f65fc009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2d954ee498061f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.997+0000 7f65eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.998+0000 7f65eb7fe640 1 -- 172.21.15.103:0/909741861 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc01b020 con 0x55a648b7ab40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.998+0000 7f65eb7fe640 1 -- 172.21.15.103:0/909741861 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f65fc017870 con 0x55a648b7ab40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.998+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f40046c0 con 0x55a648b7ab40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.998+0000 7f65eb7fe640 1 -- 172.21.15.103:0/909741861 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f65fc026050 con 0x55a648b7ab40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.998+0000 7f6614c73740 10 client.16806 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.998+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a648b92bf0 con 0x55a648b7ab40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.998+0000 7f65eb7fe640 1 -- 172.21.15.103:0/909741861 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f40046c0 con 0x55a648b7ab40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f65eb7fe640 1 -- 172.21.15.103:0/909741861 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f65fc0163c0 con 0x55a648b7ab40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f6614c73740 10 client.16806 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f6614c73740 10 client.16806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:44:39.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f6614c73740 20 client.16806 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f6614c73740 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648b96560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f6614c73740 4 client.16806 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a648aec860 con 0x55a648b94100 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f660b7fe640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648b96560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:38.999+0000 7f660b7fe640 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648b96560 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=883e9a22eaface6a server_cookie=aeeff35c5ae76edf in_seq=0 out_seq=0 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.000+0000 7f65eb7fe640 10 client.16806 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.000+0000 7f65eb7fe640 1 -- 172.21.15.103:0/909741861 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a648aec860 con 0x55a648b94100 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.000+0000 7f65eb7fe640 10 client.16806 handle_command_reply: tid=0 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.000+0000 7f6614c73740 1 client.16806 shutdown 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.000+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 msgr2=0x55a648b81060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.000+0000 7f6614c73740 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b81060 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009550 tx=0x7f65fc009a10 comp rx=0 tx=0).stop 2024-09-19T07:44:39.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 shutdown_connections 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648b96560 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f65eb7fe640 0 client.16806 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b84370 0x55a648b817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b81060 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 --2- 172.21.15.103:0/909741861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 >> 172.21.15.103:0/909741861 conn(0x55a648b42310 msgr2=0x55a648b7b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 shutdown_connections 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 1 -- 172.21.15.103:0/909741861 wait complete. 2024-09-19T07:44:39.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.001+0000 7f6614c73740 20 client.16806 trim_cache size 0 max 16384 2024-09-19T07:44:39.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.020+0000 7f6614c73740 1 Processor -- start 2024-09-19T07:44:39.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.020+0000 7f6614c73740 1 -- start start 2024-09-19T07:44:39.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f6614c73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f6614c73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b7a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f6614c73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b75700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f6614c73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec00e750 con 0x55a648b7ab40 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f6614c73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec010290 con 0x55a648b84370 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f6614c73740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec00ee20 con 0x55a648b795d0 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:57232/0 (socket says 172.21.15.103:57232) 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 -- 172.21.15.103:0/1419775100 learned_addr learned my addr 172.21.15.103:0/1419775100 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b7a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b75700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 -- 172.21.15.103:0/1419775100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 msgr2=0x55a648b75700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b75700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 -- 172.21.15.103:0/1419775100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 msgr2=0x55a648b7a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:39.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b7a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.021+0000 7f660b7fe640 1 -- 172.21.15.103:0/1419775100 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a648b76960 con 0x55a648b795d0 2024-09-19T07:44:39.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.022+0000 7f660b7fe640 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b83db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f65f4004650 tx=0x7f65f4005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9176a6e343b50904 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:39.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.022+0000 7f6601ffb640 1 -- 172.21.15.103:0/1419775100 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4035040 con 0x55a648b795d0 2024-09-19T07:44:39.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.022+0000 7f6601ffb640 1 -- 172.21.15.103:0/1419775100 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f4004da0 con 0x55a648b795d0 2024-09-19T07:44:39.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.022+0000 7f6601ffb640 1 -- 172.21.15.103:0/1419775100 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4031d50 con 0x55a648b795d0 2024-09-19T07:44:39.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 -- 172.21.15.103:0/1419775100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 msgr2=0x55a648b83db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b83db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f65f4004650 tx=0x7f65f4005710 comp rx=0 tx=0).stop 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 -- 172.21.15.103:0/1419775100 shutdown_connections 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b75700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b7a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 --2- 172.21.15.103:0/1419775100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b83db0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 -- 172.21.15.103:0/1419775100 >> 172.21.15.103:0/1419775100 conn(0x55a648b42310 msgr2=0x55a648b70a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 -- 172.21.15.103:0/1419775100 shutdown_connections 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.023+0000 7f6614c73740 1 -- 172.21.15.103:0/1419775100 wait complete. 2024-09-19T07:44:39.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.024+0000 7f6614c73740 1 -- start start 2024-09-19T07:44:39.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.024+0000 7f6614c73740 1 Processor -- start 2024-09-19T07:44:39.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f6614c73740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f6614c73740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b87d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f6614c73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b88260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f6614c73740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec00fbc0 con 0x55a648b7ab40 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f660b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f660b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b8c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:57238/0 (socket says 172.21.15.103:57238) 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f660b7fe640 1 -- 172.21.15.103:0/791938023 learned_addr learned my addr 172.21.15.103:0/791938023 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec00f4f0 con 0x55a648b84370 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f660bfff640 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b87d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec00c730 con 0x55a648b795d0 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f660b7fe640 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b88260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.025+0000 7f6614c73740 10 client.0 _ll_register_callbacks cb 0x7f65fc00a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.026+0000 7f660b7fe640 1 -- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 msgr2=0x55a648b88260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:39.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.026+0000 7f660b7fe640 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b88260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.026+0000 7f660b7fe640 1 -- 172.21.15.103:0/791938023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 msgr2=0x55a648b87d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:39.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.026+0000 7f660b7fe640 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b87d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.026+0000 7f660b7fe640 1 -- 172.21.15.103:0/791938023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a648b754f0 con 0x55a648b795d0 2024-09-19T07:44:39.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f660b7fe640 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b8c9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65f40056e0 tx=0x7f65f4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c72c8f4d6054e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:39.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f65eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:39.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f65eb7fe640 1 -- 172.21.15.103:0/791938023 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4035030 con 0x55a648b795d0 2024-09-19T07:44:39.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f4004da0 con 0x55a648b795d0 2024-09-19T07:44:39.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f65eb7fe640 1 -- 172.21.15.103:0/791938023 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f65f4005010 con 0x55a648b795d0 2024-09-19T07:44:39.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f65eb7fe640 1 -- 172.21.15.103:0/791938023 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f65f4005210 con 0x55a648b795d0 2024-09-19T07:44:39.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f6614c73740 10 client.26236 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:39.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.027+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a648bfea40 con 0x55a648b795d0 2024-09-19T07:44:39.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f65eb7fe640 1 -- 172.21.15.103:0/791938023 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65f4004da0 con 0x55a648b795d0 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f65eb7fe640 1 -- 172.21.15.103:0/791938023 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f65f404c850 con 0x55a648b795d0 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f6614c73740 10 client.26236 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f6614c73740 10 client.26236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f6614c73740 20 client.26236 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f6614c73740 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648c00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f6614c73740 4 client.26236 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a648ae6d70 con 0x55a648b94100 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.028+0000 7f660bfff640 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648c00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.029+0000 7f660bfff640 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648c00c80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=589e80f6b9d10de6 server_cookie=af14a099aa27b9a6 in_seq=0 out_seq=0 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.029+0000 7f65eb7fe640 10 client.26236 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.029+0000 7f65eb7fe640 1 -- 172.21.15.103:0/791938023 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a648ae6d70 con 0x55a648b94100 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.029+0000 7f65eb7fe640 10 client.26236 handle_command_reply: tid=0 2024-09-19T07:44:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.029+0000 7f6614c73740 1 client.26236 shutdown 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 msgr2=0x55a648b8c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b8c9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f65f40056e0 tx=0x7f65f4009680 comp rx=0 tx=0).stop 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 shutdown_connections 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a648b94100 0x55a648c00c80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a648b84370 0x55a648b88260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a648b7ab40 0x55a648b87d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 --2- 172.21.15.103:0/791938023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a648b795d0 0x55a648b8c9f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 >> 172.21.15.103:0/791938023 conn(0x55a648b42310 msgr2=0x55a648b7f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f65eb7fe640 0 client.26236 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 shutdown_connections 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.030+0000 7f6614c73740 1 -- 172.21.15.103:0/791938023 wait complete. 2024-09-19T07:44:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.031+0000 7f6614c73740 20 client.26236 trim_cache size 0 max 16384 2024-09-19T07:44:39.034 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:44:39.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.031+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f65f80a7880 msgr2=0x7f65f80a9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.031+0000 7f661289a640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f65f80a7880 0x7f65f80a9d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66040cda60 tx=0x7f660400efd0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.031+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c10ea80 msgr2=0x7f660c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.031+0000 7f661289a640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c10ea80 0x7f660c19e8b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f660400eb10 tx=0x7f660400c2d0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 shutdown_connections 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f65f80a7880 0x7f65f80a9d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f660c10ea80 0x7f660c19e8b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f660c109e00 0x7f660c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 --2- 172.21.15.103:0/2992833364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f660c109450 0x7f660c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 >> 172.21.15.103:0/2992833364 conn(0x7f660c06e2d0 msgr2=0x7f660c10c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 shutdown_connections 2024-09-19T07:44:39.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:39.033+0000 7f661289a640 1 -- 172.21.15.103:0/2992833364 wait complete. 2024-09-19T07:44:39.818 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:44:40.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:39 smithi174 ceph-mon[35370]: pgmap v203: 97 pgs: 97 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 637 op/s 2024-09-19T07:44:40.174 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:40.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:39 smithi103 ceph-mon[31201]: pgmap v203: 97 pgs: 97 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 637 op/s 2024-09-19T07:44:40.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:39 smithi134 ceph-mon[35414]: pgmap v203: 97 pgs: 97 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 637 op/s 2024-09-19T07:44:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 365.0 s, 2953.0 tps, lat 33.758 ms stddev 34.523 2024-09-19T07:44:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 -- 172.21.15.103:0/1415433104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2441036e0 msgr2=0x7fe244107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 --2- 172.21.15.103:0/1415433104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2441036e0 0x7fe244107b90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe238009930 tx=0x7fe23802a420 comp rx=0 tx=0).stop 2024-09-19T07:44:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 -- 172.21.15.103:0/1415433104 shutdown_connections 2024-09-19T07:44:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 --2- 172.21.15.103:0/1415433104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441082a0 0x7fe24410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 --2- 172.21.15.103:0/1415433104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2441036e0 0x7fe244107b90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 --2- 172.21.15.103:0/1415433104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe244102d30 0x7fe244103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 -- 172.21.15.103:0/1415433104 >> 172.21.15.103:0/1415433104 conn(0x7fe2440fc7d0 msgr2=0x7fe2440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 -- 172.21.15.103:0/1415433104 shutdown_connections 2024-09-19T07:44:41.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.083+0000 7fe249511640 1 -- 172.21.15.103:0/1415433104 wait complete. 2024-09-19T07:44:41.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 Processor -- start 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 -- start start 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe244102d30 0x7fe244198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2441036e0 0x7fe244198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441082a0 0x7fe24419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe244112060 con 0x7fe244102d30 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe244111ee0 con 0x7fe2441036e0 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe249511640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441121e0 con 0x7fe2441082a0 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe244102d30 0x7fe244198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe244102d30 0x7fe244198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53802/0 (socket says 172.21.15.103:53802) 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe242ffd640 1 -- 172.21.15.103:0/2364530434 learned_addr learned my addr 172.21.15.103:0/2364530434 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.084+0000 7fe2427fc640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2441036e0 0x7fe244198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441082a0 0x7fe24419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe242ffd640 1 -- 172.21.15.103:0/2364530434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441082a0 msgr2=0x7fe24419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe242ffd640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441082a0 0x7fe24419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe242ffd640 1 -- 172.21.15.103:0/2364530434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2441036e0 msgr2=0x7fe244198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe242ffd640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2441036e0 0x7fe244198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe242ffd640 1 -- 172.21.15.103:0/2364530434 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24419d5b0 con 0x7fe244102d30 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441082a0 0x7fe24419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe242ffd640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe244102d30 0x7fe244198600 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe23400e9f0 tx=0x7fe23400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe22bfff640 1 -- 172.21.15.103:0/2364530434 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23400cd80 con 0x7fe244102d30 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe22bfff640 1 -- 172.21.15.103:0/2364530434 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe234010910 con 0x7fe244102d30 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe22bfff640 1 -- 172.21.15.103:0/2364530434 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234018990 con 0x7fe244102d30 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24419d8a0 con 0x7fe244102d30 2024-09-19T07:44:41.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.085+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2441a5180 con 0x7fe244102d30 2024-09-19T07:44:41.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.089+0000 7fe22bfff640 1 -- 172.21.15.103:0/2364530434 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe234010ac0 con 0x7fe244102d30 2024-09-19T07:44:41.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 Processor -- start 2024-09-19T07:44:41.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 -- start start 2024-09-19T07:44:41.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe22bfff640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe22c0a33b0 0x7fe22c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaedb40 0x560bddaec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaec5d0 0x560bddaeca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaf9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560bdda522f0 con 0x560bddaedb40 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560bdd9f9cf0 con 0x560bddaf7370 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bdda4e2b0 con 0x560bddaec5d0 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe22c0a33b0 0x7fe22c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe2427fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaec5d0 0x560bddaeca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe22bfff640 1 -- 172.21.15.103:0/2364530434 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe2340d00d0 con 0x7fe244102d30 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaedb40 0x560bddaec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaedb40 0x560bddaec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53818/0 (socket says 172.21.15.103:53818) 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe242ffd640 1 -- 172.21.15.103:0/2847025690 learned_addr learned my addr 172.21.15.103:0/2847025690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaf9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe242ffd640 1 -- 172.21.15.103:0/2847025690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaec5d0 msgr2=0x560bddaeca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.090+0000 7fe242ffd640 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaec5d0 0x560bddaeca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe242ffd640 1 -- 172.21.15.103:0/2847025690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 msgr2=0x560bddaf9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe242ffd640 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaf9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe242ffd640 1 -- 172.21.15.103:0/2847025690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdd9de3a0 con 0x560bddaedb40 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe2427fc640 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaec5d0 0x560bddaeca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaf9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe242ffd640 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaedb40 0x560bddaec090 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe2340a4b00 tx=0x7fe23409bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94647756ce1e67e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe22c0a33b0 0x7fe22c0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe23c005fd0 tx=0x7fe23c009e30 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe2297fa640 1 -- 172.21.15.103:0/2847025690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340a0770 con 0x560bddaedb40 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe2297fa640 1 -- 172.21.15.103:0/2847025690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2340a5030 con 0x560bddaedb40 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.091+0000 7fe2297fa640 1 -- 172.21.15.103:0/2847025690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340ca180 con 0x560bddaedb40 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2847025690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaedb40 msgr2=0x560bddaec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaedb40 0x560bddaec090 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe2340a4b00 tx=0x7fe23409bde0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2847025690 shutdown_connections 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaf9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaec5d0 0x560bddaeca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2847025690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaedb40 0x560bddaec090 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2847025690 >> 172.21.15.103:0/2847025690 conn(0x560bddab5310 msgr2=0x560bddae8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2847025690 shutdown_connections 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2847025690 wait complete. 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- start start 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 Processor -- start 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaec5d0 0x560bddaf6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaf4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaf7370 0x560bddaf4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560bddafa070 con 0x560bddaf7370 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560bdda0d060 con 0x560bddaec5d0 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.092+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bddafa1f0 con 0x560bddaedb40 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe24b8ea740 10 client.0 _ll_register_callbacks cb 0x560bdd7608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaec5d0 0x560bddaf6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaf7370 0x560bddaf4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaf7370 0x560bddaf4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53826/0 (socket says 172.21.15.103:53826) 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 -- 172.21.15.103:0/502368509 learned_addr learned my addr 172.21.15.103:0/502368509 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2427fc640 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaf4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaec5d0 0x560bddaf6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:53024/0 (socket says 172.21.15.103:53024) 2024-09-19T07:44:41.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 -- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 msgr2=0x560bddaf4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaf4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 -- 172.21.15.103:0/502368509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaec5d0 msgr2=0x560bddaf6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaec5d0 0x560bddaf6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 -- 172.21.15.103:0/502368509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdd7c33c0 con 0x560bddaf7370 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe242ffd640 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaec5d0 0x560bddaf6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe2437fe640 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaf7370 0x560bddaf4600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe23c010db0 tx=0x7fe23c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13e61f108295540 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.093+0000 7fe226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.094+0000 7fe226ff5640 1 -- 172.21.15.103:0/502368509 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c020410 con 0x560bddaf7370 2024-09-19T07:44:41.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.094+0000 7fe226ff5640 1 -- 172.21.15.103:0/502368509 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe23c0205b0 con 0x560bddaf7370 2024-09-19T07:44:41.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.094+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2340a5030 con 0x560bddaf7370 2024-09-19T07:44:41.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.095+0000 7fe226ff5640 1 -- 172.21.15.103:0/502368509 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe23c0207b0 con 0x560bddaf7370 2024-09-19T07:44:41.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.095+0000 7fe24b8ea740 10 client.16824 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:41.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.095+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560bddb05b10 con 0x560bddaf7370 2024-09-19T07:44:41.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe226ff5640 1 -- 172.21.15.103:0/502368509 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c020be0 con 0x560bddaf7370 2024-09-19T07:44:41.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe226ff5640 1 -- 172.21.15.103:0/502368509 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fe23c01f050 con 0x560bddaf7370 2024-09-19T07:44:41.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe24b8ea740 10 client.16824 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe24b8ea740 10 client.16824 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe24b8ea740 20 client.16824 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb06f20 0x560bddb09310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe24b8ea740 4 client.16824 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe2340a5030 con 0x560bddb06f20 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.096+0000 7fe242ffd640 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb06f20 0x560bddb09310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.097+0000 7fe242ffd640 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb06f20 0x560bddb09310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8642dce0865d0fa8 server_cookie=e1b2eccecce9d8d6 in_seq=0 out_seq=0 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.097+0000 7fe226ff5640 10 client.16824 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.097+0000 7fe226ff5640 1 -- 172.21.15.103:0/502368509 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe2340a5030 con 0x560bddb06f20 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.097+0000 7fe226ff5640 10 client.16824 handle_command_reply: tid=0 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.097+0000 7fe24b8ea740 1 client.16824 shutdown 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaf7370 msgr2=0x560bddaf4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaf7370 0x560bddaf4600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe23c010db0 tx=0x7fe23c01c000 comp rx=0 tx=0).stop 2024-09-19T07:44:41.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 shutdown_connections 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb06f20 0x560bddb09310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaf7370 0x560bddaf4600 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaf4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/502368509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaec5d0 0x560bddaf6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 >> 172.21.15.103:0/502368509 conn(0x560bddab5310 msgr2=0x560bddaee990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe226ff5640 0 client.16824 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 shutdown_connections 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 1 -- 172.21.15.103:0/502368509 wait complete. 2024-09-19T07:44:41.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.098+0000 7fe24b8ea740 20 client.16824 trim_cache size 0 max 16384 2024-09-19T07:44:41.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.115+0000 7fe24b8ea740 1 Processor -- start 2024-09-19T07:44:41.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.115+0000 7fe24b8ea740 1 -- start start 2024-09-19T07:44:41.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaf5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441ab840 con 0x560bddaec5d0 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441ad380 con 0x560bddaf7370 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441abf10 con 0x560bddaedb40 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaf5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaf5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53840/0 (socket says 172.21.15.103:53840) 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe242ffd640 1 -- 172.21.15.103:0/2894761000 learned_addr learned my addr 172.21.15.103:0/2894761000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe2427fc640 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe2437fe640 1 -- 172.21.15.103:0/2894761000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 msgr2=0x560bddaef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe2437fe640 1 -- 172.21.15.103:0/2894761000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 msgr2=0x560bddaf5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaf5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe2437fe640 1 -- 172.21.15.103:0/2894761000 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bddae8ed0 con 0x560bddaf7370 2024-09-19T07:44:41.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.116+0000 7fe242ffd640 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaf5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.117+0000 7fe2437fe640 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaef6c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe23c010d80 tx=0x7fe23c0129b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f63e1698f4c771 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.117+0000 7fe228ff9640 1 -- 172.21.15.103:0/2894761000 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c020be0 con 0x560bddaf7370 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.118+0000 7fe228ff9640 1 -- 172.21.15.103:0/2894761000 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c020d80 con 0x560bddaf7370 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.118+0000 7fe228ff9640 1 -- 172.21.15.103:0/2894761000 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c02a840 con 0x560bddaf7370 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2894761000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 msgr2=0x560bddaef6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaef6c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe23c010d80 tx=0x7fe23c0129b0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2894761000 shutdown_connections 2024-09-19T07:44:41.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaf7370 0x560bddaef6c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaedb40 0x560bddaef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/2894761000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaf5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2894761000 >> 172.21.15.103:0/2894761000 conn(0x560bddab5310 msgr2=0x560bddaf5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2894761000 shutdown_connections 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 -- 172.21.15.103:0/2894761000 wait complete. 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 -- start start 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 Processor -- start 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.119+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaedb40 0x560bddaface0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe24b8ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaf7370 0x560bddafb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441accb0 con 0x560bddaec5d0 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441ac5e0 con 0x560bddaedb40 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe24b8ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441a9820 con 0x560bddaf7370 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe242ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaff9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53852/0 (socket says 172.21.15.103:53852) 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe24b8ea740 10 client.0 _ll_register_callbacks cb 0x7fe23c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe242ffd640 1 -- 172.21.15.103:0/1607470425 learned_addr learned my addr 172.21.15.103:0/1607470425 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:41.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2437fe640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaf7370 0x560bddafb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2427fc640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaedb40 0x560bddaface0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2437fe640 1 -- 172.21.15.103:0/1607470425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaedb40 msgr2=0x560bddaface0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2437fe640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaedb40 0x560bddaface0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2437fe640 1 -- 172.21.15.103:0/1607470425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 msgr2=0x560bddaff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2437fe640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2437fe640 1 -- 172.21.15.103:0/1607470425 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bddafb810 con 0x560bddaf7370 2024-09-19T07:44:41.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe242ffd640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.120+0000 7fe2437fe640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaf7370 0x560bddafb220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe23c032f80 tx=0x7fe23c009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f2f8bec388bd13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.121+0000 7fe226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.121+0000 7fe226ff5640 1 -- 172.21.15.103:0/1607470425 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c020be0 con 0x560bddaf7370 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.122+0000 7fe226ff5640 1 -- 172.21.15.103:0/1607470425 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7fe23c02a560 con 0x560bddaf7370 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.122+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23c020d80 con 0x560bddaf7370 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.122+0000 7fe226ff5640 1 -- 172.21.15.103:0/1607470425 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe23c02a760 con 0x560bddaf7370 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.122+0000 7fe24b8ea740 10 client.26254 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.122+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560bddb71950 con 0x560bddaf7370 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.122+0000 7fe226ff5640 1 -- 172.21.15.103:0/1607470425 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c020d80 con 0x560bddaf7370 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe226ff5640 1 -- 172.21.15.103:0/1607470425 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fe23c039070 con 0x560bddaf7370 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe24b8ea740 10 client.26254 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe24b8ea740 10 client.26254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:44:41.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe24b8ea740 20 client.26254 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:41.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb71fb0 0x560bddb743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:41.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe24b8ea740 4 client.26254 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:44:41.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bdda5f860 con 0x560bddb71fb0 2024-09-19T07:44:41.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.123+0000 7fe242ffd640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb71fb0 0x560bddb743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe226ff5640 10 client.26254 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:44:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe242ffd640 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb71fb0 0x560bddb743a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89adac93ab85b5e7 server_cookie=bddf3fe4203d3581 in_seq=0 out_seq=0 2024-09-19T07:44:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe226ff5640 1 -- 172.21.15.103:0/1607470425 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bdda5f860 con 0x560bddb71fb0 2024-09-19T07:44:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe226ff5640 10 client.26254 handle_command_reply: tid=0 2024-09-19T07:44:41.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 client.26254 shutdown 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaf7370 msgr2=0x560bddafb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaf7370 0x560bddafb220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe23c032f80 tx=0x7fe23c009480 comp rx=0 tx=0).stop 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 shutdown_connections 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x560bddb71fb0 0x560bddb743a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bddaf7370 0x560bddafb220 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe226ff5640 0 client.26254 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560bddaedb40 0x560bddaface0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 --2- 172.21.15.103:0/1607470425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bddaec5d0 0x560bddaff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.124+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 >> 172.21.15.103:0/1607470425 conn(0x560bddab5310 msgr2=0x560bddaf6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.125+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 shutdown_connections 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.125+0000 7fe24b8ea740 1 -- 172.21.15.103:0/1607470425 wait complete. 2024-09-19T07:44:41.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.125+0000 7fe24b8ea740 20 client.26254 trim_cache size 0 max 16384 2024-09-19T07:44:41.132 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.126+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe22c0a33b0 msgr2=0x7fe22c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe22c0a33b0 0x7fe22c0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe23c005fd0 tx=0x7fe23c009e30 comp rx=0 tx=0).stop 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe244102d30 msgr2=0x7fe244198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe244102d30 0x7fe244198600 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe23400e9f0 tx=0x7fe23400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 shutdown_connections 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe22c0a33b0 0x7fe22c0a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441082a0 0x7fe24419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2441036e0 0x7fe244198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 --2- 172.21.15.103:0/2364530434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe244102d30 0x7fe244198600 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 >> 172.21.15.103:0/2364530434 conn(0x7fe2440fc7d0 msgr2=0x7fe244106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 shutdown_connections 2024-09-19T07:44:41.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:41.127+0000 7fe249511640 1 -- 172.21.15.103:0/2364530434 wait complete. 2024-09-19T07:44:42.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:41 smithi174 ceph-mon[35370]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 945 op/s 2024-09-19T07:44:42.209 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:41 smithi103 ceph-mon[31201]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 945 op/s 2024-09-19T07:44:42.250 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:44:42.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:41 smithi134 ceph-mon[35414]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 945 op/s 2024-09-19T07:44:42.614 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.159+0000 7f36fb44e640 1 -- 172.21.15.103:0/3802237074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f406ea30 msgr2=0x7f36f406ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.159+0000 7f36fb44e640 1 --2- 172.21.15.103:0/3802237074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f406ea30 0x7f36f406ee90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009ca0 tx=0x7f36e402a8f0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.160+0000 7f36fb44e640 1 -- 172.21.15.103:0/3802237074 shutdown_connections 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.160+0000 7f36fb44e640 1 --2- 172.21.15.103:0/3802237074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f411b140 0x7f36f411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.160+0000 7f36fb44e640 1 --2- 172.21.15.103:0/3802237074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f406ea30 0x7f36f406ee90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.160+0000 7f36fb44e640 1 --2- 172.21.15.103:0/3802237074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36f4110220 0x7f36f406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.160+0000 7f36fb44e640 1 -- 172.21.15.103:0/3802237074 >> 172.21.15.103:0/3802237074 conn(0x7f36f4073210 msgr2=0x7f36f4073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.161+0000 7f36fb44e640 1 -- 172.21.15.103:0/3802237074 shutdown_connections 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.162+0000 7f36fb44e640 1 -- 172.21.15.103:0/3802237074 wait complete. 2024-09-19T07:44:43.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.162+0000 7f36fb44e640 1 Processor -- start 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36fb44e640 1 -- start start 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36fb44e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36f406ea30 0x7f36f41a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36fb44e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f4110220 0x7f36f41a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36fb44e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f411b140 0x7f36f41aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36fb44e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4120d40 con 0x7f36f4110220 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36fb44e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4120bc0 con 0x7f36f406ea30 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36fb44e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4120ec0 con 0x7f36f411b140 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.163+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f4110220 0x7f36f41a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.164+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f4110220 0x7f36f41a7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53856/0 (socket says 172.21.15.103:53856) 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.164+0000 7f36f89e9640 1 -- 172.21.15.103:0/2303032637 learned_addr learned my addr 172.21.15.103:0/2303032637 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.164+0000 7f36f99eb640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f411b140 0x7f36f41aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.165+0000 7f36f99eb640 1 -- 172.21.15.103:0/2303032637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36f406ea30 msgr2=0x7f36f41a71c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.165+0000 7f36f99eb640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36f406ea30 0x7f36f41a71c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.165+0000 7f36f99eb640 1 -- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f4110220 msgr2=0x7f36f41a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.165+0000 7f36f99eb640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f4110220 0x7f36f41a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.165+0000 7f36f99eb640 1 -- 172.21.15.103:0/2303032637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f41ac190 con 0x7f36f411b140 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.165+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f4110220 0x7f36f41a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.167+0000 7f36f99eb640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f411b140 0x7f36f41aba90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b410 tx=0x7f36e800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.169+0000 7f36f27fc640 1 -- 172.21.15.103:0/2303032637 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8015020 con 0x7f36f411b140 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.169+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f41ac480 con 0x7f36f411b140 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.169+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f4110920 con 0x7f36f411b140 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.169+0000 7f36f27fc640 1 -- 172.21.15.103:0/2303032637 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e8011d10 con 0x7f36f411b140 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.169+0000 7f36f27fc640 1 -- 172.21.15.103:0/2303032637 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e800f2e0 con 0x7f36f411b140 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.173+0000 7f36f27fc640 1 -- 172.21.15.103:0/2303032637 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f36e8018020 con 0x7f36f411b140 2024-09-19T07:44:43.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.173+0000 7f36fd827740 1 Processor -- start 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36fd827740 1 -- start start 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36fd827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e802180cf0 0x55e80217f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36fd827740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e80217fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36fd827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80218a520 0x55e80218c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36fd827740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e8020e52f0 con 0x55e802180cf0 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36fd827740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e80208ccf0 con 0x55e80217f780 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36fd827740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e8020e12b0 con 0x55e80218a520 2024-09-19T07:44:43.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36f27fc640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f36dc0a3580 0x7f36dc0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e802180cf0 0x55e80217f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e802180cf0 0x55e80217f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53862/0 (socket says 172.21.15.103:53862) 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36f89e9640 1 -- 172.21.15.103:0/1573278895 learned_addr learned my addr 172.21.15.103:0/1573278895 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36f89e9640 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80218a520 0x55e80218c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.174+0000 7f36f91ea640 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e80217fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f27fc640 1 -- 172.21.15.103:0/2303032637 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f36e80cf160 con 0x7f36f411b140 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f99eb640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f36dc0a3580 0x7f36dc0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f91ea640 1 -- 172.21.15.103:0/1573278895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80218a520 msgr2=0x55e80218c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f91ea640 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80218a520 0x55e80218c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f91ea640 1 -- 172.21.15.103:0/1573278895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e802180cf0 msgr2=0x55e80217f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f91ea640 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e802180cf0 0x55e80217f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f91ea640 1 -- 172.21.15.103:0/1573278895 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8020713a0 con 0x55e80217f780 2024-09-19T07:44:43.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f99eb640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f36dc0a3580 0x7f36dc0a5a40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b3e0 tx=0x7f36e80047d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.175+0000 7f36f91ea640 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e80217fbc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00a8c0 tx=0x7f36ec00ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196fdf35198f0010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.176+0000 7f36db7fe640 1 -- 172.21.15.103:0/1573278895 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec00ccc0 con 0x55e80217f780 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.176+0000 7f36db7fe640 1 -- 172.21.15.103:0/1573278895 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec00f930 con 0x55e80217f780 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.177+0000 7f36fd827740 1 -- 172.21.15.103:0/1573278895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 msgr2=0x55e80217fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.177+0000 7f36db7fe640 1 -- 172.21.15.103:0/1573278895 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec00ccc0 con 0x55e80217f780 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.177+0000 7f36fd827740 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e80217fbc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00a8c0 tx=0x7f36ec00ad80 comp rx=0 tx=0).stop 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.178+0000 7f36fd827740 1 -- 172.21.15.103:0/1573278895 shutdown_connections 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.178+0000 7f36fd827740 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80218a520 0x55e80218c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.178+0000 7f36fd827740 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e80217fbc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.178+0000 7f36fd827740 1 --2- 172.21.15.103:0/1573278895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e802180cf0 0x55e80217f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.178+0000 7f36fd827740 1 -- 172.21.15.103:0/1573278895 >> 172.21.15.103:0/1573278895 conn(0x55e802148310 msgr2=0x55e80217be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.178+0000 7f36fd827740 1 -- 172.21.15.103:0/1573278895 shutdown_connections 2024-09-19T07:44:43.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.178+0000 7f36fd827740 1 -- 172.21.15.103:0/1573278895 wait complete. 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.179+0000 7f36fd827740 1 -- start start 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.179+0000 7f36fd827740 1 Processor -- start 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36fd827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802189d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36fd827740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802187070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36fd827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e8021875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36fd827740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e80218d270 con 0x55e80218a520 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36fd827740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e8020a0060 con 0x55e802180cf0 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36fd827740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e80218d3f0 con 0x55e80217f780 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36fd827740 10 client.0 _ll_register_callbacks cb 0x55e801df38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802189d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f91ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802187070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f91ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802187070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:53064/0 (socket says 172.21.15.103:53064) 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f91ea640 1 -- 172.21.15.103:0/2663626794 learned_addr learned my addr 172.21.15.103:0/2663626794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e8021875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f91ea640 1 -- 172.21.15.103:0/2663626794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 msgr2=0x55e802189d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f91ea640 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802189d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.180+0000 7f36f91ea640 1 -- 172.21.15.103:0/2663626794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 msgr2=0x55e8021875b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36f91ea640 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e8021875b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36f91ea640 1 -- 172.21.15.103:0/2663626794 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e801e563c0 con 0x55e802180cf0 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802189d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36f91ea640 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802187070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f36ec000c00 tx=0x7f36ec00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0f886a9b4c25bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2663626794 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec026060 con 0x55e802180cf0 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ec00f930 con 0x55e802180cf0 2024-09-19T07:44:43.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2663626794 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f36ec005040 con 0x55e802180cf0 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2663626794 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f36ec005280 con 0x55e802180cf0 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36fd827740 10 client.26345 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.181+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e802198ca0 con 0x55e802180cf0 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.182+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2663626794 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ec00f930 con 0x55e802180cf0 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.182+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2663626794 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f36ec00e550 con 0x55e802180cf0 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36fd827740 10 client.26345 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36fd827740 10 client.26345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36fd827740 20 client.26345 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36fd827740 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e80219a050 0x55e80219c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36fd827740 4 client.26345 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8020f2860 con 0x55e80219a050 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e80219a050 0x55e80219c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36d8ff9640 10 client.26345 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:43.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.183+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e80219a050 0x55e80219c440 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991ae2a503c219b server_cookie=caa7aaace0d13d5f in_seq=0 out_seq=0 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2663626794 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8020f2860 con 0x55e80219a050 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36d8ff9640 10 client.26345 handle_command_reply: tid=0 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 client.26345 shutdown 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 msgr2=0x55e802187070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802187070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f36ec000c00 tx=0x7f36ec00cef0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 shutdown_connections 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e80219a050 0x55e80219c440 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e8021875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802187070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36d8ff9640 0 client.26345 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 --2- 172.21.15.103:0/2663626794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802189d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 >> 172.21.15.103:0/2663626794 conn(0x55e802148310 msgr2=0x55e802181ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 shutdown_connections 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 1 -- 172.21.15.103:0/2663626794 wait complete. 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.184+0000 7f36fd827740 20 client.26345 trim_cache size 0 max 16384 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.201+0000 7f36fd827740 1 Processor -- start 2024-09-19T07:44:43.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.201+0000 7f36fd827740 1 -- start start 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36fd827740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e8021965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36fd827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e802180cf0 0x55e802176b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36fd827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e80218b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36fd827740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36e000e7a0 con 0x55e80218a520 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36fd827740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36e00102e0 con 0x55e80217f780 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36fd827740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36e000ee70 con 0x55e802180cf0 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e8021965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e8021965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:53080/0 (socket says 172.21.15.103:53080) 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 -- 172.21.15.103:0/307533207 learned_addr learned my addr 172.21.15.103:0/307533207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f91ea640 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e802180cf0 0x55e802176b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 -- 172.21.15.103:0/307533207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e802180cf0 msgr2=0x55e802176b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e802180cf0 0x55e802176b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 -- 172.21.15.103:0/307533207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 msgr2=0x55e80218b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e80218b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.202+0000 7f36f89e9640 1 -- 172.21.15.103:0/307533207 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8021880b0 con 0x55e80217f780 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.203+0000 7f36f89e9640 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e8021965f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f36e402aeb0 tx=0x7f36e40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8537a377fabdfce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.203+0000 7f36daffd640 1 -- 172.21.15.103:0/307533207 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4031770 con 0x55e80217f780 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.203+0000 7f36daffd640 1 -- 172.21.15.103:0/307533207 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e4053030 con 0x55e80217f780 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.203+0000 7f36daffd640 1 -- 172.21.15.103:0/307533207 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e402f860 con 0x55e80217f780 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 -- 172.21.15.103:0/307533207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 msgr2=0x55e8021965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e8021965f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f36e402aeb0 tx=0x7f36e40095c0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 -- 172.21.15.103:0/307533207 shutdown_connections 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e80218b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e802180cf0 0x55e802176b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 --2- 172.21.15.103:0/307533207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e80217f780 0x55e8021965f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 -- 172.21.15.103:0/307533207 >> 172.21.15.103:0/307533207 conn(0x55e802148310 msgr2=0x55e802187030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 -- 172.21.15.103:0/307533207 shutdown_connections 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.204+0000 7f36fd827740 1 -- 172.21.15.103:0/307533207 wait complete. 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.205+0000 7f36fd827740 1 -- start start 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.205+0000 7f36fd827740 1 Processor -- start 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.205+0000 7f36fd827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802192a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36fd827740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802192f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36fd827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e802183900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802192a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36f89e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802192a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:57344/0 (socket says 172.21.15.103:57344) 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36f91ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802192f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36f91ea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802192f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:53086/0 (socket says 172.21.15.103:53086) 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36f89e9640 1 -- 172.21.15.103:0/2588365318 learned_addr learned my addr 172.21.15.103:0/2588365318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36e000fc10 con 0x55e80218a520 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36e000f540 con 0x55e802180cf0 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.206+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36e000c780 con 0x55e80217f780 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.207+0000 7f36fd827740 10 client.0 _ll_register_callbacks cb 0x7f36e4009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.207+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e802183900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.207+0000 7f36f89e9640 1 -- 172.21.15.103:0/2588365318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 msgr2=0x55e802192f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.207+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802192f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.207+0000 7f36f89e9640 1 -- 172.21.15.103:0/2588365318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 msgr2=0x55e802183900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.207+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e802183900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.207+0000 7f36f89e9640 1 -- 172.21.15.103:0/2588365318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e802184130 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36f89e9640 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802192a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f36e4006010 tx=0x7f36e4004830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454e36478e6394db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2588365318 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003fe0 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2588365318 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f36e4048040 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e4053030 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2588365318 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f36e4004b40 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36fd827740 10 client.26272 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.208+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e802192260 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2588365318 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e4053030 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2588365318 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f36e402f9b0 con 0x55e80217f780 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36fd827740 10 client.26272 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:43.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36fd827740 10 client.26272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36fd827740 20 client.26272 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36fd827740 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e802204c00 0x55e802207020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36fd827740 4 client.26272 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8020ecd70 con 0x55e802204c00 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.209+0000 7f36f91ea640 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e802204c00 0x55e802207020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36d8ff9640 10 client.26272 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36f91ea640 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e802204c00 0x55e802207020 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a28f2c824143af3 server_cookie=cd2dd64143e694bd in_seq=0 out_seq=0 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36d8ff9640 1 -- 172.21.15.103:0/2588365318 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8020ecd70 con 0x55e802204c00 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36d8ff9640 10 client.26272 handle_command_reply: tid=0 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 client.26272 shutdown 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 msgr2=0x55e802192a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802192a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f36e4006010 tx=0x7f36e4004830 comp rx=0 tx=0).stop 2024-09-19T07:44:43.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 shutdown_connections 2024-09-19T07:44:43.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e802204c00 0x55e802207020 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e80218a520 0x55e802183900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e802180cf0 0x55e802192f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 --2- 172.21.15.103:0/2588365318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80217f780 0x55e802192a00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 >> 172.21.15.103:0/2588365318 conn(0x55e802148310 msgr2=0x55e8021812c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:43.264 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36d8ff9640 0 client.26272 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 shutdown_connections 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 1 -- 172.21.15.103:0/2588365318 wait complete. 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.210+0000 7f36fd827740 20 client.26272 trim_cache size 0 max 16384 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f36dc0a3580 msgr2=0x7f36dc0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f36dc0a3580 0x7f36dc0a5a40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b3e0 tx=0x7f36e80047d0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f411b140 msgr2=0x7f36f41aba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f411b140 0x7f36f41aba90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b410 tx=0x7f36e800b8d0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 shutdown_connections 2024-09-19T07:44:43.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f36dc0a3580 0x7f36dc0a5a40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f411b140 0x7f36f41aba90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36f4110220 0x7f36f41a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 --2- 172.21.15.103:0/2303032637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36f406ea30 0x7f36f41a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 >> 172.21.15.103:0/2303032637 conn(0x7f36f4073210 msgr2=0x7f36f41192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 shutdown_connections 2024-09-19T07:44:43.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:43.211+0000 7f36fb44e640 1 -- 172.21.15.103:0/2303032637 wait complete. 2024-09-19T07:44:44.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:43 smithi174 ceph-mon[35370]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 614 op/s 2024-09-19T07:44:44.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:43 smithi103 ceph-mon[31201]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 614 op/s 2024-09-19T07:44:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:43 smithi134 ceph-mon[35414]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 614 op/s 2024-09-19T07:44:45.346 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:44:45.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:44:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 370.0 s, 2891.4 tps, lat 34.735 ms stddev 42.259 2024-09-19T07:44:45.710 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:46.106 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:45 smithi103 ceph-mon[31201]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 819 op/s 2024-09-19T07:44:46.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:45 smithi174 ceph-mon[35370]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 819 op/s 2024-09-19T07:44:46.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:45 smithi134 ceph-mon[35414]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 819 op/s 2024-09-19T07:44:48.006 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:44:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:44:48.006 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "847be915-e518-43db-99f3-985a35482e6e", 2024-09-19T07:44:48.007 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:44:48.007 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:44:48.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 -- 172.21.15.103:0/3165347618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 msgr2=0x7f466010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 --2- 172.21.15.103:0/3165347618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466010da40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4658009c10 tx=0x7f465802a770 comp rx=0 tx=0).stop 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 -- 172.21.15.103:0/3165347618 shutdown_connections 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 --2- 172.21.15.103:0/3165347618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466010da40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 --2- 172.21.15.103:0/3165347618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4660102570 0x7f46601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 --2- 172.21.15.103:0/3165347618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4660100e20 0x7f4660101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 -- 172.21.15.103:0/3165347618 >> 172.21.15.103:0/3165347618 conn(0x7f46600fc7d0 msgr2=0x7f46600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 -- 172.21.15.103:0/3165347618 shutdown_connections 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.489+0000 7f46653b8640 1 -- 172.21.15.103:0/3165347618 wait complete. 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.490+0000 7f46653b8640 1 Processor -- start 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.490+0000 7f46653b8640 1 -- start start 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.490+0000 7f46653b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4660100e20 0x7f46601973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f46653b8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4660102570 0x7f4660197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f46653b8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f46653b8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4660110f80 con 0x7f4660102570 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f46653b8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4660110e00 con 0x7f4660102ef0 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f46653b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4660111100 con 0x7f4660100e20 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:53106/0 (socket says 172.21.15.103:53106) 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 -- 172.21.15.103:0/1350579167 learned_addr learned my addr 172.21.15.103:0/1350579167 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 -- 172.21.15.103:0/1350579167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4660100e20 msgr2=0x7f46601973e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4660100e20 0x7f46601973e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 -- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4660102570 msgr2=0x7f4660197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4660102570 0x7f4660197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.491+0000 7f465f7fe640 1 -- 172.21.15.103:0/1350579167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466019c390 con 0x7f4660102ef0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.492+0000 7f465f7fe640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466019bcb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f465802ac90 tx=0x7f46580305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.492+0000 7f4647fff640 1 -- 172.21.15.103:0/1350579167 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658041020 con 0x7f4660102ef0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.492+0000 7f4647fff640 1 -- 172.21.15.103:0/1350579167 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4658003e50 con 0x7f4660102ef0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.492+0000 7f4647fff640 1 -- 172.21.15.103:0/1350579167 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658036040 con 0x7f4660102ef0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.492+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466019c620 con 0x7f4660102ef0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.492+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466019cab0 con 0x7f4660102ef0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.496+0000 7f4667791740 1 Processor -- start 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.496+0000 7f4647fff640 1 -- 172.21.15.103:0/1350579167 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4658044020 con 0x7f4660102ef0 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.496+0000 7f4667791740 1 -- start start 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4667791740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95de2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4667791740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95ded800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4667791740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4667791740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c95d48280 con 0x561c95de3480 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4667791740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c95cefb10 con 0x561c95de2b60 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4667791740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c95d44240 con 0x561c95dedd40 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95ded800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95ded800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53912/0 (socket says 172.21.15.103:53912) 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 -- 172.21.15.103:0/2940346920 learned_addr learned my addr 172.21.15.103:0/2940346920 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465e7fc640 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95de2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465e7fc640 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 -- 172.21.15.103:0/2940346920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 msgr2=0x561c95df0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 -- 172.21.15.103:0/2940346920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 msgr2=0x561c95de2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95de2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4647fff640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46480a3500 0x7f46480a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 -- 172.21.15.103:0/2940346920 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c95cfc1f0 con 0x561c95de3480 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465e7fc640 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95de2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f465effd640 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95ded800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4650009ac0 tx=0x7f4650009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395cb9d307e77b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.497+0000 7f4647fff640 1 -- 172.21.15.103:0/1350579167 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f46580f0820 con 0x7f4660102ef0 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.498+0000 7f46457fa640 1 -- 172.21.15.103:0/2940346920 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465000ed10 con 0x561c95de3480 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.498+0000 7f465e7fc640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46480a3500 0x7f46480a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.498+0000 7f46457fa640 1 -- 172.21.15.103:0/2940346920 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4650005500 con 0x561c95de3480 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.498+0000 7f46457fa640 1 -- 172.21.15.103:0/2940346920 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650010430 con 0x561c95de3480 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.499+0000 7f4667791740 1 -- 172.21.15.103:0/2940346920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 msgr2=0x561c95ded800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.499+0000 7f4667791740 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95ded800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4650009ac0 tx=0x7f4650009f80 comp rx=0 tx=0).stop 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.499+0000 7f465e7fc640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46480a3500 0x7f46480a59c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x561c95de0470 tx=0x7f4654009530 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:48.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 -- 172.21.15.103:0/2940346920 shutdown_connections 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95ded800 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 --2- 172.21.15.103:0/2940346920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95de2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 -- 172.21.15.103:0/2940346920 >> 172.21.15.103:0/2940346920 conn(0x561c95ddf0f0 msgr2=0x561c95ddf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 -- 172.21.15.103:0/2940346920 shutdown_connections 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 -- 172.21.15.103:0/2940346920 wait complete. 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 -- start start 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.500+0000 7f4667791740 1 Processor -- start 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f4667791740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95deab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f4667791740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95deb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f4667791740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95dedd40 0x561c95deb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f4667791740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c95df0910 con 0x561c95de3480 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f4667791740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c95d02f10 con 0x561c95dedd40 2024-09-19T07:44:48.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95deb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f4667791740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c95df0a90 con 0x561c95de2b60 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95deb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53918/0 (socket says 172.21.15.103:53918) 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465e7fc640 1 -- 172.21.15.103:0/586905303 learned_addr learned my addr 172.21.15.103:0/586905303 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465effd640 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95deab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465f7fe640 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95dedd40 0x561c95deb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f4667791740 10 client.0 _ll_register_callbacks cb 0x561c95ccba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465effd640 1 -- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95dedd40 msgr2=0x561c95deb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465effd640 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95dedd40 0x561c95deb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465effd640 1 -- 172.21.15.103:0/586905303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 msgr2=0x561c95deb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465effd640 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95deb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.501+0000 7f465effd640 1 -- 172.21.15.103:0/586905303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c95ab96a0 con 0x561c95de2b60 2024-09-19T07:44:48.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4650005500 con 0x561c95de2b60 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f465effd640 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95deab80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4650009ac0 tx=0x7f4650000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221b1ed7bc9fd713 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f4642ff5640 10 client.26281 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f4642ff5640 1 -- 172.21.15.103:0/586905303 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465001d040 con 0x561c95de2b60 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f4642ff5640 1 -- 172.21.15.103:0/586905303 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f465000eb10 con 0x561c95de2b60 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f4642ff5640 1 -- 172.21.15.103:0/586905303 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f465000ed50 con 0x561c95de2b60 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f4667791740 10 client.26281 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.502+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=40}) -- 0x561c95dfc380 con 0x561c95de2b60 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f4642ff5640 1 -- 172.21.15.103:0/586905303 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f4650004030 con 0x561c95de2b60 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f4667791740 10 client.26281 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f4667791740 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f4667791740 20 client.26281 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f4667791740 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95dfd730 0x561c95dffb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f4667791740 4 client.26281 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:48.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c95d557f0 con 0x561c95dfd730 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f465e7fc640 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95dfd730 0x561c95dffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.503+0000 7f465e7fc640 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95dfd730 0x561c95dffb20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba5aec6e6581e60 server_cookie=bee4769f96c0fb40 in_seq=0 out_seq=0 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4642ff5640 10 client.26281 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4642ff5640 1 -- 172.21.15.103:0/586905303 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c95d557f0 con 0x561c95dfd730 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4642ff5640 10 client.26281 handle_command_reply: tid=0 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 client.26281 shutdown 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 msgr2=0x561c95deab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95deab80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4650009ac0 tx=0x7f4650000bc0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 shutdown_connections 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95dfd730 0x561c95dffb20 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95dedd40 0x561c95deb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95deb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 --2- 172.21.15.103:0/586905303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95deab80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4642ff5640 0 client.26281 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 >> 172.21.15.103:0/586905303 conn(0x561c95ddf0f0 msgr2=0x561c95de4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 shutdown_connections 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.504+0000 7f4667791740 1 -- 172.21.15.103:0/586905303 wait complete. 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.505+0000 7f4667791740 20 client.26281 trim_cache size 0 max 16384 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.534+0000 7f4667791740 1 Processor -- start 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.534+0000 7f4667791740 1 -- start start 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f4667791740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95dde520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f4667791740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de3480 0x561c95deb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f4667791740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95dedd40 0x561c95e05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f4667791740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46601afaa0 con 0x561c95dedd40 2024-09-19T07:44:48.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f4667791740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46601b15e0 con 0x561c95de3480 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f4667791740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46601b0170 con 0x561c95de2b60 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f465f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95dedd40 0x561c95e05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f465f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95dedd40 0x561c95e05330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53922/0 (socket says 172.21.15.103:53922) 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f465f7fe640 1 -- 172.21.15.103:0/129016432 learned_addr learned my addr 172.21.15.103:0/129016432 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.535+0000 7f465effd640 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95dde520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f465f7fe640 1 -- 172.21.15.103:0/129016432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 msgr2=0x561c95dde520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f465f7fe640 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95dde520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f465f7fe640 1 -- 172.21.15.103:0/129016432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de3480 msgr2=0x561c95deb450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f465f7fe640 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de3480 0x561c95deb450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f465f7fe640 1 -- 172.21.15.103:0/129016432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c95ddef20 con 0x561c95dedd40 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f465f7fe640 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95dedd40 0x561c95e05330 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f46580bcaa0 tx=0x7f46580bd210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1788bebbc721b023 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:48.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f4644ff9640 1 -- 172.21.15.103:0/129016432 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580d4030 con 0x561c95dedd40 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f4644ff9640 1 -- 172.21.15.103:0/129016432 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46580c5780 con 0x561c95dedd40 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.536+0000 7f4644ff9640 1 -- 172.21.15.103:0/129016432 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658031690 con 0x561c95dedd40 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 -- 172.21.15.103:0/129016432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95dedd40 msgr2=0x561c95e05330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95dedd40 0x561c95e05330 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f46580bcaa0 tx=0x7f46580bd210 comp rx=0 tx=0).stop 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 -- 172.21.15.103:0/129016432 shutdown_connections 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95dedd40 0x561c95e05330 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de3480 0x561c95deb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 --2- 172.21.15.103:0/129016432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95de2b60 0x561c95dde520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 -- 172.21.15.103:0/129016432 >> 172.21.15.103:0/129016432 conn(0x561c95ddf0f0 msgr2=0x561c95df93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 -- 172.21.15.103:0/129016432 shutdown_connections 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.537+0000 7f4667791740 1 -- 172.21.15.103:0/129016432 wait complete. 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 -- start start 2024-09-19T07:44:48.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 Processor -- start 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95e61aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95e61fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46601b0f10 con 0x561c95de3480 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f46601b0840 con 0x561c95de2b60 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46601ada80 con 0x561c95dedd40 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f4667791740 10 client.0 _ll_register_callbacks cb 0x7f4650009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f465e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95e61fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f465e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95e61fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53936/0 (socket says 172.21.15.103:53936) 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f465e7fc640 1 -- 172.21.15.103:0/340597256 learned_addr learned my addr 172.21.15.103:0/340597256 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f465f7fe640 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.538+0000 7f465effd640 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95e61aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.540+0000 7f465f7fe640 1 -- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 msgr2=0x561c95e61aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.540+0000 7f465f7fe640 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95e61aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.540+0000 7f465f7fe640 1 -- 172.21.15.103:0/340597256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 msgr2=0x561c95e61fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.540+0000 7f465f7fe640 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95e61fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.540+0000 7f465f7fe640 1 -- 172.21.15.103:0/340597256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c95df4320 con 0x561c95dedd40 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.541+0000 7f465f7fe640 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df3d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46580bccd0 tx=0x7f46580bc4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d3b49c72a006b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.541+0000 7f4642ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.542+0000 7f4642ff5640 1 -- 172.21.15.103:0/340597256 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46580d4070 con 0x561c95dedd40 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.542+0000 7f4642ff5640 1 -- 172.21.15.103:0/340597256 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f46580db300 con 0x561c95dedd40 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.542+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46580c5780 con 0x561c95dedd40 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.542+0000 7f4642ff5640 1 -- 172.21.15.103:0/340597256 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f46580b9300 con 0x561c95dedd40 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.542+0000 7f4667791740 10 client.26284 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:48.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.542+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561c95df56d0 con 0x561c95dedd40 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4642ff5640 1 -- 172.21.15.103:0/340597256 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46580c5780 con 0x561c95dedd40 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4642ff5640 1 -- 172.21.15.103:0/340597256 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f4658004bd0 con 0x561c95dedd40 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4667791740 10 client.26284 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4667791740 10 client.26284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4667791740 20 client.26284 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4667791740 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95df5d50 0x561c95df6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4667791740 4 client.26284 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.543+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c95d4fd00 con 0x561c95df5d50 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.545+0000 7f465effd640 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95df5d50 0x561c95df6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.545+0000 7f465effd640 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95df5d50 0x561c95df6130 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebff386cd1a9dab8 server_cookie=ebeaff6a72a5186a in_seq=0 out_seq=0 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.545+0000 7f4642ff5640 10 client.26284 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.546+0000 7f4642ff5640 1 -- 172.21.15.103:0/340597256 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c95d4fd00 con 0x561c95df5d50 2024-09-19T07:44:48.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.546+0000 7f4642ff5640 10 client.26284 handle_command_reply: tid=0 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.546+0000 7f4667791740 1 client.26284 shutdown 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 msgr2=0x561c95df3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df3d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46580bccd0 tx=0x7f46580bc4c0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 shutdown_connections 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c95df5d50 0x561c95df6130 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c95dedd40 0x561c95df3d30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c95de3480 0x561c95e61fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 --2- 172.21.15.103:0/340597256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c95de2b60 0x561c95e61aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 >> 172.21.15.103:0/340597256 conn(0x561c95ddf0f0 msgr2=0x561c95dec1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4642ff5640 0 client.26284 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 shutdown_connections 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 1 -- 172.21.15.103:0/340597256 wait complete. 2024-09-19T07:44:48.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f4667791740 20 client.26284 trim_cache size 0 max 16384 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46480a3500 msgr2=0x7f46480a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f46653b8640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46480a3500 0x7f46480a59c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x561c95de0470 tx=0x7f4654009530 comp rx=0 tx=0).stop 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 msgr2=0x7f466019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f46653b8640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466019bcb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f465802ac90 tx=0x7f46580305d0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 shutdown_connections 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.547+0000 7f46653b8640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46480a3500 0x7f46480a59c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.548+0000 7f46653b8640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4660102ef0 0x7f466019bcb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.548+0000 7f46653b8640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4660102570 0x7f4660197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.548+0000 7f46653b8640 1 --2- 172.21.15.103:0/1350579167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4660100e20 0x7f46601973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.548+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 >> 172.21.15.103:0/1350579167 conn(0x7f46600fc7d0 msgr2=0x7f4660104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.548+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 shutdown_connections 2024-09-19T07:44:48.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:47.548+0000 7f46653b8640 1 -- 172.21.15.103:0/1350579167 wait complete. 2024-09-19T07:44:48.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:47 smithi174 ceph-mon[35370]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 690 op/s 2024-09-19T07:44:48.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:47 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:44:48.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:47 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:48.166 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:47 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:44:48.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:47 smithi103 ceph-mon[31201]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 690 op/s 2024-09-19T07:44:48.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:47 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:44:48.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:47 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:48.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:47 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:44:48.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:47 smithi134 ceph-mon[35414]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 690 op/s 2024-09-19T07:44:48.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:47 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:44:48.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:47 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:48.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:47 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:44:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:44:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:44:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:44:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:44:49.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:44:49.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:44:49.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:44:49.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:44:49.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:44:49.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:44:49.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:44:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:44:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:44:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:44:49.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:44:49.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:44:49.527 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:44:49.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:49 smithi103 ceph-mon[31201]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-09-19T07:44:49.936 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:44:50.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:49 smithi134 ceph-mon[35414]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-09-19T07:44:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:49 smithi174 ceph-mon[35370]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-09-19T07:44:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 375.0 s, 2881.2 tps, lat 34.553 ms stddev 43.996 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 -- 172.21.15.103:0/1868233936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 msgr2=0x7f5e081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1868233936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 0x7f5e081021c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009900 tx=0x7f5df802a3f0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 -- 172.21.15.103:0/1868233936 shutdown_connections 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1868233936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e08106f10 0x7f5e0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1868233936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 0x7f5e081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1868233936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 0x7f5e081021c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 -- 172.21.15.103:0/1868233936 >> 172.21.15.103:0/1868233936 conn(0x7f5e080fd880 msgr2=0x7f5e080ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:50.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 -- 172.21.15.103:0/1868233936 shutdown_connections 2024-09-19T07:44:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.495+0000 7f5e0e651640 1 -- 172.21.15.103:0/1868233936 wait complete. 2024-09-19T07:44:50.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.496+0000 7f5e0e651640 1 Processor -- start 2024-09-19T07:44:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.497+0000 7f5e0e651640 1 -- start start 2024-09-19T07:44:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e0e651640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 0x7f5e0810bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e0e651640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 0x7f5e0810c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e0e651640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e08106f10 0x7f5e0810c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e0e651640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e080760a0 con 0x7f5e08102790 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e0e651640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08075f20 con 0x7f5e08106f10 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e0e651640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08076220 con 0x7f5e08101de0 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 0x7f5e0810c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e07fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 0x7f5e0810bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 0x7f5e0810c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40446/0 (socket says 172.21.15.103:40446) 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e077fe640 1 -- 172.21.15.103:0/1365293751 learned_addr learned my addr 172.21.15.103:0/1365293751 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.498+0000 7f5e07fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 0x7f5e0810bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:55556/0 (socket says 172.21.15.103:55556) 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e0cbee640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e08106f10 0x7f5e0810c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e077fe640 1 -- 172.21.15.103:0/1365293751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 msgr2=0x7f5e0810bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e077fe640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 0x7f5e0810bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e077fe640 1 -- 172.21.15.103:0/1365293751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e08106f10 msgr2=0x7f5e0810c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e077fe640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e08106f10 0x7f5e0810c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e077fe640 1 -- 172.21.15.103:0/1365293751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e081aacc0 con 0x7f5e08102790 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e077fe640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 0x7f5e0810c170 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00ea90 tx=0x7f5dfc00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e057fa640 1 -- 172.21.15.103:0/1365293751 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00ce40 con 0x7f5e08102790 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e057fa640 1 -- 172.21.15.103:0/1365293751 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dfc0035f0 con 0x7f5e08102790 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e057fa640 1 -- 172.21.15.103:0/1365293751 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc010690 con 0x7f5e08102790 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e081aafb0 con 0x7f5e08102790 2024-09-19T07:44:50.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.499+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e081ab4b0 con 0x7f5e08102790 2024-09-19T07:44:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 Processor -- start 2024-09-19T07:44:50.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 -- start start 2024-09-19T07:44:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c664094a0 0x563c66409880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66414140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66416a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c6636f2e0 con 0x563c66414680 2024-09-19T07:44:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563c66316ce0 con 0x563c66409dc0 2024-09-19T07:44:50.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.503+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c6636b2a0 con 0x563c664094a0 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.504+0000 7f5e057fa640 1 -- 172.21.15.103:0/1365293751 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5dfc0037a0 con 0x7f5e08102790 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.505+0000 7f5e0cbee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c664094a0 0x563c66409880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.505+0000 7f5e0cbee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c664094a0 0x563c66409880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:55560/0 (socket says 172.21.15.103:55560) 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.505+0000 7f5e0cbee640 1 -- 172.21.15.103:0/2450469350 learned_addr learned my addr 172.21.15.103:0/2450469350 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.505+0000 7f5e057fa640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5df00a34b0 0x7f5df00a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.505+0000 7f5e057fa640 1 -- 172.21.15.103:0/1365293751 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5dfc0d0100 con 0x7f5e08102790 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.505+0000 7f5e07fff640 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66414140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.506+0000 7f5e077fe640 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66416a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.506+0000 7f5e07fff640 1 -- 172.21.15.103:0/2450469350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c664094a0 msgr2=0x563c66409880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.506+0000 7f5e07fff640 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c664094a0 0x563c66409880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.506+0000 7f5e07fff640 1 -- 172.21.15.103:0/2450469350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 msgr2=0x563c66416a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.506+0000 7f5e07fff640 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66416a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.506+0000 7f5e07fff640 1 -- 172.21.15.103:0/2450469350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c662fb440 con 0x563c66409dc0 2024-09-19T07:44:50.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.507+0000 7f5e07fff640 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66414140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5df8005940 tx=0x7f5df8005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479f9d2ae191bf7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:50.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.507+0000 7f5dee7fc640 1 -- 172.21.15.103:0/2450469350 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8040070 con 0x563c66409dc0 2024-09-19T07:44:50.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.507+0000 7f5dee7fc640 1 -- 172.21.15.103:0/2450469350 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df8031870 con 0x563c66409dc0 2024-09-19T07:44:50.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.507+0000 7f5e0cbee640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5df00a34b0 0x7f5df00a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.508+0000 7f5e10a2a740 1 -- 172.21.15.103:0/2450469350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 msgr2=0x563c66414140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.508+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66414140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5df8005940 tx=0x7f5df8005be0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.509+0000 7f5e10a2a740 1 -- 172.21.15.103:0/2450469350 shutdown_connections 2024-09-19T07:44:50.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.509+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66416a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.509+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66414140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.509+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/2450469350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c664094a0 0x563c66409880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.509+0000 7f5e10a2a740 1 -- 172.21.15.103:0/2450469350 >> 172.21.15.103:0/2450469350 conn(0x563c66405a30 msgr2=0x563c66405e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:50.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.509+0000 7f5e10a2a740 1 -- 172.21.15.103:0/2450469350 shutdown_connections 2024-09-19T07:44:50.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.510+0000 7f5e10a2a740 1 -- 172.21.15.103:0/2450469350 wait complete. 2024-09-19T07:44:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.510+0000 7f5e10a2a740 1 -- start start 2024-09-19T07:44:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.510+0000 7f5e10a2a740 1 Processor -- start 2024-09-19T07:44:50.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.511+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c66413f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.511+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c66411220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.511+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66411760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.511+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c664172f0 con 0x563c66414680 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.511+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563c6632a050 con 0x563c664094a0 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.511+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c66417470 con 0x563c66409dc0 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.511+0000 7f5e10a2a740 10 client.0 _ll_register_callbacks cb 0x563c6636a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66411760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66411760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40472/0 (socket says 172.21.15.103:40472) 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 -- 172.21.15.103:0/434160453 learned_addr learned my addr 172.21.15.103:0/434160453 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5df00a34b0 0x7f5df00a5970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009fd0 tx=0x7f5e00004ab0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e077fe640 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c66411220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 -- 172.21.15.103:0/434160453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 msgr2=0x563c66411220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:50.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c66411220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 -- 172.21.15.103:0/434160453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 msgr2=0x563c66413f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:44:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c66413f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.512+0000 7f5e0cbee640 1 -- 172.21.15.103:0/434160453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c660e03c0 con 0x563c66414680 2024-09-19T07:44:50.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.513+0000 7f5e0cbee640 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66411760 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5e00004220 tx=0x7f5e00009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac1c685049a209a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.513+0000 7f5de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:44:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.513+0000 7f5de77fe640 1 -- 172.21.15.103:0/434160453 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0001a030 con 0x563c66414680 2024-09-19T07:44:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.513+0000 7f5de77fe640 1 -- 172.21.15.103:0/434160453 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5e00011850 con 0x563c66414680 2024-09-19T07:44:50.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.514+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c6637c850 con 0x563c66414680 2024-09-19T07:44:50.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.518+0000 7f5de77fe640 1 -- 172.21.15.103:0/434160453 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5e00004850 con 0x563c66414680 2024-09-19T07:44:50.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.519+0000 7f5e10a2a740 10 client.16869 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:50.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.519+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563c66417960 con 0x563c66414680 2024-09-19T07:44:50.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.520+0000 7f5de77fe640 1 -- 172.21.15.103:0/434160453 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e00024070 con 0x563c66414680 2024-09-19T07:44:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.521+0000 7f5de77fe640 1 -- 172.21.15.103:0/434160453 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f5e00033420 con 0x563c66414680 2024-09-19T07:44:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.521+0000 7f5e10a2a740 10 client.16869 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.521+0000 7f5e10a2a740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.521+0000 7f5e10a2a740 20 client.16869 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.521+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664191b0 0x563c6641b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.521+0000 7f5e10a2a740 4 client.16869 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:44:50.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.521+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c6637c850 con 0x563c664191b0 2024-09-19T07:44:50.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.524+0000 7f5e07fff640 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664191b0 0x563c6641b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.528+0000 7f5e07fff640 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664191b0 0x563c6641b640 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652ff3eec9b13e7a server_cookie=ac0f40be8a8cf03f in_seq=0 out_seq=0 2024-09-19T07:44:50.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.529+0000 7f5de77fe640 10 client.16869 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:50.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.529+0000 7f5de77fe640 1 -- 172.21.15.103:0/434160453 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c6637c850 con 0x563c664191b0 2024-09-19T07:44:50.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.529+0000 7f5de77fe640 10 client.16869 handle_command_reply: tid=0 2024-09-19T07:44:50.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.529+0000 7f5e10a2a740 1 client.16869 shutdown 2024-09-19T07:44:50.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.533+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 msgr2=0x563c66411760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.533+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66411760 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5e00004220 tx=0x7f5e00009500 comp rx=0 tx=0).stop 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 shutdown_connections 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664191b0 0x563c6641b640 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c66411760 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c66411220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/434160453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c66413f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 >> 172.21.15.103:0/434160453 conn(0x563c66405a30 msgr2=0x563c6640b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5de77fe640 0 client.16869 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:50.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 shutdown_connections 2024-09-19T07:44:50.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 1 -- 172.21.15.103:0/434160453 wait complete. 2024-09-19T07:44:50.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.534+0000 7f5e10a2a740 20 client.16869 trim_cache size 0 max 16384 2024-09-19T07:44:50.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.563+0000 7f5e10a2a740 1 Processor -- start 2024-09-19T07:44:50.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e10a2a740 1 -- start start 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c664094a0 0x563c66420f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66421460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66414680 0x563c6640e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081ab790 con 0x563c664094a0 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081ad2d0 con 0x563c66409dc0 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e10a2a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081abe60 con 0x563c66414680 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c664094a0 0x563c66420f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e07fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c664094a0 0x563c66420f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40486/0 (socket says 172.21.15.103:40486) 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e07fff640 1 -- 172.21.15.103:0/878439516 learned_addr learned my addr 172.21.15.103:0/878439516 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.564+0000 7f5e077fe640 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66421460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5e0cbee640 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66414680 0x563c6640e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5e077fe640 1 -- 172.21.15.103:0/878439516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66414680 msgr2=0x563c6640e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5e077fe640 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66414680 0x563c6640e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5e077fe640 1 -- 172.21.15.103:0/878439516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c664094a0 msgr2=0x563c66420f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5e077fe640 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c664094a0 0x563c66420f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5e077fe640 1 -- 172.21.15.103:0/878439516 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c664142e0 con 0x563c66409dc0 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5e077fe640 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66421460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0cd800 tx=0x7f5dfc098740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2aa13ae0d73c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5dedffb640 1 -- 172.21.15.103:0/878439516 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09c390 con 0x563c66409dc0 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5dedffb640 1 -- 172.21.15.103:0/878439516 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dfc0a0740 con 0x563c66409dc0 2024-09-19T07:44:50.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.565+0000 7f5dedffb640 1 -- 172.21.15.103:0/878439516 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0d20e0 con 0x563c66409dc0 2024-09-19T07:44:50.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.566+0000 7f5e10a2a740 1 -- 172.21.15.103:0/878439516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 msgr2=0x563c66421460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.566+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66421460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0cd800 tx=0x7f5dfc098740 comp rx=0 tx=0).stop 2024-09-19T07:44:50.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 -- 172.21.15.103:0/878439516 shutdown_connections 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66414680 0x563c6640e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c66409dc0 0x563c66421460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/878439516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c664094a0 0x563c66420f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 -- 172.21.15.103:0/878439516 >> 172.21.15.103:0/878439516 conn(0x563c66405a30 msgr2=0x563c664132a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 -- 172.21.15.103:0/878439516 shutdown_connections 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 -- 172.21.15.103:0/878439516 wait complete. 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.567+0000 7f5e10a2a740 1 -- start start 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.568+0000 7f5e10a2a740 1 Processor -- start 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.568+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c6642d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.568+0000 7f5e10a2a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c6642b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.569+0000 7f5e07fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c6642d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.569+0000 7f5e07fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c6642d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42680/0 (socket says 172.21.15.103:42680) 2024-09-19T07:44:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.569+0000 7f5e07fff640 1 -- 172.21.15.103:0/535181022 learned_addr learned my addr 172.21.15.103:0/535181022 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:44:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.570+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c6642bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.570+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081acc00 con 0x563c66414680 2024-09-19T07:44:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.570+0000 7f5e077fe640 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c6642b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.570+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e081ac530 con 0x563c664094a0 2024-09-19T07:44:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0811af00 con 0x563c66409dc0 2024-09-19T07:44:50.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e10a2a740 10 client.0 _ll_register_callbacks cb 0x7f5e00009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:44:50.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e07fff640 1 -- 172.21.15.103:0/535181022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 msgr2=0x563c6642b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:50.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e07fff640 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c6642b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e07fff640 1 -- 172.21.15.103:0/535181022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 msgr2=0x563c6642bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:44:50.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e07fff640 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c6642bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e07fff640 1 -- 172.21.15.103:0/535181022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c664101d0 con 0x563c664094a0 2024-09-19T07:44:50.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5e07fff640 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c6642d3a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5df80059d0 tx=0x7f5df8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11d58d5e6f128cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:44:50.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.571+0000 7f5de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5de77fe640 1 -- 172.21.15.103:0/535181022 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8040040 con 0x563c664094a0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5de77fe640 1 -- 172.21.15.103:0/535181022 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8498+0+0 (secure 0 0 0) 0x7f5df8004030 con 0x563c664094a0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dfc0a0740 con 0x563c664094a0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5de77fe640 1 -- 172.21.15.103:0/535181022 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5df8004230 con 0x563c664094a0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5e10a2a740 10 client.26381 fetch_fsmap learned FSMap version 40 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563c664b0dd0 con 0x563c664094a0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5de77fe640 1 -- 172.21.15.103:0/535181022 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dfc0a0740 con 0x563c664094a0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5de77fe640 1 -- 172.21.15.103:0/535181022 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f5df80584f0 con 0x563c664094a0 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5e10a2a740 10 client.26381 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5e10a2a740 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5e10a2a740 20 client.26381 populate_metadata read hostname 'smithi103' 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.572+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664b10c0 0x563c664b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.573+0000 7f5e10a2a740 4 client.26381 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:44:50.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.573+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c66376d60 con 0x563c664b10c0 2024-09-19T07:44:50.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.573+0000 7f5e077fe640 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664b10c0 0x563c664b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:44:50.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.573+0000 7f5e077fe640 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664b10c0 0x563c664b34f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e37c27fc4c1cbb server_cookie=557c69955fff6400 in_seq=0 out_seq=0 2024-09-19T07:44:50.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.573+0000 7f5de77fe640 10 client.26381 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:50.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.573+0000 7f5de77fe640 1 -- 172.21.15.103:0/535181022 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563c66376d60 con 0x563c664b10c0 2024-09-19T07:44:50.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5de77fe640 10 client.26381 handle_command_reply: tid=0 2024-09-19T07:44:50.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5e10a2a740 1 client.26381 shutdown 2024-09-19T07:44:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 msgr2=0x563c6642d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c6642d3a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5df80059d0 tx=0x7f5df8005fd0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 shutdown_connections 2024-09-19T07:44:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563c664b10c0 0x563c664b34f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c66414680 0x563c6642bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c66409dc0 0x563c6642b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e10a2a740 1 --2- 172.21.15.103:0/535181022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c664094a0 0x563c6642d3a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.574+0000 7f5de77fe640 0 client.26381 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 >> 172.21.15.103:0/535181022 conn(0x563c66405a30 msgr2=0x563c66404350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 shutdown_connections 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e10a2a740 1 -- 172.21.15.103:0/535181022 wait complete. 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e10a2a740 20 client.26381 trim_cache size 0 max 16384 2024-09-19T07:44:50.578 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "847be915-e518-43db-99f3-985a35482e6e": { 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "847be915-e518-43db-99f3-985a35482e6e", 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:44:50.579 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:44:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5df00a34b0 msgr2=0x7f5df00a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5df00a34b0 0x7f5df00a5970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009fd0 tx=0x7f5e00004ab0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 msgr2=0x7f5e0810c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.575+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 0x7f5e0810c170 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00ea90 tx=0x7f5dfc00ef50 comp rx=0 tx=0).stop 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 shutdown_connections 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5df00a34b0 0x7f5df00a5970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e08106f10 0x7f5e0810c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e08102790 0x7f5e0810c170 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 --2- 172.21.15.103:0/1365293751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e08101de0 0x7f5e0810bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 >> 172.21.15.103:0/1365293751 conn(0x7f5e080fd880 msgr2=0x7f5e080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 shutdown_connections 2024-09-19T07:44:50.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:44:50.576+0000 7f5e0e651640 1 -- 172.21.15.103:0/1365293751 wait complete. 2024-09-19T07:44:51.615 INFO:tasks.cephfs.filesystem:scrub status for tag:847be915-e518-43db-99f3-985a35482e6e - {'path': '/', 'tag': '847be915-e518-43db-99f3-985a35482e6e', 'options': 'recursive,force'} 2024-09-19T07:44:52.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:51 smithi103 ceph-mon[31201]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.8 MiB/s wr, 912 op/s 2024-09-19T07:44:52.332 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:51 smithi134 ceph-mon[35414]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.8 MiB/s wr, 912 op/s 2024-09-19T07:44:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:51 smithi174 ceph-mon[35370]: pgmap v209: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.8 MiB/s wr, 912 op/s 2024-09-19T07:44:54.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:53 smithi103 ceph-mon[31201]: pgmap v210: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.6 MiB/s wr, 605 op/s 2024-09-19T07:44:54.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:53 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:54.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:53 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:54.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:53 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:54.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:53 smithi134 ceph-mon[35414]: pgmap v210: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.6 MiB/s wr, 605 op/s 2024-09-19T07:44:54.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:53 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:54.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:53 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:54.331 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:53 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:53 smithi174 ceph-mon[35370]: pgmap v210: 97 pgs: 97 active+clean; 3.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.6 MiB/s wr, 605 op/s 2024-09-19T07:44:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:53 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:54.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:53 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:54.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:53 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:55.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:54 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:55.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:54 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:44:55.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:54 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:55.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:54 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:44:55.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:54 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:55.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:54 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:44:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 380.0 s, 3604.2 tps, lat 27.675 ms stddev 31.957 2024-09-19T07:44:56.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:55 smithi103 ceph-mon[31201]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 999 op/s 2024-09-19T07:44:56.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:55 smithi134 ceph-mon[35414]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 999 op/s 2024-09-19T07:44:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:55 smithi174 ceph-mon[35370]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 15 MiB/s wr, 999 op/s 2024-09-19T07:44:58.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:57 smithi103 ceph-mon[31201]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 946 op/s 2024-09-19T07:44:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:57 smithi134 ceph-mon[35414]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 946 op/s 2024-09-19T07:44:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:57 smithi174 ceph-mon[35370]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 946 op/s 2024-09-19T07:45:00.217 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:44:59 smithi174 ceph-mon[35370]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 MiB/s wr, 854 op/s 2024-09-19T07:45:00.219 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:44:59 smithi103 ceph-mon[31201]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 MiB/s wr, 854 op/s 2024-09-19T07:45:00.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:44:59 smithi134 ceph-mon[35414]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 MiB/s wr, 854 op/s 2024-09-19T07:45:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 385.0 s, 2659.8 tps, lat 37.684 ms stddev 42.340 2024-09-19T07:45:02.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:01 smithi103 ceph-mon[31201]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-09-19T07:45:02.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:01 smithi134 ceph-mon[35414]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-09-19T07:45:02.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:01 smithi174 ceph-mon[35370]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-09-19T07:45:03.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:03 smithi103 ceph-mon[31201]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 840 op/s 2024-09-19T07:45:04.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:03 smithi134 ceph-mon[35414]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 840 op/s 2024-09-19T07:45:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:03 smithi174 ceph-mon[35370]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 840 op/s 2024-09-19T07:45:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 390.0 s, 444.4 tps, lat 216.396 ms stddev 206.544 2024-09-19T07:45:06.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:05 smithi103 ceph-mon[31201]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 997 op/s 2024-09-19T07:45:06.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:05 smithi134 ceph-mon[35414]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 997 op/s 2024-09-19T07:45:06.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:05 smithi174 ceph-mon[35370]: pgmap v216: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 997 op/s 2024-09-19T07:45:08.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:07 smithi103 ceph-mon[31201]: pgmap v217: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.4 MiB/s wr, 692 op/s 2024-09-19T07:45:08.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:07 smithi134 ceph-mon[35414]: pgmap v217: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.4 MiB/s wr, 692 op/s 2024-09-19T07:45:08.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:07 smithi174 ceph-mon[35370]: pgmap v217: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.4 MiB/s wr, 692 op/s 2024-09-19T07:45:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:09 smithi103 ceph-mon[31201]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-19T07:45:10.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:09 smithi103 ceph-mon[31201]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T07:45:10.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:09 smithi134 ceph-mon[35414]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-19T07:45:10.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:09 smithi134 ceph-mon[35414]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T07:45:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:09 smithi174 ceph-mon[35370]: pgmap v218: 97 pgs: 97 active+clean; 4.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-19T07:45:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:09 smithi174 ceph-mon[35370]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T07:45:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 395.0 s, 1597.0 tps, lat 65.065 ms stddev 129.311 2024-09-19T07:45:12.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:11 smithi103 ceph-mon[31201]: pgmap v220: 97 pgs: 97 active+clean; 3.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.49k op/s 2024-09-19T07:45:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:11 smithi134 ceph-mon[35414]: pgmap v220: 97 pgs: 97 active+clean; 3.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.49k op/s 2024-09-19T07:45:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:11 smithi174 ceph-mon[35370]: pgmap v220: 97 pgs: 97 active+clean; 3.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.49k op/s 2024-09-19T07:45:14.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:13 smithi103 ceph-mon[31201]: pgmap v221: 97 pgs: 97 active+clean; 3.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.49k op/s 2024-09-19T07:45:14.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:13 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:45:14.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:13 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:45:14.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:13 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:45:14.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:13 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:45:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:13 smithi134 ceph-mon[35414]: pgmap v221: 97 pgs: 97 active+clean; 3.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.49k op/s 2024-09-19T07:45:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:13 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:45:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:13 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:45:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:13 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:45:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:13 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:45:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:13 smithi174 ceph-mon[35370]: pgmap v221: 97 pgs: 97 active+clean; 3.8 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 MiB/s wr, 1.49k op/s 2024-09-19T07:45:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:13 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:45:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:13 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:45:14.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:13 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:45:14.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:13 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:45:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 400.0 s, 2602.2 tps, lat 38.533 ms stddev 42.668 2024-09-19T07:45:15.470 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:45:15.470 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T07:45:15.471 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:45:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:15 smithi103 ceph-mon[31201]: pgmap v222: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 MiB/s wr, 1.66k op/s 2024-09-19T07:45:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:15 smithi103 ceph-mon[31201]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T07:45:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:15 smithi134 ceph-mon[35414]: pgmap v222: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 MiB/s wr, 1.66k op/s 2024-09-19T07:45:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:15 smithi134 ceph-mon[35414]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T07:45:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:15 smithi174 ceph-mon[35370]: pgmap v222: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 40 MiB/s wr, 1.66k op/s 2024-09-19T07:45:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:15 smithi174 ceph-mon[35370]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T07:45:17.048 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:45:17.048 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":40,"btime":"2024-09-19T07:44:20:535666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:44:19.854970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 -- 172.21.15.103:0/1357106138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84106f10 msgr2=0x7f3b8410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 --2- 172.21.15.103:0/1357106138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84106f10 0x7f3b8410ea90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009ca0 tx=0x7f3b7c02a5d0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 -- 172.21.15.103:0/1357106138 shutdown_connections 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 --2- 172.21.15.103:0/1357106138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84106f10 0x7f3b8410ea90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 --2- 172.21.15.103:0/1357106138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b84102790 0x7f3b841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 --2- 172.21.15.103:0/1357106138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b84101de0 0x7f3b841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 -- 172.21.15.103:0/1357106138 >> 172.21.15.103:0/1357106138 conn(0x7f3b840fd8c0 msgr2=0x7f3b840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 -- 172.21.15.103:0/1357106138 shutdown_connections 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.473+0000 7f3b8a854640 1 -- 172.21.15.103:0/1357106138 wait complete. 2024-09-19T07:45:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.474+0000 7f3b8a854640 1 Processor -- start 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.474+0000 7f3b8a854640 1 -- start start 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.474+0000 7f3b8a854640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84101de0 0x7f3b84196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.474+0000 7f3b8a854640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b84102790 0x7f3b84196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.475+0000 7f3b8a854640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b84106f10 0x7f3b8419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.475+0000 7f3b8a854640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8410fd60 con 0x7f3b84106f10 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.475+0000 7f3b8a854640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8410fbe0 con 0x7f3b84102790 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.475+0000 7f3b8a854640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8410fee0 con 0x7f3b84101de0 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84101de0 0x7f3b84196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84101de0 0x7f3b84196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49674/0 (socket says 172.21.15.103:49674) 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 -- 172.21.15.103:0/740213321 learned_addr learned my addr 172.21.15.103:0/740213321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b837fe640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b84102790 0x7f3b84196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b88df1640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b84106f10 0x7f3b8419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 -- 172.21.15.103:0/740213321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b84102790 msgr2=0x7f3b84196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b84102790 0x7f3b84196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 -- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b84106f10 msgr2=0x7f3b8419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b84106f10 0x7f3b8419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 -- 172.21.15.103:0/740213321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8419b1f0 con 0x7f3b84101de0 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b837fe640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b84102790 0x7f3b84196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b88df1640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b84106f10 0x7f3b8419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b83fff640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84101de0 0x7f3b84196240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400ff50 tx=0x7f3b74015f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b817fa640 1 -- 172.21.15.103:0/740213321 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74022070 con 0x7f3b84101de0 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b817fa640 1 -- 172.21.15.103:0/740213321 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b74013e30 con 0x7f3b84101de0 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b817fa640 1 -- 172.21.15.103:0/740213321 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7401cb10 con 0x7f3b84101de0 2024-09-19T07:45:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.476+0000 7f3b8a854640 1 -- 172.21.15.103:0/740213321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8419b4e0 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.477+0000 7f3b8a854640 1 -- 172.21.15.103:0/740213321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b841a2dc0 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.477+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b64005290 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.479+0000 7f3b817fa640 1 -- 172.21.15.103:0/740213321 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3b74025070 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.480+0000 7f3b817fa640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b6c0a3480 0x7f3b6c0a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.480+0000 7f3b817fa640 1 -- 172.21.15.103:0/740213321 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f3b740d79b0 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.480+0000 7f3b837fe640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b6c0a3480 0x7f3b6c0a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.481+0000 7f3b837fe640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b6c0a3480 0x7f3b6c0a5940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b841087d0 tx=0x7f3b78005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.483+0000 7f3b817fa640 1 -- 172.21.15.103:0/740213321 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3b74018070 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 40 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.793+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b64005580 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.794+0000 7f3b817fa640 1 -- 172.21.15.103:0/740213321 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11996 (secure 0 0 0) 0x7f3b740a4320 con 0x7f3b84101de0 2024-09-19T07:45:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b6c0a3480 msgr2=0x7f3b6c0a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b6c0a3480 0x7f3b6c0a5940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b841087d0 tx=0x7f3b78005f90 comp rx=0 tx=0).stop 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84101de0 msgr2=0x7f3b84196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84101de0 0x7f3b84196240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400ff50 tx=0x7f3b74015f90 comp rx=0 tx=0).stop 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 shutdown_connections 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b6c0a3480 0x7f3b6c0a5940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b84106f10 0x7f3b8419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b84102790 0x7f3b84196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 --2- 172.21.15.103:0/740213321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b84101de0 0x7f3b84196240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 >> 172.21.15.103:0/740213321 conn(0x7f3b840fd8c0 msgr2=0x7f3b840feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.798+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 shutdown_connections 2024-09-19T07:45:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:16.801+0000 7f3b6affd640 1 -- 172.21.15.103:0/740213321 wait complete. 2024-09-19T07:45:17.054 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:16 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/740213321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:17.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:16 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/740213321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:16 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/740213321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:17.962 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T07:45:17.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T07:45:18.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:17 smithi103 ceph-mon[31201]: pgmap v224: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.03k op/s 2024-09-19T07:45:18.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:17 smithi134 ceph-mon[35414]: pgmap v224: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.03k op/s 2024-09-19T07:45:18.329 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:17 smithi174 ceph-mon[35370]: pgmap v224: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 50 MiB/s wr, 2.03k op/s 2024-09-19T07:45:18.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.900+0000 7ff38c8c5640 1 -- 172.21.15.103:0/432129981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109830 msgr2=0x7ff388109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.900+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/432129981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109830 0x7ff388109c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009900 tx=0x7ff37c02a410 comp rx=0 tx=0).stop 2024-09-19T07:45:18.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.900+0000 7ff38c8c5640 1 -- 172.21.15.103:0/432129981 shutdown_connections 2024-09-19T07:45:18.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.900+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/432129981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38811b140 0x7ff38811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.900+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/432129981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109830 0x7ff388109c90 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009900 tx=0x7ff37c02a410 comp rx=0 tx=0).stop 2024-09-19T07:45:18.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.900+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/432129981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff388109450 0x7ff388110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.900+0000 7ff38c8c5640 1 -- 172.21.15.103:0/432129981 >> 172.21.15.103:0/432129981 conn(0x7ff38806e140 msgr2=0x7ff38806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:18.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.901+0000 7ff38c8c5640 1 -- 172.21.15.103:0/432129981 shutdown_connections 2024-09-19T07:45:18.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.901+0000 7ff38c8c5640 1 -- 172.21.15.103:0/432129981 wait complete. 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 Processor -- start 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 -- start start 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109450 0x7ff3881a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38811b140 0x7ff3881a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3881abdc0 0x7ff3881ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff388120e50 con 0x7ff38811b140 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff388120cd0 con 0x7ff3881abdc0 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.902+0000 7ff38c8c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff388120fd0 con 0x7ff388109450 2024-09-19T07:45:18.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.903+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3881abdc0 0x7ff3881ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.903+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3881abdc0 0x7ff3881ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50686/0 (socket says 172.21.15.103:50686) 2024-09-19T07:45:18.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.903+0000 7ff386d9d640 1 -- 172.21.15.103:0/455223680 learned_addr learned my addr 172.21.15.103:0/455223680 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:18.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.903+0000 7ff38659c640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109450 0x7ff3881a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.904+0000 7ff38659c640 1 -- 172.21.15.103:0/455223680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3881abdc0 msgr2=0x7ff3881ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.904+0000 7ff38659c640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3881abdc0 0x7ff3881ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.904+0000 7ff38659c640 1 -- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38811b140 msgr2=0x7ff3881a7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.904+0000 7ff38659c640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38811b140 0x7ff3881a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.904+0000 7ff38659c640 1 -- 172.21.15.103:0/455223680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3881ac910 con 0x7ff388109450 2024-09-19T07:45:18.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.904+0000 7ff38659c640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109450 0x7ff3881a74f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff378009fd0 tx=0x7ff37800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.905+0000 7ff3777fe640 1 -- 172.21.15.103:0/455223680 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378015020 con 0x7ff388109450 2024-09-19T07:45:18.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.906+0000 7ff3777fe640 1 -- 172.21.15.103:0/455223680 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff378011d10 con 0x7ff388109450 2024-09-19T07:45:18.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.906+0000 7ff3777fe640 1 -- 172.21.15.103:0/455223680 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37800f2e0 con 0x7ff388109450 2024-09-19T07:45:18.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.906+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3881acc00 con 0x7ff388109450 2024-09-19T07:45:18.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.907+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3881b4390 con 0x7ff388109450 2024-09-19T07:45:18.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.911+0000 7ff38ec9e740 1 Processor -- start 2024-09-19T07:45:18.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.912+0000 7ff38ec9e740 1 -- start start 2024-09-19T07:45:18.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.912+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598415f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.912+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x5598416034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.912+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x559841605e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.913+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55984155d160 con 0x559841603a30 2024-09-19T07:45:18.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.913+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559841504720 con 0x5598415f8850 2024-09-19T07:45:18.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.913+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559841559120 con 0x5598415f9170 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.913+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598415f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.913+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598415f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50690/0 (socket says 172.21.15.103:50690) 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.913+0000 7ff386d9d640 1 -- 172.21.15.103:0/3594522492 learned_addr learned my addr 172.21.15.103:0/3594522492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.913+0000 7ff38659c640 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x5598416034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.914+0000 7ff386d9d640 1 -- 172.21.15.103:0/3594522492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 msgr2=0x5598416034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.914+0000 7ff386d9d640 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x5598416034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.914+0000 7ff386d9d640 1 -- 172.21.15.103:0/3594522492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 msgr2=0x559841605e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.914+0000 7ff386d9d640 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x559841605e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.914+0000 7ff386d9d640 1 -- 172.21.15.103:0/3594522492 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598412ceba0 con 0x5598415f8850 2024-09-19T07:45:18.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.914+0000 7ff3777fe640 1 -- 172.21.15.103:0/455223680 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff378018020 con 0x7ff388109450 2024-09-19T07:45:18.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.914+0000 7ff386d9d640 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598415f8c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff380011460 tx=0x7ff380011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd95f8bbcbc5dde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.915+0000 7ff374ff9640 1 -- 172.21.15.103:0/3594522492 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380012770 con 0x5598415f8850 2024-09-19T07:45:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.915+0000 7ff374ff9640 1 -- 172.21.15.103:0/3594522492 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff380012df0 con 0x5598415f8850 2024-09-19T07:45:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.915+0000 7ff374ff9640 1 -- 172.21.15.103:0/3594522492 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800193a0 con 0x5598415f8850 2024-09-19T07:45:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff3777fe640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff3700a34f0 0x7ff3700a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3594522492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 msgr2=0x5598415f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598415f8c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff380011460 tx=0x7ff380011920 comp rx=0 tx=0).stop 2024-09-19T07:45:18.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff3777fe640 1 -- 172.21.15.103:0/455223680 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7ff3780cf320 con 0x7ff388109450 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3594522492 shutdown_connections 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x559841605e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x5598416034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3594522492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598415f8c30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.916+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3594522492 >> 172.21.15.103:0/3594522492 conn(0x5598415f4de0 msgr2=0x5598415f51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.917+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3594522492 shutdown_connections 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.917+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3594522492 wait complete. 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.917+0000 7ff38659c640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff3700a34f0 0x7ff3700a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.917+0000 7ff38ec9e740 1 -- start start 2024-09-19T07:45:18.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.917+0000 7ff38659c640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff3700a34f0 0x7ff3700a59b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5598415f5c40 tx=0x7ff3780d2e10 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.917+0000 7ff38ec9e740 1 Processor -- start 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598415f8850 0x5598416009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x559841600f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559841603a30 0x559841601470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559841606600 con 0x5598415f8850 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5598414e5900 con 0x559841603a30 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559841606780 con 0x5598415f9170 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff38ec9e740 10 client.0 _ll_register_callbacks cb 0x5598414e3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598415f8850 0x5598416009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598415f8850 0x5598416009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49816/0 (socket says 172.21.15.103:49816) 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff386d9d640 1 -- 172.21.15.103:0/3088377370 learned_addr learned my addr 172.21.15.103:0/3088377370 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:18.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff386d9d640 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559841603a30 0x559841601470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.918+0000 7ff385d9b640 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x559841600f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.919+0000 7ff385d9b640 1 -- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559841603a30 msgr2=0x559841601470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.919+0000 7ff385d9b640 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559841603a30 0x559841601470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.919+0000 7ff385d9b640 1 -- 172.21.15.103:0/3088377370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598415f8850 msgr2=0x5598416009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.919+0000 7ff385d9b640 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598415f8850 0x5598416009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.919+0000 7ff385d9b640 1 -- 172.21.15.103:0/3088377370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559841525c50 con 0x5598415f9170 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.919+0000 7ff385d9b640 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x559841600f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff37c008000 tx=0x7ff37c031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba66b0f86e7425d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.919+0000 7ff36dffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.920+0000 7ff36dffb640 1 -- 172.21.15.103:0/3088377370 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c040070 con 0x5598415f9170 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.920+0000 7ff36dffb640 1 -- 172.21.15.103:0/3088377370 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7ff37c03a3f0 con 0x5598415f9170 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.920+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff380012df0 con 0x5598415f9170 2024-09-19T07:45:18.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.920+0000 7ff36dffb640 1 -- 172.21.15.103:0/3088377370 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff37c04c050 con 0x5598415f9170 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.920+0000 7ff38ec9e740 10 client.26314 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.920+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559841612370 con 0x5598415f9170 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.920+0000 7ff36dffb640 1 -- 172.21.15.103:0/3088377370 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37c03aaf0 con 0x5598415f9170 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff36dffb640 1 -- 172.21.15.103:0/3088377370 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7ff37c045860 con 0x5598415f9170 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff38ec9e740 10 client.26314 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff38ec9e740 10 client.26314 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff38ec9e740 20 client.26314 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559841613780 0x559841615b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff38ec9e740 4 client.26314 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff380012df0 con 0x559841613780 2024-09-19T07:45:18.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.921+0000 7ff386d9d640 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559841613780 0x559841615b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.922+0000 7ff386d9d640 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559841613780 0x559841615b70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fb50e3210c922 server_cookie=d94bc5e6162f55dd in_seq=0 out_seq=0 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.922+0000 7ff36dffb640 10 client.26314 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.922+0000 7ff36dffb640 1 -- 172.21.15.103:0/3088377370 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff380012df0 con 0x559841613780 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.922+0000 7ff36dffb640 10 client.26314 handle_command_reply: tid=0 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.922+0000 7ff38ec9e740 1 client.26314 shutdown 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.922+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 msgr2=0x559841600f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.922+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x559841600f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff37c008000 tx=0x7ff37c031630 comp rx=0 tx=0).stop 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 shutdown_connections 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559841613780 0x559841615b70 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559841603a30 0x559841601470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x559841600f30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/3088377370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598415f8850 0x5598416009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 >> 172.21.15.103:0/3088377370 conn(0x5598415f4de0 msgr2=0x5598415faac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:18.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff36dffb640 0 client.26314 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:18.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 shutdown_connections 2024-09-19T07:45:18.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 1 -- 172.21.15.103:0/3088377370 wait complete. 2024-09-19T07:45:18.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.923+0000 7ff38ec9e740 20 client.26314 trim_cache size 0 max 16384 2024-09-19T07:45:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 Processor -- start 2024-09-19T07:45:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 -- start start 2024-09-19T07:45:18.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598416012c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x55984161b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x55984161b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881bced0 con 0x559841603a30 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881bea10 con 0x5598415f8850 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.943+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881bd5a0 con 0x5598415f9170 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x55984161b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x55984161b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49820/0 (socket says 172.21.15.103:49820) 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 -- 172.21.15.103:0/922262817 learned_addr learned my addr 172.21.15.103:0/922262817 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598416012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff385d9b640 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x55984161b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 -- 172.21.15.103:0/922262817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 msgr2=0x55984161b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x55984161b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 -- 172.21.15.103:0/922262817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 msgr2=0x55984161b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x55984161b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 -- 172.21.15.103:0/922262817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598415f8fa0 con 0x5598415f8850 2024-09-19T07:45:18.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.944+0000 7ff386d9d640 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598416012c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff380011e20 tx=0x7ff38001aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9dac77b2d55aea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.945+0000 7ff36ffff640 1 -- 172.21.15.103:0/922262817 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38001a5f0 con 0x5598415f8850 2024-09-19T07:45:18.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.945+0000 7ff36ffff640 1 -- 172.21.15.103:0/922262817 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38001a790 con 0x5598415f8850 2024-09-19T07:45:18.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.945+0000 7ff36ffff640 1 -- 172.21.15.103:0/922262817 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380029850 con 0x5598415f8850 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 -- 172.21.15.103:0/922262817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 msgr2=0x5598416012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598416012c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff380011e20 tx=0x7ff38001aaf0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 -- 172.21.15.103:0/922262817 shutdown_connections 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x55984161b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f9170 0x55984161b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/922262817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f8850 0x5598416012c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 -- 172.21.15.103:0/922262817 >> 172.21.15.103:0/922262817 conn(0x5598415f4de0 msgr2=0x559841602a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:18.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 -- 172.21.15.103:0/922262817 shutdown_connections 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 -- 172.21.15.103:0/922262817 wait complete. 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 -- start start 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.946+0000 7ff38ec9e740 1 Processor -- start 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f8850 0x55984160b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f9170 0x55984160bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff38ec9e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x5598416089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881be340 con 0x559841603a30 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881bdc70 con 0x5598415f9170 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff38ec9e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881baed0 con 0x5598415f8850 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff38ec9e740 10 client.0 _ll_register_callbacks cb 0x7ff37c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x5598416089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff385d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f9170 0x55984160bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff386d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x5598416089a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49832/0 (socket says 172.21.15.103:49832) 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff386d9d640 1 -- 172.21.15.103:0/1023433428 learned_addr learned my addr 172.21.15.103:0/1023433428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff386d9d640 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f8850 0x55984160b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff385d9b640 1 -- 172.21.15.103:0/1023433428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f8850 msgr2=0x55984160b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff385d9b640 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f8850 0x55984160b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff385d9b640 1 -- 172.21.15.103:0/1023433428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 msgr2=0x5598416089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff385d9b640 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x5598416089a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff385d9b640 1 -- 172.21.15.103:0/1023433428 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559841608ee0 con 0x5598415f9170 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.947+0000 7ff386d9d640 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x5598416089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.948+0000 7ff385d9b640 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f9170 0x55984160bf10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009900 tx=0x7ff37c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3665495b915932 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:18.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.948+0000 7ff36dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.948+0000 7ff36dffb640 1 -- 172.21.15.103:0/1023433428 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c055060 con 0x5598415f9170 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.948+0000 7ff36dffb640 1 -- 172.21.15.103:0/1023433428 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7ff37c0055f0 con 0x5598415f9170 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55984156a890 con 0x5598415f9170 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff36dffb640 1 -- 172.21.15.103:0/1023433428 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff37c040070 con 0x5598415f9170 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 10 client.26402 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55984167eb30 con 0x5598415f9170 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff36dffb640 1 -- 172.21.15.103:0/1023433428 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55984156a890 con 0x5598415f9170 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff36dffb640 1 -- 172.21.15.103:0/1023433428 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7ff37c004a60 con 0x5598415f9170 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 10 client.26402 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 10 client.26402 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 20 client.26402 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:18.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55984167f2d0 0x5598416816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:18.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 4 client.26402 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T07:45:18.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.949+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x559841564be0 con 0x55984167f2d0 2024-09-19T07:45:18.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.950+0000 7ff386d9d640 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55984167f2d0 0x5598416816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:18.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.950+0000 7ff36dffb640 10 client.26402 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:18.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:18.950+0000 7ff386d9d640 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55984167f2d0 0x5598416816f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92806e8011d822d server_cookie=4b526040ed14fa3 in_seq=0 out_seq=0 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff36dffb640 1 -- 172.21.15.103:0/1023433428 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559841564be0 con 0x55984167f2d0 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff36dffb640 10 client.26402 handle_command_reply: tid=0 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff38ec9e740 1 client.26402 shutdown 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f9170 msgr2=0x55984160bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f9170 0x55984160bf10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009900 tx=0x7ff37c005270 comp rx=0 tx=0).stop 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 shutdown_connections 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55984167f2d0 0x5598416816f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.160+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559841603a30 0x5598416089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598415f9170 0x55984160bf10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38ec9e740 1 --2- 172.21.15.103:0/1023433428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598415f8850 0x55984160b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff36dffb640 0 client.26402 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 >> 172.21.15.103:0/1023433428 conn(0x5598415f4de0 msgr2=0x559841602250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 shutdown_connections 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38ec9e740 1 -- 172.21.15.103:0/1023433428 wait complete. 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38ec9e740 20 client.26402 trim_cache size 0 max 16384 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:45:19.162 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 40, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 26, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "80da2ce4": { 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 26, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:45:19.163 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:19.164 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:45:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff3700a34f0 msgr2=0x7ff3700a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff3700a34f0 0x7ff3700a59b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5598415f5c40 tx=0x7ff3780d2e10 comp rx=0 tx=0).stop 2024-09-19T07:45:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109450 msgr2=0x7ff3881a74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109450 0x7ff3881a74f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff378009fd0 tx=0x7ff37800b870 comp rx=0 tx=0).stop 2024-09-19T07:45:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 shutdown_connections 2024-09-19T07:45:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff3700a34f0 0x7ff3700a59b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3881abdc0 0x7ff3881ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff38811b140 0x7ff3881a7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 --2- 172.21.15.103:0/455223680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff388109450 0x7ff3881a74f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.161+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 >> 172.21.15.103:0/455223680 conn(0x7ff38806e140 msgr2=0x7ff38811ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.162+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 shutdown_connections 2024-09-19T07:45:19.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:19.162+0000 7ff38c8c5640 1 -- 172.21.15.103:0/455223680 wait complete. 2024-09-19T07:45:19.935 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '80da2ce4', quiesce took 4.5 seconds. Will release after: 21.25 2024-09-19T07:45:20.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:19 smithi103 ceph-mon[31201]: pgmap v225: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 42 MiB/s wr, 1.72k op/s 2024-09-19T07:45:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:19 smithi134 ceph-mon[35414]: pgmap v225: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 42 MiB/s wr, 1.72k op/s 2024-09-19T07:45:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:19 smithi174 ceph-mon[35370]: pgmap v225: 97 pgs: 97 active+clean; 3.9 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 42 MiB/s wr, 1.72k op/s 2024-09-19T07:45:20.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 405.0 s, 1738.4 tps, lat 39.873 ms stddev 42.142 2024-09-19T07:45:21.616 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:45:21.872 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:22.278 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:21 smithi103 ceph-mon[31201]: pgmap v226: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 752 op/s 2024-09-19T07:45:22.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:21 smithi134 ceph-mon[35414]: pgmap v226: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 752 op/s 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 -- 172.21.15.103:0/4042613936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c1002f0 msgr2=0x7fde9c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 --2- 172.21.15.103:0/4042613936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c1002f0 0x7fde9c1006d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009930 tx=0x7fde8c02a420 comp rx=0 tx=0).stop 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 -- 172.21.15.103:0/4042613936 shutdown_connections 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 --2- 172.21.15.103:0/4042613936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c106f50 0x7fde9c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 --2- 172.21.15.103:0/4042613936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde9c100ca0 0x7fde9c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 --2- 172.21.15.103:0/4042613936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c1002f0 0x7fde9c1006d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 -- 172.21.15.103:0/4042613936 >> 172.21.15.103:0/4042613936 conn(0x7fde9c077210 msgr2=0x7fde9c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 -- 172.21.15.103:0/4042613936 shutdown_connections 2024-09-19T07:45:22.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.360+0000 7fdea2c97640 1 -- 172.21.15.103:0/4042613936 wait complete. 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 Processor -- start 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 -- start start 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c1002f0 0x7fde9c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c100ca0 0x7fde9c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde9c106f50 0x7fde9c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c111fa0 con 0x7fde9c100ca0 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c111e20 con 0x7fde9c106f50 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea2c97640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112120 con 0x7fde9c1002f0 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c100ca0 0x7fde9c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c100ca0 0x7fde9c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49848/0 (socket says 172.21.15.103:49848) 2024-09-19T07:45:22.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fde9bfff640 1 -- 172.21.15.103:0/3385655849 learned_addr learned my addr 172.21.15.103:0/3385655849 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.361+0000 7fdea0a33640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c1002f0 0x7fde9c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fdea1234640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde9c106f50 0x7fde9c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde9bfff640 1 -- 172.21.15.103:0/3385655849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c1002f0 msgr2=0x7fde9c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde9bfff640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c1002f0 0x7fde9c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde9bfff640 1 -- 172.21.15.103:0/3385655849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde9c106f50 msgr2=0x7fde9c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde9bfff640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde9c106f50 0x7fde9c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde9bfff640 1 -- 172.21.15.103:0/3385655849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c1a6a20 con 0x7fde9c100ca0 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde9bfff640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c100ca0 0x7fde9c1017b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fde90005fd0 tx=0x7fde9000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde99ffb640 1 -- 172.21.15.103:0/3385655849 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90017070 con 0x7fde9c100ca0 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde99ffb640 1 -- 172.21.15.103:0/3385655849 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90009c30 con 0x7fde9c100ca0 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c1a6d10 con 0x7fde9c100ca0 2024-09-19T07:45:22.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fde99ffb640 1 -- 172.21.15.103:0/3385655849 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90010460 con 0x7fde9c100ca0 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.362+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c1a71f0 con 0x7fde9c100ca0 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.364+0000 7fde99ffb640 1 -- 172.21.15.103:0/3385655849 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fde90003e70 con 0x7fde9c100ca0 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 Processor -- start 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 -- start start 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fde99ffb640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fde840a33b0 0x7fde840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f537880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f542140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f544a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c7f49d2e0 con 0x556c7f5374a0 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c7f444ce0 con 0x556c7f542680 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.365+0000 7fdea5070740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c7f4992a0 con 0x556c7f537dc0 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f537880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea1234640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fde840a33b0 0x7fde840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f537880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49864/0 (socket says 172.21.15.103:49864) 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 -- 172.21.15.103:0/3728688357 learned_addr learned my addr 172.21.15.103:0/3728688357 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fde99ffb640 1 -- 172.21.15.103:0/3385655849 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fde900cf9f0 con 0x7fde9c100ca0 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fde9bfff640 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f542140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 -- 172.21.15.103:0/3728688357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 msgr2=0x556c7f542140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f542140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 -- 172.21.15.103:0/3728688357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 msgr2=0x556c7f544a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f544a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 -- 172.21.15.103:0/3728688357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7f429440 con 0x556c7f5374a0 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea1234640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fde840a33b0 0x7fde840a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fde940092f0 tx=0x7fde940097f0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fdea0a33640 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f537880 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde8c02a940 tx=0x7fde8c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeec7cc5944ba13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fde82ffd640 1 -- 172.21.15.103:0/3728688357 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c040070 con 0x556c7f5374a0 2024-09-19T07:45:22.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fde82ffd640 1 -- 172.21.15.103:0/3728688357 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c031930 con 0x556c7f5374a0 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.366+0000 7fde82ffd640 1 -- 172.21.15.103:0/3728688357 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c035040 con 0x556c7f5374a0 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 -- 172.21.15.103:0/3728688357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 msgr2=0x556c7f537880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f537880 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde8c02a940 tx=0x7fde8c005c00 comp rx=0 tx=0).stop 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 -- 172.21.15.103:0/3728688357 shutdown_connections 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f544a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f542140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 --2- 172.21.15.103:0/3728688357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f537880 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 -- 172.21.15.103:0/3728688357 >> 172.21.15.103:0/3728688357 conn(0x556c7f533a30 msgr2=0x556c7f533e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 -- 172.21.15.103:0/3728688357 shutdown_connections 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 -- 172.21.15.103:0/3728688357 wait complete. 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 -- start start 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.367+0000 7fdea5070740 1 Processor -- start 2024-09-19T07:45:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea5070740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f5374a0 0x556c7f541ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea5070740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f537dc0 0x556c7f53f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea5070740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f542680 0x556c7f53f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea5070740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c7f545250 con 0x556c7f542680 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea5070740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c7f458050 con 0x556c7f537dc0 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea5070740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c7f5453d0 con 0x556c7f5374a0 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea5070740 10 client.0 _ll_register_callbacks cb 0x556c7f498160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f542680 0x556c7f53f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f542680 0x556c7f53f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49876/0 (socket says 172.21.15.103:49876) 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 -- 172.21.15.103:0/191207889 learned_addr learned my addr 172.21.15.103:0/191207889 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea0a33640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f5374a0 0x556c7f541ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fde9bfff640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f537dc0 0x556c7f53f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 -- 172.21.15.103:0/191207889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f5374a0 msgr2=0x556c7f541ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f5374a0 0x556c7f541ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 -- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f537dc0 msgr2=0x556c7f53f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f537dc0 0x556c7f53f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 -- 172.21.15.103:0/191207889 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7f20e3c0 con 0x556c7f542680 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fde9bfff640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f537dc0 0x556c7f53f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fdea1234640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f542680 0x556c7f53f6e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fde940138b0 tx=0x7fde94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d52edbde9c9123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.368+0000 7fde7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fde7bfff640 1 -- 172.21.15.103:0/191207889 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9400fc70 con 0x556c7f542680 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fde7bfff640 1 -- 172.21.15.103:0/191207889 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fde94020450 con 0x556c7f542680 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8c031930 con 0x556c7f542680 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fde7bfff640 1 -- 172.21.15.103:0/191207889 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fde9402e050 con 0x556c7f542680 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 10 client.16902 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:22.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556c7f550c20 con 0x556c7f542680 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fde7bfff640 1 -- 172.21.15.103:0/191207889 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c031930 con 0x556c7f542680 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fde7bfff640 1 -- 172.21.15.103:0/191207889 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fde9400f040 con 0x556c7f542680 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 10 client.16902 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 20 client.16902 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f554580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 4 client.16902 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c7f4aa850 con 0x556c7f5520e0 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.369+0000 7fdea0a33640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f554580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.370+0000 7fdea0a33640 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f554580 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe238721a49688e server_cookie=13de5dab0835ea76 in_seq=0 out_seq=0 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.370+0000 7fde7bfff640 10 client.16902 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fde7bfff640 1 -- 172.21.15.103:0/191207889 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c7f4aa850 con 0x556c7f5520e0 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fde7bfff640 10 client.16902 handle_command_reply: tid=0 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 client.16902 shutdown 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f542680 msgr2=0x556c7f53f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f542680 0x556c7f53f6e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fde940138b0 tx=0x7fde94004ab0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 shutdown_connections 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f554580 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f542680 0x556c7f53f6e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fde7bfff640 0 client.16902 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f537dc0 0x556c7f53f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 --2- 172.21.15.103:0/191207889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f5374a0 0x556c7f541ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 >> 172.21.15.103:0/191207889 conn(0x556c7f533a30 msgr2=0x556c7f539710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 shutdown_connections 2024-09-19T07:45:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 1 -- 172.21.15.103:0/191207889 wait complete. 2024-09-19T07:45:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.371+0000 7fdea5070740 20 client.16902 trim_cache size 0 max 16384 2024-09-19T07:45:22.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.388+0000 7fdea5070740 1 Processor -- start 2024-09-19T07:45:22.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.388+0000 7fdea5070740 1 -- start start 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea5070740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f5374a0 0x556c7f541110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea5070740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f537dc0 0x556c7f559ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea5070740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f542680 0x556c7f55a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea5070740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1afb40 con 0x556c7f537dc0 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea5070740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b1680 con 0x556c7f5374a0 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea5070740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b0210 con 0x556c7f542680 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f537dc0 0x556c7f559ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f537dc0 0x556c7f559ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49882/0 (socket says 172.21.15.103:49882) 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 -- 172.21.15.103:0/135004799 learned_addr learned my addr 172.21.15.103:0/135004799 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea1234640 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f542680 0x556c7f55a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea0a33640 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f5374a0 0x556c7f541110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 -- 172.21.15.103:0/135004799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f542680 msgr2=0x556c7f55a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f542680 0x556c7f55a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 -- 172.21.15.103:0/135004799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f5374a0 msgr2=0x556c7f541110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f5374a0 0x556c7f541110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 -- 172.21.15.103:0/135004799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7f533860 con 0x556c7f537dc0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fdea0a33640 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f5374a0 0x556c7f541110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde9bfff640 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f537dc0 0x556c7f559ea0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fde900cd130 tx=0x7fde9009bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bc60074399aab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde827fc640 1 -- 172.21.15.103:0/135004799 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90009de0 con 0x556c7f537dc0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde827fc640 1 -- 172.21.15.103:0/135004799 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde900b1070 con 0x556c7f537dc0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.389+0000 7fde827fc640 1 -- 172.21.15.103:0/135004799 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900b5220 con 0x556c7f537dc0 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 -- 172.21.15.103:0/135004799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f537dc0 msgr2=0x556c7f559ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f537dc0 0x556c7f559ea0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fde900cd130 tx=0x7fde9009bb60 comp rx=0 tx=0).stop 2024-09-19T07:45:22.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 -- 172.21.15.103:0/135004799 shutdown_connections 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f542680 0x556c7f55a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f537dc0 0x556c7f559ea0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 --2- 172.21.15.103:0/135004799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f5374a0 0x556c7f541110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 -- 172.21.15.103:0/135004799 >> 172.21.15.103:0/135004799 conn(0x556c7f533a30 msgr2=0x556c7f540a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 -- 172.21.15.103:0/135004799 shutdown_connections 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 -- 172.21.15.103:0/135004799 wait complete. 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.390+0000 7fdea5070740 1 -- start start 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 1 Processor -- start 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f54aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f539ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f53a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b0fb0 con 0x556c7f5374a0 2024-09-19T07:45:22.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b08e0 con 0x556c7f542680 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1adb40 con 0x556c7f537dc0 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea5070740 10 client.0 _ll_register_callbacks cb 0x7fde94009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f54aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f54aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49884/0 (socket says 172.21.15.103:49884) 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 -- 172.21.15.103:0/766723496 learned_addr learned my addr 172.21.15.103:0/766723496 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fde9bfff640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f539ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea1234640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f53a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 -- 172.21.15.103:0/766723496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 msgr2=0x556c7f539ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f539ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 -- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 msgr2=0x556c7f53a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f53a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea0a33640 1 -- 172.21.15.103:0/766723496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7f548a00 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.391+0000 7fdea1234640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f53a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea0a33640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f54aee0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005ec0 tx=0x7fde8c031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1364e1773a89a897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fde7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fde7bfff640 1 -- 172.21.15.103:0/766723496 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c040040 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fde7bfff640 1 -- 172.21.15.103:0/766723496 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fde8c004240 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde900b1070 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fde7bfff640 1 -- 172.21.15.103:0/766723496 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fde8c035040 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 10 client.16908 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556c7f5bcfb0 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fde7bfff640 1 -- 172.21.15.103:0/766723496 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde900b1070 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fde7bfff640 1 -- 172.21.15.103:0/766723496 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fde8c057600 con 0x556c7f5374a0 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 10 client.16908 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:22.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 20 client.16908 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f5bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 4 client.16908 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.392+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c7f4a4d60 con 0x556c7f5520e0 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fde9bfff640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f5bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fde9bfff640 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f5bf1f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9001b77a0164d686 server_cookie=93086959ce59e461 in_seq=0 out_seq=0 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fde7bfff640 10 client.16908 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fde7bfff640 1 -- 172.21.15.103:0/766723496 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c7f4a4d60 con 0x556c7f5520e0 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fde7bfff640 10 client.16908 handle_command_reply: tid=0 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 client.16908 shutdown 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 msgr2=0x556c7f54aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f54aee0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005ec0 tx=0x7fde8c031ae0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 shutdown_connections 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x556c7f5520e0 0x556c7f5bf1f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c7f542680 0x556c7f53a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c7f537dc0 0x556c7f539ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 --2- 172.21.15.103:0/766723496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c7f5374a0 0x556c7f54aee0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 >> 172.21.15.103:0/766723496 conn(0x556c7f533a30 msgr2=0x556c7f541220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fde7bfff640 0 client.16908 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 shutdown_connections 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.393+0000 7fdea5070740 1 -- 172.21.15.103:0/766723496 wait complete. 2024-09-19T07:45:22.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea5070740 20 client.16908 trim_cache size 0 max 16384 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:45:22.403 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fde840a33b0 msgr2=0x7fde840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fde840a33b0 0x7fde840a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fde940092f0 tx=0x7fde940097f0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c100ca0 msgr2=0x7fde9c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c100ca0 0x7fde9c1017b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fde90005fd0 tx=0x7fde9000dde0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 shutdown_connections 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fde840a33b0 0x7fde840a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde9c106f50 0x7fde9c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde9c100ca0 0x7fde9c1017b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 --2- 172.21.15.103:0/3385655849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c1002f0 0x7fde9c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 >> 172.21.15.103:0/3385655849 conn(0x7fde9c077210 msgr2=0x7fde9c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 shutdown_connections 2024-09-19T07:45:22.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:22.394+0000 7fdea2c97640 1 -- 172.21.15.103:0/3385655849 wait complete. 2024-09-19T07:45:22.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:21 smithi174 ceph-mon[35370]: pgmap v226: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 752 op/s 2024-09-19T07:45:23.457 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:45:23.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:45:23.707 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:24.155 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:23 smithi103 ceph-mon[31201]: pgmap v227: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 752 op/s 2024-09-19T07:45:24.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.154+0000 7fc16fd02640 1 -- 172.21.15.103:0/626415679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168100a80 msgr2=0x7fc168100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:24.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.154+0000 7fc16fd02640 1 --2- 172.21.15.103:0/626415679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168100a80 0x7fc168100ee0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc158009930 tx=0x7fc15802a420 comp rx=0 tx=0).stop 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.154+0000 7fc16fd02640 1 -- 172.21.15.103:0/626415679 shutdown_connections 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.154+0000 7fc16fd02640 1 --2- 172.21.15.103:0/626415679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc168101420 0x7fc16810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.154+0000 7fc16fd02640 1 --2- 172.21.15.103:0/626415679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168100a80 0x7fc168100ee0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.154+0000 7fc16fd02640 1 --2- 172.21.15.103:0/626415679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc168106a80 0x7fc168106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.154+0000 7fc16fd02640 1 -- 172.21.15.103:0/626415679 >> 172.21.15.103:0/626415679 conn(0x7fc1680fc7a0 msgr2=0x7fc1680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.155+0000 7fc16fd02640 1 -- 172.21.15.103:0/626415679 shutdown_connections 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.155+0000 7fc16fd02640 1 -- 172.21.15.103:0/626415679 wait complete. 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.155+0000 7fc16fd02640 1 Processor -- start 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.155+0000 7fc16fd02640 1 -- start start 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.155+0000 7fc16fd02640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc168100a80 0x7fc168197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:24.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.155+0000 7fc16fd02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168101420 0x7fc168197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16fd02640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc168106a80 0x7fc16819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16fd02640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc168110e80 con 0x7fc168101420 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16fd02640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc168110d00 con 0x7fc168106a80 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16fd02640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc168111000 con 0x7fc168100a80 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168101420 0x7fc168197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168101420 0x7fc168197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49900/0 (socket says 172.21.15.103:49900) 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 -- 172.21.15.103:0/3012409084 learned_addr learned my addr 172.21.15.103:0/3012409084 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16e29f640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc168106a80 0x7fc16819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16da9e640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc168100a80 0x7fc168197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 -- 172.21.15.103:0/3012409084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc168100a80 msgr2=0x7fc168197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc168100a80 0x7fc168197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 -- 172.21.15.103:0/3012409084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc168106a80 msgr2=0x7fc16819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc168106a80 0x7fc16819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 -- 172.21.15.103:0/3012409084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16819c2b0 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16e29f640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc168106a80 0x7fc16819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16d29d640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168101420 0x7fc168197840 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc158009230 tx=0x7fc158004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc15effd640 1 -- 172.21.15.103:0/3012409084 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158046070 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc15effd640 1 -- 172.21.15.103:0/3012409084 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc158003ad0 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16819c540 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc15effd640 1 -- 172.21.15.103:0/3012409084 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158035050 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.156+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16806a790 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.157+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1681021c0 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.158+0000 7fc15effd640 1 -- 172.21.15.103:0/3012409084 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc158049070 con 0x7fc168101420 2024-09-19T07:45:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.159+0000 7fc15effd640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1540a33b0 0x7fc1540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:24.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.159+0000 7fc16da9e640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1540a33b0 0x7fc1540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:24.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.160+0000 7fc15effd640 1 -- 172.21.15.103:0/3012409084 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fc1580f1880 con 0x7fc168101420 2024-09-19T07:45:24.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.160+0000 7fc16da9e640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1540a33b0 0x7fc1540a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc164005fd0 tx=0x7fc16400b510 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:24.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.164+0000 7fc15effd640 1 -- 172.21.15.103:0/3012409084 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1580be250 con 0x7fc168101420 2024-09-19T07:45:24.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:24 smithi134 ceph-mon[35414]: pgmap v227: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 752 op/s 2024-09-19T07:45:24.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.366+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc16806b1b0 con 0x7fc168101420 2024-09-19T07:45:24.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.367+0000 7fc15effd640 1 -- 172.21.15.103:0/3012409084 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11996 (secure 0 0 0) 0x7fc1580bdb50 con 0x7fc168101420 2024-09-19T07:45:24.367 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:45:24.367 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":40,"btime":"2024-09-19T07:44:20:535666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:44:19.854970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:45:24.367 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 40 2024-09-19T07:45:24.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1540a33b0 msgr2=0x7fc1540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1540a33b0 0x7fc1540a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc164005fd0 tx=0x7fc16400b510 comp rx=0 tx=0).stop 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168101420 msgr2=0x7fc168197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168101420 0x7fc168197840 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc158009230 tx=0x7fc158004270 comp rx=0 tx=0).stop 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 shutdown_connections 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1540a33b0 0x7fc1540a5870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc168106a80 0x7fc16819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc168101420 0x7fc168197840 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 --2- 172.21.15.103:0/3012409084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc168100a80 0x7fc168197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 >> 172.21.15.103:0/3012409084 conn(0x7fc1680fc7a0 msgr2=0x7fc1680fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 shutdown_connections 2024-09-19T07:45:24.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:24.369+0000 7fc16fd02640 1 -- 172.21.15.103:0/3012409084 wait complete. 2024-09-19T07:45:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:23 smithi174 ceph-mon[35370]: pgmap v227: 97 pgs: 97 active+clean; 3.9 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 752 op/s 2024-09-19T07:45:25.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:45:25.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:25 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3012409084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:25.357 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:25 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3012409084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:25 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3012409084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:25.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 410.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:45:25.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 -- 172.21.15.103:0/1647515156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101420 msgr2=0x7f24ac10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 --2- 172.21.15.103:0/1647515156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101420 0x7f24ac10d950 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009cd0 tx=0x7f24a402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 -- 172.21.15.103:0/1647515156 shutdown_connections 2024-09-19T07:45:25.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 --2- 172.21.15.103:0/1647515156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac101420 0x7f24ac10d950 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 --2- 172.21.15.103:0/1647515156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac100a80 0x7f24ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 --2- 172.21.15.103:0/1647515156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24ac106a80 0x7f24ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 -- 172.21.15.103:0/1647515156 >> 172.21.15.103:0/1647515156 conn(0x7f24ac0fc7a0 msgr2=0x7f24ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 -- 172.21.15.103:0/1647515156 shutdown_connections 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.833+0000 7f24b326e640 1 -- 172.21.15.103:0/1647515156 wait complete. 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 Processor -- start 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 -- start start 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac100a80 0x7f24ac1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac101420 0x7f24ac197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24ac106a80 0x7f24ac19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac110ea0 con 0x7f24ac100a80 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac110d20 con 0x7f24ac101420 2024-09-19T07:45:25.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b326e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111020 con 0x7f24ac106a80 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac100a80 0x7f24ac1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac100a80 0x7f24ac1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49920/0 (socket says 172.21.15.103:49920) 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 -- 172.21.15.103:0/3633769796 learned_addr learned my addr 172.21.15.103:0/3633769796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b0809640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac101420 0x7f24ac197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b180b640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24ac106a80 0x7f24ac19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 -- 172.21.15.103:0/3633769796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24ac106a80 msgr2=0x7f24ac19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24ac106a80 0x7f24ac19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 -- 172.21.15.103:0/3633769796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac101420 msgr2=0x7f24ac197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac101420 0x7f24ac197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 -- 172.21.15.103:0/3633769796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac19c270 con 0x7f24ac100a80 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b0809640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac101420 0x7f24ac197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:25.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.834+0000 7f24b100a640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac100a80 0x7f24ac1972c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d990 tx=0x7f24a800de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.835+0000 7f24a27fc640 1 -- 172.21.15.103:0/3633769796 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a801b070 con 0x7f24ac100a80 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.835+0000 7f24a27fc640 1 -- 172.21.15.103:0/3633769796 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a800edf0 con 0x7f24ac100a80 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.835+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac19c560 con 0x7f24ac100a80 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.835+0000 7f24a27fc640 1 -- 172.21.15.103:0/3633769796 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8014870 con 0x7f24ac100a80 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.835+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac06a790 con 0x7f24ac100a80 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24a27fc640 1 -- 172.21.15.103:0/3633769796 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f24a801e070 con 0x7f24ac100a80 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 Processor -- start 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 -- start start 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69ada090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69adaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ae7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d69a402f0 con 0x563d69adbb40 2024-09-19T07:45:25.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563d699e7cf0 con 0x563d69ae5370 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b5647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d69a3c2b0 con 0x563d69ada5d0 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b180b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69ada090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b180b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69ada090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49936/0 (socket says 172.21.15.103:49936) 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b180b640 1 -- 172.21.15.103:0/403976021 learned_addr learned my addr 172.21.15.103:0/403976021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b100a640 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69adaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24a27fc640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24980a33b0 0x7f24980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.838+0000 7f24b0809640 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ae7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24b180b640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24980a33b0 0x7f24980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24a27fc640 1 -- 172.21.15.103:0/3633769796 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f24a80d1410 con 0x7f24ac100a80 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24b180b640 1 -- 172.21.15.103:0/403976021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 msgr2=0x563d69adaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24b180b640 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69adaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24b180b640 1 -- 172.21.15.103:0/403976021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 msgr2=0x563d69ae7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24b180b640 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ae7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24b180b640 1 -- 172.21.15.103:0/403976021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d699cc3a0 con 0x563d69adbb40 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24b180b640 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69ada090 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f24a4005cc0 tx=0x7f24a4031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2873d33b76932e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24977fe640 1 -- 172.21.15.103:0/403976021 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a404d070 con 0x563d69adbb40 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24977fe640 1 -- 172.21.15.103:0/403976021 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a4036040 con 0x563d69adbb40 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.839+0000 7f24977fe640 1 -- 172.21.15.103:0/403976021 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4030d20 con 0x563d69adbb40 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 -- 172.21.15.103:0/403976021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 msgr2=0x563d69ada090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b180b640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24980a33b0 0x7f24980a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24a404a000 tx=0x7f24a4046000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69ada090 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f24a4005cc0 tx=0x7f24a4031760 comp rx=0 tx=0).stop 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 -- 172.21.15.103:0/403976021 shutdown_connections 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ae7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69adaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 --2- 172.21.15.103:0/403976021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69ada090 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 -- 172.21.15.103:0/403976021 >> 172.21.15.103:0/403976021 conn(0x563d69aa3310 msgr2=0x563d69ad6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 -- 172.21.15.103:0/403976021 shutdown_connections 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 -- 172.21.15.103:0/403976021 wait complete. 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 -- start start 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.840+0000 7f24b5647740 1 Processor -- start 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b5647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ada5d0 0x563d69ae4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b5647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69ae1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b5647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ae5370 0x563d69ae25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b5647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d69ae8070 con 0x563d69ae5370 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b5647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563d699fb060 con 0x563d69ada5d0 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b5647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d69ae81f0 con 0x563d69adbb40 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b5647740 10 client.0 _ll_register_callbacks cb 0x563d6974e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:25.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ae5370 0x563d69ae25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ae5370 0x563d69ae25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49944/0 (socket says 172.21.15.103:49944) 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 -- 172.21.15.103:0/419004051 learned_addr learned my addr 172.21.15.103:0/419004051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b100a640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ada5d0 0x563d69ae4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b0809640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69ae1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 -- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 msgr2=0x563d69ae1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69ae1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 -- 172.21.15.103:0/419004051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ada5d0 msgr2=0x563d69ae4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ada5d0 0x563d69ae4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 -- 172.21.15.103:0/419004051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d697b13c0 con 0x563d69ae5370 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b100a640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ada5d0 0x563d69ae4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b0809640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69ae1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f24b180b640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ae5370 0x563d69ae25d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f24a402ae00 tx=0x7f24a4031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ed6acd324b76e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:25.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.841+0000 7f2494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f2494ff9640 1 -- 172.21.15.103:0/419004051 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a404d040 con 0x563d69ae5370 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f2494ff9640 1 -- 172.21.15.103:0/419004051 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f24a4003e00 con 0x563d69ae5370 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a4036040 con 0x563d69ae5370 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f2494ff9640 1 -- 172.21.15.103:0/419004051 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f24a405f050 con 0x563d69ae5370 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 10 client.16926 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563d69ae8070 con 0x563d69ae5370 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f2494ff9640 1 -- 172.21.15.103:0/419004051 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a4036040 con 0x563d69ae5370 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f2494ff9640 1 -- 172.21.15.103:0/419004051 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f24a4030070 con 0x563d69ae5370 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 10 client.16926 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 10 client.16926 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 20 client.16926 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69af4c50 0x563d69af7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 4 client.16926 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d69a4d860 con 0x563d69af4c50 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.842+0000 7f24b100a640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69af4c50 0x563d69af7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.843+0000 7f24b100a640 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69af4c50 0x563d69af7040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5dc96512aaf1574b server_cookie=9898efca5eed5ec in_seq=0 out_seq=0 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.843+0000 7f2494ff9640 10 client.16926 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.843+0000 7f2494ff9640 1 -- 172.21.15.103:0/419004051 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d69a4d860 con 0x563d69af4c50 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.843+0000 7f2494ff9640 10 client.16926 handle_command_reply: tid=0 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.843+0000 7f24b5647740 1 client.16926 shutdown 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ae5370 msgr2=0x563d69ae25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ae5370 0x563d69ae25d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f24a402ae00 tx=0x7f24a4031e40 comp rx=0 tx=0).stop 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 shutdown_connections 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69af4c50 0x563d69af7040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ae5370 0x563d69ae25d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69ae1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 --2- 172.21.15.103:0/419004051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ada5d0 0x563d69ae4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 >> 172.21.15.103:0/419004051 conn(0x563d69aa3310 msgr2=0x563d69adc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:25.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f2494ff9640 0 client.16926 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:45:25.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 shutdown_connections 2024-09-19T07:45:25.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 1 -- 172.21.15.103:0/419004051 wait complete. 2024-09-19T07:45:25.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.844+0000 7f24b5647740 20 client.16926 trim_cache size 0 max 16384 2024-09-19T07:45:25.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 Processor -- start 2024-09-19T07:45:25.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 -- start start 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ada5d0 0x563d69af1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69add120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69add660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1ab640 con 0x563d69ada5d0 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1ad180 con 0x563d69ae5370 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b5647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1abd10 con 0x563d69adbb40 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ada5d0 0x563d69af1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ada5d0 0x563d69af1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49954/0 (socket says 172.21.15.103:49954) 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 -- 172.21.15.103:0/3104549445 learned_addr learned my addr 172.21.15.103:0/3104549445 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b0809640 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69add120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 -- 172.21.15.103:0/3104549445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 msgr2=0x563d69add120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b180b640 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69add660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69add120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 -- 172.21.15.103:0/3104549445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 msgr2=0x563d69add660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69add660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 -- 172.21.15.103:0/3104549445 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d69af7660 con 0x563d69ada5d0 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b180b640 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69add660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.858+0000 7f24b100a640 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ada5d0 0x563d69af1310 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f24a809d4f0 tx=0x7f24a80d3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd4295e9c80f741 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f2496ffd640 1 -- 172.21.15.103:0/3104549445 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80a1ba0 con 0x563d69ada5d0 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f2496ffd640 1 -- 172.21.15.103:0/3104549445 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a80a6460 con 0x563d69ada5d0 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f2496ffd640 1 -- 172.21.15.103:0/3104549445 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8099e60 con 0x563d69ada5d0 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 -- 172.21.15.103:0/3104549445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ada5d0 msgr2=0x563d69af1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ada5d0 0x563d69af1310 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f24a809d4f0 tx=0x7f24a80d3920 comp rx=0 tx=0).stop 2024-09-19T07:45:25.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 -- 172.21.15.103:0/3104549445 shutdown_connections 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69add660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69adbb40 0x563d69add120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 --2- 172.21.15.103:0/3104549445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69ada5d0 0x563d69af1310 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 -- 172.21.15.103:0/3104549445 >> 172.21.15.103:0/3104549445 conn(0x563d69aa3310 msgr2=0x563d69ae10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 -- 172.21.15.103:0/3104549445 shutdown_connections 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.859+0000 7f24b5647740 1 -- 172.21.15.103:0/3104549445 wait complete. 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 -- start start 2024-09-19T07:45:25.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 Processor -- start 2024-09-19T07:45:25.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69aed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69aedc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ade950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1acab0 con 0x563d69adbb40 2024-09-19T07:45:25.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1ac3e0 con 0x563d69ae5370 2024-09-19T07:45:25.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1a9640 con 0x563d69ada5d0 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b5647740 10 client.0 _ll_register_callbacks cb 0x7f24a404b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b0809640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69aedc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b0809640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69aedc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49956/0 (socket says 172.21.15.103:49956) 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b0809640 1 -- 172.21.15.103:0/2343042628 learned_addr learned my addr 172.21.15.103:0/2343042628 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b100a640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69aed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.860+0000 7f24b180b640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ade950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b0809640 1 -- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 msgr2=0x563d69aed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b0809640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69aed710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b0809640 1 -- 172.21.15.103:0/2343042628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 msgr2=0x563d69ade950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b0809640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ade950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b0809640 1 -- 172.21.15.103:0/2343042628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d69adf180 con 0x563d69adbb40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b100a640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69aed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b0809640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69aedc50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f249c005fd0 tx=0x7f249c012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99495c555976d9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f2494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f2494ff9640 1 -- 172.21.15.103:0/2343042628 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c010bb0 con 0x563d69adbb40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f2494ff9640 1 -- 172.21.15.103:0/2343042628 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f249c010d50 con 0x563d69adbb40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a80a6460 con 0x563d69adbb40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f2494ff9640 1 -- 172.21.15.103:0/2343042628 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f249c004020 con 0x563d69adbb40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b5647740 10 client.16932 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563d69b5f350 con 0x563d69adbb40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.861+0000 7f2494ff9640 1 -- 172.21.15.103:0/2343042628 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a80a6460 con 0x563d69adbb40 2024-09-19T07:45:25.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f2494ff9640 1 -- 172.21.15.103:0/2343042628 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f249c004a00 con 0x563d69adbb40 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b5647740 10 client.16932 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b5647740 10 client.16932 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b5647740 20 client.16932 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b5647740 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69b5faf0 0x563d69b61f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b5647740 4 client.16932 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d69a47d70 con 0x563d69b5faf0 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b100a640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69b5faf0 0x563d69b61f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f24b100a640 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69b5faf0 0x563d69b61f10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a1153c786e1fc9e server_cookie=c4933fbf2c328456 in_seq=0 out_seq=0 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f2494ff9640 10 client.16932 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f2494ff9640 1 -- 172.21.15.103:0/2343042628 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d69a47d70 con 0x563d69b5faf0 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.862+0000 7f2494ff9640 10 client.16932 handle_command_reply: tid=0 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 client.16932 shutdown 2024-09-19T07:45:25.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 msgr2=0x563d69aedc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69aedc50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f249c005fd0 tx=0x7f249c012ed0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 shutdown_connections 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x563d69b5faf0 0x563d69b61f10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d69ae5370 0x563d69ade950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d69adbb40 0x563d69aedc50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f2494ff9640 0 client.16932 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 --2- 172.21.15.103:0/2343042628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69ada5d0 0x563d69aed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 >> 172.21.15.103:0/2343042628 conn(0x563d69aa3310 msgr2=0x563d69ae3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 shutdown_connections 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 1 -- 172.21.15.103:0/2343042628 wait complete. 2024-09-19T07:45:25.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.863+0000 7f24b5647740 20 client.16932 trim_cache size 0 max 16384 2024-09-19T07:45:25.878 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:45:25.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24980a33b0 msgr2=0x7f24980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24980a33b0 0x7f24980a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24a404a000 tx=0x7f24a4046000 comp rx=0 tx=0).stop 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac100a80 msgr2=0x7f24ac1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac100a80 0x7f24ac1972c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f24a800d990 tx=0x7f24a800de50 comp rx=0 tx=0).stop 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 shutdown_connections 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24980a33b0 0x7f24980a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24ac106a80 0x7f24ac19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24ac101420 0x7f24ac197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 --2- 172.21.15.103:0/3633769796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24ac100a80 0x7f24ac1972c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 >> 172.21.15.103:0/3633769796 conn(0x7f24ac0fc7a0 msgr2=0x7f24ac0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:25.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 shutdown_connections 2024-09-19T07:45:25.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:25.864+0000 7f24b326e640 1 -- 172.21.15.103:0/3633769796 wait complete. 2024-09-19T07:45:26.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:26 smithi134 ceph-mon[35414]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-09-19T07:45:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:26 smithi174 ceph-mon[35370]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-09-19T07:45:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:26 smithi103 ceph-mon[31201]: pgmap v228: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.37k op/s 2024-09-19T07:45:26.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:45:26.898 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:27.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.470+0000 7f60e4dac640 1 -- 172.21.15.103:0/997140966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01036e0 msgr2=0x7f60e0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.470+0000 7f60e4dac640 1 --2- 172.21.15.103:0/997140966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01036e0 0x7f60e0107b90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009930 tx=0x7f60d402a420 comp rx=0 tx=0).stop 2024-09-19T07:45:27.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 -- 172.21.15.103:0/997140966 shutdown_connections 2024-09-19T07:45:27.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 --2- 172.21.15.103:0/997140966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e01082a0 0x7f60e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 --2- 172.21.15.103:0/997140966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01036e0 0x7f60e0107b90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 --2- 172.21.15.103:0/997140966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102d30 0x7f60e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 -- 172.21.15.103:0/997140966 >> 172.21.15.103:0/997140966 conn(0x7f60e00fc7d0 msgr2=0x7f60e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:27.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 -- 172.21.15.103:0/997140966 shutdown_connections 2024-09-19T07:45:27.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 -- 172.21.15.103:0/997140966 wait complete. 2024-09-19T07:45:27.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 Processor -- start 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.471+0000 7f60e4dac640 1 -- start start 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102d30 0x7f60e0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01036e0 0x7f60e0198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01082a0 0x7f60e019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01120c0 con 0x7f60e01082a0 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0111f40 con 0x7f60e01036e0 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0112240 con 0x7f60e0102d30 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01082a0 0x7f60e019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01082a0 0x7f60e019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49964/0 (socket says 172.21.15.103:49964) 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 -- 172.21.15.103:0/761636730 learned_addr learned my addr 172.21.15.103:0/761636730 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:27.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01036e0 0x7f60e0198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60de59c640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102d30 0x7f60e0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 -- 172.21.15.103:0/761636730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102d30 msgr2=0x7f60e0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102d30 0x7f60e0198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 -- 172.21.15.103:0/761636730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01036e0 msgr2=0x7f60e0198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01036e0 0x7f60e0198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 -- 172.21.15.103:0/761636730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e019d5d0 con 0x7f60e01082a0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60ded9d640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01082a0 0x7f60e019cef0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b970 tx=0x7f60d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60cf7fe640 1 -- 172.21.15.103:0/761636730 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8019070 con 0x7f60e01082a0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e019d8c0 con 0x7f60e01082a0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60cf7fe640 1 -- 172.21.15.103:0/761636730 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d800cb40 con 0x7f60e01082a0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60cf7fe640 1 -- 172.21.15.103:0/761636730 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8007650 con 0x7f60e01082a0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.472+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01a51a0 con 0x7f60e01082a0 2024-09-19T07:45:27.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.474+0000 7f60cf7fe640 1 -- 172.21.15.103:0/761636730 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f60d801c070 con 0x7f60e01082a0 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.475+0000 7f60e7185740 1 Processor -- start 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.475+0000 7f60e7185740 1 -- start start 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.475+0000 7f60cf7fe640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a33b0 0x7f60c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.475+0000 7f60e7185740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.475+0000 7f60e7185740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.475+0000 7f60e7185740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c93760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60e7185740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c342bec2f0 con 0x55c342c865d0 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a33b0 0x7f60c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60e7185740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c342b93cf0 con 0x55c342c91370 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60e7185740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c342be82b0 con 0x55c342c87b40 2024-09-19T07:45:27.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60cf7fe640 1 -- 172.21.15.103:0/761636730 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f60d80cf290 con 0x7f60e01082a0 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ddd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60de59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ddd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c86a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49970/0 (socket says 172.21.15.103:49970) 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ddd9b640 1 -- 172.21.15.103:0/3222221706 learned_addr learned my addr 172.21.15.103:0/3222221706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ded9d640 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c93760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60de59c640 1 -- 172.21.15.103:0/3222221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 msgr2=0x55c342c93760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60de59c640 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c93760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60de59c640 1 -- 172.21.15.103:0/3222221706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 msgr2=0x55c342c86a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60de59c640 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c86a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60de59c640 1 -- 172.21.15.103:0/3222221706 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c342b783a0 con 0x55c342c87b40 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:27.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a33b0 0x7f60c80a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60d4031550 tx=0x7f60d4009680 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60de59c640 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c86090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009ab0 tx=0x7f60d0009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa930590aa5ca214 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.476+0000 7f60ccff9640 1 -- 172.21.15.103:0/3222221706 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0016070 con 0x55c342c87b40 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60ccff9640 1 -- 172.21.15.103:0/3222221706 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d0005500 con 0x55c342c87b40 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60ccff9640 1 -- 172.21.15.103:0/3222221706 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0011470 con 0x55c342c87b40 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 -- 172.21.15.103:0/3222221706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 msgr2=0x55c342c86090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c86090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009ab0 tx=0x7f60d0009f70 comp rx=0 tx=0).stop 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 -- 172.21.15.103:0/3222221706 shutdown_connections 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c93760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 --2- 172.21.15.103:0/3222221706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c86090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 -- 172.21.15.103:0/3222221706 >> 172.21.15.103:0/3222221706 conn(0x55c342c4f310 msgr2=0x55c342c82c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 -- 172.21.15.103:0/3222221706 shutdown_connections 2024-09-19T07:45:27.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.477+0000 7f60e7185740 1 -- 172.21.15.103:0/3222221706 wait complete. 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 -- start start 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 Processor -- start 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c8df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c8e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c342c98480 con 0x55c342c865d0 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c342ba7060 con 0x55c342c91370 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c342c98600 con 0x55c342c87b40 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60e7185740 10 client.0 _ll_register_callbacks cb 0x55c3428fa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c90c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49976/0 (socket says 172.21.15.103:49976) 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 -- 172.21.15.103:0/1223233448 learned_addr learned my addr 172.21.15.103:0/1223233448 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:27.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c8df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60ded9d640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c8e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 -- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 msgr2=0x55c342c8df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c8df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 -- 172.21.15.103:0/1223233448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 msgr2=0x55c342c8e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c8e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60de59c640 1 -- 172.21.15.103:0/1223233448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c34295d3c0 con 0x55c342c865d0 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.478+0000 7f60ded9d640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c8e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60de59c640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c90c60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006010 tx=0x7f60d0004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4031af53fb4835e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1223233448 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0028060 con 0x55c342c865d0 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1223233448 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f60d0005280 con 0x55c342c865d0 2024-09-19T07:45:27.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d0005500 con 0x55c342c865d0 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1223233448 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f60d0016040 con 0x55c342c865d0 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 10 client.16944 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c342ca3e70 con 0x55c342c865d0 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1223233448 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d0005500 con 0x55c342c865d0 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1223233448 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f60d000f040 con 0x55c342c865d0 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 10 client.16944 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 10 client.16944 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 20 client.16944 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342ca5270 0x55c342ca7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 4 client.16944 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.479+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c342bf9860 con 0x55c342ca5270 2024-09-19T07:45:27.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.480+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342ca5270 0x55c342ca7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.480+0000 7f60c5ffb640 10 client.16944 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.480+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342ca5270 0x55c342ca7660 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9beb419c74967837 server_cookie=5ee1ad1668cff3ed in_seq=0 out_seq=0 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1223233448 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c342bf9860 con 0x55c342ca5270 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60c5ffb640 10 client.16944 handle_command_reply: tid=0 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 client.16944 shutdown 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 msgr2=0x55c342c90c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c90c60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006010 tx=0x7f60d0004e50 comp rx=0 tx=0).stop 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 shutdown_connections 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342ca5270 0x55c342ca7660 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c8e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c8df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 --2- 172.21.15.103:0/1223233448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342c90c60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60c5ffb640 0 client.16944 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:45:27.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 >> 172.21.15.103:0/1223233448 conn(0x55c342c4f310 msgr2=0x55c342c88960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:27.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 shutdown_connections 2024-09-19T07:45:27.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 1 -- 172.21.15.103:0/1223233448 wait complete. 2024-09-19T07:45:27.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.481+0000 7f60e7185740 20 client.16944 trim_cache size 0 max 16384 2024-09-19T07:45:27.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 Processor -- start 2024-09-19T07:45:27.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 -- start start 2024-09-19T07:45:27.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342ca1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c89120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c89660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ab750 con 0x55c342c865d0 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ad290 con 0x55c342c91370 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60e7185740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01abe20 con 0x55c342c87b40 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342ca1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342ca1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49992/0 (socket says 172.21.15.103:49992) 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 -- 172.21.15.103:0/3807300037 learned_addr learned my addr 172.21.15.103:0/3807300037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60ded9d640 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c89660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 -- 172.21.15.103:0/3807300037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 msgr2=0x55c342c89120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c89120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c89120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 -- 172.21.15.103:0/3807300037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 msgr2=0x55c342c89660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c89660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60de59c640 1 -- 172.21.15.103:0/3807300037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c342c89ba0 con 0x55c342c865d0 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60ded9d640 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c89660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.494+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c89120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60de59c640 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342ca1720 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f60d0004bf0 tx=0x7f60d00092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ba61070f875f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60c7fff640 1 -- 172.21.15.103:0/3807300037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0028070 con 0x55c342c865d0 2024-09-19T07:45:27.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60c7fff640 1 -- 172.21.15.103:0/3807300037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d00096b0 con 0x55c342c865d0 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60c7fff640 1 -- 172.21.15.103:0/3807300037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0011900 con 0x55c342c865d0 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 -- 172.21.15.103:0/3807300037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 msgr2=0x55c342ca1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342ca1720 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f60d0004bf0 tx=0x7f60d00092a0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 -- 172.21.15.103:0/3807300037 shutdown_connections 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342c89660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c87b40 0x55c342c89120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 --2- 172.21.15.103:0/3807300037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c865d0 0x55c342ca1720 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 -- 172.21.15.103:0/3807300037 >> 172.21.15.103:0/3807300037 conn(0x55c342c4f310 msgr2=0x55c342c8d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.495+0000 7f60e7185740 1 -- 172.21.15.103:0/3807300037 shutdown_connections 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 -- 172.21.15.103:0/3807300037 wait complete. 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 -- start start 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 Processor -- start 2024-09-19T07:45:27.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c865d0 0x55c342c98a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c87b40 0x55c342c98f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342d0dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01acbc0 con 0x55c342c87b40 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ac4f0 con 0x55c342c91370 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01a9750 con 0x55c342c865d0 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60e7185740 10 client.0 _ll_register_callbacks cb 0x7f60d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60ddd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c87b40 0x55c342c98f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60ddd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c87b40 0x55c342c98f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50006/0 (socket says 172.21.15.103:50006) 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60ddd9b640 1 -- 172.21.15.103:0/3472298510 learned_addr learned my addr 172.21.15.103:0/3472298510 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60ded9d640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342d0dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.496+0000 7f60de59c640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c865d0 0x55c342c98a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60ddd9b640 1 -- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c865d0 msgr2=0x55c342c98a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c865d0 0x55c342c98a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60ddd9b640 1 -- 172.21.15.103:0/3472298510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 msgr2=0x55c342d0dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342d0dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60ddd9b640 1 -- 172.21.15.103:0/3472298510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c342d0e1c0 con 0x55c342c87b40 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60de59c640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c865d0 0x55c342c98a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60ddd9b640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c87b40 0x55c342c98f50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f60d4032e10 tx=0x7f60d4045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b2cbe30e067ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60c5ffb640 1 -- 172.21.15.103:0/3472298510 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4058060 con 0x55c342c87b40 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60c5ffb640 1 -- 172.21.15.103:0/3472298510 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f60d4005500 con 0x55c342c87b40 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d00096b0 con 0x55c342c87b40 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60c5ffb640 1 -- 172.21.15.103:0/3472298510 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f60d4035040 con 0x55c342c87b40 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60e7185740 10 client.16950 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c342d0f380 con 0x55c342c87b40 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60c5ffb640 1 -- 172.21.15.103:0/3472298510 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d00096b0 con 0x55c342c87b40 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.497+0000 7f60c5ffb640 1 -- 172.21.15.103:0/3472298510 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f60d403a040 con 0x55c342c87b40 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 10 client.16950 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 10 client.16950 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 20 client.16950 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342d0fb40 0x55c342d11f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 4 client.16950 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c342bf3d70 con 0x55c342d0fb40 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60de59c640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342d0fb40 0x55c342d11f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60de59c640 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342d0fb40 0x55c342d11f60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ef38cff338df22b server_cookie=b55ed676f52baaa1 in_seq=0 out_seq=0 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60c5ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:45:27.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60c5ffb640 1 -- 172.21.15.103:0/3472298510 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c342bf3d70 con 0x55c342d0fb40 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60c5ffb640 10 client.16950 handle_command_reply: tid=0 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 client.16950 shutdown 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c87b40 msgr2=0x55c342c98f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c87b40 0x55c342c98f50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f60d4032e10 tx=0x7f60d4045000 comp rx=0 tx=0).stop 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 shutdown_connections 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55c342d0fb40 0x55c342d11f60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c342c91370 0x55c342d0dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c342c87b40 0x55c342c98f50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60c5ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 --2- 172.21.15.103:0/3472298510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c342c865d0 0x55c342c98a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.498+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 >> 172.21.15.103:0/3472298510 conn(0x55c342c4f310 msgr2=0x55c342c814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:27.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 shutdown_connections 2024-09-19T07:45:27.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e7185740 1 -- 172.21.15.103:0/3472298510 wait complete. 2024-09-19T07:45:27.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e7185740 20 client.16950 trim_cache size 0 max 16384 2024-09-19T07:45:27.516 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:45:27.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a33b0 msgr2=0x7f60c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a33b0 0x7f60c80a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60d4031550 tx=0x7f60d4009680 comp rx=0 tx=0).stop 2024-09-19T07:45:27.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01082a0 msgr2=0x7f60e019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01082a0 0x7f60e019cef0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b970 tx=0x7f60d800be30 comp rx=0 tx=0).stop 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 shutdown_connections 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a33b0 0x7f60c80a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e01082a0 0x7f60e019cef0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e01036e0 0x7f60e0198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 --2- 172.21.15.103:0/761636730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102d30 0x7f60e0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 >> 172.21.15.103:0/761636730 conn(0x7f60e00fc7d0 msgr2=0x7f60e0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 shutdown_connections 2024-09-19T07:45:27.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:27.499+0000 7f60e4dac640 1 -- 172.21.15.103:0/761636730 wait complete. 2024-09-19T07:45:28.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:28 smithi103 ceph-mon[31201]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-09-19T07:45:28.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:45:28.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:28 smithi134 ceph-mon[35414]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-09-19T07:45:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:28 smithi174 ceph-mon[35370]: pgmap v229: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.54k op/s 2024-09-19T07:45:28.557 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:29.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 -- 172.21.15.103:0/477090090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0102d40 msgr2=0x7f9be0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 --2- 172.21.15.103:0/477090090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0102d40 0x7f9be0103120 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009900 tx=0x7f9bc802a480 comp rx=0 tx=0).stop 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 -- 172.21.15.103:0/477090090 shutdown_connections 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 --2- 172.21.15.103:0/477090090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9be010c170 0x7f9be010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 --2- 172.21.15.103:0/477090090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9be0103660 0x7f9be010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 --2- 172.21.15.103:0/477090090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0102d40 0x7f9be0103120 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 -- 172.21.15.103:0/477090090 >> 172.21.15.103:0/477090090 conn(0x7f9be00fc7a0 msgr2=0x7f9be00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 -- 172.21.15.103:0/477090090 shutdown_connections 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.550+0000 7f9be50b4640 1 -- 172.21.15.103:0/477090090 wait complete. 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 Processor -- start 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 -- start start 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9be0102d40 0x7f9be019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0103660 0x7f9be019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9be010c170 0x7f9be019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0111f70 con 0x7f9be0103660 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9be0111df0 con 0x7f9be0102d40 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.551+0000 7f9be50b4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9be01120f0 con 0x7f9be010c170 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0103660 0x7f9be019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0103660 0x7f9be019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57382/0 (socket says 172.21.15.103:57382) 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 -- 172.21.15.103:0/1271380962 learned_addr learned my addr 172.21.15.103:0/1271380962 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bded9d640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9be0102d40 0x7f9be019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9be010c170 0x7f9be019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 -- 172.21.15.103:0/1271380962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9be010c170 msgr2=0x7f9be019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9be010c170 0x7f9be019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 -- 172.21.15.103:0/1271380962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9be0102d40 msgr2=0x7f9be019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9be0102d40 0x7f9be019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 -- 172.21.15.103:0/1271380962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be019f620 con 0x7f9be0103660 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9be010c170 0x7f9be019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bde59c640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0103660 0x7f9be019abb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000d740 tx=0x7f9bd000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bd7fff640 1 -- 172.21.15.103:0/1271380962 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0017070 con 0x7f9be0103660 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be019f910 con 0x7f9be0103660 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bd7fff640 1 -- 172.21.15.103:0/1271380962 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd000de50 con 0x7f9be0103660 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9bd7fff640 1 -- 172.21.15.103:0/1271380962 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd000f620 con 0x7f9be0103660 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.552+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be01a71f0 con 0x7f9be0103660 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.554+0000 7f9bd7fff640 1 -- 172.21.15.103:0/1271380962 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9bd0009c30 con 0x7f9be0103660 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 Processor -- start 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 -- start start 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f1b40 0x55aca35f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca35f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aca35562f0 con 0x55aca35fb370 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aca34fdcf0 con 0x55aca35f05d0 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9be748d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aca35522b0 con 0x55aca35f1b40 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.555+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bdf59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f1b40 0x55aca35f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bd7fff640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9bcc0a33b0 0x7f9bcc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57390/0 (socket says 172.21.15.103:57390) 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bde59c640 1 -- 172.21.15.103:0/2449650410 learned_addr learned my addr 172.21.15.103:0/2449650410 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bded9d640 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca35f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9bcc0a33b0 0x7f9bcc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bd7fff640 1 -- 172.21.15.103:0/1271380962 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f9bd00cf990 con 0x7f9be0103660 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bded9d640 1 -- 172.21.15.103:0/2449650410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f1b40 msgr2=0x55aca35f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bded9d640 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f1b40 0x55aca35f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bded9d640 1 -- 172.21.15.103:0/2449650410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 msgr2=0x55aca35fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bded9d640 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35fd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bded9d640 1 -- 172.21.15.103:0/2449650410 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca34e23a0 con 0x55aca35f05d0 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bde59c640 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9bcc0a33b0 0x7f9bcc0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8005fd0 tx=0x7f9bd8004ab0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bded9d640 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca35f0a10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005940 tx=0x7f9bc8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a44cb07cbcd879 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.556+0000 7f9bd57fa640 1 -- 172.21.15.103:0/2449650410 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8040070 con 0x55aca35f05d0 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9bd57fa640 1 -- 172.21.15.103:0/2449650410 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc80319f0 con 0x55aca35f05d0 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9bd57fa640 1 -- 172.21.15.103:0/2449650410 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8035040 con 0x55aca35f05d0 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 -- 172.21.15.103:0/2449650410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 msgr2=0x55aca35f0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca35f0a10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005940 tx=0x7f9bc8005e80 comp rx=0 tx=0).stop 2024-09-19T07:45:29.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 -- 172.21.15.103:0/2449650410 shutdown_connections 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca35f0a10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 --2- 172.21.15.103:0/2449650410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f1b40 0x55aca35f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 -- 172.21.15.103:0/2449650410 >> 172.21.15.103:0/2449650410 conn(0x55aca35b9310 msgr2=0x55aca35ecc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 -- 172.21.15.103:0/2449650410 shutdown_connections 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.557+0000 7f9be748d740 1 -- 172.21.15.103:0/2449650410 wait complete. 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 -- start start 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 Processor -- start 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca35fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35fb370 0x55aca35f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aca35fe010 con 0x55aca35f1b40 2024-09-19T07:45:29.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aca3511060 con 0x55aca35fb370 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aca35fe190 con 0x55aca35f05d0 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9be748d740 10 client.0 _ll_register_callbacks cb 0x55aca32648c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57404/0 (socket says 172.21.15.103:57404) 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 -- 172.21.15.103:0/51556766 learned_addr learned my addr 172.21.15.103:0/51556766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35fb370 0x55aca35f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bded9d640 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca35fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 -- 172.21.15.103:0/51556766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 msgr2=0x55aca35fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca35fad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 -- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35fb370 msgr2=0x55aca35f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35fb370 0x55aca35f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.558+0000 7f9bde59c640 1 -- 172.21.15.103:0/51556766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca32c73c0 con 0x55aca35f1b40 2024-09-19T07:45:29.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9bde59c640 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35f7ff0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009610 tx=0x7f9bd0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ee89a6dd65ff76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9bc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9bc67fc640 1 -- 172.21.15.103:0/51556766 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0020040 con 0x55aca35f1b40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9bc67fc640 1 -- 172.21.15.103:0/51556766 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f9bd00860b0 con 0x55aca35f1b40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc80319f0 con 0x55aca35f1b40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9bc67fc640 1 -- 172.21.15.103:0/51556766 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9bd00cd4f0 con 0x55aca35f1b40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 10 client.16962 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55aca3609ad0 con 0x55aca35f1b40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9bc67fc640 1 -- 172.21.15.103:0/51556766 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd00b2070 con 0x55aca35f1b40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9bc67fc640 1 -- 172.21.15.103:0/51556766 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f9bd009aac0 con 0x55aca35f1b40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 10 client.16962 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 10 client.16962 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:45:29.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 20 client.16962 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca360aee0 0x55aca360d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 4 client.16962 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.559+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aca3563860 con 0x55aca360aee0 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.560+0000 7f9bded9d640 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca360aee0 0x55aca360d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.560+0000 7f9bc67fc640 10 client.16962 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.560+0000 7f9bded9d640 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca360aee0 0x55aca360d320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d97396100137002a server_cookie=e9472bdbda07c15e in_seq=0 out_seq=0 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9bc67fc640 1 -- 172.21.15.103:0/51556766 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aca3563860 con 0x55aca360aee0 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9bc67fc640 10 client.16962 handle_command_reply: tid=0 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 client.16962 shutdown 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 msgr2=0x55aca35f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35f7ff0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009610 tx=0x7f9bd0000bc0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 shutdown_connections 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca360aee0 0x55aca360d320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35fb370 0x55aca35f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35f7ff0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9bc67fc640 0 client.16962 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 --2- 172.21.15.103:0/51556766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca35fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 >> 172.21.15.103:0/51556766 conn(0x55aca35b9310 msgr2=0x55aca35f29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 shutdown_connections 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 1 -- 172.21.15.103:0/51556766 wait complete. 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.561+0000 7f9be748d740 20 client.16962 trim_cache size 0 max 16384 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 Processor -- start 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 -- start start 2024-09-19T07:45:29.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca36072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f1b40 0x55aca35f1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9be01afb40 con 0x55aca35fb370 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9be01b1680 con 0x55aca35f1b40 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9be748d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9be01b0210 con 0x55aca35f05d0 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35ec700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57416/0 (socket says 172.21.15.103:57416) 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 -- 172.21.15.103:0/62617656 learned_addr learned my addr 172.21.15.103:0/62617656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bde59c640 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f1b40 0x55aca35f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 -- 172.21.15.103:0/62617656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 msgr2=0x55aca36072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bded9d640 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca36072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca36072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 -- 172.21.15.103:0/62617656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f1b40 msgr2=0x55aca35f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f1b40 0x55aca35f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bdf59e640 1 -- 172.21.15.103:0/62617656 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca35f3bc0 con 0x55aca35fb370 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.574+0000 7f9bded9d640 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca36072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.575+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35ec700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800a7f0 tx=0x7f9bd80097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6b3d5acd67b845 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.575+0000 7f9bd4ff9640 1 -- 172.21.15.103:0/62617656 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8004d20 con 0x55aca35fb370 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.575+0000 7f9bd4ff9640 1 -- 172.21.15.103:0/62617656 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd800ae30 con 0x55aca35fb370 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.575+0000 7f9bd4ff9640 1 -- 172.21.15.103:0/62617656 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8005860 con 0x55aca35fb370 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.575+0000 7f9be748d740 1 -- 172.21.15.103:0/62617656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 msgr2=0x55aca35ec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.575+0000 7f9be748d740 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35ec700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800a7f0 tx=0x7f9bd80097a0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- 172.21.15.103:0/62617656 shutdown_connections 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35fb370 0x55aca35ec700 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f1b40 0x55aca35f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 --2- 172.21.15.103:0/62617656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35f05d0 0x55aca36072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- 172.21.15.103:0/62617656 >> 172.21.15.103:0/62617656 conn(0x55aca35b9310 msgr2=0x55aca35e7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- 172.21.15.103:0/62617656 shutdown_connections 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- 172.21.15.103:0/62617656 wait complete. 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- start start 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 Processor -- start 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca3603b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35fb370 0x55aca35ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9be01b0fb0 con 0x55aca35f1b40 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9be01b08e0 con 0x55aca35f05d0 2024-09-19T07:45:29.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9be01adb40 con 0x55aca35fb370 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9be748d740 10 client.0 _ll_register_callbacks cb 0x7f9bd00cd010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57422/0 (socket says 172.21.15.103:57422) 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9bde59c640 1 -- 172.21.15.103:0/674543025 learned_addr learned my addr 172.21.15.103:0/674543025 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.576+0000 7f9bded9d640 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca3603b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bdf59e640 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35fb370 0x55aca35ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bde59c640 1 -- 172.21.15.103:0/674543025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35fb370 msgr2=0x55aca35ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bde59c640 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35fb370 0x55aca35ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bde59c640 1 -- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 msgr2=0x55aca3603b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bde59c640 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca3603b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bde59c640 1 -- 172.21.15.103:0/674543025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca35ffc50 con 0x55aca35f1b40 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bde59c640 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35feec0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9bd009afe0 tx=0x7f9bd009b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120712605473058a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:29.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bc67fc640 1 -- 172.21.15.103:0/674543025 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0014070 con 0x55aca35f1b40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bc67fc640 1 -- 172.21.15.103:0/674543025 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f9bd00a12b0 con 0x55aca35f1b40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd800ae30 con 0x55aca35f1b40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bc67fc640 1 -- 172.21.15.103:0/674543025 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9bd009acd0 con 0x55aca35f1b40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9be748d740 10 client.16968 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55aca3675b60 con 0x55aca35f1b40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.577+0000 7f9bc67fc640 1 -- 172.21.15.103:0/674543025 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd800ae30 con 0x55aca35f1b40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9bc67fc640 1 -- 172.21.15.103:0/674543025 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f9bd00860b0 con 0x55aca35f1b40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9be748d740 10 client.16968 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9be748d740 10 client.16968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9be748d740 20 client.16968 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9be748d740 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca3676300 0x55aca3678720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:29.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9be748d740 4 client.16968 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aca355dd70 con 0x55aca3676300 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9bded9d640 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca3676300 0x55aca3678720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9bc67fc640 10 client.16968 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9bded9d640 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca3676300 0x55aca3678720 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50361051d9fc0694 server_cookie=7cea7d6319f3f1a4 in_seq=0 out_seq=0 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9bc67fc640 1 -- 172.21.15.103:0/674543025 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aca355dd70 con 0x55aca3676300 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9bc67fc640 10 client.16968 handle_command_reply: tid=0 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.578+0000 7f9be748d740 1 client.16968 shutdown 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 msgr2=0x55aca35feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35feec0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9bd009afe0 tx=0x7f9bd009b360 comp rx=0 tx=0).stop 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 shutdown_connections 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55aca3676300 0x55aca3678720 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aca35fb370 0x55aca35ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9bc67fc640 0 client.16968 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:45:29.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aca35f1b40 0x55aca35feec0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 --2- 172.21.15.103:0/674543025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aca35f05d0 0x55aca3603b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 >> 172.21.15.103:0/674543025 conn(0x55aca35b9310 msgr2=0x55aca35ec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:29.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 shutdown_connections 2024-09-19T07:45:29.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 1 -- 172.21.15.103:0/674543025 wait complete. 2024-09-19T07:45:29.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.579+0000 7f9be748d740 20 client.16968 trim_cache size 0 max 16384 2024-09-19T07:45:29.597 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:45:29.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9bcc0a33b0 msgr2=0x7f9bcc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9bcc0a33b0 0x7f9bcc0a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8005fd0 tx=0x7f9bd8004ab0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0103660 msgr2=0x7f9be019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0103660 0x7f9be019abb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000d740 tx=0x7f9bd000dc00 comp rx=0 tx=0).stop 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 shutdown_connections 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9bcc0a33b0 0x7f9bcc0a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9be010c170 0x7f9be019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9be0103660 0x7f9be019abb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 --2- 172.21.15.103:0/1271380962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9be0102d40 0x7f9be019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 >> 172.21.15.103:0/1271380962 conn(0x7f9be00fc7a0 msgr2=0x7f9be00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 shutdown_connections 2024-09-19T07:45:29.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:29.580+0000 7f9be50b4640 1 -- 172.21.15.103:0/1271380962 wait complete. 2024-09-19T07:45:30.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:30 smithi103 ceph-mon[31201]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-09-19T07:45:30.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:30 smithi134 ceph-mon[35414]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-09-19T07:45:30.360 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:45:30.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:30 smithi174 ceph-mon[35370]: pgmap v230: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 1.35k op/s 2024-09-19T07:45:30.472 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 415.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:45:30.606 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:31.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 -- 172.21.15.103:0/1591595275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 msgr2=0x7fb9841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 --2- 172.21.15.103:0/1591595275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841029b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009ca0 tx=0x7fb96c02a950 comp rx=0 tx=0).stop 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 -- 172.21.15.103:0/1591595275 shutdown_connections 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 --2- 172.21.15.103:0/1591595275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb984102ef0 0x7fb98410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 --2- 172.21.15.103:0/1591595275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841029b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 --2- 172.21.15.103:0/1591595275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb984100e20 0x7fb984101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 -- 172.21.15.103:0/1591595275 >> 172.21.15.103:0/1591595275 conn(0x7fb9840fc7d0 msgr2=0x7fb9840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 -- 172.21.15.103:0/1591595275 shutdown_connections 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 -- 172.21.15.103:0/1591595275 wait complete. 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.117+0000 7fb989ec2640 1 Processor -- start 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb989ec2640 1 -- start start 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb989ec2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb984100e20 0x7fb984199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb989ec2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb989ec2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb984102ef0 0x7fb9841a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb989ec2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb984110f80 con 0x7fb984102570 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb989ec2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb984110e00 con 0x7fb984100e20 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb989ec2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb984111100 con 0x7fb984102ef0 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57428/0 (socket says 172.21.15.103:57428) 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 -- 172.21.15.103:0/2178438107 learned_addr learned my addr 172.21.15.103:0/2178438107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb983fff640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb984102ef0 0x7fb9841a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb9837fe640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb984100e20 0x7fb984199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 -- 172.21.15.103:0/2178438107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb984102ef0 msgr2=0x7fb9841a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb984102ef0 0x7fb9841a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 -- 172.21.15.103:0/2178438107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb984100e20 msgr2=0x7fb984199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb984100e20 0x7fb984199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb982ffd640 1 -- 172.21.15.103:0/2178438107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9841a4850 con 0x7fb984102570 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.118+0000 7fb9837fe640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb984100e20 0x7fb984199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.119+0000 7fb982ffd640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841997d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005cc0 tx=0x7fb96c02f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.119+0000 7fb980ff9640 1 -- 172.21.15.103:0/2178438107 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c034040 con 0x7fb984102570 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.119+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9841a4a20 con 0x7fb984102570 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.119+0000 7fb980ff9640 1 -- 172.21.15.103:0/2178438107 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c030920 con 0x7fb984102570 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.119+0000 7fb980ff9640 1 -- 172.21.15.103:0/2178438107 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c004830 con 0x7fb984102570 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.119+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9841a4ed0 con 0x7fb984102570 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb980ff9640 1 -- 172.21.15.103:0/2178438107 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb96c037020 con 0x7fb984102570 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 Processor -- start 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 -- start start 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4aceca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bd4ac522f0 con 0x55bd4acf7370 2024-09-19T07:45:31.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bd4abf9cf0 con 0x55bd4acedb40 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb98c29b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bd4ac4e2b0 con 0x55bd4acec5d0 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb982ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb982ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57432/0 (socket says 172.21.15.103:57432) 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb982ffd640 1 -- 172.21.15.103:0/2582009143 learned_addr learned my addr 172.21.15.103:0/2582009143 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb9837fe640 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4aceca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb983fff640 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.122+0000 7fb980ff9640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb9700a33b0 0x7fb9700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb982ffd640 1 -- 172.21.15.103:0/2582009143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 msgr2=0x55bd4aceca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb982ffd640 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4aceca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb982ffd640 1 -- 172.21.15.103:0/2582009143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 msgr2=0x55bd4acec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb982ffd640 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb982ffd640 1 -- 172.21.15.103:0/2582009143 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4abde3a0 con 0x55bd4acf7370 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb9837fe640 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4aceca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb983fff640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb9700a33b0 0x7fb9700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb980ff9640 1 -- 172.21.15.103:0/2178438107 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fb96c0f0c20 con 0x7fb984102570 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb982ffd640 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf9760 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0ee3e0 tx=0x7fb96c087940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825d89600c582704 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb979ffb640 1 -- 172.21.15.103:0/2582009143 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0bc960 con 0x55bd4acf7370 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb979ffb640 1 -- 172.21.15.103:0/2582009143 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c0dd0d0 con 0x55bd4acf7370 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.123+0000 7fb979ffb640 1 -- 172.21.15.103:0/2582009143 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0dd3c0 con 0x55bd4acf7370 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 -- 172.21.15.103:0/2582009143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 msgr2=0x55bd4acf9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb983fff640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb9700a33b0 0x7fb9700a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0045a0 tx=0x7fb97c0046c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf9760 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0ee3e0 tx=0x7fb96c087940 comp rx=0 tx=0).stop 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 -- 172.21.15.103:0/2582009143 shutdown_connections 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf9760 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4aceca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 --2- 172.21.15.103:0/2582009143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 -- 172.21.15.103:0/2582009143 >> 172.21.15.103:0/2582009143 conn(0x55bd4acb5310 msgr2=0x55bd4ace8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 -- 172.21.15.103:0/2582009143 shutdown_connections 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 -- 172.21.15.103:0/2582009143 wait complete. 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 -- start start 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.124+0000 7fb98c29b740 1 Processor -- start 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4acf6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acf3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb98c29b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bd4ac522f0 con 0x55bd4acf7370 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb98c29b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bd4ac0d060 con 0x55bd4acedb40 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb98c29b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bd4acf9e30 con 0x55bd4acec5d0 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb98c29b740 10 client.0 _ll_register_callbacks cb 0x55bd4a9608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57448/0 (socket says 172.21.15.103:57448) 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 -- 172.21.15.103:0/3974956585 learned_addr learned my addr 172.21.15.103:0/3974956585 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb9837fe640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4acf6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb982ffd640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acf3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 -- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 msgr2=0x55bd4acf6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4acf6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 -- 172.21.15.103:0/3974956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 msgr2=0x55bd4acf3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acf3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 -- 172.21.15.103:0/3974956585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4a9c33c0 con 0x55bd4acf7370 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb982ffd640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acf3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb983fff640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf4570 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0043d0 tx=0x7fb97c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8521c13a41a4dcde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.125+0000 7fb962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb962ffd640 1 -- 172.21.15.103:0/3974956585 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00f660 con 0x55bd4acf7370 2024-09-19T07:45:31.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb962ffd640 1 -- 172.21.15.103:0/3974956585 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fb97c00f800 con 0x55bd4acf7370 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96c0dd0d0 con 0x55bd4acf7370 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb962ffd640 1 -- 172.21.15.103:0/3974956585 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb97c024020 con 0x55bd4acf7370 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 10 client.16980 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bd4ad05790 con 0x55bd4acf7370 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb962ffd640 1 -- 172.21.15.103:0/3974956585 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96c0dd0d0 con 0x55bd4acf7370 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb962ffd640 1 -- 172.21.15.103:0/3974956585 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fb97c010c80 con 0x55bd4acf7370 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 10 client.16980 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:31.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 10 client.16980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 20 client.16980 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad06cf0 0x55bd4ad09140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 4 client.16980 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd4ac5f860 con 0x55bd4ad06cf0 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.126+0000 7fb9837fe640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad06cf0 0x55bd4ad09140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.127+0000 7fb9837fe640 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad06cf0 0x55bd4ad09140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34b10d85d51ab77 server_cookie=9fd8c79222cc175b in_seq=0 out_seq=0 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.127+0000 7fb962ffd640 10 client.16980 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.127+0000 7fb962ffd640 1 -- 172.21.15.103:0/3974956585 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd4ac5f860 con 0x55bd4ad06cf0 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.127+0000 7fb962ffd640 10 client.16980 handle_command_reply: tid=0 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.127+0000 7fb98c29b740 1 client.16980 shutdown 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 msgr2=0x55bd4acf4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf4570 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0043d0 tx=0x7fb97c004730 comp rx=0 tx=0).stop 2024-09-19T07:45:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 shutdown_connections 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad06cf0 0x55bd4ad09140 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acf7370 0x55bd4acf4570 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acf3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 --2- 172.21.15.103:0/3974956585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acec5d0 0x55bd4acf6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb962ffd640 0 client.16980 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 >> 172.21.15.103:0/3974956585 conn(0x55bd4acb5310 msgr2=0x55bd4acee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 shutdown_connections 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 1 -- 172.21.15.103:0/3974956585 wait complete. 2024-09-19T07:45:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.128+0000 7fb98c29b740 20 client.16980 trim_cache size 0 max 16384 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 Processor -- start 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 -- start start 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acf5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acf7370 0x55bd4acef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841ad8e0 con 0x55bd4acec5d0 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841af420 con 0x55bd4acedb40 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb98c29b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841adfb0 con 0x55bd4acf7370 2024-09-19T07:45:31.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acf5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acf5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57464/0 (socket says 172.21.15.103:57464) 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 -- 172.21.15.103:0/1832615341 learned_addr learned my addr 172.21.15.103:0/1832615341 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb982ffd640 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 -- 172.21.15.103:0/1832615341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acf7370 msgr2=0x55bd4acef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acf7370 0x55bd4acef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 -- 172.21.15.103:0/1832615341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 msgr2=0x55bd4acef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 -- 172.21.15.103:0/1832615341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd4acefba0 con 0x55bd4acec5d0 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.141+0000 7fb9837fe640 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acf5f40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55bd4ad099b0 tx=0x7fb9740039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f9272ae31215b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb9797fa640 1 -- 172.21.15.103:0/1832615341 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974015070 con 0x55bd4acec5d0 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb9797fa640 1 -- 172.21.15.103:0/1832615341 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb974005500 con 0x55bd4acec5d0 2024-09-19T07:45:31.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb9797fa640 1 -- 172.21.15.103:0/1832615341 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97401d610 con 0x55bd4acec5d0 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 -- 172.21.15.103:0/1832615341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 msgr2=0x55bd4acf5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acf5f40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55bd4ad099b0 tx=0x7fb9740039c0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 -- 172.21.15.103:0/1832615341 shutdown_connections 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acf7370 0x55bd4acef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acedb40 0x55bd4acef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1832615341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acf5f40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 -- 172.21.15.103:0/1832615341 >> 172.21.15.103:0/1832615341 conn(0x55bd4acb5310 msgr2=0x55bd4acf5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 -- 172.21.15.103:0/1832615341 shutdown_connections 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.142+0000 7fb98c29b740 1 -- 172.21.15.103:0/1832615341 wait complete. 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 -- start start 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 Processor -- start 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acfb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acedb40 0x55bd4acfb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acf7370 0x55bd4ad709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841aed50 con 0x55bd4acec5d0 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841ae680 con 0x55bd4acf7370 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841ab8e0 con 0x55bd4acedb40 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb98c29b740 10 client.0 _ll_register_callbacks cb 0x7fb97c005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb9837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acfb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb9837fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acfb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57466/0 (socket says 172.21.15.103:57466) 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb9837fe640 1 -- 172.21.15.103:0/1898820392 learned_addr learned my addr 172.21.15.103:0/1898820392 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb982ffd640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acedb40 0x55bd4acfb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.143+0000 7fb983fff640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acf7370 0x55bd4ad709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb9837fe640 1 -- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acedb40 msgr2=0x55bd4acfb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb9837fe640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acedb40 0x55bd4acfb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb9837fe640 1 -- 172.21.15.103:0/1898820392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acf7370 msgr2=0x55bd4ad709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb9837fe640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acf7370 0x55bd4ad709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb9837fe640 1 -- 172.21.15.103:0/1898820392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4ad70f20 con 0x55bd4acec5d0 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb982ffd640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acedb40 0x55bd4acfb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb9837fe640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acfb200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb974004450 tx=0x7fb97400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d213a216c5c5ee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb962ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb962ffd640 1 -- 172.21.15.103:0/1898820392 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97402d020 con 0x55bd4acec5d0 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb962ffd640 1 -- 172.21.15.103:0/1898820392 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fb974005200 con 0x55bd4acec5d0 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb974005500 con 0x55bd4acec5d0 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb962ffd640 1 -- 172.21.15.103:0/1898820392 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb974038050 con 0x55bd4acec5d0 2024-09-19T07:45:31.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb98c29b740 10 client.16986 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bd4ad72070 con 0x55bd4acec5d0 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.144+0000 7fb962ffd640 1 -- 172.21.15.103:0/1898820392 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb974005500 con 0x55bd4acec5d0 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb962ffd640 1 -- 172.21.15.103:0/1898820392 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fb974011440 con 0x55bd4acec5d0 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb98c29b740 10 client.16986 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb98c29b740 10 client.16986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb98c29b740 20 client.16986 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad72720 0x55bd4ad74b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb98c29b740 4 client.16986 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd4ac59d70 con 0x55bd4ad72720 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb982ffd640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad72720 0x55bd4ad74b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb962ffd640 10 client.16986 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:45:31.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb982ffd640 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad72720 0x55bd4ad74b40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76f281fbfa9f4919 server_cookie=89dbeb9a6324831d in_seq=0 out_seq=0 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb962ffd640 1 -- 172.21.15.103:0/1898820392 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd4ac59d70 con 0x55bd4ad72720 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.145+0000 7fb962ffd640 10 client.16986 handle_command_reply: tid=0 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 client.16986 shutdown 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 msgr2=0x55bd4acfb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acfb200 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb974004450 tx=0x7fb97400b900 comp rx=0 tx=0).stop 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 shutdown_connections 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55bd4ad72720 0x55bd4ad74b40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd4acf7370 0x55bd4ad709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd4acedb40 0x55bd4acfb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb962ffd640 0 client.16986 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 --2- 172.21.15.103:0/1898820392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bd4acec5d0 0x55bd4acfb200 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 >> 172.21.15.103:0/1898820392 conn(0x55bd4acb5310 msgr2=0x55bd4acf6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 shutdown_connections 2024-09-19T07:45:31.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 1 -- 172.21.15.103:0/1898820392 wait complete. 2024-09-19T07:45:31.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb98c29b740 20 client.16986 trim_cache size 0 max 16384 2024-09-19T07:45:31.159 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb9700a33b0 msgr2=0x7fb9700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb989ec2640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb9700a33b0 0x7fb9700a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0045a0 tx=0x7fb97c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 msgr2=0x7fb9841997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb989ec2640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841997d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005cc0 tx=0x7fb96c02f650 comp rx=0 tx=0).stop 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 shutdown_connections 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb989ec2640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb9700a33b0 0x7fb9700a5870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.146+0000 7fb989ec2640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb984102ef0 0x7fb9841a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.147+0000 7fb989ec2640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb984102570 0x7fb9841997d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.147+0000 7fb989ec2640 1 --2- 172.21.15.103:0/2178438107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb984100e20 0x7fb984199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.147+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 >> 172.21.15.103:0/2178438107 conn(0x7fb9840fc7d0 msgr2=0x7fb9840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.147+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 shutdown_connections 2024-09-19T07:45:31.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:31.147+0000 7fb989ec2640 1 -- 172.21.15.103:0/2178438107 wait complete. 2024-09-19T07:45:32.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:32 smithi103 ceph-mon[31201]: pgmap v231: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-09-19T07:45:32.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:32 smithi134 ceph-mon[35414]: pgmap v231: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-09-19T07:45:32.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:45:32.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:32 smithi174 ceph-mon[35370]: pgmap v231: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 15 MiB/s wr, 1.36k op/s 2024-09-19T07:45:32.633 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 -- 172.21.15.103:0/2344947965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8100e20 msgr2=0x7f58d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2344947965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8100e20 0x7f58d8101200 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009900 tx=0x7f58c802a480 comp rx=0 tx=0).stop 2024-09-19T07:45:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 -- 172.21.15.103:0/2344947965 shutdown_connections 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2344947965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d8102ef0 0x7f58d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2344947965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8102570 0x7f58d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2344947965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8100e20 0x7f58d8101200 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 -- 172.21.15.103:0/2344947965 >> 172.21.15.103:0/2344947965 conn(0x7f58d80fc7d0 msgr2=0x7f58d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 -- 172.21.15.103:0/2344947965 shutdown_connections 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 -- 172.21.15.103:0/2344947965 wait complete. 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 Processor -- start 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.098+0000 7f58e07e8640 1 -- start start 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58e07e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d8100e20 0x7f58d8197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58e07e8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8102570 0x7f58d8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58e07e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8102ef0 0x7f58d819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58e07e8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8110f80 con 0x7f58d8102ef0 2024-09-19T07:45:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58e07e8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8110e00 con 0x7f58d8100e20 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58e07e8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8111100 con 0x7f58d8102570 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8102ef0 0x7f58d819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8102ef0 0x7f58d819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57492/0 (socket says 172.21.15.103:57492) 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 -- 172.21.15.103:0/2915416368 learned_addr learned my addr 172.21.15.103:0/2915416368 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58de584640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d8100e20 0x7f58d8197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ddd83640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8102570 0x7f58d8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 -- 172.21.15.103:0/2915416368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8102570 msgr2=0x7f58d8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8102570 0x7f58d8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 -- 172.21.15.103:0/2915416368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d8100e20 msgr2=0x7f58d8197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d8100e20 0x7f58d8197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 -- 172.21.15.103:0/2915416368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d819c3c0 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58de584640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d8100e20 0x7f58d8197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ddd83640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8102570 0x7f58d8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58ded85640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8102ef0 0x7f58d819bce0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f58d000eaa0 tx=0x7f58d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58cf7fe640 1 -- 172.21.15.103:0/2915416368 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d000cdb0 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58cf7fe640 1 -- 172.21.15.103:0/2915416368 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d0005020 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d819c6b0 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.099+0000 7f58cf7fe640 1 -- 172.21.15.103:0/2915416368 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00116b0 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.100+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d81a3f90 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.102+0000 7f58cf7fe640 1 -- 172.21.15.103:0/2915416368 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f58d0005560 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.102+0000 7f58cf7fe640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58c40a3430 0x7f58c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.102+0000 7f58e2bc1740 1 Processor -- start 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.102+0000 7f58de584640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58c40a3430 0x7f58c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.102+0000 7f58cf7fe640 1 -- 172.21.15.103:0/2915416368 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f58d00cffd0 con 0x7f58d8102ef0 2024-09-19T07:45:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58e2bc1740 1 -- start start 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eaba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0b0e112f0 con 0x55d0b0eacb40 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0b0db8cf0 con 0x55d0b0eb6370 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0b0e0d2b0 con 0x55d0b0eab5d0 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57506/0 (socket says 172.21.15.103:57506) 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58ddd83640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eaba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 -- 172.21.15.103:0/3491482151 learned_addr learned my addr 172.21.15.103:0/3491482151 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58ded85640 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58c40a3430 0x7f58c40a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58c8005ab0 tx=0x7f58c8004160 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 -- 172.21.15.103:0/3491482151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eab5d0 msgr2=0x55d0b0eaba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eaba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 -- 172.21.15.103:0/3491482151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 msgr2=0x55d0b0eb8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 -- 172.21.15.103:0/3491482151 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b0d9d3a0 con 0x55d0b0eacb40 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58ded85640 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.103+0000 7f58de584640 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eab090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009900 tx=0x7f58c8044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52935a8945f0327 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58ccff9640 1 -- 172.21.15.103:0/3491482151 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8035050 con 0x55d0b0eacb40 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58ccff9640 1 -- 172.21.15.103:0/3491482151 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8004da0 con 0x55d0b0eacb40 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58ccff9640 1 -- 172.21.15.103:0/3491482151 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8003f00 con 0x55d0b0eacb40 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58e2bc1740 1 -- 172.21.15.103:0/3491482151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 msgr2=0x55d0b0eab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eab090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009900 tx=0x7f58c8044000 comp rx=0 tx=0).stop 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58e2bc1740 1 -- 172.21.15.103:0/3491482151 shutdown_connections 2024-09-19T07:45:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eaba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/3491482151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eab090 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.104+0000 7f58e2bc1740 1 -- 172.21.15.103:0/3491482151 >> 172.21.15.103:0/3491482151 conn(0x55d0b0e74310 msgr2=0x55d0b0ea7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 -- 172.21.15.103:0/3491482151 shutdown_connections 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 -- 172.21.15.103:0/3491482151 wait complete. 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 -- start start 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 Processor -- start 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eb2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0b0ebb220 con 0x55d0b0eacb40 2024-09-19T07:45:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0b0dcc060 con 0x55d0b0eab5d0 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0b0ebb3a0 con 0x55d0b0eb6370 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58e2bc1740 10 client.0 _ll_register_callbacks cb 0x55d0b0b1f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58ddd83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eb2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58ddd83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eb2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57512/0 (socket says 172.21.15.103:57512) 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58ddd83640 1 -- 172.21.15.103:0/286371261 learned_addr learned my addr 172.21.15.103:0/286371261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58de584640 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.105+0000 7f58ded85640 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58ddd83640 1 -- 172.21.15.103:0/286371261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eb6370 msgr2=0x55d0b0eb34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58ddd83640 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58ddd83640 1 -- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 msgr2=0x55d0b0eb5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58ddd83640 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58ddd83640 1 -- 172.21.15.103:0/286371261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b0b823c0 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58ddd83640 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eb2f80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009fa0 tx=0x7f58d4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e300c85f04c64372 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58c1ffb640 1 -- 172.21.15.103:0/286371261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4012920 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58c1ffb640 1 -- 172.21.15.103:0/286371261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f58d4012ac0 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c8004da0 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58c1ffb640 1 -- 172.21.15.103:0/286371261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f58d4012cc0 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58e2bc1740 10 client.16998 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d0b0ec6c90 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58c1ffb640 1 -- 172.21.15.103:0/286371261 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8004da0 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58c1ffb640 1 -- 172.21.15.103:0/286371261 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f58d400f040 con 0x55d0b0eacb40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58e2bc1740 10 client.16998 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.106+0000 7f58e2bc1740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:33.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.107+0000 7f58e2bc1740 20 client.16998 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.107+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0ec81a0 0x55d0b0eca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.107+0000 7f58e2bc1740 4 client.16998 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.107+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b0e1e860 con 0x55d0b0ec81a0 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.107+0000 7f58de584640 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0ec81a0 0x55d0b0eca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.107+0000 7f58c1ffb640 10 client.16998 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.107+0000 7f58de584640 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0ec81a0 0x55d0b0eca5c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1efb2084f63bd6e0 server_cookie=59dbedd9044a2e7f in_seq=0 out_seq=0 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58c1ffb640 1 -- 172.21.15.103:0/286371261 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0b0e1e860 con 0x55d0b0ec81a0 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58c1ffb640 10 client.16998 handle_command_reply: tid=0 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 client.16998 shutdown 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 msgr2=0x55d0b0eb2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eb2f80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009fa0 tx=0x7f58d4011870 comp rx=0 tx=0).stop 2024-09-19T07:45:33.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 shutdown_connections 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0ec81a0 0x55d0b0eca5c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eb6370 0x55d0b0eb34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eacb40 0x55d0b0eb2f80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58c1ffb640 0 client.16998 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/286371261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 >> 172.21.15.103:0/286371261 conn(0x55d0b0e74310 msgr2=0x55d0b0ead960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 shutdown_connections 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 1 -- 172.21.15.103:0/286371261 wait complete. 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.108+0000 7f58e2bc1740 20 client.16998 trim_cache size 0 max 16384 2024-09-19T07:45:33.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 Processor -- start 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 -- start start 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 0x55d0b0eae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81ab6f0 con 0x55d0b0eab5d0 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81ad230 con 0x55d0b0eb6370 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81abdc0 con 0x55d0b0eacb40 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58de584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58de584640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57516/0 (socket says 172.21.15.103:57516) 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58de584640 1 -- 172.21.15.103:0/2902472057 learned_addr learned my addr 172.21.15.103:0/2902472057 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.121+0000 7f58ded85640 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58de584640 1 -- 172.21.15.103:0/2902472057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 msgr2=0x55d0b0eae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58de584640 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 0x55d0b0eae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58de584640 1 -- 172.21.15.103:0/2902472057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 msgr2=0x55d0b0eae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58de584640 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58de584640 1 -- 172.21.15.103:0/2902472057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b0ea74f0 con 0x55d0b0eab5d0 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58de584640 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb4ef0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f58c8003b10 tx=0x7f58c8003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f06092e4e7236e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58c3fff640 1 -- 172.21.15.103:0/2902472057 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80035f0 con 0x55d0b0eab5d0 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58c3fff640 1 -- 172.21.15.103:0/2902472057 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8003790 con 0x55d0b0eab5d0 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.122+0000 7f58c3fff640 1 -- 172.21.15.103:0/2902472057 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80329a0 con 0x55d0b0eab5d0 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- 172.21.15.103:0/2902472057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eab5d0 msgr2=0x55d0b0eb4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb4ef0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f58c8003b10 tx=0x7f58c8003b40 comp rx=0 tx=0).stop 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- 172.21.15.103:0/2902472057 shutdown_connections 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eb6370 0x55d0b0eae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 0x55d0b0eae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/2902472057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eab5d0 0x55d0b0eb4ef0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- 172.21.15.103:0/2902472057 >> 172.21.15.103:0/2902472057 conn(0x55d0b0e74310 msgr2=0x55d0b0eb47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- 172.21.15.103:0/2902472057 shutdown_connections 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- 172.21.15.103:0/2902472057 wait complete. 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- start start 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 Processor -- start 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0ebbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 0x55d0b0ebc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eb6370 0x55d0b0ec0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81acb60 con 0x55d0b0eb6370 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81ac490 con 0x55d0b0eab5d0 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81a96f0 con 0x55d0b0eacb40 2024-09-19T07:45:33.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58e2bc1740 10 client.0 _ll_register_callbacks cb 0x7f58d40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58ded85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eb6370 0x55d0b0ec0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.123+0000 7f58ded85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eb6370 0x55d0b0ec0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57520/0 (socket says 172.21.15.103:57520) 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ded85640 1 -- 172.21.15.103:0/553908861 learned_addr learned my addr 172.21.15.103:0/553908861 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58de584640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0ebbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ddd83640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 0x55d0b0ebc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ded85640 1 -- 172.21.15.103:0/553908861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 msgr2=0x55d0b0ebc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ded85640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 0x55d0b0ebc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ded85640 1 -- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 msgr2=0x55d0b0ebbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ded85640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0ebbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ded85640 1 -- 172.21.15.103:0/553908861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b0ec0b00 con 0x55d0b0eb6370 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58de584640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0ebbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58ded85640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eb6370 0x55d0b0ec0460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f58d00cd770 tx=0x7f58d009a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4e25e1dffd5fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:33.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58c1ffb640 1 -- 172.21.15.103:0/553908861 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00b4020 con 0x55d0b0eb6370 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58c1ffb640 1 -- 172.21.15.103:0/553908861 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f58d0098910 con 0x55d0b0eb6370 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c8003790 con 0x55d0b0eb6370 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58c1ffb640 1 -- 172.21.15.103:0/553908861 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f58d001f040 con 0x55d0b0eb6370 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58e2bc1740 10 client.17004 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.124+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d0b0f329e0 con 0x55d0b0eb6370 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58c1ffb640 1 -- 172.21.15.103:0/553908861 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8003790 con 0x55d0b0eb6370 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58c1ffb640 1 -- 172.21.15.103:0/553908861 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f58d000f040 con 0x55d0b0eb6370 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58e2bc1740 10 client.17004 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58e2bc1740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58e2bc1740 20 client.17004 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0f33100 0x55d0b0f35520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:33.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58e2bc1740 4 client.17004 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0b0e18d70 con 0x55d0b0f33100 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58de584640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0f33100 0x55d0b0f35520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58c1ffb640 10 client.17004 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.125+0000 7f58de584640 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0f33100 0x55d0b0f35520 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5773521cca5bf3af server_cookie=3b811395c0562bb2 in_seq=0 out_seq=0 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58c1ffb640 1 -- 172.21.15.103:0/553908861 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0b0e18d70 con 0x55d0b0f33100 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58c1ffb640 10 client.17004 handle_command_reply: tid=0 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 client.17004 shutdown 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eb6370 msgr2=0x55d0b0ec0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eb6370 0x55d0b0ec0460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f58d00cd770 tx=0x7f58d009a7b0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 shutdown_connections 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0b0f33100 0x55d0b0f35520 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b0eb6370 0x55d0b0ec0460 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b0eacb40 0x55d0b0ebc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 --2- 172.21.15.103:0/553908861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b0eab5d0 0x55d0b0ebbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 >> 172.21.15.103:0/553908861 conn(0x55d0b0e74310 msgr2=0x55d0b0eb5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:33.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58c1ffb640 0 client.17004 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:33.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 shutdown_connections 2024-09-19T07:45:33.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 1 -- 172.21.15.103:0/553908861 wait complete. 2024-09-19T07:45:33.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e2bc1740 20 client.17004 trim_cache size 0 max 16384 2024-09-19T07:45:33.144 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:45:33.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58c40a3430 msgr2=0x7f58c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58c40a3430 0x7f58c40a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f58c8005ab0 tx=0x7f58c8004160 comp rx=0 tx=0).stop 2024-09-19T07:45:33.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8102ef0 msgr2=0x7f58d819bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:33.145 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.126+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8102ef0 0x7f58d819bce0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f58d000eaa0 tx=0x7f58d000ef60 comp rx=0 tx=0).stop 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 shutdown_connections 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f58c40a3430 0x7f58c40a58f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58d8102ef0 0x7f58d819bce0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8102570 0x7f58d8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 --2- 172.21.15.103:0/2915416368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58d8100e20 0x7f58d8197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 >> 172.21.15.103:0/2915416368 conn(0x7f58d80fc7d0 msgr2=0x7f58d80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 shutdown_connections 2024-09-19T07:45:33.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:33.127+0000 7f58e07e8640 1 -- 172.21.15.103:0/2915416368 wait complete. 2024-09-19T07:45:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:34 smithi103 ceph-mon[31201]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-09-19T07:45:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:34 smithi134 ceph-mon[35414]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-09-19T07:45:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:34 smithi174 ceph-mon[35370]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-09-19T07:45:34.909 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:45:34.909 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:45:35.173 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:35.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 420.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:45:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 -- 172.21.15.103:0/211022926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8103660 msgr2=0x7f63e810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 --2- 172.21.15.103:0/211022926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8103660 0x7f63e810bb50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009900 tx=0x7f63d002a480 comp rx=0 tx=0).stop 2024-09-19T07:45:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 -- 172.21.15.103:0/211022926 shutdown_connections 2024-09-19T07:45:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 --2- 172.21.15.103:0/211022926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63e810c170 0x7f63e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 --2- 172.21.15.103:0/211022926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8103660 0x7f63e810bb50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 --2- 172.21.15.103:0/211022926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63e8102d40 0x7f63e8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 -- 172.21.15.103:0/211022926 >> 172.21.15.103:0/211022926 conn(0x7f63e80fc7a0 msgr2=0x7f63e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 -- 172.21.15.103:0/211022926 shutdown_connections 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 -- 172.21.15.103:0/211022926 wait complete. 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 Processor -- start 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.669+0000 7f63ee587640 1 -- start start 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ee587640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8102d40 0x7f63e81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ee587640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63e8103660 0x7f63e8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ee587640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63e810c170 0x7f63e8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ee587640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8111f70 con 0x7f63e8102d40 2024-09-19T07:45:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ee587640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63e8111df0 con 0x7f63e8103660 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ee587640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63e81120f0 con 0x7f63e810c170 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8102d40 0x7f63e81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8102d40 0x7f63e81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57532/0 (socket says 172.21.15.103:57532) 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 -- 172.21.15.103:0/3987686647 learned_addr learned my addr 172.21.15.103:0/3987686647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e77fe640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63e8103660 0x7f63e8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ecb24640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63e810c170 0x7f63e8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 -- 172.21.15.103:0/3987686647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63e810c170 msgr2=0x7f63e8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63e810c170 0x7f63e8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 -- 172.21.15.103:0/3987686647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63e8103660 msgr2=0x7f63e8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63e8103660 0x7f63e8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 -- 172.21.15.103:0/3987686647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e81a68a0 con 0x7f63e8102d40 2024-09-19T07:45:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e77fe640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63e8103660 0x7f63e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63ecb24640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63e810c170 0x7f63e8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.670+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8102d40 0x7f63e81009a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c890 tx=0x7f63d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.671+0000 7f63e57fa640 1 -- 172.21.15.103:0/3987686647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8004220 con 0x7f63e8102d40 2024-09-19T07:45:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.671+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e81a6b90 con 0x7f63e8102d40 2024-09-19T07:45:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.671+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e81a7120 con 0x7f63e8102d40 2024-09-19T07:45:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.671+0000 7f63e57fa640 1 -- 172.21.15.103:0/3987686647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d80043c0 con 0x7f63e8102d40 2024-09-19T07:45:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.671+0000 7f63e57fa640 1 -- 172.21.15.103:0/3987686647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d801d060 con 0x7f63e8102d40 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63e57fa640 1 -- 172.21.15.103:0/3987686647 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f63d8005020 con 0x7f63e8102d40 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 Processor -- start 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 -- start start 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63e57fa640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63d40a33b0 0x7f63d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fe97300280 con 0x55fe9739ab60 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe972a7b10 con 0x55fe9739b480 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.673+0000 7f63f0960740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe972fc240 con 0x55fe973a5d40 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57544/0 (socket says 172.21.15.103:57544) 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 -- 172.21.15.103:0/2650334194 learned_addr learned my addr 172.21.15.103:0/2650334194 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63d40a33b0 0x7f63d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63e57fa640 1 -- 172.21.15.103:0/3987686647 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f63d80cf790 con 0x7f63e8102d40 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63e7fff640 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63e77fe640 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 -- 172.21.15.103:0/2650334194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 msgr2=0x55fe973a8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973a8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 -- 172.21.15.103:0/2650334194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 msgr2=0x55fe973a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 -- 172.21.15.103:0/2650334194 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe972b41f0 con 0x55fe9739ab60 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63e77fe640 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739af40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f63e0012d80 tx=0x7f63e000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0cfe0e2dc33811 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63ecb24640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63d40a33b0 0x7f63d40a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63e000fc30 tx=0x7f63e001a000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63de7fc640 1 -- 172.21.15.103:0/2650334194 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0020070 con 0x55fe9739ab60 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63de7fc640 1 -- 172.21.15.103:0/2650334194 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e000ab70 con 0x55fe9739ab60 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.674+0000 7f63de7fc640 1 -- 172.21.15.103:0/2650334194 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e000d3d0 con 0x55fe9739ab60 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 -- 172.21.15.103:0/2650334194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 msgr2=0x55fe9739af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739af40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f63e0012d80 tx=0x7f63e000f730 comp rx=0 tx=0).stop 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 -- 172.21.15.103:0/2650334194 shutdown_connections 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 --2- 172.21.15.103:0/2650334194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739af40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 -- 172.21.15.103:0/2650334194 >> 172.21.15.103:0/2650334194 conn(0x55fe973970f0 msgr2=0x55fe973974d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 -- 172.21.15.103:0/2650334194 shutdown_connections 2024-09-19T07:45:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.675+0000 7f63f0960740 1 -- 172.21.15.103:0/2650334194 wait complete. 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 -- start start 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 Processor -- start 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe973a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739b480 0x55fe973a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973a3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fe973a8910 con 0x55fe9739ab60 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe972baf10 con 0x55fe973a5d40 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe973a8a90 con 0x55fe9739b480 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63f0960740 10 client.0 _ll_register_callbacks cb 0x55fe97283a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe973a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe973a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57560/0 (socket says 172.21.15.103:57560) 2024-09-19T07:45:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63e7fff640 1 -- 172.21.15.103:0/4071518504 learned_addr learned my addr 172.21.15.103:0/4071518504 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63e77fe640 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739b480 0x55fe973a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.676+0000 7f63ecb24640 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63e7fff640 1 -- 172.21.15.103:0/4071518504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739b480 msgr2=0x55fe973a30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63e7fff640 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739b480 0x55fe973a30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63e7fff640 1 -- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 msgr2=0x55fe973a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63e7fff640 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63e7fff640 1 -- 172.21.15.103:0/4071518504 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe970716a0 con 0x55fe9739ab60 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63e7fff640 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe973a2b90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f63d8097e60 tx=0x7f63d809b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f399aed60efa904 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63c77fe640 1 -- 172.21.15.103:0/4071518504 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80cd2f0 con 0x55fe9739ab60 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63c77fe640 1 -- 172.21.15.103:0/4071518504 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f63d809ae50 con 0x55fe9739ab60 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e000ab70 con 0x55fe9739ab60 2024-09-19T07:45:35.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63c77fe640 1 -- 172.21.15.103:0/4071518504 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f63d809fdb0 con 0x55fe9739ab60 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63f0960740 10 client.17016 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fe973b4500 con 0x55fe9739ab60 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63c77fe640 1 -- 172.21.15.103:0/4071518504 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63e000ab70 con 0x55fe9739ab60 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.677+0000 7f63c77fe640 1 -- 172.21.15.103:0/4071518504 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f63d8020060 con 0x55fe9739ab60 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63f0960740 10 client.17016 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63f0960740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63f0960740 20 client.17016 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63f0960740 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe973b59c0 0x55fe973b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63f0960740 4 client.17016 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe9730d7f0 con 0x55fe973b59c0 2024-09-19T07:45:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63e77fe640 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe973b59c0 0x55fe973b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63e77fe640 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe973b59c0 0x55fe973b7e40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13edc3bbede41e70 server_cookie=b77a408f9862e15f in_seq=0 out_seq=0 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.678+0000 7f63c77fe640 10 client.17016 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63c77fe640 1 -- 172.21.15.103:0/4071518504 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe9730d7f0 con 0x55fe973b59c0 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63c77fe640 10 client.17016 handle_command_reply: tid=0 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 client.17016 shutdown 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 msgr2=0x55fe973a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe973a2b90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f63d8097e60 tx=0x7f63d809b360 comp rx=0 tx=0).stop 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 shutdown_connections 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe973b59c0 0x55fe973b7e40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739b480 0x55fe973a30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 --2- 172.21.15.103:0/4071518504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe973a2b90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63c77fe640 0 client.17016 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:35.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 >> 172.21.15.103:0/4071518504 conn(0x55fe973970f0 msgr2=0x55fe9739ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:35.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 shutdown_connections 2024-09-19T07:45:35.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 1 -- 172.21.15.103:0/4071518504 wait complete. 2024-09-19T07:45:35.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.679+0000 7f63f0960740 20 client.17016 trim_cache size 0 max 16384 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 Processor -- start 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 -- start start 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63c800e7e0 con 0x55fe9739ab60 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8010320 con 0x55fe9739b480 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63f0960740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63c800eeb0 con 0x55fe973a5d40 2024-09-19T07:45:35.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57564/0 (socket says 172.21.15.103:57564) 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.696+0000 7f63e7fff640 1 -- 172.21.15.103:0/3755352821 learned_addr learned my addr 172.21.15.103:0/3755352821 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63e77fe640 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63ecb24640 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63e7fff640 1 -- 172.21.15.103:0/3755352821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 msgr2=0x55fe973bdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973bdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63e7fff640 1 -- 172.21.15.103:0/3755352821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 msgr2=0x55fe973bd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973bd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63e7fff640 1 -- 172.21.15.103:0/3755352821 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe973b84c0 con 0x55fe9739ab60 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739d5c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f63d800a460 tx=0x7f63d8097670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e6d61b084b1591 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63ddffb640 1 -- 172.21.15.103:0/3755352821 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80a4670 con 0x55fe9739ab60 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63ddffb640 1 -- 172.21.15.103:0/3755352821 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d80b8070 con 0x55fe9739ab60 2024-09-19T07:45:35.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.697+0000 7f63ddffb640 1 -- 172.21.15.103:0/3755352821 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d809b0b0 con 0x55fe9739ab60 2024-09-19T07:45:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- 172.21.15.103:0/3755352821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 msgr2=0x55fe9739d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739d5c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f63d800a460 tx=0x7f63d8097670 comp rx=0 tx=0).stop 2024-09-19T07:45:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- 172.21.15.103:0/3755352821 shutdown_connections 2024-09-19T07:45:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe973a5d40 0x55fe973bdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe9739b480 0x55fe973bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 --2- 172.21.15.103:0/3755352821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739ab60 0x55fe9739d5c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- 172.21.15.103:0/3755352821 >> 172.21.15.103:0/3755352821 conn(0x55fe973970f0 msgr2=0x55fe97396270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- 172.21.15.103:0/3755352821 shutdown_connections 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- 172.21.15.103:0/3755352821 wait complete. 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- start start 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 Processor -- start 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739ab60 0x55fe973add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739b480 0x55fe973ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63c800fc50 con 0x55fe9739b480 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63c800f580 con 0x55fe973a5d40 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.698+0000 7f63f0960740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63c800c7e0 con 0x55fe9739ab60 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63f0960740 10 client.0 _ll_register_callbacks cb 0x7f63d809b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:35.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739b480 0x55fe973ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739b480 0x55fe973ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57574/0 (socket says 172.21.15.103:57574) 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 -- 172.21.15.103:0/3629308626 learned_addr learned my addr 172.21.15.103:0/3629308626 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739ab60 0x55fe973add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63ecb24640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 -- 172.21.15.103:0/3629308626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739ab60 msgr2=0x55fe973add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739ab60 0x55fe973add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 -- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 msgr2=0x55fe973aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 -- 172.21.15.103:0/3629308626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe973ae8f0 con 0x55fe9739b480 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63ecb24640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63e77fe640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739b480 0x55fe973ae280 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009230 tx=0x7f63d00041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c337844456be6063 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63c77fe640 1 -- 172.21.15.103:0/3629308626 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d004b060 con 0x55fe9739b480 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63c77fe640 1 -- 172.21.15.103:0/3629308626 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f63d0005240 con 0x55fe9739b480 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d80b8070 con 0x55fe9739b480 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63c77fe640 1 -- 172.21.15.103:0/3629308626 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f63d0035040 con 0x55fe9739b480 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63f0960740 10 client.17022 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.699+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fe97420d30 con 0x55fe9739b480 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63c77fe640 1 -- 172.21.15.103:0/3629308626 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d80b8070 con 0x55fe9739b480 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63c77fe640 1 -- 172.21.15.103:0/3629308626 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f63d0045070 con 0x55fe9739b480 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63f0960740 10 client.17022 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63f0960740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63f0960740 20 client.17022 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63f0960740 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe974214e0 0x55fe97423900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63f0960740 4 client.17022 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:45:35.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe97307d00 con 0x55fe974214e0 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.700+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe974214e0 0x55fe97423900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.701+0000 7f63e7fff640 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe974214e0 0x55fe97423900 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bad3de006bfdd91 server_cookie=1f7b8001cc346886 in_seq=0 out_seq=0 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.701+0000 7f63c77fe640 10 client.17022 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63c77fe640 1 -- 172.21.15.103:0/3629308626 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe97307d00 con 0x55fe974214e0 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63c77fe640 10 client.17022 handle_command_reply: tid=0 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 client.17022 shutdown 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739b480 msgr2=0x55fe973ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739b480 0x55fe973ae280 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009230 tx=0x7f63d00041a0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 shutdown_connections 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55fe974214e0 0x55fe97423900 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe973a5d40 0x55fe973aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe9739b480 0x55fe973ae280 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 --2- 172.21.15.103:0/3629308626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe9739ab60 0x55fe973add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 >> 172.21.15.103:0/3629308626 conn(0x55fe973970f0 msgr2=0x55fe97395a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63c77fe640 0 client.17022 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 shutdown_connections 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 1 -- 172.21.15.103:0/3629308626 wait complete. 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.702+0000 7f63f0960740 20 client.17022 trim_cache size 0 max 16384 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "34036f79-2c30-443e-8e69-f77e702abf68", 2024-09-19T07:45:35.708 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:45:35.709 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:45:35.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63d40a33b0 msgr2=0x7f63d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63d40a33b0 0x7f63d40a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63e000fc30 tx=0x7f63e001a000 comp rx=0 tx=0).stop 2024-09-19T07:45:35.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8102d40 msgr2=0x7f63e81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:35.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8102d40 0x7f63e81009a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c890 tx=0x7f63d800cd50 comp rx=0 tx=0).stop 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 shutdown_connections 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f63d40a33b0 0x7f63d40a5870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63e810c170 0x7f63e8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63e8103660 0x7f63e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 --2- 172.21.15.103:0/3987686647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63e8102d40 0x7f63e81009a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 >> 172.21.15.103:0/3987686647 conn(0x7f63e80fc7a0 msgr2=0x7f63e80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 shutdown_connections 2024-09-19T07:45:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:35.703+0000 7f63ee587640 1 -- 172.21.15.103:0/3987686647 wait complete. 2024-09-19T07:45:36.310 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:36 smithi103 ceph-mon[31201]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-09-19T07:45:36.310 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:36 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:45:36.310 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:36 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:36.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:36 smithi134 ceph-mon[35414]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-09-19T07:45:36.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:36 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:45:36.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:36 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:36.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:45:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:36 smithi174 ceph-mon[35370]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-09-19T07:45:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:36 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:45:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:36 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:36.642 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:37.289 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.290 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:37 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:45:37.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:45:37.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:37 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 -- 172.21.15.103:0/3867344347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102ef0 msgr2=0x7f4e6810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 --2- 172.21.15.103:0/3867344347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102ef0 0x7f4e6810da40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009ca0 tx=0x7f4e6002a950 comp rx=0 tx=0).stop 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 -- 172.21.15.103:0/3867344347 shutdown_connections 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 --2- 172.21.15.103:0/3867344347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102ef0 0x7f4e6810da40 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 --2- 172.21.15.103:0/3867344347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e68102570 0x7f4e681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 --2- 172.21.15.103:0/3867344347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e68100e20 0x7f4e68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 -- 172.21.15.103:0/3867344347 >> 172.21.15.103:0/3867344347 conn(0x7f4e680fc7d0 msgr2=0x7f4e680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:37.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 -- 172.21.15.103:0/3867344347 shutdown_connections 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.375+0000 7f4e6f074640 1 -- 172.21.15.103:0/3867344347 wait complete. 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 Processor -- start 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 -- start start 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e68100e20 0x7f4e68198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102570 0x7f4e68198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e68102ef0 0x7f4e6819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68110f80 con 0x7f4e68102570 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68110e00 con 0x7f4e68100e20 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6f074640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68111100 con 0x7f4e68102ef0 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102570 0x7f4e68198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102570 0x7f4e68198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57592/0 (socket says 172.21.15.103:57592) 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 -- 172.21.15.103:0/2715257352 learned_addr learned my addr 172.21.15.103:0/2715257352 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6d611640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e68102ef0 0x7f4e6819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e68100e20 0x7f4e68198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 -- 172.21.15.103:0/2715257352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e68102ef0 msgr2=0x7f4e6819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e68102ef0 0x7f4e6819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 -- 172.21.15.103:0/2715257352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e68100e20 msgr2=0x7f4e68198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e68100e20 0x7f4e68198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e67fff640 1 -- 172.21.15.103:0/2715257352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6819d400 con 0x7f4e68102570 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.376+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e68100e20 0x7f4e68198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.377+0000 7f4e67fff640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102570 0x7f4e68198990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d950 tx=0x7f4e5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.377+0000 7f4e65ffb640 1 -- 172.21.15.103:0/2715257352 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54017030 con 0x7f4e68102570 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.377+0000 7f4e65ffb640 1 -- 172.21.15.103:0/2715257352 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e54004020 con 0x7f4e68102570 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.377+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6819d6f0 con 0x7f4e68102570 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.377+0000 7f4e65ffb640 1 -- 172.21.15.103:0/2715257352 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540047a0 con 0x7f4e68102570 2024-09-19T07:45:37.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.377+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e681a4fd0 con 0x7f4e68102570 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.379+0000 7f4e65ffb640 1 -- 172.21.15.103:0/2715257352 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4e540041d0 con 0x7f4e68102570 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 Processor -- start 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 -- start start 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e65ffb640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4e580a33b0 0x7f4e580a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55722161a2e0 con 0x5572216b4ca0 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5572215c1ce0 con 0x5572216b4380 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e7144d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5572216162a0 con 0x5572216c3a90 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e6d611640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4e580a33b0 0x7f4e580a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e65ffb640 1 -- 172.21.15.103:0/2715257352 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f4e540cf720 con 0x7f4e68102570 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57598/0 (socket says 172.21.15.103:57598) 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e67fff640 1 -- 172.21.15.103:0/3754745542 learned_addr learned my addr 172.21.15.103:0/3754745542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e6d611640 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.380+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e67fff640 1 -- 172.21.15.103:0/3754745542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 msgr2=0x5572216c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e67fff640 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e6d611640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4e580a33b0 0x7f4e580a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005e30 tx=0x7f4e6002fa10 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e67fff640 1 -- 172.21.15.103:0/3754745542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 msgr2=0x5572216b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e67fff640 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e67fff640 1 -- 172.21.15.103:0/3754745542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572215a6440 con 0x5572216b4ca0 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:37.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e67fff640 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c3550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4e54005fd0 tx=0x7f4e5409b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cee86e44bfaff6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:37.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e52ffd640 1 -- 172.21.15.103:0/3754745542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54049c80 con 0x5572216b4ca0 2024-09-19T07:45:37.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e52ffd640 1 -- 172.21.15.103:0/3754745542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5409c0f0 con 0x5572216b4ca0 2024-09-19T07:45:37.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.381+0000 7f4e52ffd640 1 -- 172.21.15.103:0/3754745542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54049010 con 0x5572216b4ca0 2024-09-19T07:45:37.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 -- 172.21.15.103:0/3754745542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 msgr2=0x5572216c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c3550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4e54005fd0 tx=0x7f4e5409b530 comp rx=0 tx=0).stop 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 -- 172.21.15.103:0/3754745542 shutdown_connections 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c3550 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 --2- 172.21.15.103:0/3754745542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 -- 172.21.15.103:0/3754745542 >> 172.21.15.103:0/3754745542 conn(0x5572216b0a30 msgr2=0x5572216b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 -- 172.21.15.103:0/3754745542 shutdown_connections 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 -- 172.21.15.103:0/3754745542 wait complete. 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 -- start start 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.382+0000 7f4e7144d740 1 Processor -- start 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e7144d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5572216c6660 con 0x5572216b4ca0 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e7144d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5572215d5050 con 0x5572216b4380 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e7144d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5572216c67e0 con 0x5572216c3a90 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e7144d740 10 client.0 _ll_register_callbacks cb 0x557221615160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57612/0 (socket says 172.21.15.103:57612) 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 -- 172.21.15.103:0/4152351867 learned_addr learned my addr 172.21.15.103:0/4152351867 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e6d611640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 -- 172.21.15.103:0/4152351867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 msgr2=0x5572216c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 -- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 msgr2=0x5572216c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216c3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 -- 172.21.15.103:0/4152351867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722138b3c0 con 0x5572216b4ca0 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e6d611640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e67fff640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c0740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e540cd460 tx=0x7f4e54099d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db758bd8168da7af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e47fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.383+0000 7f4e47fff640 1 -- 172.21.15.103:0/4152351867 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540b4060 con 0x5572216b4ca0 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e47fff640 1 -- 172.21.15.103:0/4152351867 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f4e5409ae40 con 0x5572216b4ca0 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5409c0f0 con 0x5572216b4ca0 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e47fff640 1 -- 172.21.15.103:0/4152351867 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4e54049c80 con 0x5572216b4ca0 2024-09-19T07:45:37.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 10 client.17034 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5572216d2100 con 0x5572216b4ca0 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e47fff640 1 -- 172.21.15.103:0/4152351867 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5409c0f0 con 0x5572216b4ca0 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e47fff640 1 -- 172.21.15.103:0/4152351867 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f4e540ba200 con 0x5572216b4ca0 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 10 client.17034 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 20 client.17034 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5572216d3450 0x5572216d5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 4 client.17034 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557221627850 con 0x5572216d3450 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.384+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5572216d3450 0x5572216d5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.385+0000 7f4e47fff640 10 client.17034 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.385+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5572216d3450 0x5572216d5840 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2763877f511bd369 server_cookie=33447d335dde4346 in_seq=0 out_seq=0 2024-09-19T07:45:37.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e47fff640 1 -- 172.21.15.103:0/4152351867 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557221627850 con 0x5572216d3450 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e47fff640 10 client.17034 handle_command_reply: tid=0 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 client.17034 shutdown 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 msgr2=0x5572216c0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c0740 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e540cd460 tx=0x7f4e54099d60 comp rx=0 tx=0).stop 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 shutdown_connections 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5572216d3450 0x5572216d5840 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216c3a90 0x5572216c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4ca0 0x5572216c0740 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 --2- 172.21.15.103:0/4152351867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 >> 172.21.15.103:0/4152351867 conn(0x5572216b0a30 msgr2=0x5572216b6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e47fff640 0 client.17034 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 shutdown_connections 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 1 -- 172.21.15.103:0/4152351867 wait complete. 2024-09-19T07:45:37.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.386+0000 7f4e7144d740 20 client.17034 trim_cache size 0 max 16384 2024-09-19T07:45:37.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 Processor -- start 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 -- start start 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4380 0x5572216c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216c3a90 0x5572216d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681ab6a0 con 0x5572216b4380 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681ad1e0 con 0x5572216c3a90 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e7144d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681abd70 con 0x5572216b4ca0 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4380 0x5572216c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4380 0x5572216c0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57624/0 (socket says 172.21.15.103:57624) 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 -- 172.21.15.103:0/903182904 learned_addr learned my addr 172.21.15.103:0/903182904 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e67fff640 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6d611640 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216c3a90 0x5572216d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 -- 172.21.15.103:0/903182904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 msgr2=0x5572216d8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 -- 172.21.15.103:0/903182904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216c3a90 msgr2=0x5572216d89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216c3a90 0x5572216d89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.403+0000 7f4e6ce10640 1 -- 172.21.15.103:0/903182904 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572216c37c0 con 0x5572216b4380 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4380 0x5572216c0710 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b370 tx=0x7f4e5c0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc4bd99bde871ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e527fc640 1 -- 172.21.15.103:0/903182904 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004030 con 0x5572216b4380 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e527fc640 1 -- 172.21.15.103:0/903182904 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c0041d0 con 0x5572216b4380 2024-09-19T07:45:37.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e527fc640 1 -- 172.21.15.103:0/903182904 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c011730 con 0x5572216b4380 2024-09-19T07:45:37.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e7144d740 1 -- 172.21.15.103:0/903182904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4380 msgr2=0x5572216c0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e7144d740 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4380 0x5572216c0710 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00b370 tx=0x7f4e5c0138f0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e7144d740 1 -- 172.21.15.103:0/903182904 shutdown_connections 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e7144d740 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216c3a90 0x5572216d89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e7144d740 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e7144d740 1 --2- 172.21.15.103:0/903182904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216b4380 0x5572216c0710 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.404+0000 7f4e7144d740 1 -- 172.21.15.103:0/903182904 >> 172.21.15.103:0/903182904 conn(0x5572216b0a30 msgr2=0x5572216c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 -- 172.21.15.103:0/903182904 shutdown_connections 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 -- 172.21.15.103:0/903182904 wait complete. 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 -- start start 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 Processor -- start 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216d64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216c3a90 0x5572216d6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681acb10 con 0x5572216c3a90 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681ac440 con 0x5572216b4380 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e681a96a0 con 0x5572216b4ca0 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e7144d740 10 client.0 _ll_register_callbacks cb 0x7f4e5c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e6d611640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216c3a90 0x5572216d6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e6ce10640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216d64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.405+0000 7f4e6d611640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216c3a90 0x5572216d6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57632/0 (socket says 172.21.15.103:57632) 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6d611640 1 -- 172.21.15.103:0/2139056571 learned_addr learned my addr 172.21.15.103:0/2139056571 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e67fff640 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6ce10640 1 -- 172.21.15.103:0/2139056571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 msgr2=0x5572216d6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6ce10640 1 -- 172.21.15.103:0/2139056571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216c3a90 msgr2=0x5572216d6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216c3a90 0x5572216d6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6ce10640 1 -- 172.21.15.103:0/2139056571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722173cfe0 con 0x5572216b4380 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6d611640 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216c3a90 0x5572216d6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e6ce10640 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216d64c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00bdd0 tx=0x7f4e5c00bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de438a534f522b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e47fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e47fff640 1 -- 172.21.15.103:0/2139056571 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004030 con 0x5572216b4380 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5c0041d0 con 0x5572216b4380 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e47fff640 1 -- 172.21.15.103:0/2139056571 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f4e5c005030 con 0x5572216b4380 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.406+0000 7f4e47fff640 1 -- 172.21.15.103:0/2139056571 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c005230 con 0x5572216b4380 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 10 client.26516 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55722173e2f0 con 0x5572216b4380 2024-09-19T07:45:37.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e47fff640 1 -- 172.21.15.103:0/2139056571 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e5c0041d0 con 0x5572216b4380 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e47fff640 1 -- 172.21.15.103:0/2139056571 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f4e5c037080 con 0x5572216b4380 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 10 client.26516 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 20 client.26516 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55722173e8b0 0x557221740cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 4 client.26516 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557221621d60 con 0x55722173e8b0 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.407+0000 7f4e67fff640 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55722173e8b0 0x557221740cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e47fff640 10 client.26516 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e67fff640 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55722173e8b0 0x557221740cf0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec6488b9c716029 server_cookie=1071d68d81b2bfec in_seq=0 out_seq=0 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e47fff640 1 -- 172.21.15.103:0/2139056571 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557221621d60 con 0x55722173e8b0 2024-09-19T07:45:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e47fff640 10 client.26516 handle_command_reply: tid=0 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 client.26516 shutdown 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 msgr2=0x5572216d64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216d64c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00bdd0 tx=0x7f4e5c00bbf0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 shutdown_connections 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55722173e8b0 0x557221740cf0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572216c3a90 0x5572216d6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572216b4ca0 0x5572216d6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 --2- 172.21.15.103:0/2139056571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572216b4380 0x5572216d64c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e47fff640 0 client.26516 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 >> 172.21.15.103:0/2139056571 conn(0x5572216b0a30 msgr2=0x5572216c1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 shutdown_connections 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 1 -- 172.21.15.103:0/2139056571 wait complete. 2024-09-19T07:45:37.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.408+0000 7f4e7144d740 20 client.26516 trim_cache size 0 max 16384 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "34036f79-2c30-443e-8e69-f77e702abf68": { 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "34036f79-2c30-443e-8e69-f77e702abf68", 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:37.416 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:45:37.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4e580a33b0 msgr2=0x7f4e580a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4e580a33b0 0x7f4e580a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e60005e30 tx=0x7f4e6002fa10 comp rx=0 tx=0).stop 2024-09-19T07:45:37.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102570 msgr2=0x7f4e68198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:37.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102570 0x7f4e68198990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d950 tx=0x7f4e5400de10 comp rx=0 tx=0).stop 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 shutdown_connections 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4e580a33b0 0x7f4e580a5870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e68102ef0 0x7f4e6819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e68102570 0x7f4e68198990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 --2- 172.21.15.103:0/2715257352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e68100e20 0x7f4e68198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 >> 172.21.15.103:0/2715257352 conn(0x7f4e680fc7d0 msgr2=0x7f4e6806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 shutdown_connections 2024-09-19T07:45:37.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:37.409+0000 7f4e6f074640 1 -- 172.21.15.103:0/2715257352 wait complete. 2024-09-19T07:45:37.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:37.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:37 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:45:38.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:38 smithi134 ceph-mon[35414]: pgmap v234: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 340 op/s 2024-09-19T07:45:38.404 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:38 smithi103 ceph-mon[31201]: pgmap v234: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 340 op/s 2024-09-19T07:45:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:38 smithi174 ceph-mon[35370]: pgmap v234: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 340 op/s 2024-09-19T07:45:38.436 INFO:tasks.cephfs.filesystem:scrub status for tag:34036f79-2c30-443e-8e69-f77e702abf68 - {'path': '/', 'tag': '34036f79-2c30-443e-8e69-f77e702abf68', 'options': 'recursive,force'} 2024-09-19T07:45:40.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:40 smithi134 ceph-mon[35414]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 918 KiB/s wr, 66 op/s 2024-09-19T07:45:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:40 smithi174 ceph-mon[35370]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 918 KiB/s wr, 66 op/s 2024-09-19T07:45:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:40 smithi103 ceph-mon[31201]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 918 KiB/s wr, 66 op/s 2024-09-19T07:45:40.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 425.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:45:41.186 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:45:41.446 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4201587589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4102790 msgr2=0x7f17d41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4201587589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4102790 0x7f17d41067e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009900 tx=0x7f17c802a410 comp rx=0 tx=0).stop 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4201587589 shutdown_connections 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4201587589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17d4106f10 0x7f17d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4201587589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4102790 0x7f17d41067e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4201587589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17d4101de0 0x7f17d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4201587589 >> 172.21.15.103:0/4201587589 conn(0x7f17d40fd8c0 msgr2=0x7f17d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4201587589 shutdown_connections 2024-09-19T07:45:41.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4201587589 wait complete. 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 Processor -- start 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.944+0000 7f17dbdd2640 1 -- start start 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17dbdd2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4101de0 0x7f17d41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17dbdd2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17d4102790 0x7f17d41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17dbdd2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17d4106f10 0x7f17d419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17dbdd2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4111f70 con 0x7f17d4101de0 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17dbdd2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4111df0 con 0x7f17d4102790 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17dbdd2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41120f0 con 0x7f17d4106f10 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4101de0 0x7f17d41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4101de0 0x7f17d41a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52320/0 (socket says 172.21.15.103:52320) 2024-09-19T07:45:41.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 -- 172.21.15.103:0/4020422829 learned_addr learned my addr 172.21.15.103:0/4020422829 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d936d640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17d4102790 0x7f17d41a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 -- 172.21.15.103:0/4020422829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17d4106f10 msgr2=0x7f17d419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17d4106f10 0x7f17d419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 -- 172.21.15.103:0/4020422829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17d4102790 msgr2=0x7f17d41a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17d4102790 0x7f17d41a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 -- 172.21.15.103:0/4020422829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d419acf0 con 0x7f17d4101de0 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d9b6e640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4101de0 0x7f17d41a0400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d950 tx=0x7f17cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.945+0000 7f17d2ffd640 1 -- 172.21.15.103:0/4020422829 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc017070 con 0x7f17d4101de0 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.946+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d419afe0 con 0x7f17d4101de0 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.946+0000 7f17d2ffd640 1 -- 172.21.15.103:0/4020422829 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17cc009c30 con 0x7f17d4101de0 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.946+0000 7f17d2ffd640 1 -- 172.21.15.103:0/4020422829 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00f6b0 con 0x7f17d4101de0 2024-09-19T07:45:41.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.946+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41a72d0 con 0x7f17d4101de0 2024-09-19T07:45:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.947+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d41021c0 con 0x7f17d4101de0 2024-09-19T07:45:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.948+0000 7f17d2ffd640 1 -- 172.21.15.103:0/4020422829 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f17cc0109d0 con 0x7f17d4101de0 2024-09-19T07:45:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.948+0000 7f17d2ffd640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17bc0a33b0 0x7f17bc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.948+0000 7f17d936d640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17bc0a33b0 0x7f17bc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.948+0000 7f17d2ffd640 1 -- 172.21.15.103:0/4020422829 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f17cc0cf9a0 con 0x7f17d4101de0 2024-09-19T07:45:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.949+0000 7f17d936d640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17bc0a33b0 0x7f17bc0a5870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009230 tx=0x7f17c80058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:41.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:41.953+0000 7f17d2ffd640 1 -- 172.21.15.103:0/4020422829 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f17cc009de0 con 0x7f17d4101de0 2024-09-19T07:45:42.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.159+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17d41a74a0 con 0x7f17d4101de0 2024-09-19T07:45:42.160 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:42 smithi103 ceph-mon[31201]: pgmap v236: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 930 KiB/s wr, 73 op/s 2024-09-19T07:45:42.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.160+0000 7f17d2ffd640 1 -- 172.21.15.103:0/4020422829 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11996 (secure 0 0 0) 0x7f17cc09c370 con 0x7f17d4101de0 2024-09-19T07:45:42.161 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:45:42.161 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":40,"btime":"2024-09-19T07:44:20:535666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:44:19.854970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:45:42.162 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 40 2024-09-19T07:45:42.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17bc0a33b0 msgr2=0x7f17bc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:42.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17bc0a33b0 0x7f17bc0a5870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009230 tx=0x7f17c80058d0 comp rx=0 tx=0).stop 2024-09-19T07:45:42.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4101de0 msgr2=0x7f17d41a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:42.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4101de0 0x7f17d41a0400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d950 tx=0x7f17cc00de10 comp rx=0 tx=0).stop 2024-09-19T07:45:42.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 shutdown_connections 2024-09-19T07:45:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17bc0a33b0 0x7f17bc0a5870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17d4106f10 0x7f17d419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17d4102790 0x7f17d41a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 --2- 172.21.15.103:0/4020422829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17d4101de0 0x7f17d41a0400 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.162+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 >> 172.21.15.103:0/4020422829 conn(0x7f17d40fd8c0 msgr2=0x7f17d40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.163+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 shutdown_connections 2024-09-19T07:45:42.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:42.163+0000 7f17dbdd2640 1 -- 172.21.15.103:0/4020422829 wait complete. 2024-09-19T07:45:42.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:42 smithi134 ceph-mon[35414]: pgmap v236: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 930 KiB/s wr, 73 op/s 2024-09-19T07:45:42.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:42 smithi174 ceph-mon[35370]: pgmap v236: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 930 KiB/s wr, 73 op/s 2024-09-19T07:45:42.859 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 80da2ce4 --release --await' 2024-09-19T07:45:42.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 80da2ce4 --release --await 2024-09-19T07:45:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:43.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:43 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4020422829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:43.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:43 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4020422829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:43.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:43 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4020422829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:43.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 -- 172.21.15.103:0/1016289175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c105a90 msgr2=0x7f602c105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 --2- 172.21.15.103:0/1016289175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c105a90 0x7f602c105e70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6028009900 tx=0x7f602802a410 comp rx=0 tx=0).stop 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 -- 172.21.15.103:0/1016289175 shutdown_connections 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 --2- 172.21.15.103:0/1016289175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c101bc0 0x7f602c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 --2- 172.21.15.103:0/1016289175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c101240 0x7f602c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 --2- 172.21.15.103:0/1016289175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c105a90 0x7f602c105e70 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 -- 172.21.15.103:0/1016289175 >> 172.21.15.103:0/1016289175 conn(0x7f602c077220 msgr2=0x7f602c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 -- 172.21.15.103:0/1016289175 shutdown_connections 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.587+0000 7f6033465640 1 -- 172.21.15.103:0/1016289175 wait complete. 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 Processor -- start 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 -- start start 2024-09-19T07:45:43.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c101240 0x7f602c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c101bc0 0x7f602c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c105a90 0x7f602c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f602c110f70 con 0x7f602c101240 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f602c110df0 con 0x7f602c101bc0 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6033465640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1110f0 con 0x7f602c105a90 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6031201640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c101240 0x7f602c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6031201640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c101240 0x7f602c197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52338/0 (socket says 172.21.15.103:52338) 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.588+0000 7f6031201640 1 -- 172.21.15.103:0/17155964 learned_addr learned my addr 172.21.15.103:0/17155964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6030a00640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c101bc0 0x7f602c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6031a02640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c105a90 0x7f602c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6031201640 1 -- 172.21.15.103:0/17155964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c105a90 msgr2=0x7f602c19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6031201640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c105a90 0x7f602c19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6031201640 1 -- 172.21.15.103:0/17155964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c101bc0 msgr2=0x7f602c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6031201640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c101bc0 0x7f602c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6031201640 1 -- 172.21.15.103:0/17155964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c19c320 con 0x7f602c101240 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6030a00640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c101bc0 0x7f602c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6031201640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c101240 0x7f602c197370 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6028006010 tx=0x7f6028003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f60227fc640 1 -- 172.21.15.103:0/17155964 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028035070 con 0x7f602c101240 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602c19c5b0 con 0x7f602c101240 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f60227fc640 1 -- 172.21.15.103:0/17155964 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6028031a20 con 0x7f602c101240 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f60227fc640 1 -- 172.21.15.103:0/17155964 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028038ae0 con 0x7f602c101240 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.589+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602c100130 con 0x7f602c101240 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.591+0000 7f60227fc640 1 -- 172.21.15.103:0/17155964 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6028031bd0 con 0x7f602c101240 2024-09-19T07:45:43.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 Processor -- start 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 -- start start 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f60227fc640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60180a33b0 0x7f60180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d060 0x56541813d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541814a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5654180a2620 con 0x56541813d060 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565418049e50 con 0x56541813d980 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f603583e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56541809e5e0 con 0x565418148240 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f60227fc640 1 -- 172.21.15.103:0/17155964 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f60280f1ea0 con 0x7f602c101240 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.592+0000 7f6031a02640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60180a33b0 0x7f60180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d060 0x56541813d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d060 0x56541813d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52342/0 (socket says 172.21.15.103:52342) 2024-09-19T07:45:43.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 -- 172.21.15.103:0/1837094933 learned_addr learned my addr 172.21.15.103:0/1837094933 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6030a00640 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 -- 172.21.15.103:0/1837094933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 msgr2=0x56541814a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541814a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 -- 172.21.15.103:0/1837094933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 msgr2=0x565418147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 -- 172.21.15.103:0/1837094933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541803ad40 con 0x56541813d060 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031a02640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60180a33b0 0x7f60180a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6024005fd0 tx=0x7f6024010740 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f6031201640 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d060 0x56541813d440 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f60280baae0 tx=0x7f6028031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a3caf95439d196 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f60177fe640 1 -- 172.21.15.103:0/1837094933 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280d7070 con 0x56541813d060 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f60177fe640 1 -- 172.21.15.103:0/1837094933 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60280be810 con 0x56541813d060 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.593+0000 7f60177fe640 1 -- 172.21.15.103:0/1837094933 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028038d70 con 0x56541813d060 2024-09-19T07:45:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 -- 172.21.15.103:0/1837094933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d060 msgr2=0x56541813d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d060 0x56541813d440 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f60280baae0 tx=0x7f6028031f10 comp rx=0 tx=0).stop 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 -- 172.21.15.103:0/1837094933 shutdown_connections 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541814a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 --2- 172.21.15.103:0/1837094933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d060 0x56541813d440 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 -- 172.21.15.103:0/1837094933 >> 172.21.15.103:0/1837094933 conn(0x5654181395f0 msgr2=0x5654181399d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 -- 172.21.15.103:0/1837094933 shutdown_connections 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 -- 172.21.15.103:0/1837094933 wait complete. 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 -- start start 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.594+0000 7f603583e740 1 Processor -- start 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541813d060 0x565418145220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418145760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565418148240 0x565418145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56541814abd0 con 0x565418148240 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56541802af10 con 0x56541813d980 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56541814ad50 con 0x56541813d060 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 10 client.0 _ll_register_callbacks cb 0x56541804edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565418148240 0x565418145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565418148240 0x565418145ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52350/0 (socket says 172.21.15.103:52350) 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 -- 172.21.15.103:0/2568980485 learned_addr learned my addr 172.21.15.103:0/2568980485 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6030a00640 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031201640 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541813d060 0x565418145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 -- 172.21.15.103:0/2568980485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541813d060 msgr2=0x565418145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541813d060 0x565418145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 -- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 msgr2=0x565418145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 -- 172.21.15.103:0/2568980485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565417db1580 con 0x565418148240 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6031a02640 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565418148240 0x565418145ca0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6024012810 tx=0x7f6024004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e120bfd3d5a466b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6014ff9640 1 -- 172.21.15.103:0/2568980485 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024011b60 con 0x565418148240 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f6014ff9640 1 -- 172.21.15.103:0/2568980485 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f6024011d00 con 0x565418148240 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.595+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60280be810 con 0x565418148240 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f6014ff9640 1 -- 172.21.15.103:0/2568980485 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f602402e050 con 0x565418148240 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 10 client.17058 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565417e7b960 con 0x565418148240 2024-09-19T07:45:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f6014ff9640 1 -- 172.21.15.103:0/2568980485 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60280be810 con 0x565418148240 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f6014ff9640 1 -- 172.21.15.103:0/2568980485 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f602400f040 con 0x565418148240 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 10 client.17058 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 10 client.17058 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 20 client.17058 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x56541815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 4 client.17058 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654180afb90 con 0x565418157c70 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.596+0000 7f6031201640 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x56541815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.597+0000 7f6031201640 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x56541815a0d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=946a4745c933d004 server_cookie=1895403d7b046525 in_seq=0 out_seq=0 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.597+0000 7f6014ff9640 10 client.17058 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.597+0000 7f6014ff9640 1 -- 172.21.15.103:0/2568980485 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654180afb90 con 0x565418157c70 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.597+0000 7f6014ff9640 10 client.17058 handle_command_reply: tid=0 2024-09-19T07:45:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 client.17058 shutdown 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565418148240 msgr2=0x565418145ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565418148240 0x565418145ca0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6024012810 tx=0x7f6024004ab0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 shutdown_connections 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x56541815a0d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f6014ff9640 0 client.17058 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565418148240 0x565418145ca0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d980 0x565418145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 --2- 172.21.15.103:0/2568980485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541813d060 0x565418145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 >> 172.21.15.103:0/2568980485 conn(0x5654181395f0 msgr2=0x565418138750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 shutdown_connections 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 1 -- 172.21.15.103:0/2568980485 wait complete. 2024-09-19T07:45:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.598+0000 7f603583e740 20 client.17058 trim_cache size 0 max 16384 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 Processor -- start 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 -- start start 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x565418140420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541813f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1ab6a0 con 0x56541813d980 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1ad1e0 con 0x56541813d060 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f603583e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1abd70 con 0x565418148240 2024-09-19T07:45:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541813f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541813f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52358/0 (socket says 172.21.15.103:52358) 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 -- 172.21.15.103:0/1175845647 learned_addr learned my addr 172.21.15.103:0/1175845647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 -- 172.21.15.103:0/1175845647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 msgr2=0x56541815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6031a02640 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6031201640 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x565418140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 -- 172.21.15.103:0/1175845647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 msgr2=0x565418140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x565418140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 -- 172.21.15.103:0/1175845647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565418147160 con 0x56541813d980 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6031201640 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x565418140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6031a02640 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.615+0000 7f6030a00640 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541813f210 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d70 tx=0x7f601c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500e311ee73e68ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:43.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f6016ffd640 1 -- 172.21.15.103:0/1175845647 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c009e60 con 0x56541813d980 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f6016ffd640 1 -- 172.21.15.103:0/1175845647 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c0035f0 con 0x56541813d980 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f6016ffd640 1 -- 172.21.15.103:0/1175845647 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c011710 con 0x56541813d980 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 -- 172.21.15.103:0/1175845647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 msgr2=0x56541813f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541813f210 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d70 tx=0x7f601c013830 comp rx=0 tx=0).stop 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 -- 172.21.15.103:0/1175845647 shutdown_connections 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x56541815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541813f210 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 --2- 172.21.15.103:0/1175845647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x565418140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 -- 172.21.15.103:0/1175845647 >> 172.21.15.103:0/1175845647 conn(0x5654181395f0 msgr2=0x56541813eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 -- 172.21.15.103:0/1175845647 shutdown_connections 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.616+0000 7f603583e740 1 -- 172.21.15.103:0/1175845647 wait complete. 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 -- start start 2024-09-19T07:45:43.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 Processor -- start 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x56541814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541814c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x5654181bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1acb10 con 0x56541813d980 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1ac440 con 0x56541813d060 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f602c1a96a0 con 0x565418148240 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f603583e740 10 client.0 _ll_register_callbacks cb 0x7f6024009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f6030a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541814c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f6030a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541814c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52360/0 (socket says 172.21.15.103:52360) 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f6030a00640 1 -- 172.21.15.103:0/804421177 learned_addr learned my addr 172.21.15.103:0/804421177 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f6031201640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x56541814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.617+0000 7f6031a02640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x5654181bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6031201640 1 -- 172.21.15.103:0/804421177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 msgr2=0x5654181bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6031201640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x5654181bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6031201640 1 -- 172.21.15.103:0/804421177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 msgr2=0x56541814c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6031201640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541814c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6031201640 1 -- 172.21.15.103:0/804421177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541813d7b0 con 0x56541813d060 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6031a02640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x5654181bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6030a00640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541814c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6031201640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x56541814c3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x56541813df20 tx=0x7f6028038570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f47aa9fb71c599 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6014ff9640 1 -- 172.21.15.103:0/804421177 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280d7040 con 0x56541813d060 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601c0035f0 con 0x56541813d060 2024-09-19T07:45:43.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6014ff9640 1 -- 172.21.15.103:0/804421177 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f60280c2170 con 0x56541813d060 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6014ff9640 1 -- 172.21.15.103:0/804421177 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f60280c6d80 con 0x56541813d060 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f603583e740 10 client.26537 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5654181c30c0 con 0x56541813d060 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.618+0000 7f6014ff9640 1 -- 172.21.15.103:0/804421177 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c0035f0 con 0x56541813d060 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f6014ff9640 1 -- 172.21.15.103:0/804421177 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f60280ef800 con 0x56541813d060 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f603583e740 10 client.26537 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f603583e740 10 client.26537 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f603583e740 20 client.26537 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f603583e740 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x5654181c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f603583e740 4 client.26537 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "80da2ce4", "release": true, "await": true}] 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "80da2ce4", "release": true, "await": true}) -- 0x5654180aa0a0 con 0x565418157c70 2024-09-19T07:45:43.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f6030a00640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x5654181c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:43.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f6014ff9640 10 client.26537 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:43.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.619+0000 7f6030a00640 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x5654181c5650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b539e001805e4c04 server_cookie=1c92534fa2ee6416 in_seq=0 out_seq=0 2024-09-19T07:45:43.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f6014ff9640 1 -- 172.21.15.103:0/804421177 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5654180aa0a0 con 0x565418157c70 2024-09-19T07:45:43.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f6014ff9640 10 client.26537 handle_command_reply: tid=0 2024-09-19T07:45:43.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 client.26537 shutdown 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 msgr2=0x56541814c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x56541814c3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x56541813df20 tx=0x7f6028038570 comp rx=0 tx=0).stop 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 shutdown_connections 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565418157c70 0x5654181c5650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565418148240 0x5654181bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541813d980 0x56541814c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 --2- 172.21.15.103:0/804421177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56541813d060 0x56541814c3d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 >> 172.21.15.103:0/804421177 conn(0x5654181395f0 msgr2=0x565418138750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f6014ff9640 0 client.26537 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 shutdown_connections 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 1 -- 172.21.15.103:0/804421177 wait complete. 2024-09-19T07:45:43.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.671+0000 7f603583e740 20 client.26537 trim_cache size 0 max 16384 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 40, 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 28, 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "80da2ce4": { 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 28, 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:45:43.674 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:43.675 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:43.676 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:43.676 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:43.676 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:43.676 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:45:43.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60180a33b0 msgr2=0x7f60180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60180a33b0 0x7f60180a5870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6024005fd0 tx=0x7f6024010740 comp rx=0 tx=0).stop 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c101240 msgr2=0x7f602c197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c101240 0x7f602c197370 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6028006010 tx=0x7f6028003b90 comp rx=0 tx=0).stop 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 shutdown_connections 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60180a33b0 0x7f60180a5870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f602c105a90 0x7f602c19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f602c101bc0 0x7f602c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 --2- 172.21.15.103:0/17155964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f602c101240 0x7f602c197370 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 >> 172.21.15.103:0/17155964 conn(0x7f602c077220 msgr2=0x7f602c103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 shutdown_connections 2024-09-19T07:45:43.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:43.672+0000 7f6033465640 1 -- 172.21.15.103:0/17155964 wait complete. 2024-09-19T07:45:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:44 smithi134 ceph-mon[35414]: pgmap v237: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 576 KiB/s wr, 62 op/s 2024-09-19T07:45:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:45:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:45:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:45:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:45:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:44 smithi174 ceph-mon[35370]: pgmap v237: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 576 KiB/s wr, 62 op/s 2024-09-19T07:45:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:45:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:45:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:45:44.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:45:44.417 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:44 smithi103 ceph-mon[31201]: pgmap v237: 97 pgs: 97 active+clean; 4.0 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 576 KiB/s wr, 62 op/s 2024-09-19T07:45:44.417 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:45:44.417 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:45:44.417 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:45:44.417 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:45:44.451 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '80da2ce4', total seconds elapsed: 29.0 2024-09-19T07:45:44.451 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T07:45:44.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:45:44.825 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:45.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:45 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:45:45.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:45 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:45:45.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:45 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:45:45.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 -- 172.21.15.103:0/2539405300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8106f10 msgr2=0x7f60d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 --2- 172.21.15.103:0/2539405300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8106f10 0x7f60d810ea90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009900 tx=0x7f60c802a420 comp rx=0 tx=0).stop 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 -- 172.21.15.103:0/2539405300 shutdown_connections 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 --2- 172.21.15.103:0/2539405300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8106f10 0x7f60d810ea90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 --2- 172.21.15.103:0/2539405300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60d8102790 0x7f60d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 --2- 172.21.15.103:0/2539405300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60d8101de0 0x7f60d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 -- 172.21.15.103:0/2539405300 >> 172.21.15.103:0/2539405300 conn(0x7f60d80fd880 msgr2=0x7f60d80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 -- 172.21.15.103:0/2539405300 shutdown_connections 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.435+0000 7f60dd65c640 1 -- 172.21.15.103:0/2539405300 wait complete. 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 Processor -- start 2024-09-19T07:45:45.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 -- start start 2024-09-19T07:45:45.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60d8101de0 0x7f60d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:45.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8102790 0x7f60d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:45.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60d8106f10 0x7f60d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8111fb0 con 0x7f60d8102790 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8111e30 con 0x7f60d8101de0 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60dd65c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112130 con 0x7f60d8106f10 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60d67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8102790 0x7f60d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60d67fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8102790 0x7f60d8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52378/0 (socket says 172.21.15.103:52378) 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60d67fc640 1 -- 172.21.15.103:0/1767402985 learned_addr learned my addr 172.21.15.103:0/1767402985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60d77fe640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60d8106f10 0x7f60d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.436+0000 7f60d6ffd640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60d8101de0 0x7f60d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d77fe640 1 -- 172.21.15.103:0/1767402985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60d8101de0 msgr2=0x7f60d80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d77fe640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60d8101de0 0x7f60d80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d77fe640 1 -- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8102790 msgr2=0x7f60d8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:45.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d77fe640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8102790 0x7f60d8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d77fe640 1 -- 172.21.15.103:0/1767402985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d81a6910 con 0x7f60d8106f10 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d6ffd640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60d8101de0 0x7f60d80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d67fc640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8102790 0x7f60d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60d77fe640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60d8106f10 0x7f60d8074f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009230 tx=0x7f60c8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60bffff640 1 -- 172.21.15.103:0/1767402985 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c803f030 con 0x7f60d8106f10 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60dd65c640 1 -- 172.21.15.103:0/1767402985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d81a6ba0 con 0x7f60d8106f10 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60dd65c640 1 -- 172.21.15.103:0/1767402985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d81a70d0 con 0x7f60d8106f10 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.437+0000 7f60bffff640 1 -- 172.21.15.103:0/1767402985 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c8004440 con 0x7f60d8106f10 2024-09-19T07:45:45.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.438+0000 7f60bffff640 1 -- 172.21.15.103:0/1767402985 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8035070 con 0x7f60d8106f10 2024-09-19T07:45:45.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.439+0000 7f60dd65c640 1 -- 172.21.15.103:0/1767402985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c4005290 con 0x7f60d8106f10 2024-09-19T07:45:45.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.440+0000 7f60bffff640 1 -- 172.21.15.103:0/1767402985 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f60c8042030 con 0x7f60d8106f10 2024-09-19T07:45:45.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.440+0000 7f60bffff640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c00a3430 0x7f60c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:45.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.441+0000 7f60bffff640 1 -- 172.21.15.103:0/1767402985 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f60c80eff00 con 0x7f60d8106f10 2024-09-19T07:45:45.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.441+0000 7f60d6ffd640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c00a3430 0x7f60c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:45.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.442+0000 7f60d6ffd640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c00a3430 0x7f60c00a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0095f0 tx=0x7f60cc005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:45.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.445+0000 7f60bffff640 1 -- 172.21.15.103:0/1767402985 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f60c80bc280 con 0x7f60d8106f10 2024-09-19T07:45:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 430.0 s, 986.2 tps, lat 538.286 ms stddev 3488.644 2024-09-19T07:45:45.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.763+0000 7f60dd65c640 1 -- 172.21.15.103:0/1767402985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60c4005850 con 0x7f60d8106f10 2024-09-19T07:45:45.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.765+0000 7f60bffff640 1 -- 172.21.15.103:0/1767402985 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11996 (secure 0 0 0) 0x7f60c80c0580 con 0x7f60d8106f10 2024-09-19T07:45:45.767 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:45:45.767 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":40,"btime":"2024-09-19T07:44:20:535666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:44:19.854970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:45:45.767 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 40 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 -- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c00a3430 msgr2=0x7f60c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c00a3430 0x7f60c00a58f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0095f0 tx=0x7f60cc005f90 comp rx=0 tx=0).stop 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 -- 172.21.15.103:0/1767402985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60d8106f10 msgr2=0x7f60d8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60d8106f10 0x7f60d8074f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009230 tx=0x7f60c8004040 comp rx=0 tx=0).stop 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 -- 172.21.15.103:0/1767402985 shutdown_connections 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c00a3430 0x7f60c00a58f0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0095f0 tx=0x7f60cc005f90 comp rx=0 tx=0).stop 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60d8106f10 0x7f60d8074f70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d8102790 0x7f60d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 --2- 172.21.15.103:0/1767402985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60d8101de0 0x7f60d80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.770+0000 7f60bdffb640 1 -- 172.21.15.103:0/1767402985 >> 172.21.15.103:0/1767402985 conn(0x7f60d80fd880 msgr2=0x7f60d80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:45.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.771+0000 7f60bdffb640 1 -- 172.21.15.103:0/1767402985 shutdown_connections 2024-09-19T07:45:45.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:45.773+0000 7f60bdffb640 1 -- 172.21.15.103:0/1767402985 wait complete. 2024-09-19T07:45:46.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:46 smithi134 ceph-mon[35414]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 940 KiB/s wr, 103 op/s 2024-09-19T07:45:46.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:46 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1767402985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:46 smithi174 ceph-mon[35370]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 940 KiB/s wr, 103 op/s 2024-09-19T07:45:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:46 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1767402985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:46.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:46 smithi103 ceph-mon[31201]: pgmap v238: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 940 KiB/s wr, 103 op/s 2024-09-19T07:45:46.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:46 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1767402985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:45:46.459 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T07:45:46.460 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T07:45:46.823 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:45:47.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/2846453699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 msgr2=0x7fa8f010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/2846453699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 0x7fa8f010ea20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009ca0 tx=0x7fa8e802a8d0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/2846453699 shutdown_connections 2024-09-19T07:45:47.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/2846453699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 0x7fa8f010ea20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/2846453699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8f0100100 0x7fa8f0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/2846453699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8f0106b80 0x7fa8f0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/2846453699 >> 172.21.15.103:0/2846453699 conn(0x7fa8f0077230 msgr2=0x7fa8f00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:47.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/2846453699 shutdown_connections 2024-09-19T07:45:47.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.401+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/2846453699 wait complete. 2024-09-19T07:45:47.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.402+0000 7fa8f6aa3640 1 Processor -- start 2024-09-19T07:45:47.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.402+0000 7fa8f6aa3640 1 -- start start 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.402+0000 7fa8f6aa3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8f0100100 0x7fa8f01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f6aa3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 0x7fa8f0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f6aa3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8f0106b80 0x7fa8f019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f6aa3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0112090 con 0x7fa8f0100100 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f6aa3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0111f10 con 0x7fa8f0106b80 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f6aa3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0112210 con 0x7fa8f0100aa0 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f5040640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8f0106b80 0x7fa8f019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f5040640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8f0106b80 0x7fa8f019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:46818/0 (socket says 172.21.15.103:46818) 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8f5040640 1 -- 172.21.15.103:0/3940478922 learned_addr learned my addr 172.21.15.103:0/3940478922 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:47.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.403+0000 7fa8effff640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 0x7fa8f0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8effff640 1 -- 172.21.15.103:0/3940478922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8f0106b80 msgr2=0x7fa8f019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8effff640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8f0106b80 0x7fa8f019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8effff640 1 -- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8f0100100 msgr2=0x7fa8f01985f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8effff640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8f0100100 0x7fa8f01985f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8effff640 1 -- 172.21.15.103:0/3940478922 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f019d5c0 con 0x7fa8f0100aa0 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8effff640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 0x7fa8f0198b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d6d0 tx=0x7fa8e400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8edffb640 1 -- 172.21.15.103:0/3940478922 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4017070 con 0x7fa8f0100aa0 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8edffb640 1 -- 172.21.15.103:0/3940478922 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e4009820 con 0x7fa8f0100aa0 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8edffb640 1 -- 172.21.15.103:0/3940478922 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4004c40 con 0x7fa8f0100aa0 2024-09-19T07:45:47.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f019d8b0 con 0x7fa8f0100aa0 2024-09-19T07:45:47.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.404+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f0101210 con 0x7fa8f0100aa0 2024-09-19T07:45:47.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.408+0000 7fa8f8e7c740 1 Processor -- start 2024-09-19T07:45:47.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.408+0000 7fa8f8e7c740 1 -- start start 2024-09-19T07:45:47.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df95c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016dfa0a30 0x56016dfa2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56016defa160 con 0x56016dfa0a30 2024-09-19T07:45:47.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56016dea1720 con 0x56016df95850 2024-09-19T07:45:47.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56016def6120 con 0x56016df96170 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f483f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f5040640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df95c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f5040640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df95c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:46834/0 (socket says 172.21.15.103:46834) 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.409+0000 7fa8f5040640 1 -- 172.21.15.103:0/1129868721 learned_addr learned my addr 172.21.15.103:0/1129868721 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8f5040640 1 -- 172.21.15.103:0/1129868721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 msgr2=0x56016dfa04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8f5040640 1 --2- 172.21.15.103:0/1129868721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8f5040640 1 -- 172.21.15.103:0/1129868721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016dfa0a30 msgr2=0x56016dfa2e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8f5040640 1 --2- 172.21.15.103:0/1129868721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016dfa0a30 0x56016dfa2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8f5040640 1 -- 172.21.15.103:0/1129868721 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016dc6bba0 con 0x56016df95850 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8edffb640 1 -- 172.21.15.103:0/3940478922 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa8e4005220 con 0x7fa8f0100aa0 2024-09-19T07:45:47.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8f5040640 1 --2- 172.21.15.103:0/1129868721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df95c30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009960 tx=0x7fa8e8009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89cfda1b32b335b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:47.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8d6ffd640 1 -- 172.21.15.103:0/1129868721 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8031e00 con 0x56016df95850 2024-09-19T07:45:47.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8d6ffd640 1 -- 172.21.15.103:0/1129868721 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e8036030 con 0x56016df95850 2024-09-19T07:45:47.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.410+0000 7fa8d6ffd640 1 -- 172.21.15.103:0/1129868721 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8004440 con 0x56016df95850 2024-09-19T07:45:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8edffb640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8d80a3500 0x7fa8d80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/1129868721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 msgr2=0x56016df95c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/1129868721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df95c30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009960 tx=0x7fa8e8009990 comp rx=0 tx=0).stop 2024-09-19T07:45:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/1129868721 shutdown_connections 2024-09-19T07:45:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/1129868721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016dfa0a30 0x56016dfa2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/1129868721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/1129868721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df95c30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8edffb640 1 -- 172.21.15.103:0/3940478922 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fa8e40cf900 con 0x7fa8f0100aa0 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.411+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/1129868721 >> 172.21.15.103:0/1129868721 conn(0x56016df91de0 msgr2=0x56016df921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.412+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/1129868721 shutdown_connections 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.412+0000 7fa8f483f640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8d80a3500 0x7fa8d80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.412+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/1129868721 wait complete. 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.412+0000 7fa8f8e7c740 1 -- start start 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.412+0000 7fa8f8e7c740 1 Processor -- start 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f483f640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8d80a3500 0x7fa8d80a59c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x56016df92c40 tx=0x7fa8e0009060 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016df9d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df96170 0x56016df9def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016df9e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56016dfa3600 con 0x56016df95850 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56016de82900 con 0x56016df96170 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56016dfa3780 con 0x56016dfa0a30 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f5040640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016df9e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f5040640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016df9e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:59668/0 (socket says 172.21.15.103:59668) 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8f5040640 1 -- 172.21.15.103:0/150172685 learned_addr learned my addr 172.21.15.103:0/150172685 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.413+0000 7fa8effff640 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df96170 0x56016df9def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8f8e7c740 10 client.0 _ll_register_callbacks cb 0x56016de80780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8f5040640 1 -- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df96170 msgr2=0x56016df9def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8f5040640 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df96170 0x56016df9def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8f5040640 1 -- 172.21.15.103:0/150172685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 msgr2=0x56016df9d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8f5040640 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016df9d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8f5040640 1 -- 172.21.15.103:0/150172685 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016dec2c50 con 0x56016dfa0a30 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8f5040640 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016df9e430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009bf0 tx=0x7fa8e802f490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44bc3bec31146352 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.414+0000 7fa8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.415+0000 7fa8cffff640 1 -- 172.21.15.103:0/150172685 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8004140 con 0x56016dfa0a30 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.415+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e8036030 con 0x56016dfa0a30 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.415+0000 7fa8cffff640 1 -- 172.21.15.103:0/150172685 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fa8e80042e0 con 0x56016dfa0a30 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.415+0000 7fa8cffff640 1 -- 172.21.15.103:0/150172685 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa8e8004520 con 0x56016dfa0a30 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.415+0000 7fa8f8e7c740 10 client.26458 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.415+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56016dfa3600 con 0x56016dfa0a30 2024-09-19T07:45:47.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.415+0000 7fa8cffff640 1 -- 172.21.15.103:0/150172685 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e8036030 con 0x56016dfa0a30 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8cffff640 1 -- 172.21.15.103:0/150172685 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fa8e804e070 con 0x56016dfa0a30 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8f8e7c740 10 client.26458 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8f8e7c740 10 client.26458 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8f8e7c740 20 client.26458 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016dfb0690 0x56016dfb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8f8e7c740 4 client.26458 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56016df07890 con 0x56016dfb0690 2024-09-19T07:45:47.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.416+0000 7fa8f483f640 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016dfb0690 0x56016dfb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.417+0000 7fa8cffff640 10 client.26458 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.417+0000 7fa8f483f640 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016dfb0690 0x56016dfb2b00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749f34aeadfc8775 server_cookie=8b69addde08736b8 in_seq=0 out_seq=0 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8cffff640 1 -- 172.21.15.103:0/150172685 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56016df07890 con 0x56016dfb0690 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8cffff640 10 client.26458 handle_command_reply: tid=0 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 client.26458 shutdown 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 msgr2=0x56016df9e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016df9e430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009bf0 tx=0x7fa8e802f490 comp rx=0 tx=0).stop 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 shutdown_connections 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016dfb0690 0x56016dfb2b00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016df9e430 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df96170 0x56016df9def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/150172685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016df9d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 >> 172.21.15.103:0/150172685 conn(0x56016df91de0 msgr2=0x56016df97b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:47.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.418+0000 7fa8cffff640 0 client.26458 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:47.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.419+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 shutdown_connections 2024-09-19T07:45:47.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.419+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/150172685 wait complete. 2024-09-19T07:45:47.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.419+0000 7fa8f8e7c740 20 client.26458 trim_cache size 0 max 16384 2024-09-19T07:45:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 Processor -- start 2024-09-19T07:45:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 -- start start 2024-09-19T07:45:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df9c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df96170 0x56016dfb8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016dfb88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b3ee0 con 0x56016df96170 2024-09-19T07:45:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df96170 0x56016dfb8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b5a20 con 0x56016df95850 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b45b0 con 0x56016dfa0a30 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8effff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df96170 0x56016dfb8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52406/0 (socket says 172.21.15.103:52406) 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.455+0000 7fa8effff640 1 -- 172.21.15.103:0/2697865730 learned_addr learned my addr 172.21.15.103:0/2697865730 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.456+0000 7fa8f5040640 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016dfb88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.457+0000 7fa8effff640 1 -- 172.21.15.103:0/2697865730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 msgr2=0x56016dfb88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.457+0000 7fa8effff640 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016dfb88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.457+0000 7fa8effff640 1 -- 172.21.15.103:0/2697865730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 msgr2=0x56016df9c930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.457+0000 7fa8effff640 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df9c930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.457+0000 7fa8effff640 1 -- 172.21.15.103:0/2697865730 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016df95fa0 con 0x56016df96170 2024-09-19T07:45:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.457+0000 7fa8effff640 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df96170 0x56016dfb8370 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d6a0 tx=0x7fa8e409c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67e690ed04f81a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.460+0000 7fa8d67fc640 1 -- 172.21.15.103:0/2697865730 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4020040 con 0x56016df96170 2024-09-19T07:45:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.460+0000 7fa8d67fc640 1 -- 172.21.15.103:0/2697865730 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e40a47e0 con 0x56016df96170 2024-09-19T07:45:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.460+0000 7fa8d67fc640 1 -- 172.21.15.103:0/2697865730 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40a4350 con 0x56016df96170 2024-09-19T07:45:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.462+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/2697865730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df96170 msgr2=0x56016dfb8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.462+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df96170 0x56016dfb8370 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d6a0 tx=0x7fa8e409c330 comp rx=0 tx=0).stop 2024-09-19T07:45:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.464+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/2697865730 shutdown_connections 2024-09-19T07:45:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.464+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016dfa0a30 0x56016dfb88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.464+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df96170 0x56016dfb8370 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.464+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/2697865730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016df95850 0x56016df9c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.464+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/2697865730 >> 172.21.15.103:0/2697865730 conn(0x56016df91de0 msgr2=0x56016df97b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.466+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/2697865730 shutdown_connections 2024-09-19T07:45:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.466+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/2697865730 wait complete. 2024-09-19T07:45:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.466+0000 7fa8f8e7c740 1 -- start start 2024-09-19T07:45:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.466+0000 7fa8f8e7c740 1 Processor -- start 2024-09-19T07:45:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016e016b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f8e7c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016dfa0a30 0x56016dfa8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b5350 con 0x56016df95850 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b4c80 con 0x56016dfa0a30 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f8e7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01b1ee0 con 0x56016df96170 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f8e7c740 10 client.0 _ll_register_callbacks cb 0x7fa8e0006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f483f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016e016b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f483f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016e016b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52408/0 (socket says 172.21.15.103:52408) 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f483f640 1 -- 172.21.15.103:0/4057792044 learned_addr learned my addr 172.21.15.103:0/4057792044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8effff640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.467+0000 7fa8f5040640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016dfa0a30 0x56016dfa8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8f5040640 1 -- 172.21.15.103:0/4057792044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 msgr2=0x56016dfa89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8f5040640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8f5040640 1 -- 172.21.15.103:0/4057792044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 msgr2=0x56016e016b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8f5040640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016e016b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8f5040640 1 -- 172.21.15.103:0/4057792044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016df9a860 con 0x56016dfa0a30 2024-09-19T07:45:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8f483f640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016e016b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:45:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8f5040640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016dfa0a30 0x56016dfa8f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8030040 tx=0x7fa8e8005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb85959630361f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:45:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:45:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8cffff640 1 -- 172.21.15.103:0/4057792044 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8031670 con 0x56016dfa0a30 2024-09-19T07:45:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.468+0000 7fa8cffff640 1 -- 172.21.15.103:0/4057792044 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fa8e8031810 con 0x56016dfa0a30 2024-09-19T07:45:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.469+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e40a47e0 con 0x56016dfa0a30 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.469+0000 7fa8cffff640 1 -- 172.21.15.103:0/4057792044 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa8e8030b60 con 0x56016dfa0a30 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.469+0000 7fa8f8e7c740 10 client.26552 fetch_fsmap learned FSMap version 40 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.469+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56016e02c440 con 0x56016dfa0a30 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8cffff640 1 -- 172.21.15.103:0/4057792044 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e40a47e0 con 0x56016dfa0a30 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8cffff640 1 -- 172.21.15.103:0/4057792044 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fa8e8043070 con 0x56016dfa0a30 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8f8e7c740 10 client.26552 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8f8e7c740 10 client.26552 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8f8e7c740 20 client.26552 populate_metadata read hostname 'smithi103' 2024-09-19T07:45:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016e02ca70 0x56016e02ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:45:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8f8e7c740 4 client.26552 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T07:45:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x56016df01be0 con 0x56016e02ca70 2024-09-19T07:45:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.470+0000 7fa8f483f640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016e02ca70 0x56016e02ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:45:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.471+0000 7fa8cffff640 10 client.26552 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.471+0000 7fa8f483f640 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016e02ca70 0x56016e02ee90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef263fd9b3c533b server_cookie=8f6649b1cb49c217 in_seq=0 out_seq=0 2024-09-19T07:45:47.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8cffff640 1 -- 172.21.15.103:0/4057792044 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56016df01be0 con 0x56016e02ca70 2024-09-19T07:45:47.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8cffff640 10 client.26552 handle_command_reply: tid=0 2024-09-19T07:45:47.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 client.26552 shutdown 2024-09-19T07:45:47.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016dfa0a30 msgr2=0x56016dfa8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016dfa0a30 0x56016dfa8f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8030040 tx=0x7fa8e8005c50 comp rx=0 tx=0).stop 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 shutdown_connections 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56016e02ca70 0x56016e02ee90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56016dfa0a30 0x56016dfa8f10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56016df96170 0x56016dfa89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8cffff640 0 client.26552 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 --2- 172.21.15.103:0/4057792044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56016df95850 0x56016e016b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 >> 172.21.15.103:0/4057792044 conn(0x56016df91de0 msgr2=0x56016df972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 shutdown_connections 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 1 -- 172.21.15.103:0/4057792044 wait complete. 2024-09-19T07:45:47.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.641+0000 7fa8f8e7c740 20 client.26552 trim_cache size 0 max 16384 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 40, 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 30, 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "868342b1": { 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 30, 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:45:47.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:45:47.645 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:45:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:47.646 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:45:47.646 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:45:47.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8d80a3500 msgr2=0x7fa8d80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8d80a3500 0x7fa8d80a59c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x56016df92c40 tx=0x7fa8e0009060 comp rx=0 tx=0).stop 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 msgr2=0x7fa8f0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 0x7fa8f0198b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d6d0 tx=0x7fa8e400db90 comp rx=0 tx=0).stop 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 shutdown_connections 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8d80a3500 0x7fa8d80a59c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8f0106b80 0x7fa8f019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f0100aa0 0x7fa8f0198b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 --2- 172.21.15.103:0/3940478922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8f0100100 0x7fa8f01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 >> 172.21.15.103:0/3940478922 conn(0x7fa8f0077230 msgr2=0x7fa8f0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 shutdown_connections 2024-09-19T07:45:47.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:45:47.642+0000 7fa8f6aa3640 1 -- 172.21.15.103:0/3940478922 wait complete. 2024-09-19T07:45:48.300 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '868342b1', quiesce took 3.8 seconds. Will release after: 21.95 2024-09-19T07:45:48.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:48 smithi134 ceph-mon[35414]: pgmap v239: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 512 KiB/s wr, 62 op/s 2024-09-19T07:45:48.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:48 smithi174 ceph-mon[35370]: pgmap v239: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 512 KiB/s wr, 62 op/s 2024-09-19T07:45:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:48 smithi103 ceph-mon[31201]: pgmap v239: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 512 KiB/s wr, 62 op/s 2024-09-19T07:45:50.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:50 smithi134 ceph-mon[35414]: pgmap v240: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 446 KiB/s wr, 55 op/s 2024-09-19T07:45:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:50 smithi174 ceph-mon[35370]: pgmap v240: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 446 KiB/s wr, 55 op/s 2024-09-19T07:45:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 435.0 s, 996.3 tps, lat 40.168 ms stddev 40.778 2024-09-19T07:45:50.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:50 smithi103 ceph-mon[31201]: pgmap v240: 97 pgs: 97 active+clean; 4.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 446 KiB/s wr, 55 op/s 2024-09-19T07:45:52.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:52 smithi134 ceph-mon[35414]: pgmap v241: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.2 MiB/s wr, 242 op/s 2024-09-19T07:45:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:52 smithi174 ceph-mon[35370]: pgmap v241: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.2 MiB/s wr, 242 op/s 2024-09-19T07:45:52.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:52 smithi103 ceph-mon[31201]: pgmap v241: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 2.2 MiB/s wr, 242 op/s 2024-09-19T07:45:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:54 smithi174 ceph-mon[35370]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.1 MiB/s wr, 236 op/s 2024-09-19T07:45:54.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:54 smithi103 ceph-mon[31201]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.1 MiB/s wr, 236 op/s 2024-09-19T07:45:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:54 smithi134 ceph-mon[35414]: pgmap v242: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.1 MiB/s wr, 236 op/s 2024-09-19T07:45:55.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 440.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:45:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:56 smithi174 ceph-mon[35370]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-19T07:45:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:56 smithi103 ceph-mon[31201]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-19T07:45:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:56 smithi134 ceph-mon[35414]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-19T07:45:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:45:58 smithi174 ceph-mon[35370]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-09-19T07:45:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:45:58 smithi103 ceph-mon[31201]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-09-19T07:45:58.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:45:58 smithi134 ceph-mon[35414]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-09-19T07:46:00.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:00 smithi134 ceph-mon[35414]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-19T07:46:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:00 smithi174 ceph-mon[35370]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-19T07:46:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:00 smithi103 ceph-mon[31201]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-19T07:46:00.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 445.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:46:02.361 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:02 smithi103 ceph-mon[31201]: pgmap v246: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-09-19T07:46:02.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:02 smithi174 ceph-mon[35370]: pgmap v246: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-09-19T07:46:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:02 smithi134 ceph-mon[35414]: pgmap v246: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-09-19T07:46:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:04 smithi174 ceph-mon[35370]: pgmap v247: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 255 KiB/s wr, 40 op/s 2024-09-19T07:46:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:04 smithi103 ceph-mon[31201]: pgmap v247: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 255 KiB/s wr, 40 op/s 2024-09-19T07:46:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:04 smithi134 ceph-mon[35414]: pgmap v247: 97 pgs: 97 active+clean; 4.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 255 KiB/s wr, 40 op/s 2024-09-19T07:46:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:05 smithi174 ceph-mon[35370]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T07:46:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:05 smithi103 ceph-mon[31201]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T07:46:05.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 450.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:46:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:05 smithi134 ceph-mon[35414]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T07:46:06.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:06 smithi174 ceph-mon[35370]: pgmap v249: 97 pgs: 97 active+clean; 3.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 151 KiB/s wr, 43 op/s 2024-09-19T07:46:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:06 smithi103 ceph-mon[31201]: pgmap v249: 97 pgs: 97 active+clean; 3.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 151 KiB/s wr, 43 op/s 2024-09-19T07:46:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:06 smithi134 ceph-mon[35414]: pgmap v249: 97 pgs: 97 active+clean; 3.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 151 KiB/s wr, 43 op/s 2024-09-19T07:46:08.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:08 smithi174 ceph-mon[35370]: pgmap v250: 97 pgs: 97 active+clean; 3.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-19T07:46:08.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:46:08.467 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:08 smithi103 ceph-mon[31201]: pgmap v250: 97 pgs: 97 active+clean; 3.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-19T07:46:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:08 smithi134 ceph-mon[35414]: pgmap v250: 97 pgs: 97 active+clean; 3.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-19T07:46:08.703 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:09.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 -- 172.21.15.103:0/2795192245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c102790 msgr2=0x7f780c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2795192245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c102790 0x7f780c1067e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009900 tx=0x7f77f802a410 comp rx=0 tx=0).stop 2024-09-19T07:46:09.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 -- 172.21.15.103:0/2795192245 shutdown_connections 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2795192245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f780c106f10 0x7f780c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2795192245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c102790 0x7f780c1067e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2795192245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c101de0 0x7f780c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 -- 172.21.15.103:0/2795192245 >> 172.21.15.103:0/2795192245 conn(0x7f780c0fd8c0 msgr2=0x7f780c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 -- 172.21.15.103:0/2795192245 shutdown_connections 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.285+0000 7f7812bc1640 1 -- 172.21.15.103:0/2795192245 wait complete. 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 Processor -- start 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 -- start start 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f780c101de0 0x7f780c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c102790 0x7f780c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c106f10 0x7f780c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f780c111f70 con 0x7f780c106f10 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f780c111df0 con 0x7f780c101de0 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f7812bc1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1120f0 con 0x7f780c102790 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f781115e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c106f10 0x7f780c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f781115e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c106f10 0x7f780c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43376/0 (socket says 172.21.15.103:43376) 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f781115e640 1 -- 172.21.15.103:0/2940061043 learned_addr learned my addr 172.21.15.103:0/2940061043 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.286+0000 7f781095d640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f780c101de0 0x7f780c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f780bfff640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c102790 0x7f780c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f781115e640 1 -- 172.21.15.103:0/2940061043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c102790 msgr2=0x7f780c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f781115e640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c102790 0x7f780c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f781115e640 1 -- 172.21.15.103:0/2940061043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f780c101de0 msgr2=0x7f780c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f781115e640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f780c101de0 0x7f780c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f781115e640 1 -- 172.21.15.103:0/2940061043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c19d380 con 0x7f780c106f10 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f781115e640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c106f10 0x7f780c19cca0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f780400cce0 tx=0x7f780400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f7809ffb640 1 -- 172.21.15.103:0/2940061043 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780400f450 con 0x7f780c106f10 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c19d670 con 0x7f780c106f10 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f7809ffb640 1 -- 172.21.15.103:0/2940061043 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780400fad0 con 0x7f780c106f10 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f7809ffb640 1 -- 172.21.15.103:0/2940061043 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804018c10 con 0x7f780c106f10 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.287+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c1a4f50 con 0x7f780c106f10 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.289+0000 7f7809ffb640 1 -- 172.21.15.103:0/2940061043 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f780400f5f0 con 0x7f780c106f10 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 Processor -- start 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 -- start start 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7809ffb640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f77fc0a33b0 0x7f77fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e6584a0 0x55d97e658880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e658dc0 0x55d97e663140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e663680 0x55d97e665a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d97e5be2e0 con 0x55d97e6584a0 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d97e565ce0 con 0x55d97e663680 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7814f9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d97e5ba2a0 con 0x55d97e658dc0 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f7809ffb640 1 -- 172.21.15.103:0/2940061043 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8586+0+0 (secure 0 0 0) 0x7f78040d0390 con 0x7f780c106f10 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f780bfff640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f77fc0a33b0 0x7f77fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f781095d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e6584a0 0x55d97e658880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f781095d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e6584a0 0x55d97e658880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43388/0 (socket says 172.21.15.103:43388) 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f781095d640 1 -- 172.21.15.103:0/873314560 learned_addr learned my addr 172.21.15.103:0/873314560 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f780bfff640 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e658dc0 0x55d97e663140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.290+0000 7f781115e640 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e663680 0x55d97e665a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f781095d640 1 -- 172.21.15.103:0/873314560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e658dc0 msgr2=0x55d97e663140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f781095d640 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e658dc0 0x55d97e663140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f781095d640 1 -- 172.21.15.103:0/873314560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e663680 msgr2=0x55d97e665a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f781095d640 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e663680 0x55d97e665a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f781095d640 1 -- 172.21.15.103:0/873314560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97e54a440 con 0x55d97e6584a0 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f780bfff640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f77fc0a33b0 0x7f77fc0a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009230 tx=0x7f77f80318e0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f781095d640 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e6584a0 0x55d97e658880 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f780000b760 tx=0x7f780000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28541b9eb272c3fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f77f6ffd640 1 -- 172.21.15.103:0/873314560 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800003d50 con 0x55d97e6584a0 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f77f6ffd640 1 -- 172.21.15.103:0/873314560 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7800015070 con 0x55d97e6584a0 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.291+0000 7f77f6ffd640 1 -- 172.21.15.103:0/873314560 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800004de0 con 0x55d97e6584a0 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 -- 172.21.15.103:0/873314560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e6584a0 msgr2=0x55d97e658880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e6584a0 0x55d97e658880 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f780000b760 tx=0x7f780000bc20 comp rx=0 tx=0).stop 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 -- 172.21.15.103:0/873314560 shutdown_connections 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e663680 0x55d97e665a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e658dc0 0x55d97e663140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 --2- 172.21.15.103:0/873314560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e6584a0 0x55d97e658880 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 -- 172.21.15.103:0/873314560 >> 172.21.15.103:0/873314560 conn(0x55d97e654a30 msgr2=0x55d97e654e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 -- 172.21.15.103:0/873314560 shutdown_connections 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 -- 172.21.15.103:0/873314560 wait complete. 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 -- start start 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.292+0000 7f7814f9a740 1 Processor -- start 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 0x55d97e662fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e660250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e663680 0x55d97e660790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d97e666250 con 0x55d97e663680 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d97e579050 con 0x55d97e658dc0 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d97e6663d0 con 0x55d97e6584a0 2024-09-19T07:46:09.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 10 client.0 _ll_register_callbacks cb 0x55d97e5b9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e663680 0x55d97e660790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e663680 0x55d97e660790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43400/0 (socket says 172.21.15.103:43400) 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 -- 172.21.15.103:0/2046083225 learned_addr learned my addr 172.21.15.103:0/2046083225 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f780bfff640 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e660250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 -- 172.21.15.103:0/2046083225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 msgr2=0x55d97e662fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 0x55d97e662fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 -- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 msgr2=0x55d97e660250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e660250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 -- 172.21.15.103:0/2046083225 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97e32f3c0 con 0x55d97e663680 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f780bfff640 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e660250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f781115e640 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e663680 0x55d97e660790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f780400ab20 tx=0x7f780409bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db22373e5a2c64d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:09.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f77ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f77ebfff640 1 -- 172.21.15.103:0/2046083225 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78040ba060 con 0x55d97e663680 2024-09-19T07:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f77ebfff640 1 -- 172.21.15.103:0/2046083225 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8586+0+0 (secure 0 0 0) 0x7f78040a4db0 con 0x55d97e663680 2024-09-19T07:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.293+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7800015070 con 0x55d97e663680 2024-09-19T07:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f77ebfff640 1 -- 172.21.15.103:0/2046083225 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f78040b5070 con 0x55d97e663680 2024-09-19T07:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 10 client.17091 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d97e671c30 con 0x55d97e663680 2024-09-19T07:46:09.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f77ebfff640 1 -- 172.21.15.103:0/2046083225 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7800015070 con 0x55d97e663680 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f77ebfff640 1 -- 172.21.15.103:0/2046083225 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f780409ba20 con 0x55d97e663680 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 10 client.17091 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 20 client.17091 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e6730f0 0x55d97e675520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 4 client.17091 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d97e5cb850 con 0x55d97e6730f0 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.294+0000 7f781095d640 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e6730f0 0x55d97e675520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f781095d640 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e6730f0 0x55d97e675520 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f58079b1b593471 server_cookie=4ec5b68f71938b61 in_seq=0 out_seq=0 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f77ebfff640 10 client.17091 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f77ebfff640 1 -- 172.21.15.103:0/2046083225 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d97e5cb850 con 0x55d97e6730f0 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f77ebfff640 10 client.17091 handle_command_reply: tid=0 2024-09-19T07:46:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 client.17091 shutdown 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e663680 msgr2=0x55d97e660790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e663680 0x55d97e660790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f780400ab20 tx=0x7f780409bf30 comp rx=0 tx=0).stop 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 shutdown_connections 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e6730f0 0x55d97e675520 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e663680 0x55d97e660790 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e660250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f77ebfff640 0 client.17091 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 --2- 172.21.15.103:0/2046083225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 0x55d97e662fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.295+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 >> 172.21.15.103:0/2046083225 conn(0x55d97e654a30 msgr2=0x55d97e65a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.296+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 shutdown_connections 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.296+0000 7f7814f9a740 1 -- 172.21.15.103:0/2046083225 wait complete. 2024-09-19T07:46:09.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.296+0000 7f7814f9a740 20 client.17091 trim_cache size 0 max 16384 2024-09-19T07:46:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.312+0000 7f7814f9a740 1 Processor -- start 2024-09-19T07:46:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.312+0000 7f7814f9a740 1 -- start start 2024-09-19T07:46:09.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e654a30 0x55d97e6621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e6584a0 0x55d97e67adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e658dc0 0x55d97e67b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f7814f9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1ad8c0 con 0x55d97e658dc0 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f7814f9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1af400 con 0x55d97e6584a0 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f7814f9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1adf90 con 0x55d97e654a30 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e658dc0 0x55d97e67b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e658dc0 0x55d97e67b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43414/0 (socket says 172.21.15.103:43414) 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 -- 172.21.15.103:0/3088494322 learned_addr learned my addr 172.21.15.103:0/3088494322 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f780bfff640 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e6584a0 0x55d97e67adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781095d640 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e654a30 0x55d97e6621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 -- 172.21.15.103:0/3088494322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e654a30 msgr2=0x55d97e6621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e654a30 0x55d97e6621c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 -- 172.21.15.103:0/3088494322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e6584a0 msgr2=0x55d97e67adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e6584a0 0x55d97e67adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 -- 172.21.15.103:0/3088494322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97e65e7a0 con 0x55d97e658dc0 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f780bfff640 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e6584a0 0x55d97e67adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f781115e640 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e658dc0 0x55d97e67b330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f780409c560 tx=0x7f780409c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5108cdeaedec20a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.313+0000 7f77f67fc640 1 -- 172.21.15.103:0/3088494322 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780409ccd0 con 0x55d97e658dc0 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f77f67fc640 1 -- 172.21.15.103:0/3088494322 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78040a0980 con 0x55d97e658dc0 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f77f67fc640 1 -- 172.21.15.103:0/3088494322 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804098c40 con 0x55d97e658dc0 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 -- 172.21.15.103:0/3088494322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e658dc0 msgr2=0x55d97e67b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e658dc0 0x55d97e67b330 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f780409c560 tx=0x7f780409c060 comp rx=0 tx=0).stop 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 -- 172.21.15.103:0/3088494322 shutdown_connections 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e658dc0 0x55d97e67b330 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e6584a0 0x55d97e67adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 --2- 172.21.15.103:0/3088494322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e654a30 0x55d97e6621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 -- 172.21.15.103:0/3088494322 >> 172.21.15.103:0/3088494322 conn(0x55d97e6608f0 msgr2=0x55d97e661ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 -- 172.21.15.103:0/3088494322 shutdown_connections 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.314+0000 7f7814f9a740 1 -- 172.21.15.103:0/3088494322 wait complete. 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 -- start start 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 Processor -- start 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e654a30 0x55d97e669510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 0x55d97e669a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e669f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1aed30 con 0x55d97e654a30 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1ae660 con 0x55d97e658dc0 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1ab8c0 con 0x55d97e6584a0 2024-09-19T07:46:09.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f7814f9a740 10 client.0 _ll_register_callbacks cb 0x7f7804098200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e654a30 0x55d97e669510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e654a30 0x55d97e669510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43416/0 (socket says 172.21.15.103:43416) 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 -- 172.21.15.103:0/1634514358 learned_addr learned my addr 172.21.15.103:0/1634514358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781115e640 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e669f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 -- 172.21.15.103:0/1634514358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 msgr2=0x55d97e669a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 0x55d97e669a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 -- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 msgr2=0x55d97e669f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e669f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.315+0000 7f781095d640 1 -- 172.21.15.103:0/1634514358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97e6dcf70 con 0x55d97e654a30 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f781095d640 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e654a30 0x55d97e669510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f780000bc60 tx=0x7f7800004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e124593cb0c61e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f77ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f77ebfff640 1 -- 172.21.15.103:0/1634514358 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800003e80 con 0x55d97e654a30 2024-09-19T07:46:09.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f77ebfff640 1 -- 172.21.15.103:0/1634514358 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8586+0+0 (secure 0 0 0) 0x7f7800025430 con 0x55d97e654a30 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78040a0980 con 0x55d97e654a30 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f77ebfff640 1 -- 172.21.15.103:0/1634514358 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7800034050 con 0x55d97e654a30 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 10 client.17097 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d97e6de280 con 0x55d97e654a30 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f77ebfff640 1 -- 172.21.15.103:0/1634514358 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78040a0980 con 0x55d97e654a30 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f77ebfff640 1 -- 172.21.15.103:0/1634514358 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f7800011790 con 0x55d97e654a30 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 10 client.17097 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 20 client.17097 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e663680 0x55d97e6e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 4 client.17097 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.316+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d97e5c5d60 con 0x55d97e663680 2024-09-19T07:46:09.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f780bfff640 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e663680 0x55d97e6e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f780bfff640 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e663680 0x55d97e6e0760 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f617d66dd5a2145d server_cookie=97b9d9fa76469916 in_seq=0 out_seq=0 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f77ebfff640 10 client.17097 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f77ebfff640 1 -- 172.21.15.103:0/1634514358 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d97e5c5d60 con 0x55d97e663680 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f77ebfff640 10 client.17097 handle_command_reply: tid=0 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 client.17097 shutdown 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e654a30 msgr2=0x55d97e669510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e654a30 0x55d97e669510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f780000bc60 tx=0x7f7800004920 comp rx=0 tx=0).stop 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 shutdown_connections 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d97e663680 0x55d97e6e0760 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d97e658dc0 0x55d97e669f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97e6584a0 0x55d97e669a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f77ebfff640 0 client.17097 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:09.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 --2- 172.21.15.103:0/1634514358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d97e654a30 0x55d97e669510 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.317+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 >> 172.21.15.103:0/1634514358 conn(0x55d97e6608f0 msgr2=0x55d97e6622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:09.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 shutdown_connections 2024-09-19T07:46:09.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7814f9a740 1 -- 172.21.15.103:0/1634514358 wait complete. 2024-09-19T07:46:09.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7814f9a740 20 client.17097 trim_cache size 0 max 16384 2024-09-19T07:46:09.332 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:46:09.332 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:46:09.333 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:46:09.333 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:46:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f77fc0a33b0 msgr2=0x7f77fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f77fc0a33b0 0x7f77fc0a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009230 tx=0x7f77f80318e0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c106f10 msgr2=0x7f780c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c106f10 0x7f780c19cca0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f780400cce0 tx=0x7f780400a620 comp rx=0 tx=0).stop 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 shutdown_connections 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f77fc0a33b0 0x7f77fc0a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f780c106f10 0x7f780c19cca0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c102790 0x7f780c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 --2- 172.21.15.103:0/2940061043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f780c101de0 0x7f780c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 >> 172.21.15.103:0/2940061043 conn(0x7f780c0fd8c0 msgr2=0x7f780c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 shutdown_connections 2024-09-19T07:46:09.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:09.318+0000 7f7812bc1640 1 -- 172.21.15.103:0/2940061043 wait complete. 2024-09-19T07:46:10.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:46:10.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:46:10.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:46:10.408 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:10 smithi174 ceph-mon[35370]: pgmap v251: 97 pgs: 97 active+clean; 3.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-19T07:46:10.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:10 smithi103 ceph-mon[31201]: pgmap v251: 97 pgs: 97 active+clean; 3.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-19T07:46:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 455.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:46:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:10 smithi134 ceph-mon[35414]: pgmap v251: 97 pgs: 97 active+clean; 3.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-19T07:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 -- 172.21.15.103:0/1386094754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 msgr2=0x7fa2a810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 --2- 172.21.15.103:0/1386094754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a810ea90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009ca0 tx=0x7fa2a002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 -- 172.21.15.103:0/1386094754 shutdown_connections 2024-09-19T07:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 --2- 172.21.15.103:0/1386094754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a810ea90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 --2- 172.21.15.103:0/1386094754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a8102790 0x7fa2a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 --2- 172.21.15.103:0/1386094754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2a8101de0 0x7fa2a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:10.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 -- 172.21.15.103:0/1386094754 >> 172.21.15.103:0/1386094754 conn(0x7fa2a80fd8c0 msgr2=0x7fa2a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 -- 172.21.15.103:0/1386094754 shutdown_connections 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.867+0000 7fa2ae753640 1 -- 172.21.15.103:0/1386094754 wait complete. 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.868+0000 7fa2ae753640 1 Processor -- start 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.868+0000 7fa2ae753640 1 -- start start 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2ae753640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2a8101de0 0x7fa2a8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2ae753640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a8102790 0x7fa2a8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2ae753640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2ae753640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111fb0 con 0x7fa2a8106f10 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2ae753640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111e30 con 0x7fa2a8101de0 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2ae753640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8112130 con 0x7fa2a8102790 2024-09-19T07:46:10.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43436/0 (socket says 172.21.15.103:43436) 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 -- 172.21.15.103:0/3228998253 learned_addr learned my addr 172.21.15.103:0/3228998253 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2a7fff640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2a8101de0 0x7fa2a8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2a77fe640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a8102790 0x7fa2a8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 -- 172.21.15.103:0/3228998253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a8102790 msgr2=0x7fa2a8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a8102790 0x7fa2a8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 -- 172.21.15.103:0/3228998253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2a8101de0 msgr2=0x7fa2a8198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2a8101de0 0x7fa2a8198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 -- 172.21.15.103:0/3228998253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a819d4c0 con 0x7fa2a8106f10 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2a77fe640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a8102790 0x7fa2a8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:10.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2a7fff640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2a8101de0 0x7fa2a8198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2accf0640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a819cde0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0031a20 tx=0x7fa2a0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2a57fa640 1 -- 172.21.15.103:0/3228998253 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0003c30 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2a57fa640 1 -- 172.21.15.103:0/3228998253 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0003dd0 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.869+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a819d750 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.870+0000 7fa2a57fa640 1 -- 172.21.15.103:0/3228998253 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002fcf0 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.870+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a5090 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.871+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a81021c0 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.872+0000 7fa2a57fa640 1 -- 172.21.15.103:0/3228998253 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa2a0004050 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.872+0000 7fa2a57fa640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2900a33b0 0x7fa2900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.873+0000 7fa2a7fff640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2900a33b0 0x7fa2900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.873+0000 7fa2a57fa640 1 -- 172.21.15.103:0/3228998253 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8586+0+0 (secure 0 0 0) 0x7fa2a00f10a0 con 0x7fa2a8106f10 2024-09-19T07:46:10.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.873+0000 7fa2a7fff640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2900a33b0 0x7fa2900a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa298009930 tx=0x7fa298008040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:10.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:10.877+0000 7fa2a57fa640 1 -- 172.21.15.103:0/3228998253 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa2a00bdbe0 con 0x7fa2a8106f10 2024-09-19T07:46:10.974 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:11.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.108+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2a810a0f0 con 0x7fa2a8106f10 2024-09-19T07:46:11.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.108+0000 7fa2a57fa640 1 -- 172.21.15.103:0/3228998253 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11996 (secure 0 0 0) 0x7fa2a002fe90 con 0x7fa2a8106f10 2024-09-19T07:46:11.108 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:46:11.108 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":40,"btime":"2024-09-19T07:44:20:535666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:44:19.854970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:46:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 40 2024-09-19T07:46:11.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2900a33b0 msgr2=0x7fa2900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:11.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2900a33b0 0x7fa2900a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa298009930 tx=0x7fa298008040 comp rx=0 tx=0).stop 2024-09-19T07:46:11.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 msgr2=0x7fa2a819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:11.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a819cde0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0031a20 tx=0x7fa2a0031a50 comp rx=0 tx=0).stop 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 shutdown_connections 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa2900a33b0 0x7fa2900a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2a8106f10 0x7fa2a819cde0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a8102790 0x7fa2a8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 --2- 172.21.15.103:0/3228998253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2a8101de0 0x7fa2a8198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 >> 172.21.15.103:0/3228998253 conn(0x7fa2a80fd8c0 msgr2=0x7fa2a80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 shutdown_connections 2024-09-19T07:46:11.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:11.111+0000 7fa2ae753640 1 -- 172.21.15.103:0/3228998253 wait complete. 2024-09-19T07:46:12.019 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:46:12.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:11 smithi174 ceph-mon[35370]: pgmap v252: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 576 op/s 2024-09-19T07:46:12.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:11 smithi174 ceph-mon[35370]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T07:46:12.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:11 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3228998253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:46:12.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:11 smithi103 ceph-mon[31201]: pgmap v252: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 576 op/s 2024-09-19T07:46:12.188 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:11 smithi103 ceph-mon[31201]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T07:46:12.189 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:11 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3228998253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 -- 172.21.15.103:0/769449699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108102570 msgr2=0x7f91081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 --2- 172.21.15.103:0/769449699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108102570 0x7f91081029b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009900 tx=0x7f90fc02a480 comp rx=0 tx=0).stop 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 -- 172.21.15.103:0/769449699 shutdown_connections 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 --2- 172.21.15.103:0/769449699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9108102ef0 0x7f910810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 --2- 172.21.15.103:0/769449699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108102570 0x7f91081029b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 --2- 172.21.15.103:0/769449699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9108100e20 0x7f9108101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 -- 172.21.15.103:0/769449699 >> 172.21.15.103:0/769449699 conn(0x7f91080fc7d0 msgr2=0x7f91080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 -- 172.21.15.103:0/769449699 shutdown_connections 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.188+0000 7f910e71f640 1 -- 172.21.15.103:0/769449699 wait complete. 2024-09-19T07:46:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 Processor -- start 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 -- start start 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108100e20 0x7f9108197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9108102570 0x7f9108197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9108102ef0 0x7f910819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9108110f80 con 0x7f9108100e20 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9108110e00 con 0x7f9108102570 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.189+0000 7f910e71f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9108111100 con 0x7f9108102ef0 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108100e20 0x7f9108197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108100e20 0x7f9108197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43450/0 (socket says 172.21.15.103:43450) 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 -- 172.21.15.103:0/412605494 learned_addr learned my addr 172.21.15.103:0/412605494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f91077fe640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9108102570 0x7f9108197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:12.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f910ccbc640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9108102ef0 0x7f910819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 -- 172.21.15.103:0/412605494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9108102ef0 msgr2=0x7f910819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9108102ef0 0x7f910819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 -- 172.21.15.103:0/412605494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9108102570 msgr2=0x7f9108197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9108102570 0x7f9108197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 -- 172.21.15.103:0/412605494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910819c3d0 con 0x7f9108100e20 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f9107fff640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108100e20 0x7f9108197420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f90f800d770 tx=0x7f90f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f91057fa640 1 -- 172.21.15.103:0/412605494 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8017070 con 0x7f9108100e20 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910819c6c0 con 0x7f9108100e20 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f91057fa640 1 -- 172.21.15.103:0/412605494 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8005500 con 0x7f9108100e20 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f91057fa640 1 -- 172.21.15.103:0/412605494 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8004e00 con 0x7f9108100e20 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.190+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91081a3fa0 con 0x7f9108100e20 2024-09-19T07:46:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.192+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9108101200 con 0x7f9108100e20 2024-09-19T07:46:12.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.193+0000 7f91057fa640 1 -- 172.21.15.103:0/412605494 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f90f8004fa0 con 0x7f9108100e20 2024-09-19T07:46:12.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.193+0000 7f91057fa640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f90f00a33b0 0x7f90f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:12.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.193+0000 7f91077fe640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f90f00a33b0 0x7f90f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:12.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.193+0000 7f91057fa640 1 -- 172.21.15.103:0/412605494 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f90f80cf770 con 0x7f9108100e20 2024-09-19T07:46:12.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.194+0000 7f91077fe640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f90f00a33b0 0x7f90f00a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009230 tx=0x7f90fc0058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:12.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.198+0000 7f91057fa640 1 -- 172.21.15.103:0/412605494 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f90f809c140 con 0x7f9108100e20 2024-09-19T07:46:12.275 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:11 smithi134 ceph-mon[35414]: pgmap v252: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 576 op/s 2024-09-19T07:46:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:11 smithi134 ceph-mon[35414]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T07:46:12.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:11 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3228998253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:46:12.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.415+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9108109090 con 0x7f9108100e20 2024-09-19T07:46:12.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.416+0000 7f91057fa640 1 -- 172.21.15.103:0/412605494 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11996 (secure 0 0 0) 0x7f90f809ba40 con 0x7f9108100e20 2024-09-19T07:46:12.416 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:46:12.416 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":40,"btime":"2024-09-19T07:44:20:535666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:44:19.854970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:46:12.416 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 40 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f90f00a33b0 msgr2=0x7f90f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f90f00a33b0 0x7f90f00a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009230 tx=0x7f90fc0058d0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108100e20 msgr2=0x7f9108197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108100e20 0x7f9108197420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f90f800d770 tx=0x7f90f800dc30 comp rx=0 tx=0).stop 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 shutdown_connections 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f90f00a33b0 0x7f90f00a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9108102ef0 0x7f910819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9108102570 0x7f9108197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 --2- 172.21.15.103:0/412605494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9108100e20 0x7f9108197420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:12.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 >> 172.21.15.103:0/412605494 conn(0x7f91080fc7d0 msgr2=0x7f9108105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:12.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 shutdown_connections 2024-09-19T07:46:12.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:12.418+0000 7f910e71f640 1 -- 172.21.15.103:0/412605494 wait complete. 2024-09-19T07:46:13.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:12 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/412605494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:46:13.159 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 868342b1 --release --await' 2024-09-19T07:46:13.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 868342b1 --release --await 2024-09-19T07:46:13.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:12 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/412605494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:46:13.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:12 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/412605494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:46:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 -- 172.21.15.103:0/3262597271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34100a80 msgr2=0x7f3d34100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 --2- 172.21.15.103:0/3262597271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34100a80 0x7f3d34100ee0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009930 tx=0x7f3d2402a420 comp rx=0 tx=0).stop 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 -- 172.21.15.103:0/3262597271 shutdown_connections 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 --2- 172.21.15.103:0/3262597271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d34101420 0x7f3d3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 --2- 172.21.15.103:0/3262597271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34100a80 0x7f3d34100ee0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 --2- 172.21.15.103:0/3262597271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d34106a80 0x7f3d34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 -- 172.21.15.103:0/3262597271 >> 172.21.15.103:0/3262597271 conn(0x7f3d340fc7a0 msgr2=0x7f3d340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 -- 172.21.15.103:0/3262597271 shutdown_connections 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.367+0000 7f3d3956c640 1 -- 172.21.15.103:0/3262597271 wait complete. 2024-09-19T07:46:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 Processor -- start 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 -- start start 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d34100a80 0x7f3d341985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d34101420 0x7f3d34198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34106a80 0x7f3d3419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341120b0 con 0x7f3d34106a80 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34111f30 con 0x7f3d34100a80 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.368+0000 7f3d3956c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34112230 con 0x7f3d34101420 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34106a80 0x7f3d3419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34106a80 0x7f3d3419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43470/0 (socket says 172.21.15.103:43470) 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 -- 172.21.15.103:0/1793004096 learned_addr learned my addr 172.21.15.103:0/1793004096 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d34100a80 0x7f3d341985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d34101420 0x7f3d34198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 -- 172.21.15.103:0/1793004096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d34101420 msgr2=0x7f3d34198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d34101420 0x7f3d34198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 -- 172.21.15.103:0/1793004096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d34100a80 msgr2=0x7f3d341985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d34100a80 0x7f3d341985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 -- 172.21.15.103:0/1793004096 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3419d5a0 con 0x7f3d34106a80 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34106a80 0x7f3d3419cec0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00b970 tx=0x7f3d2c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d1bfff640 1 -- 172.21.15.103:0/1793004096 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c019070 con 0x7f3d34106a80 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d1bfff640 1 -- 172.21.15.103:0/1793004096 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c014440 con 0x7f3d34106a80 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3419d890 con 0x7f3d34106a80 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d1bfff640 1 -- 172.21.15.103:0/1793004096 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0075d0 con 0x7f3d34106a80 2024-09-19T07:46:13.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.369+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341a5170 con 0x7f3d34106a80 2024-09-19T07:46:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.371+0000 7f3d1bfff640 1 -- 172.21.15.103:0/1793004096 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3d2c01c070 con 0x7f3d34106a80 2024-09-19T07:46:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 Processor -- start 2024-09-19T07:46:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 -- start start 2024-09-19T07:46:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa35b40 0x56197aa34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d1bfff640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3d1c0a33b0 0x7f3d1c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa3f370 0x56197aa41760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56197a99a2f0 con 0x56197aa345d0 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56197a941cf0 con 0x56197aa35b40 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d3b945740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56197a9962b0 con 0x56197aa3f370 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa34a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43484/0 (socket says 172.21.15.103:43484) 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d32ffd640 1 -- 172.21.15.103:0/1127041270 learned_addr learned my addr 172.21.15.103:0/1127041270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.372+0000 7f3d1bfff640 1 -- 172.21.15.103:0/1793004096 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f3d2c0cf290 con 0x7f3d34106a80 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3d1c0a33b0 0x7f3d1c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa35b40 0x56197aa34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa3f370 0x56197aa41760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d32ffd640 1 -- 172.21.15.103:0/1127041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa3f370 msgr2=0x56197aa41760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa3f370 0x56197aa41760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d32ffd640 1 -- 172.21.15.103:0/1127041270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa35b40 msgr2=0x56197aa34090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa35b40 0x56197aa34090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d32ffd640 1 -- 172.21.15.103:0/1127041270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197a9263a0 con 0x56197aa345d0 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa34a10 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009a70 tx=0x7f3d28009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0613da41ed069b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3d1c0a33b0 0x7f3d1c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d240314c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d197fa640 1 -- 172.21.15.103:0/1127041270 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28017070 con 0x56197aa345d0 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d197fa640 1 -- 172.21.15.103:0/1127041270 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d28013470 con 0x56197aa345d0 2024-09-19T07:46:13.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.373+0000 7f3d197fa640 1 -- 172.21.15.103:0/1127041270 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28012610 con 0x56197aa345d0 2024-09-19T07:46:13.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 -- 172.21.15.103:0/1127041270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 msgr2=0x56197aa34a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa34a10 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009a70 tx=0x7f3d28009f30 comp rx=0 tx=0).stop 2024-09-19T07:46:13.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 -- 172.21.15.103:0/1127041270 shutdown_connections 2024-09-19T07:46:13.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa3f370 0x56197aa41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa34a10 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1127041270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa35b40 0x56197aa34090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 -- 172.21.15.103:0/1127041270 >> 172.21.15.103:0/1127041270 conn(0x56197a9fd310 msgr2=0x56197aa30c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 -- 172.21.15.103:0/1127041270 shutdown_connections 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.374+0000 7f3d3b945740 1 -- 172.21.15.103:0/1127041270 wait complete. 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 -- start start 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 Processor -- start 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa3ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa3bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa3c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56197aa46480 con 0x56197aa345d0 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56197a955060 con 0x56197aa3f370 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56197aa46600 con 0x56197aa35b40 2024-09-19T07:46:13.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d3b945740 10 client.0 _ll_register_callbacks cb 0x56197a6a88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa3ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa3ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43490/0 (socket says 172.21.15.103:43490) 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d32ffd640 1 -- 172.21.15.103:0/3604837936 learned_addr learned my addr 172.21.15.103:0/3604837936 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d327fc640 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa3bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.375+0000 7f3d337fe640 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa3c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d32ffd640 1 -- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 msgr2=0x56197aa3bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa3bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d32ffd640 1 -- 172.21.15.103:0/3604837936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 msgr2=0x56197aa3c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa3c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d32ffd640 1 -- 172.21.15.103:0/3604837936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56197a70b3c0 con 0x56197aa345d0 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa3ece0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3d28004810 tx=0x7f3d28013d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661900aace9321c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:46:13.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d16ff5640 1 -- 172.21.15.103:0/3604837936 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28017050 con 0x56197aa345d0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d16ff5640 1 -- 172.21.15.103:0/3604837936 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f3d2801c450 con 0x56197aa345d0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d28013470 con 0x56197aa345d0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d16ff5640 1 -- 172.21.15.103:0/3604837936 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3d2801c650 con 0x56197aa345d0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d3b945740 10 client.17121 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56197aa51e90 con 0x56197aa345d0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.376+0000 7f3d16ff5640 1 -- 172.21.15.103:0/3604837936 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d28013470 con 0x56197aa345d0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d16ff5640 1 -- 172.21.15.103:0/3604837936 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f3d280269e0 con 0x56197aa345d0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d3b945740 10 client.17121 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d3b945740 10 client.17121 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d3b945740 20 client.17121 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d3b945740 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aa532f0 0x56197aa556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d3b945740 4 client.17121 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56197a9a7860 con 0x56197aa532f0 2024-09-19T07:46:13.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.377+0000 7f3d327fc640 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aa532f0 0x56197aa556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.378+0000 7f3d327fc640 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aa532f0 0x56197aa556e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=785b895e21c2d485 server_cookie=794d40ed188fd02d in_seq=0 out_seq=0 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.378+0000 7f3d16ff5640 10 client.17121 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.378+0000 7f3d16ff5640 1 -- 172.21.15.103:0/3604837936 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56197a9a7860 con 0x56197aa532f0 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.378+0000 7f3d16ff5640 10 client.17121 handle_command_reply: tid=0 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.378+0000 7f3d3b945740 1 client.17121 shutdown 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.378+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 msgr2=0x56197aa3ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.378+0000 7f3d3b945740 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa3ece0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3d28004810 tx=0x7f3d28013d30 comp rx=0 tx=0).stop 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 shutdown_connections 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aa532f0 0x56197aa556e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa3c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa3bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d16ff5640 0 client.17121 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 --2- 172.21.15.103:0/3604837936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa345d0 0x56197aa3ece0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 >> 172.21.15.103:0/3604837936 conn(0x56197a9fd310 msgr2=0x56197aa36960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:13.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 shutdown_connections 2024-09-19T07:46:13.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 1 -- 172.21.15.103:0/3604837936 wait complete. 2024-09-19T07:46:13.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.379+0000 7f3d3b945740 20 client.17121 trim_cache size 0 max 16384 2024-09-19T07:46:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.392+0000 7f3d3b945740 1 Processor -- start 2024-09-19T07:46:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.392+0000 7f3d3b945740 1 -- start start 2024-09-19T07:46:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa345d0 0x56197aa4f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa5ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa3f370 0x56197aa5b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d3b945740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341ab840 con 0x56197aa3f370 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d3b945740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341ad380 con 0x56197aa345d0 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d3b945740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341abf10 con 0x56197aa35b40 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa3f370 0x56197aa5b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa3f370 0x56197aa5b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43504/0 (socket says 172.21.15.103:43504) 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 -- 172.21.15.103:0/1676111040 learned_addr learned my addr 172.21.15.103:0/1676111040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa5ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 -- 172.21.15.103:0/1676111040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 msgr2=0x56197aa5ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa5ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 -- 172.21.15.103:0/1676111040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa345d0 msgr2=0x56197aa4f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa345d0 0x56197aa4f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa345d0 0x56197aa4f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 -- 172.21.15.103:0/1676111040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197aa342d0 con 0x56197aa3f370 2024-09-19T07:46:13.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.393+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa3f370 0x56197aa5b250 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c09b370 tx=0x7f3d2c09b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c94696474d5a5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:13.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.394+0000 7f3d18ff9640 1 -- 172.21.15.103:0/1676111040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0a42e0 con 0x56197aa3f370 2024-09-19T07:46:13.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.394+0000 7f3d18ff9640 1 -- 172.21.15.103:0/1676111040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c099970 con 0x56197aa3f370 2024-09-19T07:46:13.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.394+0000 7f3d18ff9640 1 -- 172.21.15.103:0/1676111040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c097e70 con 0x56197aa3f370 2024-09-19T07:46:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 -- 172.21.15.103:0/1676111040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa3f370 msgr2=0x56197aa5b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa3f370 0x56197aa5b250 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c09b370 tx=0x7f3d2c09b090 comp rx=0 tx=0).stop 2024-09-19T07:46:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 -- 172.21.15.103:0/1676111040 shutdown_connections 2024-09-19T07:46:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa3f370 0x56197aa5b250 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa35b40 0x56197aa5ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1676111040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa345d0 0x56197aa4f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 -- 172.21.15.103:0/1676111040 >> 172.21.15.103:0/1676111040 conn(0x56197a9fd310 msgr2=0x56197aa3b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 -- 172.21.15.103:0/1676111040 shutdown_connections 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 -- 172.21.15.103:0/1676111040 wait complete. 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 -- start start 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.395+0000 7f3d3b945740 1 Processor -- start 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa345d0 0x56197aa39030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa35b40 0x56197aa39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d3b945740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa35b40 0x56197aa39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa35b40 0x56197aa39570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43512/0 (socket says 172.21.15.103:43512) 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa345d0 0x56197aa39030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa345d0 0x56197aa39030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:32824/0 (socket says 172.21.15.103:32824) 2024-09-19T07:46:13.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 -- 172.21.15.103:0/1938408048 learned_addr learned my addr 172.21.15.103:0/1938408048 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341accb0 con 0x56197aa35b40 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341ac5e0 con 0x56197aa3f370 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341a9840 con 0x56197aa345d0 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d3b945740 10 client.0 _ll_register_callbacks cb 0x7f3d280098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 -- 172.21.15.103:0/1938408048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa345d0 msgr2=0x56197aa39030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa345d0 0x56197aa39030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 -- 172.21.15.103:0/1938408048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 msgr2=0x56197aa39ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa39ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.396+0000 7f3d327fc640 1 -- 172.21.15.103:0/1938408048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56197aabcd90 con 0x56197aa35b40 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d337fe640 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2c099970 con 0x56197aa35b40 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d327fc640 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa35b40 0x56197aa39570 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3d240038e0 tx=0x7f3d24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92065d2777ae105 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:13.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d16ff5640 10 client.17127 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d16ff5640 1 -- 172.21.15.103:0/1938408048 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24060050 con 0x56197aa35b40 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d16ff5640 1 -- 172.21.15.103:0/1938408048 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f3d240315b0 con 0x56197aa35b40 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d16ff5640 1 -- 172.21.15.103:0/1938408048 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3d24046040 con 0x56197aa35b40 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d3b945740 10 client.17127 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.397+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=40}) -- 0x56197aabdf40 con 0x56197aa35b40 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d16ff5640 1 -- 172.21.15.103:0/1938408048 <== mon.0 v2:172.21.15.103:3300/0 4 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f3d240317b0 con 0x56197aa35b40 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d3b945740 10 client.17127 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d3b945740 10 client.17127 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d3b945740 20 client.17127 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aabe5e0 0x56197aac0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d3b945740 4 client.17127 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:46:13.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56197a9a1d70 con 0x56197aabe5e0 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aabe5e0 0x56197aac0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d16ff5640 10 client.17127 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.398+0000 7f3d32ffd640 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aabe5e0 0x56197aac0a00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abe53dc8fee4749d server_cookie=10f975f78f7a131b in_seq=0 out_seq=0 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d16ff5640 1 -- 172.21.15.103:0/1938408048 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56197a9a1d70 con 0x56197aabe5e0 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d16ff5640 10 client.17127 handle_command_reply: tid=0 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 client.17127 shutdown 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa35b40 msgr2=0x56197aa39570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa35b40 0x56197aa39570 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3d240038e0 tx=0x7f3d24000bc0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 shutdown_connections 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56197aabe5e0 0x56197aac0a00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56197aa3f370 0x56197aa39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d16ff5640 0 client.17127 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56197aa35b40 0x56197aa39570 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 --2- 172.21.15.103:0/1938408048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56197aa345d0 0x56197aa39030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 >> 172.21.15.103:0/1938408048 conn(0x56197a9fd310 msgr2=0x56197aa2f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:13.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 shutdown_connections 2024-09-19T07:46:13.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 1 -- 172.21.15.103:0/1938408048 wait complete. 2024-09-19T07:46:13.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.399+0000 7f3d3b945740 20 client.17127 trim_cache size 0 max 16384 2024-09-19T07:46:13.404 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3d1c0a33b0 msgr2=0x7f3d1c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3d1c0a33b0 0x7f3d1c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d240314c0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34106a80 msgr2=0x7f3d3419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34106a80 0x7f3d3419cec0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00b970 tx=0x7f3d2c00bd70 comp rx=0 tx=0).stop 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 shutdown_connections 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3d1c0a33b0 0x7f3d1c0a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d34106a80 0x7f3d3419cec0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d34101420 0x7f3d34198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 --2- 172.21.15.103:0/1793004096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d34100a80 0x7f3d341985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 >> 172.21.15.103:0/1793004096 conn(0x7f3d340fc7a0 msgr2=0x7f3d340fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 shutdown_connections 2024-09-19T07:46:13.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:13.400+0000 7f3d3956c640 1 -- 172.21.15.103:0/1793004096 wait complete. 2024-09-19T07:46:13.668 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:14.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:13 smithi174 ceph-mon[35370]: pgmap v254: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 693 op/s 2024-09-19T07:46:14.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:13 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:46:14.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:13 smithi103 ceph-mon[31201]: pgmap v254: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 693 op/s 2024-09-19T07:46:14.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:13 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:46:14.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:46:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:13 smithi134 ceph-mon[35414]: pgmap v254: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 693 op/s 2024-09-19T07:46:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:13 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:46:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 -- 172.21.15.103:0/2743871152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1082a0 msgr2=0x7fa4fc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 --2- 172.21.15.103:0/2743871152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1082a0 0x7fa4fc10ec40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009cd0 tx=0x7fa4f402a8e0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 -- 172.21.15.103:0/2743871152 shutdown_connections 2024-09-19T07:46:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 --2- 172.21.15.103:0/2743871152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1082a0 0x7fa4fc10ec40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 --2- 172.21.15.103:0/2743871152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4fc1036e0 0x7fa4fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 --2- 172.21.15.103:0/2743871152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4fc102d30 0x7fa4fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 -- 172.21.15.103:0/2743871152 >> 172.21.15.103:0/2743871152 conn(0x7fa4fc0fc7d0 msgr2=0x7fa4fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 -- 172.21.15.103:0/2743871152 shutdown_connections 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 -- 172.21.15.103:0/2743871152 wait complete. 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.761+0000 7fa503f76640 1 Processor -- start 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa503f76640 1 -- start start 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa503f76640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4fc102d30 0x7fa4fc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa503f76640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1036e0 0x7fa4fc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa503f76640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4fc1082a0 0x7fa4fc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa503f76640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1120a0 con 0x7fa4fc1036e0 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa503f76640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc111f20 con 0x7fa4fc102d30 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa503f76640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc112220 con 0x7fa4fc1082a0 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1036e0 0x7fa4fc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1036e0 0x7fa4fc198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43530/0 (socket says 172.21.15.103:43530) 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 -- 172.21.15.103:0/1720892742 learned_addr learned my addr 172.21.15.103:0/1720892742 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501d12640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4fc102d30 0x7fa4fc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa502513640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4fc1082a0 0x7fa4fc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 -- 172.21.15.103:0/1720892742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4fc1082a0 msgr2=0x7fa4fc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4fc1082a0 0x7fa4fc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 -- 172.21.15.103:0/1720892742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4fc102d30 msgr2=0x7fa4fc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4fc102d30 0x7fa4fc198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.762+0000 7fa501511640 1 -- 172.21.15.103:0/1720892742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc19d4d0 con 0x7fa4fc1036e0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.763+0000 7fa501511640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1036e0 0x7fa4fc198a60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00d740 tx=0x7fa4ec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.763+0000 7fa4f2ffd640 1 -- 172.21.15.103:0/1720892742 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec017070 con 0x7fa4fc1036e0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.763+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc19d7c0 con 0x7fa4fc1036e0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.763+0000 7fa4f2ffd640 1 -- 172.21.15.103:0/1720892742 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec00de50 con 0x7fa4fc1036e0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.763+0000 7fa4f2ffd640 1 -- 172.21.15.103:0/1720892742 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00f6b0 con 0x7fa4fc1036e0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.763+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc1a50a0 con 0x7fa4fc1036e0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.765+0000 7fa4f2ffd640 1 -- 172.21.15.103:0/1720892742 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa4ec01a070 con 0x7fa4fc1036e0 2024-09-19T07:46:14.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.765+0000 7fa50634f740 1 Processor -- start 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa50634f740 1 -- start start 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa50634f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa50634f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa50634f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94d0650 0x560cc94d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa50634f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560cc9426620 con 0x560cc94c0f60 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa50634f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560cc93cde50 con 0x560cc94d0650 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa50634f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560cc94225e0 con 0x560cc94c1880 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa4f2ffd640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa4e80a33b0 0x7fa4e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa502513640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa501d12640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa501d12640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:32862/0 (socket says 172.21.15.103:32862) 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa501d12640 1 -- 172.21.15.103:0/2083811496 learned_addr learned my addr 172.21.15.103:0/2083811496 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:14.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa501511640 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94d0650 0x560cc94d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa502513640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa4e80a33b0 0x7fa4e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.766+0000 7fa4f2ffd640 1 -- 172.21.15.103:0/1720892742 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fa4ec0cf720 con 0x7fa4fc1036e0 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa501511640 1 -- 172.21.15.103:0/2083811496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 msgr2=0x560cc94d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa501511640 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa501511640 1 -- 172.21.15.103:0/2083811496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 msgr2=0x560cc94c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa501511640 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa501511640 1 -- 172.21.15.103:0/2083811496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc93bed40 con 0x560cc94d0650 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa502513640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa4e80a33b0 0x7fa4e80a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4004af0 tx=0x7fa4f40095c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa501511640 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94d0650 0x560cc94d2a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0cce00 tx=0x7fa4ec09c090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f4c1aaf7dcd161 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa4e7fff640 1 -- 172.21.15.103:0/2083811496 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec021030 con 0x560cc94d0650 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa4e7fff640 1 -- 172.21.15.103:0/2083811496 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec0a4600 con 0x560cc94d0650 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.767+0000 7fa4e7fff640 1 -- 172.21.15.103:0/2083811496 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec09b040 con 0x560cc94d0650 2024-09-19T07:46:14.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 -- 172.21.15.103:0/2083811496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94d0650 msgr2=0x560cc94d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94d0650 0x560cc94d2a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0cce00 tx=0x7fa4ec09c090 comp rx=0 tx=0).stop 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 -- 172.21.15.103:0/2083811496 shutdown_connections 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94d0650 0x560cc94d2a40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94d0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 --2- 172.21.15.103:0/2083811496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 -- 172.21.15.103:0/2083811496 >> 172.21.15.103:0/2083811496 conn(0x560cc94bd5f0 msgr2=0x560cc94bd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 -- 172.21.15.103:0/2083811496 shutdown_connections 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 -- 172.21.15.103:0/2083811496 wait complete. 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 -- start start 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.768+0000 7fa50634f740 1 Processor -- start 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa50634f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c0f60 0x560cc94cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa50634f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa50634f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa50634f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560cc94d3220 con 0x560cc94d0650 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa50634f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560cc93aef10 con 0x560cc94c1880 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa50634f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560cc94d33a0 con 0x560cc94c0f60 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa50634f740 10 client.0 _ll_register_callbacks cb 0x560cc93d2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43544/0 (socket says 172.21.15.103:43544) 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 -- 172.21.15.103:0/2829489131 learned_addr learned my addr 172.21.15.103:0/2829489131 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa501d12640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c0f60 0x560cc94cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa501511640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 -- 172.21.15.103:0/2829489131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c0f60 msgr2=0x560cc94cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c0f60 0x560cc94cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 -- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 msgr2=0x560cc94cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa502513640 1 -- 172.21.15.103:0/2829489131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc9135580 con 0x560cc94d0650 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.769+0000 7fa501511640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa502513640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94ce0e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009cd0 tx=0x7fa4f4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6628f5247d4a3b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa4e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2829489131 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4051070 con 0x560cc94d0650 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2829489131 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fa4f4004310 con 0x560cc94d0650 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ec0a4600 con 0x560cc94d0650 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2829489131 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4f405d050 con 0x560cc94d0650 2024-09-19T07:46:14.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa50634f740 10 client.17139 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560cc91ff960 con 0x560cc94d0650 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.770+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2829489131 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec0a4600 con 0x560cc94d0650 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2829489131 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fa4f4049040 con 0x560cc94d0650 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa50634f740 10 client.17139 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa50634f740 10 client.17139 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa50634f740 20 client.17139 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa50634f740 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc94e02c0 0x560cc94e2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa50634f740 4 client.17139 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cc9433b90 con 0x560cc94e02c0 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa501d12640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc94e02c0 0x560cc94e2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa4e57fa640 10 client.17139 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:14.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.771+0000 7fa501d12640 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc94e02c0 0x560cc94e2740 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6d3fe2f5465076 server_cookie=82378c6190e6c9cc in_seq=0 out_seq=0 2024-09-19T07:46:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2829489131 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560cc9433b90 con 0x560cc94e02c0 2024-09-19T07:46:14.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa4e57fa640 10 client.17139 handle_command_reply: tid=0 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 client.17139 shutdown 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 msgr2=0x560cc94ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94ce0e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009cd0 tx=0x7fa4f4009900 comp rx=0 tx=0).stop 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 shutdown_connections 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc94e02c0 0x560cc94e2740 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94ce0e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa4e57fa640 0 client.17139 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 --2- 172.21.15.103:0/2829489131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c0f60 0x560cc94cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.772+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 >> 172.21.15.103:0/2829489131 conn(0x560cc94bd5f0 msgr2=0x560cc94c3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.773+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 shutdown_connections 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.773+0000 7fa50634f740 1 -- 172.21.15.103:0/2829489131 wait complete. 2024-09-19T07:46:14.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.773+0000 7fa50634f740 20 client.17139 trim_cache size 0 max 16384 2024-09-19T07:46:14.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 Processor -- start 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 -- start start 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c0f60 0x560cc94c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94e8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94e85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ab720 con 0x560cc94d0650 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ad260 con 0x560cc94c0f60 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa50634f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1abdf0 con 0x560cc94c1880 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.793+0000 7fa501d12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c0f60 0x560cc94c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501d12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c0f60 0x560cc94c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56136/0 (socket says 172.21.15.103:56136) 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501d12640 1 -- 172.21.15.103:0/2549841186 learned_addr learned my addr 172.21.15.103:0/2549841186 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa502513640 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94e85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501511640 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94e8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501511640 1 -- 172.21.15.103:0/2549841186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c0f60 msgr2=0x560cc94c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501511640 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c0f60 0x560cc94c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501511640 1 -- 172.21.15.103:0/2549841186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 msgr2=0x560cc94e85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501511640 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94e85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501511640 1 -- 172.21.15.103:0/2549841186 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc94c9240 con 0x560cc94c1880 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa502513640 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501d12640 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c0f60 0x560cc94c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa501511640 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94e8060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0862c0 tx=0x7fa4ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d191a01b19098042 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa4e77fe640 1 -- 172.21.15.103:0/2549841186 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec014030 con 0x560cc94c1880 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa4e77fe640 1 -- 172.21.15.103:0/2549841186 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec060040 con 0x560cc94c1880 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.794+0000 7fa4e77fe640 1 -- 172.21.15.103:0/2549841186 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec021040 con 0x560cc94c1880 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 -- 172.21.15.103:0/2549841186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 msgr2=0x560cc94e8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94e8060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0862c0 tx=0x7fa4ec000bc0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 -- 172.21.15.103:0/2549841186 shutdown_connections 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94d0650 0x560cc94e85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.798 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94c1880 0x560cc94e8060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 --2- 172.21.15.103:0/2549841186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c0f60 0x560cc94c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 -- 172.21.15.103:0/2549841186 >> 172.21.15.103:0/2549841186 conn(0x560cc94bd5f0 msgr2=0x560cc94c2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:14.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 -- 172.21.15.103:0/2549841186 shutdown_connections 2024-09-19T07:46:14.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.795+0000 7fa50634f740 1 -- 172.21.15.103:0/2549841186 wait complete. 2024-09-19T07:46:14.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 -- start start 2024-09-19T07:46:14.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 Processor -- start 2024-09-19T07:46:14.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94d0650 0x560cc94d5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1acb90 con 0x560cc94c0f60 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ac4c0 con 0x560cc94c1880 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa50634f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1a9720 con 0x560cc94d0650 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa501d12640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94d8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa501511640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa501d12640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94d8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43564/0 (socket says 172.21.15.103:43564) 2024-09-19T07:46:14.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.796+0000 7fa501d12640 1 -- 172.21.15.103:0/2824415885 learned_addr learned my addr 172.21.15.103:0/2824415885 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa50634f740 10 client.0 _ll_register_callbacks cb 0x7fa4f4031da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa502513640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94d0650 0x560cc94d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa501511640 1 -- 172.21.15.103:0/2824415885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94d0650 msgr2=0x560cc94d5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa501511640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94d0650 0x560cc94d5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa501511640 1 -- 172.21.15.103:0/2824415885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 msgr2=0x560cc94d8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa501511640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94d8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa501511640 1 -- 172.21.15.103:0/2824415885 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc94d92b0 con 0x560cc94c1880 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa502513640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94d0650 0x560cc94d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa501d12640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa501511640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94d8b70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560cc94c26c0 tx=0x7fa4ec005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1802d63a647236b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:14.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa4e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2824415885 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0b3020 con 0x560cc94c1880 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ec060040 con 0x560cc94c1880 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2824415885 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fa4ec098000 con 0x560cc94c1880 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2824415885 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4ec021070 con 0x560cc94c1880 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa50634f740 10 client.26600 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.797+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560cc954b6a0 con 0x560cc94c1880 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2824415885 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec060040 con 0x560cc94c1880 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2824415885 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fa4ec09a850 con 0x560cc94c1880 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa50634f740 10 client.26600 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa50634f740 10 client.26600 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:46:14.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa50634f740 20 client.26600 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:14.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa50634f740 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc954b930 0x560cc954dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:14.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa50634f740 4 client.26600 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "868342b1", "release": true, "await": true}] 2024-09-19T07:46:14.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "868342b1", "release": true, "await": true}) -- 0x560cc942e0a0 con 0x560cc954b930 2024-09-19T07:46:14.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa501d12640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc954b930 0x560cc954dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:14.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.798+0000 7fa501d12640 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc954b930 0x560cc954dd20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e7d09120c08af server_cookie=d2016224e9a2b599 in_seq=0 out_seq=0 2024-09-19T07:46:14.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.799+0000 7fa4e57fa640 10 client.26600 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:14.809 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:14.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa4e57fa640 1 -- 172.21.15.103:0/2824415885 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560cc942e0a0 con 0x560cc954b930 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa4e57fa640 10 client.26600 handle_command_reply: tid=0 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 client.26600 shutdown 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 msgr2=0x560cc94d8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94d8b70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560cc94c26c0 tx=0x7fa4ec005cc0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 shutdown_connections 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560cc954b930 0x560cc954dd20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cc94d0650 0x560cc94d5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560cc94c1880 0x560cc94d8b70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 --2- 172.21.15.103:0/2824415885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560cc94c0f60 0x560cc94d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 >> 172.21.15.103:0/2824415885 conn(0x560cc94bd5f0 msgr2=0x560cc94c2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa4e57fa640 0 client.26600 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:14.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 shutdown_connections 2024-09-19T07:46:14.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.832+0000 7fa50634f740 1 -- 172.21.15.103:0/2824415885 wait complete. 2024-09-19T07:46:14.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa50634f740 20 client.26600 trim_cache size 0 max 16384 2024-09-19T07:46:14.834 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 40, 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 32, 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "868342b1": { 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 32, 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:46:14.835 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.8, 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:46:14.836 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:46:14.837 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:46:14.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa4e80a33b0 msgr2=0x7fa4e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa4e80a33b0 0x7fa4e80a5870 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4004af0 tx=0x7fa4f40095c0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1036e0 msgr2=0x7fa4fc198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:14.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1036e0 0x7fa4fc198a60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00d740 tx=0x7fa4ec00dc00 comp rx=0 tx=0).stop 2024-09-19T07:46:14.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 shutdown_connections 2024-09-19T07:46:14.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa4e80a33b0 0x7fa4e80a5870 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4fc1082a0 0x7fa4fc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4fc1036e0 0x7fa4fc198a60 secure :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00d740 tx=0x7fa4ec00dc00 comp rx=0 tx=0).stop 2024-09-19T07:46:14.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 --2- 172.21.15.103:0/1720892742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4fc102d30 0x7fa4fc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:14.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.833+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 >> 172.21.15.103:0/1720892742 conn(0x7fa4fc0fc7d0 msgr2=0x7fa4fc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:14.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.835+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 shutdown_connections 2024-09-19T07:46:14.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:14.835+0000 7fa503f76640 1 -- 172.21.15.103:0/1720892742 wait complete. 2024-09-19T07:46:15.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:14 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:46:15.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:14 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:46:15.198 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:14 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:46:15.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:14 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:46:15.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:14 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:46:15.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:14 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:46:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:14 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:46:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:14 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:46:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:14 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:46:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 460.0 s, 198.2 tps, lat 2827.385 ms stddev 8251.446 2024-09-19T07:46:15.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.826+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/3373073181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 msgr2=0x7fdc88101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.826+0000 7fdc877fe640 1 -- 172.21.15.103:0/3373073181 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0309c0 con 0x7fdc88100e20 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.826+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/3373073181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 0x7fdc88101200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009890 tx=0x7fdc7c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/3373073181 shutdown_connections 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/3373073181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc88102ef0 0x7fdc8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/3373073181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88102570 0x7fdc881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/3373073181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 0x7fdc88101200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/3373073181 >> 172.21.15.103:0/3373073181 conn(0x7fdc880fc7d0 msgr2=0x7fdc880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/3373073181 shutdown_connections 2024-09-19T07:46:15.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/3373073181 wait complete. 2024-09-19T07:46:15.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 Processor -- start 2024-09-19T07:46:15.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.827+0000 7fdc8ef6c640 1 -- start start 2024-09-19T07:46:15.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.828+0000 7fdc8ef6c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 0x7fdc881963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc8ef6c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc88102570 0x7fdc88196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc8ef6c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88102ef0 0x7fdc8819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc8ef6c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8810fe80 con 0x7fdc88102ef0 2024-09-19T07:46:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc8ef6c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8810fd00 con 0x7fdc88102570 2024-09-19T07:46:15.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc8ef6c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88110000 con 0x7fdc88100e20 2024-09-19T07:46:15.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc88102570 0x7fdc88196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc87fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc88102570 0x7fdc88196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56168/0 (socket says 172.21.15.103:56168) 2024-09-19T07:46:15.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc87fff640 1 -- 172.21.15.103:0/1346999415 learned_addr learned my addr 172.21.15.103:0/1346999415 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:15.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 0x7fdc881963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.829+0000 7fdc8d509640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88102ef0 0x7fdc8819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc8d509640 1 -- 172.21.15.103:0/1346999415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 msgr2=0x7fdc881963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc8d509640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 0x7fdc881963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc8d509640 1 -- 172.21.15.103:0/1346999415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc88102570 msgr2=0x7fdc88196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc8d509640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc88102570 0x7fdc88196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc8d509640 1 -- 172.21.15.103:0/1346999415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8819b380 con 0x7fdc88102ef0 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc8d509640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88102ef0 0x7fdc8819aca0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009d20 tx=0x7fdc80007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc85ffb640 1 -- 172.21.15.103:0/1346999415 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80007ca0 con 0x7fdc88102ef0 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8819b670 con 0x7fdc88102ef0 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc85ffb640 1 -- 172.21.15.103:0/1346999415 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc80003e50 con 0x7fdc88102ef0 2024-09-19T07:46:15.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.830+0000 7fdc85ffb640 1 -- 172.21.15.103:0/1346999415 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8001b4b0 con 0x7fdc88102ef0 2024-09-19T07:46:15.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.831+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a2f50 con 0x7fdc88102ef0 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 Processor -- start 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc85ffb640 1 -- 172.21.15.103:0/1346999415 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdc80003970 con 0x7fdc88102ef0 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 -- start start 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af55f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af55fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af57db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561af4c52f0 con 0x5561af560b40 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5561af46ccf0 con 0x5561af56a370 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.835+0000 7fdc91345740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561af4c12b0 con 0x5561af55f5d0 2024-09-19T07:46:15.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af55fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af55fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:32912/0 (socket says 172.21.15.103:32912) 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2814551798 learned_addr learned my addr 172.21.15.103:0/2814551798 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2814551798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 msgr2=0x5561af57db40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2814551798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af57db40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2814551798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 msgr2=0x5561af55f090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2814551798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af55f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2814551798 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561af4513a0 con 0x5561af55f5d0 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc85ffb640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdc780a3470 0x7fdc780a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.836+0000 7fdc85ffb640 1 -- 172.21.15.103:0/1346999415 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fdc800d3b50 con 0x7fdc88102ef0 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.837+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2814551798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af55fa10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c034000 tx=0x7fdc7c005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6826fd800f2a4d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.837+0000 7fdc87fff640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdc780a3470 0x7fdc780a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.837+0000 7fdc72ffd640 1 -- 172.21.15.103:0/2814551798 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c037040 con 0x5561af55f5d0 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.837+0000 7fdc72ffd640 1 -- 172.21.15.103:0/2814551798 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7c004230 con 0x5561af55f5d0 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.837+0000 7fdc72ffd640 1 -- 172.21.15.103:0/2814551798 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c030940 con 0x5561af55f5d0 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc91345740 1 -- 172.21.15.103:0/2814551798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 msgr2=0x5561af55fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc87fff640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdc780a3470 0x7fdc780a5930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc880fed50 tx=0x7fdc740095e0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:15.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc91345740 1 --2- 172.21.15.103:0/2814551798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af55fa10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c034000 tx=0x7fdc7c005d70 comp rx=0 tx=0).stop 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc91345740 1 -- 172.21.15.103:0/2814551798 shutdown_connections 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc91345740 1 --2- 172.21.15.103:0/2814551798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af57db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc91345740 1 --2- 172.21.15.103:0/2814551798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af55fa10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc91345740 1 --2- 172.21.15.103:0/2814551798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af55f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.838+0000 7fdc91345740 1 -- 172.21.15.103:0/2814551798 >> 172.21.15.103:0/2814551798 conn(0x5561af528310 msgr2=0x5561af55bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.839+0000 7fdc91345740 1 -- 172.21.15.103:0/2814551798 shutdown_connections 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.839+0000 7fdc91345740 1 -- 172.21.15.103:0/2814551798 wait complete. 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.839+0000 7fdc91345740 1 -- start start 2024-09-19T07:46:15.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.839+0000 7fdc91345740 1 Processor -- start 2024-09-19T07:46:15.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.840+0000 7fdc91345740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af55f5d0 0x5561af569ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.840+0000 7fdc91345740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af566f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.840+0000 7fdc91345740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc91345740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561af57e080 con 0x5561af55f5d0 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc91345740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5561af480060 con 0x5561af56a370 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8cd08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af55f5d0 0x5561af569ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8cd08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af55f5d0 0x5561af569ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43614/0 (socket says 172.21.15.103:43614) 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8cd08640 1 -- 172.21.15.103:0/3545902318 learned_addr learned my addr 172.21.15.103:0/3545902318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561af57e200 con 0x5561af560b40 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8d509640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc87fff640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af566f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc91345740 10 client.0 _ll_register_callbacks cb 0x5561af1d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8d509640 1 -- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 msgr2=0x5561af566f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8d509640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af566f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8d509640 1 -- 172.21.15.103:0/3545902318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af55f5d0 msgr2=0x5561af569ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8d509640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af55f5d0 0x5561af569ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc8d509640 1 -- 172.21.15.103:0/3545902318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561af2363c0 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.841+0000 7fdc87fff640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af566f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc8d509640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5674f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc800d1230 tx=0x7fdc8009f720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a9bea60ba16a66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc67fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc67fff640 1 -- 172.21.15.103:0/3545902318 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc800d16b0 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc7c004230 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc67fff640 1 -- 172.21.15.103:0/3545902318 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fdc800a85a0 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc67fff640 1 -- 172.21.15.103:0/3545902318 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdc8009fdc0 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc91345740 10 client.26609 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5561af589ac0 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc67fff640 1 -- 172.21.15.103:0/3545902318 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7c004230 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc67fff640 1 -- 172.21.15.103:0/3545902318 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fdc8000c020 con 0x5561af56a370 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc91345740 10 client.26609 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:15.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc91345740 10 client.26609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:46:15.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc91345740 20 client.26609 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:15.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.842+0000 7fdc91345740 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af58aed0 0x5561af58d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.843+0000 7fdc91345740 4 client.26609 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:15.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.843+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561af4d2860 con 0x5561af58aed0 2024-09-19T07:46:15.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.843+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af58aed0 0x5561af58d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.844+0000 7fdc67fff640 10 client.26609 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.845+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af58aed0 0x5561af58d350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=682366c837d8931 server_cookie=7076d59e51ab96ec in_seq=0 out_seq=0 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.845+0000 7fdc67fff640 1 -- 172.21.15.103:0/3545902318 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561af4d2860 con 0x5561af58aed0 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.845+0000 7fdc67fff640 10 client.26609 handle_command_reply: tid=0 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.845+0000 7fdc91345740 1 client.26609 shutdown 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 msgr2=0x5561af5674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5674f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc800d1230 tx=0x7fdc8009f720 comp rx=0 tx=0).stop 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 shutdown_connections 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af58aed0 0x5561af58d350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5674f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af566f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 --2- 172.21.15.103:0/3545902318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af55f5d0 0x5561af569ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc67fff640 0 client.26609 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:46:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 >> 172.21.15.103:0/3545902318 conn(0x5561af528310 msgr2=0x5561af561960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 shutdown_connections 2024-09-19T07:46:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 1 -- 172.21.15.103:0/3545902318 wait complete. 2024-09-19T07:46:15.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.846+0000 7fdc91345740 20 client.26609 trim_cache size 0 max 16384 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 Processor -- start 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 -- start start 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af55f5d0 0x5561af587320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af55abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af56a370 0x5561af5620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ab750 con 0x5561af56a370 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ad290 con 0x5561af55f5d0 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc91345740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881abe20 con 0x5561af560b40 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc8d509640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af56a370 0x5561af5620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc8d509640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af56a370 0x5561af5620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43626/0 (socket says 172.21.15.103:43626) 2024-09-19T07:46:15.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.865+0000 7fdc8d509640 1 -- 172.21.15.103:0/2101736879 learned_addr learned my addr 172.21.15.103:0/2101736879 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc87fff640 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af55abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af55f5d0 0x5561af587320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2101736879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 msgr2=0x5561af55abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af55abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2101736879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af56a370 msgr2=0x5561af5620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af56a370 0x5561af5620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2101736879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561af5654e0 con 0x5561af55f5d0 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af55f5d0 0x5561af587320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c003ea0 tx=0x7fdc7c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0e2c48fcc2d897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc727fc640 1 -- 172.21.15.103:0/2101736879 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c053070 con 0x5561af55f5d0 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc727fc640 1 -- 172.21.15.103:0/2101736879 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7c003ad0 con 0x5561af55f5d0 2024-09-19T07:46:15.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.866+0000 7fdc727fc640 1 -- 172.21.15.103:0/2101736879 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c037040 con 0x5561af55f5d0 2024-09-19T07:46:15.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 -- 172.21.15.103:0/2101736879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af55f5d0 msgr2=0x5561af587320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af55f5d0 0x5561af587320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c003ea0 tx=0x7fdc7c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 -- 172.21.15.103:0/2101736879 shutdown_connections 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af56a370 0x5561af5620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af560b40 0x5561af55abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 --2- 172.21.15.103:0/2101736879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af55f5d0 0x5561af587320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 -- 172.21.15.103:0/2101736879 >> 172.21.15.103:0/2101736879 conn(0x5561af528310 msgr2=0x5561af55a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 -- 172.21.15.103:0/2101736879 shutdown_connections 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.867+0000 7fdc91345740 1 -- 172.21.15.103:0/2101736879 wait complete. 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.868+0000 7fdc91345740 1 -- start start 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.868+0000 7fdc91345740 1 Processor -- start 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc91345740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af583670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc91345740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af583bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc91345740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc91345740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881acbc0 con 0x5561af560b40 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc91345740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ac4f0 con 0x5561af56a370 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc91345740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881a9750 con 0x5561af55f5d0 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc91345740 10 client.0 _ll_register_callbacks cb 0x7fdc8000f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af583670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af583670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:32932/0 (socket says 172.21.15.103:32932) 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2822480764 learned_addr learned my addr 172.21.15.103:0/2822480764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc87fff640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af583bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8d509640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2822480764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 msgr2=0x5561af5637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2822480764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 msgr2=0x5561af583bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af583bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 -- 172.21.15.103:0/2822480764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561af563e30 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc87fff640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af583bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc8cd08640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af583670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0095f0 tx=0x7fdc7c0054a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db8e5f5152134bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.869+0000 7fdc67fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc67fff640 1 -- 172.21.15.103:0/2822480764 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c053070 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc67fff640 1 -- 172.21.15.103:0/2822480764 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fdc7c004160 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc7c003ad0 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc67fff640 1 -- 172.21.15.103:0/2822480764 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c05c050 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 10 client.26533 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5561af5f5570 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc67fff640 1 -- 172.21.15.103:0/2822480764 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc7c005250 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc67fff640 1 -- 172.21.15.103:0/2822480764 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fdc7c056020 con 0x5561af55f5d0 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 10 client.26533 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:15.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 10 client.26533 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:46:15.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 20 client.26533 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:15.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af5f5840 0x5561af5f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:15.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 4 client.26533 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:46:15.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdc7c003ad0 con 0x5561af5f5840 2024-09-19T07:46:15.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.870+0000 7fdc87fff640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af5f5840 0x5561af5f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:15.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.871+0000 7fdc87fff640 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af5f5840 0x5561af5f7c30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d27408f1b62192ab server_cookie=633020164c55f4c in_seq=0 out_seq=0 2024-09-19T07:46:15.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.871+0000 7fdc67fff640 10 client.26533 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:46:15.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.871+0000 7fdc67fff640 1 -- 172.21.15.103:0/2822480764 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdc7c003ad0 con 0x5561af5f5840 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.871+0000 7fdc67fff640 10 client.26533 handle_command_reply: tid=0 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.871+0000 7fdc91345740 1 client.26533 shutdown 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 msgr2=0x5561af583670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af583670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0095f0 tx=0x7fdc7c0054a0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 shutdown_connections 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5561af5f5840 0x5561af5f7c30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5561af56a370 0x5561af5637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc67fff640 0 client.26533 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561af560b40 0x5561af583bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 --2- 172.21.15.103:0/2822480764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561af55f5d0 0x5561af583670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.872+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 >> 172.21.15.103:0/2822480764 conn(0x5561af528310 msgr2=0x5561af569010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:15.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.873+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 shutdown_connections 2024-09-19T07:46:15.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.873+0000 7fdc91345740 1 -- 172.21.15.103:0/2822480764 wait complete. 2024-09-19T07:46:15.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.874+0000 7fdc91345740 20 client.26533 trim_cache size 0 max 16384 2024-09-19T07:46:15.877 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.874+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdc780a3470 msgr2=0x7fdc780a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.874+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdc780a3470 0x7fdc780a5930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc880fed50 tx=0x7fdc740095e0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.874+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88102ef0 msgr2=0x7fdc8819aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.874+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88102ef0 0x7fdc8819aca0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009d20 tx=0x7fdc80007590 comp rx=0 tx=0).stop 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 shutdown_connections 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdc780a3470 0x7fdc780a5930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88102ef0 0x7fdc8819aca0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc88102570 0x7fdc88196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 --2- 172.21.15.103:0/1346999415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc88100e20 0x7fdc881963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 >> 172.21.15.103:0/1346999415 conn(0x7fdc880fc7d0 msgr2=0x7fdc88104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 shutdown_connections 2024-09-19T07:46:15.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:15.875+0000 7fdc8ef6c640 1 -- 172.21.15.103:0/1346999415 wait complete. 2024-09-19T07:46:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:15 smithi103 ceph-mon[31201]: pgmap v255: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:15 smithi134 ceph-mon[35414]: pgmap v255: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:15 smithi174 ceph-mon[35370]: pgmap v255: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:16.504 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '868342b1', total seconds elapsed: 32.1 2024-09-19T07:46:17.250 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:46:17.588 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:18.028 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:17 smithi103 ceph-mon[31201]: pgmap v256: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:18.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.146+0000 7f86be883640 1 -- 172.21.15.103:0/2197468961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 msgr2=0x7f86b810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.147+0000 7f86be883640 1 --2- 172.21.15.103:0/2197468961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 0x7f86b810eaa0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009ca0 tx=0x7f86b002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.147+0000 7f86be883640 1 -- 172.21.15.103:0/2197468961 shutdown_connections 2024-09-19T07:46:18.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.147+0000 7f86be883640 1 --2- 172.21.15.103:0/2197468961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 0x7f86b810eaa0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.147+0000 7f86be883640 1 --2- 172.21.15.103:0/2197468961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b81027a0 0x7f86b81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.147+0000 7f86be883640 1 --2- 172.21.15.103:0/2197468961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b8101df0 0x7f86b81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.147+0000 7f86be883640 1 -- 172.21.15.103:0/2197468961 >> 172.21.15.103:0/2197468961 conn(0x7f86b80fd8f0 msgr2=0x7f86b80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:18.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.148+0000 7f86be883640 1 -- 172.21.15.103:0/2197468961 shutdown_connections 2024-09-19T07:46:18.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.148+0000 7f86be883640 1 -- 172.21.15.103:0/2197468961 wait complete. 2024-09-19T07:46:18.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 Processor -- start 2024-09-19T07:46:18.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 -- start start 2024-09-19T07:46:18.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8101df0 0x7f86b8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b81027a0 0x7f86b8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 0x7f86b819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8111f80 con 0x7f86b8106f20 2024-09-19T07:46:18.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8111e00 con 0x7f86b81027a0 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86be883640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8112100 con 0x7f86b8101df0 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86bce20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 0x7f86b819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8101df0 0x7f86b8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.149+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b81027a0 0x7f86b8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86bce20640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 0x7f86b819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43652/0 (socket says 172.21.15.103:43652) 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86bce20640 1 -- 172.21.15.103:0/3771750030 learned_addr learned my addr 172.21.15.103:0/3771750030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b7fff640 1 -- 172.21.15.103:0/3771750030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b81027a0 msgr2=0x7f86b8198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b7fff640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b81027a0 0x7f86b8198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b7fff640 1 -- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 msgr2=0x7f86b819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b7fff640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 0x7f86b819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b7fff640 1 -- 172.21.15.103:0/3771750030 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b819d160 con 0x7f86b8101df0 2024-09-19T07:46:18.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b7fff640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8101df0 0x7f86b8198350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86a800c970 tx=0x7f86a800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:18.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b57fa640 1 -- 172.21.15.103:0/3771750030 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8007bb0 con 0x7f86b8101df0 2024-09-19T07:46:18.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b57fa640 1 -- 172.21.15.103:0/3771750030 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a8007d50 con 0x7f86b8101df0 2024-09-19T07:46:18.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.150+0000 7f86b57fa640 1 -- 172.21.15.103:0/3771750030 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8018760 con 0x7f86b8101df0 2024-09-19T07:46:18.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.151+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b819d3f0 con 0x7f86b8101df0 2024-09-19T07:46:18.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.151+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b819d8e0 con 0x7f86b8101df0 2024-09-19T07:46:18.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.154+0000 7f86b57fa640 1 -- 172.21.15.103:0/3771750030 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f86a800f450 con 0x7f86b8101df0 2024-09-19T07:46:18.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 Processor -- start 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 -- start start 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f5f5d0 0x5643a6f5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f6a370 0x5643a6f6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5643a6ec52f0 con 0x5643a6f5f5d0 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5643a6e6ccf0 con 0x5643a6f60b40 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643a6ec12b0 con 0x5643a6f6a370 2024-09-19T07:46:18.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.155+0000 7f86b57fa640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f86a00a3500 0x7f86a00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f5f5d0 0x5643a6f5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f86a00a3500 0x7f86a00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f5f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56222/0 (socket says 172.21.15.103:56222) 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 -- 172.21.15.103:0/73688198 learned_addr learned my addr 172.21.15.103:0/73688198 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 -- 172.21.15.103:0/73688198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f6a370 msgr2=0x5643a6f6c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f6a370 0x5643a6f6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 -- 172.21.15.103:0/73688198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f5f5d0 msgr2=0x5643a6f5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f5f5d0 0x5643a6f5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86b57fa640 1 -- 172.21.15.103:0/3771750030 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f86a80cfce0 con 0x7f86b8101df0 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 -- 172.21.15.103:0/73688198 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a6e513a0 con 0x5643a6f60b40 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f86a00a3500 0x7f86a00a59c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009a60 tx=0x7f86b0005f60 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:18.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86b7fff640 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f5f5d0 0x5643a6f5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:46:18.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.156+0000 7f86bce20640 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f5f090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f86b00039c0 tx=0x7f86b004b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dfb86e320bd31bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:18.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.157+0000 7f869e7fc640 1 -- 172.21.15.103:0/73688198 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0042030 con 0x5643a6f60b40 2024-09-19T07:46:18.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.157+0000 7f869e7fc640 1 -- 172.21.15.103:0/73688198 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b004f070 con 0x5643a6f60b40 2024-09-19T07:46:18.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.157+0000 7f869e7fc640 1 -- 172.21.15.103:0/73688198 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0045040 con 0x5643a6f60b40 2024-09-19T07:46:18.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.157+0000 7f86c0c5c740 1 -- 172.21.15.103:0/73688198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 msgr2=0x5643a6f5f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.157+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f5f090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f86b00039c0 tx=0x7f86b004b000 comp rx=0 tx=0).stop 2024-09-19T07:46:18.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.158+0000 7f86c0c5c740 1 -- 172.21.15.103:0/73688198 shutdown_connections 2024-09-19T07:46:18.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.158+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f6a370 0x5643a6f6c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.158+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f5f5d0 0x5643a6f5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.158+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/73688198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f5f090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.158+0000 7f86c0c5c740 1 -- 172.21.15.103:0/73688198 >> 172.21.15.103:0/73688198 conn(0x5643a6f28310 msgr2=0x5643a6f5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.158+0000 7f86c0c5c740 1 -- 172.21.15.103:0/73688198 shutdown_connections 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.158+0000 7f86c0c5c740 1 -- 172.21.15.103:0/73688198 wait complete. 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.159+0000 7f86c0c5c740 1 -- start start 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.159+0000 7f86c0c5c740 1 Processor -- start 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f69d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f67030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f67640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f67030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f67030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:47588/0 (socket says 172.21.15.103:47588) 2024-09-19T07:46:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f69d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b77fe640 1 -- 172.21.15.103:0/3846816466 learned_addr learned my addr 172.21.15.103:0/3846816466 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5643a6f6d010 con 0x5643a6f6a370 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5643a6e80060 con 0x5643a6f60b40 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643a6f6d190 con 0x5643a6f5f5d0 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86bce20640 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f67640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86c0c5c740 10 client.0 _ll_register_callbacks cb 0x5643a6bd38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b7fff640 1 -- 172.21.15.103:0/3846816466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 msgr2=0x5643a6f67030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b7fff640 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f67030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b7fff640 1 -- 172.21.15.103:0/3846816466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 msgr2=0x5643a6f67640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b7fff640 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f67640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.160+0000 7f86b7fff640 1 -- 172.21.15.103:0/3846816466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a6c363c0 con 0x5643a6f5f5d0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b004f070 con 0x5643a6f5f5d0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86b7fff640 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f69d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f86a80cd3c0 tx=0x7f86a809be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2133ef6b2cb0a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86977fe640 10 client.26542 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86977fe640 1 -- 172.21.15.103:0/3846816466 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a809bf50 con 0x5643a6f5f5d0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86977fe640 1 -- 172.21.15.103:0/3846816466 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f86a80985c0 con 0x5643a6f5f5d0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86977fe640 1 -- 172.21.15.103:0/3846816466 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f86a80a0300 con 0x5643a6f5f5d0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86c0c5c740 10 client.26542 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.161+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=40}) -- 0x5643a6f78a00 con 0x5643a6f5f5d0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86977fe640 1 -- 172.21.15.103:0/3846816466 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f86a80a4730 con 0x5643a6f5f5d0 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86c0c5c740 10 client.26542 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:18.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86c0c5c740 10 client.26542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:46:18.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86c0c5c740 20 client.26542 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:18.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6f7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86c0c5c740 4 client.26542 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:18.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643a6ed2860 con 0x5643a6f79db0 2024-09-19T07:46:18.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.162+0000 7f86b77fe640 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6f7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.163+0000 7f86977fe640 10 client.26542 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:46:18.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.163+0000 7f86b77fe640 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6f7c1e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc6709016f67d180 server_cookie=e25ad5c2e9a80c0a in_seq=0 out_seq=0 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86977fe640 1 -- 172.21.15.103:0/3846816466 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643a6ed2860 con 0x5643a6f79db0 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86977fe640 10 client.26542 handle_command_reply: tid=0 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 client.26542 shutdown 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 msgr2=0x5643a6f69d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f69d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f86a80cd3c0 tx=0x7f86a809be10 comp rx=0 tx=0).stop 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 shutdown_connections 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6f7c1e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86977fe640 0 client.26542 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f67640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f60b40 0x5643a6f67030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/3846816466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f69d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 >> 172.21.15.103:0/3846816466 conn(0x5643a6f28310 msgr2=0x5643a6f61960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:18.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 shutdown_connections 2024-09-19T07:46:18.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 1 -- 172.21.15.103:0/3846816466 wait complete. 2024-09-19T07:46:18.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.164+0000 7f86c0c5c740 20 client.26542 trim_cache size 0 max 16384 2024-09-19T07:46:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 Processor -- start 2024-09-19T07:46:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 -- start start 2024-09-19T07:46:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f5f5d0 0x5643a6f62120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f60b40 0x5643a6f62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f81ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81b3e80 con 0x5643a6f6a370 2024-09-19T07:46:18.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81b59c0 con 0x5643a6f5f5d0 2024-09-19T07:46:18.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.182+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81b4550 con 0x5643a6f60b40 2024-09-19T07:46:18.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.183+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f60b40 0x5643a6f62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.183+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f60b40 0x5643a6f62660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46994/0 (socket says 172.21.15.103:46994) 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.183+0000 7f86b77fe640 1 -- 172.21.15.103:0/4091880927 learned_addr learned my addr 172.21.15.103:0/4091880927 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.183+0000 7f86bce20640 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f81ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f86b77fe640 1 -- 172.21.15.103:0/4091880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f5f5d0 msgr2=0x5643a6f62120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f86b77fe640 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f5f5d0 0x5643a6f62120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f86b77fe640 1 -- 172.21.15.103:0/4091880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 msgr2=0x5643a6f81ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f86b77fe640 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f81ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f86b77fe640 1 -- 172.21.15.103:0/4091880927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a6f65580 con 0x5643a6f60b40 2024-09-19T07:46:18.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f86b77fe640 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f60b40 0x5643a6f62660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5643a6f668c0 tx=0x7f86ac010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ce06bf3d050885 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:18.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f869dffb640 1 -- 172.21.15.103:0/4091880927 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac010e60 con 0x5643a6f60b40 2024-09-19T07:46:18.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f869dffb640 1 -- 172.21.15.103:0/4091880927 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac01db50 con 0x5643a6f60b40 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.184+0000 7f869dffb640 1 -- 172.21.15.103:0/4091880927 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac01ce60 con 0x5643a6f60b40 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4091880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f60b40 msgr2=0x5643a6f62660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f60b40 0x5643a6f62660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5643a6f668c0 tx=0x7f86ac010800 comp rx=0 tx=0).stop 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4091880927 shutdown_connections 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f6a370 0x5643a6f81ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f60b40 0x5643a6f62660 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4091880927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f5f5d0 0x5643a6f62120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4091880927 >> 172.21.15.103:0/4091880927 conn(0x5643a6f28310 msgr2=0x5643a6f5ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4091880927 shutdown_connections 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.185+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4091880927 wait complete. 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.186+0000 7f86c0c5c740 1 -- start start 2024-09-19T07:46:18.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.186+0000 7f86c0c5c740 1 Processor -- start 2024-09-19T07:46:18.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.186+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f6e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.186+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f60b40 0x5643a6f6ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.186+0000 7f86c0c5c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f6a370 0x5643a6fe3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.186+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81b52f0 con 0x5643a6f60b40 2024-09-19T07:46:18.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81b4c20 con 0x5643a6f6a370 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86c0c5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81b1e80 con 0x5643a6f5f5d0 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f6e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86c0c5c740 10 client.0 _ll_register_callbacks cb 0x7f86a800fd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f6e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46996/0 (socket says 172.21.15.103:46996) 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b7fff640 1 -- 172.21.15.103:0/4166781819 learned_addr learned my addr 172.21.15.103:0/4166781819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b77fe640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f60b40 0x5643a6f6ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86bce20640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f6a370 0x5643a6fe3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b77fe640 1 -- 172.21.15.103:0/4166781819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 msgr2=0x5643a6f6e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b77fe640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f6e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b77fe640 1 -- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f6a370 msgr2=0x5643a6fe3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:18.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b77fe640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f6a370 0x5643a6fe3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b77fe640 1 -- 172.21.15.103:0/4166781819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a6fe3fc0 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b7fff640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86b77fe640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f60b40 0x5643a6f6ed50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x5643a6f67c80 tx=0x7f86ac009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03a9aca6e2a2696 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86977fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86977fe640 1 -- 172.21.15.103:0/4166781819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac01ccf0 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86977fe640 1 -- 172.21.15.103:0/4166781819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f86ac01c3d0 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.187+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac01db50 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86977fe640 1 -- 172.21.15.103:0/4166781819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f86ac036050 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 10 client.17175 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5643a6fe51c0 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86977fe640 1 -- 172.21.15.103:0/4166781819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac01db50 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86977fe640 1 -- 172.21.15.103:0/4166781819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f86ac0051e0 con 0x5643a6f60b40 2024-09-19T07:46:18.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 10 client.17175 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 10 client.17175 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 20 client.17175 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6fe77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 4 client.17175 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643a6eccd70 con 0x5643a6f79db0 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.188+0000 7f86b7fff640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6fe77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.189+0000 7f86b7fff640 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6fe77e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=676abc6e8348b5c4 server_cookie=24d2b46b773bd2a5 in_seq=0 out_seq=0 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.189+0000 7f86977fe640 10 client.17175 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.189+0000 7f86977fe640 1 -- 172.21.15.103:0/4166781819 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643a6eccd70 con 0x5643a6f79db0 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.189+0000 7f86977fe640 10 client.17175 handle_command_reply: tid=0 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.189+0000 7f86c0c5c740 1 client.17175 shutdown 2024-09-19T07:46:18.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f60b40 msgr2=0x5643a6f6ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f60b40 0x5643a6f6ed50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x5643a6f67c80 tx=0x7f86ac009f40 comp rx=0 tx=0).stop 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 shutdown_connections 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5643a6f79db0 0x5643a6fe77e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643a6f6a370 0x5643a6fe3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5643a6f60b40 0x5643a6f6ed50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 --2- 172.21.15.103:0/4166781819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a6f5f5d0 0x5643a6f6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 >> 172.21.15.103:0/4166781819 conn(0x5643a6f28310 msgr2=0x5643a6f68890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86977fe640 0 client.17175 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 shutdown_connections 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 1 -- 172.21.15.103:0/4166781819 wait complete. 2024-09-19T07:46:18.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.190+0000 7f86c0c5c740 20 client.17175 trim_cache size 0 max 16384 2024-09-19T07:46:18.196 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:46:18.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f86a00a3500 msgr2=0x7f86a00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f86a00a3500 0x7f86a00a59c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009a60 tx=0x7f86b0005f60 comp rx=0 tx=0).stop 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8101df0 msgr2=0x7f86b8198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8101df0 0x7f86b8198350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86a800c970 tx=0x7f86a800ce30 comp rx=0 tx=0).stop 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 shutdown_connections 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f86a00a3500 0x7f86a00a59c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86b8106f20 0x7f86b819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b81027a0 0x7f86b8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 --2- 172.21.15.103:0/3771750030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8101df0 0x7f86b8198350 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 >> 172.21.15.103:0/3771750030 conn(0x7f86b80fd8f0 msgr2=0x7f86b80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 shutdown_connections 2024-09-19T07:46:18.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:18.191+0000 7f86be883640 1 -- 172.21.15.103:0/3771750030 wait complete. 2024-09-19T07:46:18.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:17 smithi134 ceph-mon[35414]: pgmap v256: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:17 smithi174 ceph-mon[35370]: pgmap v256: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:19.027 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:46:19.404 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:20.047 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:19 smithi103 ceph-mon[31201]: pgmap v257: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:20.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.177+0000 7f8d3020d640 1 -- 172.21.15.103:0/145490250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d28109450 msgr2=0x7f8d28110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.177+0000 7f8d3020d640 1 --2- 172.21.15.103:0/145490250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d28109450 0x7f8d28110250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-09-19T07:46:20.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.177+0000 7f8d3020d640 1 -- 172.21.15.103:0/145490250 shutdown_connections 2024-09-19T07:46:20.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.177+0000 7f8d3020d640 1 --2- 172.21.15.103:0/145490250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d2811b140 0x7f8d2811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.177+0000 7f8d3020d640 1 --2- 172.21.15.103:0/145490250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d28109830 0x7f8d28109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.177+0000 7f8d3020d640 1 --2- 172.21.15.103:0/145490250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d28109450 0x7f8d28110250 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009900 tx=0x7f8d1c02a410 comp rx=0 tx=0).stop 2024-09-19T07:46:20.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.177+0000 7f8d3020d640 1 -- 172.21.15.103:0/145490250 >> 172.21.15.103:0/145490250 conn(0x7f8d2806e140 msgr2=0x7f8d2806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:20.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.179+0000 7f8d3020d640 1 -- 172.21.15.103:0/145490250 shutdown_connections 2024-09-19T07:46:20.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.179+0000 7f8d3020d640 1 -- 172.21.15.103:0/145490250 wait complete. 2024-09-19T07:46:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.180+0000 7f8d3020d640 1 Processor -- start 2024-09-19T07:46:20.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.180+0000 7f8d3020d640 1 -- start start 2024-09-19T07:46:20.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d3020d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d28109830 0x7f8d281a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d3020d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2811b140 0x7f8d281a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d3020d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d281abd80 0x7f8d281ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d3020d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d28120e10 con 0x7f8d2811b140 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d3020d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d28120c90 con 0x7f8d28109830 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d3020d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d28120f90 con 0x7f8d281abd80 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d2e7aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d281abd80 0x7f8d281ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d2e7aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d281abd80 0x7f8d281ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47016/0 (socket says 172.21.15.103:47016) 2024-09-19T07:46:20.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.184+0000 7f8d2e7aa640 1 -- 172.21.15.103:0/3541225065 learned_addr learned my addr 172.21.15.103:0/3541225065 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2811b140 0x7f8d281a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2e7aa640 1 -- 172.21.15.103:0/3541225065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d28109830 msgr2=0x7f8d281a74b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2e7aa640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d28109830 0x7f8d281a74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2e7aa640 1 -- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2811b140 msgr2=0x7f8d281a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2e7aa640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2811b140 0x7f8d281a79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2e7aa640 1 -- 172.21.15.103:0/3541225065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d281ac840 con 0x7f8d281abd80 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2811b140 0x7f8d281a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:46:20.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d2e7aa640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d281abd80 0x7f8d281ac230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000c410 tx=0x7f8d20001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d26ffd640 1 -- 172.21.15.103:0/3541225065 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20004020 con 0x7f8d281abd80 2024-09-19T07:46:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d26ffd640 1 -- 172.21.15.103:0/3541225065 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d200041c0 con 0x7f8d281abd80 2024-09-19T07:46:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.185+0000 7f8d26ffd640 1 -- 172.21.15.103:0/3541225065 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20011a10 con 0x7f8d281abd80 2024-09-19T07:46:20.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.186+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d281acb30 con 0x7f8d281abd80 2024-09-19T07:46:20.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.186+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d281b42c0 con 0x7f8d281abd80 2024-09-19T07:46:20.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.193+0000 7f8d325e6740 1 Processor -- start 2024-09-19T07:46:20.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.193+0000 7f8d26ffd640 1 -- 172.21.15.103:0/3541225065 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8d20019030 con 0x7f8d281abd80 2024-09-19T07:46:20.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.194+0000 7f8d325e6740 1 -- start start 2024-09-19T07:46:20.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.195+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 0x5581bcaf4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.195+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcaf4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.195+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.195+0000 7f8d325e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5581bca5a2f0 con 0x5581bcaf5b40 2024-09-19T07:46:20.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.195+0000 7f8d325e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5581bca01cf0 con 0x5581bcaf45d0 2024-09-19T07:46:20.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.195+0000 7f8d325e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581bca562b0 con 0x5581bcaff370 2024-09-19T07:46:20.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.196+0000 7f8d26ffd640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8d100a3580 0x7f8d100a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.196+0000 7f8d26ffd640 1 -- 172.21.15.103:0/3541225065 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f8d200ce840 con 0x7f8d281abd80 2024-09-19T07:46:20.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.196+0000 7f8d2dfa9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcaf4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.196+0000 7f8d2dfa9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcaf4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:47642/0 (socket says 172.21.15.103:47642) 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.196+0000 7f8d2dfa9640 1 -- 172.21.15.103:0/3208215906 learned_addr learned my addr 172.21.15.103:0/3208215906 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.196+0000 7f8d2e7aa640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8d100a3580 0x7f8d100a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.197+0000 7f8d2e7aa640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8d100a3580 0x7f8d100a5a40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8d200cc090 tx=0x7f8d20000f70 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.197+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.197+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 0x5581bcaf4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.197+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/3208215906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 msgr2=0x5581bcaf4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.197+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcaf4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.197+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/3208215906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 msgr2=0x5581bcaf4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.197+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 0x5581bcaf4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.198+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/3208215906 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581bc9e63a0 con 0x5581bcaff370 2024-09-19T07:46:20.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.198+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb01760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005fd0 tx=0x7f8d18009a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b4c3b90669ff75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:20.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.198+0000 7f8d0ffff640 1 -- 172.21.15.103:0/3208215906 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18019030 con 0x5581bcaff370 2024-09-19T07:46:20.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.198+0000 7f8d0ffff640 1 -- 172.21.15.103:0/3208215906 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d18013df0 con 0x5581bcaff370 2024-09-19T07:46:20.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.199+0000 7f8d325e6740 1 -- 172.21.15.103:0/3208215906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 msgr2=0x5581bcb01760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.199+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb01760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005fd0 tx=0x7f8d18009a30 comp rx=0 tx=0).stop 2024-09-19T07:46:20.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.202+0000 7f8d325e6740 1 -- 172.21.15.103:0/3208215906 shutdown_connections 2024-09-19T07:46:20.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.202+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb01760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.202+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcaf4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.202+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3208215906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 0x5581bcaf4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.202+0000 7f8d325e6740 1 -- 172.21.15.103:0/3208215906 >> 172.21.15.103:0/3208215906 conn(0x5581bcabd310 msgr2=0x5581bcaf0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:20.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.202+0000 7f8d325e6740 1 -- 172.21.15.103:0/3208215906 shutdown_connections 2024-09-19T07:46:20.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.202+0000 7f8d325e6740 1 -- 172.21.15.103:0/3208215906 wait complete. 2024-09-19T07:46:20.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 -- start start 2024-09-19T07:46:20.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 Processor -- start 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcafec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf5b40 0x5581bcafbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcafc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5581bcb020c0 con 0x5581bcaff370 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5581bca15060 con 0x5581bcaf45d0 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581bcb02240 con 0x5581bcaf5b40 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.203+0000 7f8d325e6740 10 client.0 _ll_register_callbacks cb 0x5581bc7688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2d7a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcafec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf5b40 0x5581bcafbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2d7a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcafec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:47658/0 (socket says 172.21.15.103:47658) 2024-09-19T07:46:20.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf5b40 0x5581bcafbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47032/0 (socket says 172.21.15.103:47032) 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 -- 172.21.15.103:0/2222910113 learned_addr learned my addr 172.21.15.103:0/2222910113 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcafc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 -- 172.21.15.103:0/2222910113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 msgr2=0x5581bcafec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcafec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 -- 172.21.15.103:0/2222910113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 msgr2=0x5581bcafc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcafc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 -- 172.21.15.103:0/2222910113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581bc7cb3c0 con 0x5581bcaf5b40 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcafc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d2dfa9640 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf5b40 0x5581bcafbf10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c0042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5cde78fcb071bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.204+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2222910113 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c040070 con 0x5581bcaf5b40 2024-09-19T07:46:20.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.205+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d18013df0 con 0x5581bcaf5b40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.205+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2222910113 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f8d1c005040 con 0x5581bcaf5b40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.205+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2222910113 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8d1c04a050 con 0x5581bcaf5b40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.205+0000 7f8d325e6740 10 client.26560 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.205+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5581bcb0db50 con 0x5581bcaf5b40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2222910113 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d18013df0 con 0x5581bcaf5b40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2222910113 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f8d1c0035f0 con 0x5581bcaf5b40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d325e6740 10 client.26560 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d325e6740 10 client.26560 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d325e6740 20 client.26560 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb0efb0 0x5581bcb113d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d325e6740 4 client.26560 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.206+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581bca67860 con 0x5581bcb0efb0 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.207+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb0efb0 0x5581bcb113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.207+0000 7f8d0d7fa640 10 client.26560 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.207+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb0efb0 0x5581bcb113d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16116df54c4ec1c1 server_cookie=33d5a437cecad8a8 in_seq=0 out_seq=0 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2222910113 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581bca67860 con 0x5581bcb0efb0 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d0d7fa640 10 client.26560 handle_command_reply: tid=0 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 client.26560 shutdown 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf5b40 msgr2=0x5581bcafbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf5b40 0x5581bcafbf10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009230 tx=0x7f8d1c0042e0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 shutdown_connections 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb0efb0 0x5581bcb113d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcafc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf5b40 0x5581bcafbf10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2222910113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcafec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d0d7fa640 0 client.26560 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:46:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 >> 172.21.15.103:0/2222910113 conn(0x5581bcabd310 msgr2=0x5581bcaf6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 shutdown_connections 2024-09-19T07:46:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 1 -- 172.21.15.103:0/2222910113 wait complete. 2024-09-19T07:46:20.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.208+0000 7f8d325e6740 20 client.26560 trim_cache size 0 max 16384 2024-09-19T07:46:20.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.223+0000 7f8d325e6740 1 Processor -- start 2024-09-19T07:46:20.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d325e6740 1 -- start start 2024-09-19T07:46:20.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf45d0 0x5581bcb0b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf5b40 0x5581bcafde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcb00760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d2d7a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf45d0 0x5581bcb0b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d325e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1400eab0 con 0x5581bcaff370 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d325e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d140105f0 con 0x5581bcaf5b40 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d325e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1400f180 con 0x5581bcaf45d0 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d2d7a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf45d0 0x5581bcb0b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47036/0 (socket says 172.21.15.103:47036) 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.224+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/3886693917 learned_addr learned my addr 172.21.15.103:0/3886693917 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2dfa9640 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf5b40 0x5581bcafde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/3886693917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf5b40 msgr2=0x5581bcafde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf5b40 0x5581bcafde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/3886693917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 msgr2=0x5581bcb00760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcb00760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/3886693917 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581bcafa460 con 0x5581bcaf45d0 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2dfa9640 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf5b40 0x5581bcafde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf45d0 0x5581bcb0b440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005c40 tx=0x7f8d18011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95da62c291ac78fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:20.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d0f7fe640 1 -- 172.21.15.103:0/3886693917 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18019040 con 0x5581bcaf45d0 2024-09-19T07:46:20.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d0f7fe640 1 -- 172.21.15.103:0/3886693917 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d18003970 con 0x5581bcaf45d0 2024-09-19T07:46:20.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.225+0000 7f8d0f7fe640 1 -- 172.21.15.103:0/3886693917 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18005020 con 0x5581bcaf45d0 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.226+0000 7f8d325e6740 1 -- 172.21.15.103:0/3886693917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf45d0 msgr2=0x5581bcb0b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.226+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf45d0 0x5581bcb0b440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005c40 tx=0x7f8d18011e20 comp rx=0 tx=0).stop 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.226+0000 7f8d325e6740 1 -- 172.21.15.103:0/3886693917 shutdown_connections 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.226+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaff370 0x5581bcb00760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.226+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf5b40 0x5581bcafde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.226+0000 7f8d325e6740 1 --2- 172.21.15.103:0/3886693917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaf45d0 0x5581bcb0b440 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.226+0000 7f8d325e6740 1 -- 172.21.15.103:0/3886693917 >> 172.21.15.103:0/3886693917 conn(0x5581bcabd310 msgr2=0x5581bcafd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:20.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.227+0000 7f8d325e6740 1 -- 172.21.15.103:0/3886693917 shutdown_connections 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.227+0000 7f8d325e6740 1 -- 172.21.15.103:0/3886693917 wait complete. 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.227+0000 7f8d325e6740 1 -- start start 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.227+0000 7f8d325e6740 1 Processor -- start 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.228+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcb07d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.228+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 0x5581bcb165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.228+0000 7f8d325e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb16ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.228+0000 7f8d325e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1400ff20 con 0x5581bcaf5b40 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.228+0000 7f8d325e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1400f850 con 0x5581bcaf45d0 2024-09-19T07:46:20.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.228+0000 7f8d325e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1400ca30 con 0x5581bcaff370 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb16ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d325e6740 10 client.0 _ll_register_callbacks cb 0x7f8d1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb16ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47040/0 (socket says 172.21.15.103:47040) 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/2874569176 learned_addr learned my addr 172.21.15.103:0/2874569176 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcb07d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/2874569176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 msgr2=0x5581bcb07d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcb07d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/2874569176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 msgr2=0x5581bcb165a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 0x5581bcb165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.229+0000 7f8d2d7a8640 1 -- 172.21.15.103:0/2874569176 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581bcb17310 con 0x5581bcaff370 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d2d7a8640 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb16ae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5581bcaff170 tx=0x7f8d18005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc07a21642292c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:46:20.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2874569176 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18019040 con 0x5581bcaff370 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2874569176 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f8d18003c80 con 0x5581bcaff370 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d18003970 con 0x5581bcaff370 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2874569176 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8d18003e80 con 0x5581bcaff370 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d325e6740 10 client.26566 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5581bcb79c50 con 0x5581bcaff370 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.230+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2874569176 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d180056e0 con 0x5581bcaff370 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2874569176 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f8d180048a0 con 0x5581bcaff370 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d325e6740 10 client.26566 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d325e6740 10 client.26566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d325e6740 20 client.26566 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb7a020 0x5581bcb7c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:20.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d325e6740 4 client.26566 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:46:20.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8d18003970 con 0x5581bcb7a020 2024-09-19T07:46:20.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.231+0000 7f8d2dfa9640 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb7a020 0x5581bcb7c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.232+0000 7f8d2dfa9640 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb7a020 0x5581bcb7c410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c4899af93a23e03 server_cookie=66581ce8ac4505ee in_seq=0 out_seq=0 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.232+0000 7f8d0d7fa640 10 client.26566 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.232+0000 7f8d0d7fa640 1 -- 172.21.15.103:0/2874569176 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8d18003970 con 0x5581bcb7a020 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.232+0000 7f8d0d7fa640 10 client.26566 handle_command_reply: tid=0 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.232+0000 7f8d325e6740 1 client.26566 shutdown 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.232+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 msgr2=0x5581bcb16ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.232+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb16ae0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5581bcaff170 tx=0x7f8d18005bd0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 shutdown_connections 2024-09-19T07:46:20.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5581bcb7a020 0x5581bcb7c410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581bcaff370 0x5581bcb16ae0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581bcaf5b40 0x5581bcb165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d0d7fa640 0 client.26566 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 --2- 172.21.15.103:0/2874569176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5581bcaf45d0 0x5581bcb07d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 >> 172.21.15.103:0/2874569176 conn(0x5581bcabd310 msgr2=0x5581bcafdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 shutdown_connections 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 1 -- 172.21.15.103:0/2874569176 wait complete. 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.233+0000 7f8d325e6740 20 client.26566 trim_cache size 0 max 16384 2024-09-19T07:46:20.239 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:46:20.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8d100a3580 msgr2=0x7f8d100a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8d100a3580 0x7f8d100a5a40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8d200cc090 tx=0x7f8d20000f70 comp rx=0 tx=0).stop 2024-09-19T07:46:20.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d281abd80 msgr2=0x7f8d281ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d281abd80 0x7f8d281ac230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000c410 tx=0x7f8d20001e70 comp rx=0 tx=0).stop 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 shutdown_connections 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8d100a3580 0x7f8d100a5a40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d281abd80 0x7f8d281ac230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2811b140 0x7f8d281a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 --2- 172.21.15.103:0/3541225065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d28109830 0x7f8d281a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 >> 172.21.15.103:0/3541225065 conn(0x7f8d2806e140 msgr2=0x7f8d2811c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 shutdown_connections 2024-09-19T07:46:20.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:20.234+0000 7f8d3020d640 1 -- 172.21.15.103:0/3541225065 wait complete. 2024-09-19T07:46:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:19 smithi134 ceph-mon[35414]: pgmap v257: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:19 smithi174 ceph-mon[35370]: pgmap v257: 97 pgs: 97 active+clean; 3.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-09-19T07:46:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 465.0 s, 2035.0 tps, lat 49.135 ms stddev 58.258 2024-09-19T07:46:21.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:46:22.031 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:22.197 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:21 smithi103 ceph-mon[31201]: pgmap v258: 97 pgs: 97 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.7 MiB/s wr, 425 op/s 2024-09-19T07:46:22.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:21 smithi134 ceph-mon[35414]: pgmap v258: 97 pgs: 97 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.7 MiB/s wr, 425 op/s 2024-09-19T07:46:22.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:21 smithi174 ceph-mon[35370]: pgmap v258: 97 pgs: 97 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.7 MiB/s wr, 425 op/s 2024-09-19T07:46:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.613+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2890938059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc4109120 msgr2=0x7f5fc4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.613+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2890938059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc4109120 0x7f5fc4110ca0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009ca0 tx=0x7f5fbc02a8c0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2890938059 shutdown_connections 2024-09-19T07:46:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2890938059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc4109120 0x7f5fc4110ca0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2890938059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc41049a0 0x7f5fc41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2890938059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4103ff0 0x7f5fc41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2890938059 >> 172.21.15.103:0/2890938059 conn(0x7f5fc40ffab0 msgr2=0x7f5fc4101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2890938059 shutdown_connections 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2890938059 wait complete. 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.614+0000 7f5fca7f7640 1 Processor -- start 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.615+0000 7f5fca7f7640 1 -- start start 2024-09-19T07:46:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.615+0000 7f5fca7f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4103ff0 0x7f5fc4110950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.615+0000 7f5fca7f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc41049a0 0x7f5fc410da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.615+0000 7f5fca7f7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc4109120 0x7f5fc410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4103ff0 0x7f5fc4110950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fca7f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4114130 con 0x7f5fc41049a0 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fca7f7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4113fb0 con 0x7f5fc4109120 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fca7f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41142b0 con 0x7f5fc4103ff0 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc41049a0 0x7f5fc410da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4103ff0 0x7f5fc4110950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47044/0 (socket says 172.21.15.103:47044) 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc3fff640 1 -- 172.21.15.103:0/2014564008 learned_addr learned my addr 172.21.15.103:0/2014564008 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc4109120 0x7f5fc410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc37fe640 1 -- 172.21.15.103:0/2014564008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4103ff0 msgr2=0x7f5fc4110950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4103ff0 0x7f5fc4110950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc37fe640 1 -- 172.21.15.103:0/2014564008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc4109120 msgr2=0x7f5fc410df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc4109120 0x7f5fc410df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.616+0000 7f5fc37fe640 1 -- 172.21.15.103:0/2014564008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc410e640 con 0x7f5fc41049a0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.617+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc41049a0 0x7f5fc410da00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800d6d0 tx=0x7f5fb800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.617+0000 7f5fc17fa640 1 -- 172.21.15.103:0/2014564008 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8017070 con 0x7f5fc41049a0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.617+0000 7f5fc17fa640 1 -- 172.21.15.103:0/2014564008 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb8009820 con 0x7f5fc41049a0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.617+0000 7f5fc17fa640 1 -- 172.21.15.103:0/2014564008 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8004d00 con 0x7f5fc41049a0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.617+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc41a2660 con 0x7f5fc41049a0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.617+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc41a2b50 con 0x7f5fc41049a0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.620+0000 7f5fccbd0740 1 Processor -- start 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.620+0000 7f5fc17fa640 1 -- 172.21.15.103:0/2014564008 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5fb80099d0 con 0x7f5fc41049a0 2024-09-19T07:46:22.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.620+0000 7f5fccbd0740 1 -- start start 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff888090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff888a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff895760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566ff7ee2f0 con 0x5566ff8885d0 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5566ff795cf0 con 0x5566ff893370 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566ff7ea2b0 con 0x5566ff889b40 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff888090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff888090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47056/0 (socket says 172.21.15.103:47056) 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff888a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 -- 172.21.15.103:0/181040577 learned_addr learned my addr 172.21.15.103:0/181040577 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc17fa640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5fac0a3500 0x7f5fac0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 -- 172.21.15.103:0/181040577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 msgr2=0x5566ff895760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/181040577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff895760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 -- 172.21.15.103:0/181040577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 msgr2=0x5566ff888a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/181040577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff888a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.621+0000 7f5fc8d94640 1 -- 172.21.15.103:0/181040577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ff77a3a0 con 0x5566ff889b40 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.622+0000 7f5fc17fa640 1 -- 172.21.15.103:0/2014564008 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f5fb80cf480 con 0x7f5fc41049a0 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.623+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5fac0a3500 0x7f5fac0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.623+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/181040577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff888090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc035000 tx=0x7f5fbc009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ed24c8f05f8f7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.624+0000 7f5faa7fc640 1 -- 172.21.15.103:0/181040577 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc031db0 con 0x5566ff889b40 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.624+0000 7f5faa7fc640 1 -- 172.21.15.103:0/181040577 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc036040 con 0x5566ff889b40 2024-09-19T07:46:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.624+0000 7f5faa7fc640 1 -- 172.21.15.103:0/181040577 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc030820 con 0x5566ff889b40 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 -- 172.21.15.103:0/181040577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 msgr2=0x5566ff888090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/181040577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff888090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc035000 tx=0x7f5fbc009540 comp rx=0 tx=0).stop 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5fac0a3500 0x7f5fac0a59c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc04a000 tx=0x7f5fbc030e50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 -- 172.21.15.103:0/181040577 shutdown_connections 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/181040577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff895760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/181040577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff888a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/181040577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff888090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 -- 172.21.15.103:0/181040577 >> 172.21.15.103:0/181040577 conn(0x5566ff851310 msgr2=0x5566ff884c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 -- 172.21.15.103:0/181040577 shutdown_connections 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.625+0000 7f5fccbd0740 1 -- 172.21.15.103:0/181040577 wait complete. 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.626+0000 7f5fccbd0740 1 -- start start 2024-09-19T07:46:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.626+0000 7f5fccbd0740 1 Processor -- start 2024-09-19T07:46:22.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff892d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff889b40 0x5566ff890040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff893370 0x5566ff890650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566ff895ef0 con 0x5566ff8885d0 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5566ff7a9060 con 0x5566ff889b40 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566ff896070 con 0x5566ff893370 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff892d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fccbd0740 10 client.0 _ll_register_callbacks cb 0x5566ff4fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff892d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39754/0 (socket says 172.21.15.103:39754) 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fc3fff640 1 -- 172.21.15.103:0/4253795300 learned_addr learned my addr 172.21.15.103:0/4253795300 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff889b40 0x5566ff890040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.627+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff893370 0x5566ff890650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.628+0000 7f5fc37fe640 1 -- 172.21.15.103:0/4253795300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff893370 msgr2=0x5566ff890650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.628+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff893370 0x5566ff890650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.628+0000 7f5fc37fe640 1 -- 172.21.15.103:0/4253795300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 msgr2=0x5566ff892d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.628+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff892d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.628+0000 7f5fc37fe640 1 -- 172.21.15.103:0/4253795300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ff55f3c0 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.628+0000 7f5fc3fff640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff892d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.629+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff889b40 0x5566ff890040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80ccbc0 tx=0x7f5fb809b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a72cbbce371ff02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.629+0000 7f5fa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.629+0000 7f5fa37fe640 1 -- 172.21.15.103:0/4253795300 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8020040 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.629+0000 7f5fa37fe640 1 -- 172.21.15.103:0/4253795300 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f5fb8097e70 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fbc036040 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fa37fe640 1 -- 172.21.15.103:0/4253795300 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5fb809fb00 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 10 client.26648 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5566ff8a19c0 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fa37fe640 1 -- 172.21.15.103:0/4253795300 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc036040 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fa37fe640 1 -- 172.21.15.103:0/4253795300 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f5fb809ac50 con 0x5566ff889b40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 10 client.26648 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 10 client.26648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:22.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 20 client.26648 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff8a2e80 0x5566ff8a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 4 client.26648 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.630+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ff7fb860 con 0x5566ff8a2e80 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.631+0000 7f5fc3fff640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff8a2e80 0x5566ff8a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.631+0000 7f5fc3fff640 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff8a2e80 0x5566ff8a5270 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4adbd53a21d3b28 server_cookie=7bd4d0c7325c376f in_seq=0 out_seq=0 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.631+0000 7f5fa37fe640 10 client.26648 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fa37fe640 1 -- 172.21.15.103:0/4253795300 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566ff7fb860 con 0x5566ff8a2e80 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fa37fe640 10 client.26648 handle_command_reply: tid=0 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 client.26648 shutdown 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff889b40 msgr2=0x5566ff890040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff889b40 0x5566ff890040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80ccbc0 tx=0x7f5fb809b6c0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 shutdown_connections 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff8a2e80 0x5566ff8a5270 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fa37fe640 0 client.26648 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff893370 0x5566ff890650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff889b40 0x5566ff890040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/4253795300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff892d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.632+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 >> 172.21.15.103:0/4253795300 conn(0x5566ff851310 msgr2=0x5566ff88a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.633+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 shutdown_connections 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.633+0000 7f5fccbd0740 1 -- 172.21.15.103:0/4253795300 wait complete. 2024-09-19T07:46:22.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.633+0000 7f5fccbd0740 20 client.26648 trim_cache size 0 max 16384 2024-09-19T07:46:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.651+0000 7f5fccbd0740 1 Processor -- start 2024-09-19T07:46:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.651+0000 7f5fccbd0740 1 -- start start 2024-09-19T07:46:22.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.651+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff89f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.651+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff8aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.651+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41ab7c0 con 0x5566ff8885d0 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff89f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff89f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39758/0 (socket says 172.21.15.103:39758) 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff8aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc3fff640 1 -- 172.21.15.103:0/254547405 learned_addr learned my addr 172.21.15.103:0/254547405 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fccbd0740 1 -- 172.21.15.103:0/254547405 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41ad300 con 0x5566ff893370 2024-09-19T07:46:22.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fccbd0740 1 -- 172.21.15.103:0/254547405 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41abe90 con 0x5566ff889b40 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc8d94640 1 -- 172.21.15.103:0/254547405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 msgr2=0x5566ff8aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff8aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc8d94640 1 -- 172.21.15.103:0/254547405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 msgr2=0x5566ff89f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff89f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc8d94640 1 -- 172.21.15.103:0/254547405 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ff893100 con 0x5566ff893370 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc3fff640 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff89f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.652+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8aafc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0011d0 tx=0x7f5fbc053000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58df8cf095bf30c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.653+0000 7f5fa9ffb640 1 -- 172.21.15.103:0/254547405 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0305c0 con 0x5566ff893370 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.653+0000 7f5fa9ffb640 1 -- 172.21.15.103:0/254547405 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc036070 con 0x5566ff893370 2024-09-19T07:46:22.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.653+0000 7f5fa9ffb640 1 -- 172.21.15.103:0/254547405 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc045950 con 0x5566ff893370 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 -- 172.21.15.103:0/254547405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 msgr2=0x5566ff8aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8aafc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0011d0 tx=0x7f5fbc053000 comp rx=0 tx=0).stop 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 -- 172.21.15.103:0/254547405 shutdown_connections 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8aafc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff889b40 0x5566ff8aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/254547405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff8885d0 0x5566ff89f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 -- 172.21.15.103:0/254547405 >> 172.21.15.103:0/254547405 conn(0x5566ff851310 msgr2=0x5566ff88f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 -- 172.21.15.103:0/254547405 shutdown_connections 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 -- 172.21.15.103:0/254547405 wait complete. 2024-09-19T07:46:22.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 -- start start 2024-09-19T07:46:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.654+0000 7f5fccbd0740 1 Processor -- start 2024-09-19T07:46:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff8885d0 0x5566ff896f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff889b40 0x5566ff8974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fccbd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41acc30 con 0x5566ff889b40 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41ac560 con 0x5566ff893370 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fccbd0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4105510 con 0x5566ff8885d0 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fccbd0740 10 client.0 _ll_register_callbacks cb 0x7f5fb80057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff8885d0 0x5566ff896f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff8885d0 0x5566ff896f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47090/0 (socket says 172.21.15.103:47090) 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fc3fff640 1 -- 172.21.15.103:0/189945797 learned_addr learned my addr 172.21.15.103:0/189945797 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.655+0000 7f5fc8d94640 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff889b40 0x5566ff8974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fc3fff640 1 -- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 msgr2=0x5566ff8979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fc3fff640 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fc3fff640 1 -- 172.21.15.103:0/189945797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff889b40 msgr2=0x5566ff8974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:22.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fc3fff640 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff889b40 0x5566ff8974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fc3fff640 1 -- 172.21.15.103:0/189945797 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ff91da60 con 0x5566ff8885d0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fc3fff640 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff8885d0 0x5566ff896f70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4010650 tx=0x7f5fb4010b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44951dbc2153194a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fa37fe640 1 -- 172.21.15.103:0/189945797 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb400f8c0 con 0x5566ff8885d0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fa37fe640 1 -- 172.21.15.103:0/189945797 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f5fb400fa60 con 0x5566ff8885d0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fbc036070 con 0x5566ff8885d0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fa37fe640 1 -- 172.21.15.103:0/189945797 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5fb402e050 con 0x5566ff8885d0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fccbd0740 10 client.26581 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.656+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5566ff91ecb0 con 0x5566ff8885d0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fa37fe640 1 -- 172.21.15.103:0/189945797 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc036070 con 0x5566ff8885d0 2024-09-19T07:46:22.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fa37fe640 1 -- 172.21.15.103:0/189945797 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f5fb401e3f0 con 0x5566ff8885d0 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fccbd0740 10 client.26581 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fccbd0740 10 client.26581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fccbd0740 20 client.26581 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff91f480 0x5566ff9218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fccbd0740 4 client.26581 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566ff7f5d70 con 0x5566ff91f480 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.657+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff91f480 0x5566ff9218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.658+0000 7f5fc37fe640 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff91f480 0x5566ff9218a0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea6afae4816cdb0 server_cookie=11a4014c061297b7 in_seq=0 out_seq=0 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.658+0000 7f5fa37fe640 10 client.26581 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.658+0000 7f5fa37fe640 1 -- 172.21.15.103:0/189945797 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566ff7f5d70 con 0x5566ff91f480 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.658+0000 7f5fa37fe640 10 client.26581 handle_command_reply: tid=0 2024-09-19T07:46:22.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.658+0000 7f5fccbd0740 1 client.26581 shutdown 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff8885d0 msgr2=0x5566ff896f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff8885d0 0x5566ff896f70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4010650 tx=0x7f5fb4010b10 comp rx=0 tx=0).stop 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 shutdown_connections 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5566ff91f480 0x5566ff9218a0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566ff893370 0x5566ff8979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fa37fe640 0 client.26581 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566ff889b40 0x5566ff8974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 --2- 172.21.15.103:0/189945797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566ff8885d0 0x5566ff896f70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.659+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 >> 172.21.15.103:0/189945797 conn(0x5566ff851310 msgr2=0x5566ff88a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.660+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 shutdown_connections 2024-09-19T07:46:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.660+0000 7f5fccbd0740 1 -- 172.21.15.103:0/189945797 wait complete. 2024-09-19T07:46:22.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.660+0000 7f5fccbd0740 20 client.26581 trim_cache size 0 max 16384 2024-09-19T07:46:22.665 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:46:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.660+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5fac0a3500 msgr2=0x7f5fac0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.660+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5fac0a3500 0x7f5fac0a59c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc04a000 tx=0x7f5fbc030e50 comp rx=0 tx=0).stop 2024-09-19T07:46:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.660+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc41049a0 msgr2=0x7f5fc410da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.660+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc41049a0 0x7f5fc410da00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800d6d0 tx=0x7f5fb800db90 comp rx=0 tx=0).stop 2024-09-19T07:46:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 shutdown_connections 2024-09-19T07:46:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5fac0a3500 0x7f5fac0a59c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc4109120 0x7f5fc410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5fc41049a0 0x7f5fc410da00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 --2- 172.21.15.103:0/2014564008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4103ff0 0x7f5fc4110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 >> 172.21.15.103:0/2014564008 conn(0x7f5fc40ffab0 msgr2=0x7f5fc4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 shutdown_connections 2024-09-19T07:46:22.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:22.661+0000 7f5fca7f7640 1 -- 172.21.15.103:0/2014564008 wait complete. 2024-09-19T07:46:24.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:23 smithi103 ceph-mon[31201]: pgmap v259: 97 pgs: 97 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-09-19T07:46:24.611 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:46:24.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:46:24.612 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:23 smithi174 ceph-mon[35370]: pgmap v259: 97 pgs: 97 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-09-19T07:46:24.612 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:23 smithi134 ceph-mon[35414]: pgmap v259: 97 pgs: 97 active+clean; 3.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-09-19T07:46:24.980 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 470.0 s, 3229.6 tps, lat 30.944 ms stddev 33.614 2024-09-19T07:46:26.164 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:25 smithi103 ceph-mon[31201]: pgmap v260: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 560 op/s 2024-09-19T07:46:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2189851980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 msgr2=0x7f31d010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2189851980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 0x7f31d010d0e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31c800adb0 tx=0x7f31c802b7a0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2189851980 shutdown_connections 2024-09-19T07:46:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2189851980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 0x7f31d010d0e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2189851980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d0100de0 0x7f31d0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2189851980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31d0100430 0x7f31d0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2189851980 >> 172.21.15.103:0/2189851980 conn(0x7f31d0077230 msgr2=0x7f31d00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.185+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2189851980 shutdown_connections 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.186+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2189851980 wait complete. 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.186+0000 7f31d7bf0640 1 Processor -- start 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.186+0000 7f31d7bf0640 1 -- start start 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.186+0000 7f31d7bf0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31d0100430 0x7f31d01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d7bf0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d0100de0 0x7f31d0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d7bf0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 0x7f31d019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d7bf0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31d010fd40 con 0x7f31d0100de0 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d518b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d0100de0 0x7f31d0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d518b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d0100de0 0x7f31d0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39764/0 (socket says 172.21.15.103:39764) 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d518b640 1 -- 172.21.15.103:0/2949997270 learned_addr learned my addr 172.21.15.103:0/2949997270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:26.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31d010fbc0 con 0x7f31d0105560 2024-09-19T07:46:26.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31d010fec0 con 0x7f31d0100430 2024-09-19T07:46:26.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d598c640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31d0100430 0x7f31d01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d618d640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 0x7f31d019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d598c640 1 -- 172.21.15.103:0/2949997270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 msgr2=0x7f31d019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d598c640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 0x7f31d019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d598c640 1 -- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d0100de0 msgr2=0x7f31d0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d598c640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d0100de0 0x7f31d0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.187+0000 7f31d598c640 1 -- 172.21.15.103:0/2949997270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d019d3a0 con 0x7f31d0100430 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.188+0000 7f31d598c640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31d0100430 0x7f31d01983f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00d6d0 tx=0x7f31cc00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.189+0000 7f31c6ffd640 1 -- 172.21.15.103:0/2949997270 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc017070 con 0x7f31d0100430 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.189+0000 7f31c6ffd640 1 -- 172.21.15.103:0/2949997270 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc009820 con 0x7f31d0100430 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.189+0000 7f31c6ffd640 1 -- 172.21.15.103:0/2949997270 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc004d00 con 0x7f31d0100430 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.190+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d019d690 con 0x7f31d0100430 2024-09-19T07:46:26.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.190+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d0195f40 con 0x7f31d0100430 2024-09-19T07:46:26.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.193+0000 7f31c6ffd640 1 -- 172.21.15.103:0/2949997270 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f31cc0099d0 con 0x7f31d0100430 2024-09-19T07:46:26.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.193+0000 7f31d9fc9740 1 Processor -- start 2024-09-19T07:46:26.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.193+0000 7f31d9fc9740 1 -- start start 2024-09-19T07:46:26.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e292710280 con 0x55e2927ab480 2024-09-19T07:46:26.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e2926b7b10 con 0x55e2927aab60 2024-09-19T07:46:26.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e29270c240 con 0x55e2927b5d40 2024-09-19T07:46:26.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d598c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d618d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d598c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39774/0 (socket says 172.21.15.103:39774) 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d518b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d598c640 1 -- 172.21.15.103:0/403114325 learned_addr learned my addr 172.21.15.103:0/403114325 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31c6ffd640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f31bc0a3500 0x7f31bc0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d518b640 1 -- 172.21.15.103:0/403114325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 msgr2=0x55e2927aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d518b640 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d518b640 1 -- 172.21.15.103:0/403114325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 msgr2=0x55e2927b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d518b640 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d518b640 1 -- 172.21.15.103:0/403114325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2926c41f0 con 0x55e2927b5d40 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d598c640 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31d618d640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f31bc0a3500 0x7f31bc0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.194+0000 7f31c6ffd640 1 -- 172.21.15.103:0/2949997270 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f31cc0cf820 con 0x7f31d0100430 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.195+0000 7f31d518b640 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b8130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f31d00ff7f0 tx=0x7f31c00045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826f865c46003bee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.195+0000 7f31d618d640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f31bc0a3500 0x7f31bc0a59c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005cc0 tx=0x7f31c800a6d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.195+0000 7f31bbfff640 1 -- 172.21.15.103:0/403114325 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0003d20 con 0x55e2927b5d40 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.195+0000 7f31bbfff640 1 -- 172.21.15.103:0/403114325 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c0005500 con 0x55e2927b5d40 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.197+0000 7f31bbfff640 1 -- 172.21.15.103:0/403114325 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0003d20 con 0x55e2927b5d40 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 -- 172.21.15.103:0/403114325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 msgr2=0x55e2927b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b8130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f31d00ff7f0 tx=0x7f31c00045d0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 -- 172.21.15.103:0/403114325 shutdown_connections 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b8130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/403114325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 -- 172.21.15.103:0/403114325 >> 172.21.15.103:0/403114325 conn(0x55e2927a70f0 msgr2=0x55e2927a74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:26.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 -- 172.21.15.103:0/403114325 shutdown_connections 2024-09-19T07:46:26.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 -- 172.21.15.103:0/403114325 wait complete. 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.198+0000 7f31d9fc9740 1 -- start start 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 1 Processor -- start 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927aab60 0x55e2927b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927ab480 0x55e2927b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e2927b8960 con 0x55e2927aab60 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e2926caf10 con 0x55e2927ab480 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e2927b8ae0 con 0x55e2927b5d40 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d9fc9740 10 client.0 _ll_register_callbacks cb 0x55e292693a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d598c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927aab60 0x55e2927b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d598c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927aab60 0x55e2927b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39778/0 (socket says 172.21.15.103:39778) 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d598c640 1 -- 172.21.15.103:0/4105661673 learned_addr learned my addr 172.21.15.103:0/4105661673 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d518b640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927ab480 0x55e2927b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.199+0000 7f31d618d640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31d618d640 1 -- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927ab480 msgr2=0x55e2927b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31d618d640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927ab480 0x55e2927b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31d618d640 1 -- 172.21.15.103:0/4105661673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927aab60 msgr2=0x55e2927b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31d618d640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927aab60 0x55e2927b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31d618d640 1 -- 172.21.15.103:0/4105661673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2924816a0 con 0x55e2927b5d40 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31d598c640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927aab60 0x55e2927b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31d618d640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b3080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31c802bcc0 tx=0x7f31c802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63167782a3a8011b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.200+0000 7f31b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:46:26.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31b97fa640 1 -- 172.21.15.103:0/4105661673 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8031ae0 con 0x55e2927b5d40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c0005500 con 0x55e2927b5d40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31b97fa640 1 -- 172.21.15.103:0/4105661673 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f31c8031c80 con 0x55e2927b5d40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31b97fa640 1 -- 172.21.15.103:0/4105661673 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f31c8031e80 con 0x55e2927b5d40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 10 client.26593 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e2927c4460 con 0x55e2927b5d40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31b97fa640 1 -- 172.21.15.103:0/4105661673 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c0005500 con 0x55e2927b5d40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31b97fa640 1 -- 172.21.15.103:0/4105661673 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f31c80537d0 con 0x55e2927b5d40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 10 client.26593 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:26.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 20 client.26593 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:26.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2927c5870 0x55e2927c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 4 client.26593 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:26.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.201+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e29271d7f0 con 0x55e2927c5870 2024-09-19T07:46:26.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.202+0000 7f31d598c640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2927c5870 0x55e2927c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.204+0000 7f31b97fa640 10 client.26593 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.204+0000 7f31d598c640 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2927c5870 0x55e2927c7c80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c3e34c9bf033a4 server_cookie=8fe851197aa9dd07 in_seq=0 out_seq=0 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.205+0000 7f31b97fa640 1 -- 172.21.15.103:0/4105661673 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e29271d7f0 con 0x55e2927c5870 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.205+0000 7f31b97fa640 10 client.26593 handle_command_reply: tid=0 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.205+0000 7f31d9fc9740 1 client.26593 shutdown 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.205+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 msgr2=0x55e2927b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.205+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b3080 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31c802bcc0 tx=0x7f31c802b7a0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 shutdown_connections 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2927c5870 0x55e2927c7c80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927b5d40 0x55e2927b3080 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927ab480 0x55e2927b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31b97fa640 0 client.26593 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/4105661673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927aab60 0x55e2927b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 >> 172.21.15.103:0/4105661673 conn(0x55e2927a70f0 msgr2=0x55e2927ace20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 shutdown_connections 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 1 -- 172.21.15.103:0/4105661673 wait complete. 2024-09-19T07:46:26.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.206+0000 7f31d9fc9740 20 client.26593 trim_cache size 0 max 16384 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.229+0000 7f31d9fc9740 1 Processor -- start 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.229+0000 7f31d9fc9740 1 -- start start 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927aab60 0x55e2927c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927b5d40 0x55e2927cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01ab710 con 0x55e2927ab480 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01ad250 con 0x55e2927b5d40 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d9fc9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01abde0 con 0x55e2927aab60 2024-09-19T07:46:26.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d518b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d598c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927aab60 0x55e2927c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d598c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927aab60 0x55e2927c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:47168/0 (socket says 172.21.15.103:47168) 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.230+0000 7f31d598c640 1 -- 172.21.15.103:0/2493827670 learned_addr learned my addr 172.21.15.103:0/2493827670 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31d618d640 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927b5d40 0x55e2927cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31d598c640 1 -- 172.21.15.103:0/2493827670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927b5d40 msgr2=0x55e2927cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31d598c640 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927b5d40 0x55e2927cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31d598c640 1 -- 172.21.15.103:0/2493827670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 msgr2=0x55e2927cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31d598c640 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31d598c640 1 -- 172.21.15.103:0/2493827670 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2927a6f20 con 0x55e2927aab60 2024-09-19T07:46:26.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31d598c640 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927aab60 0x55e2927c1c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0047e0 tx=0x7f31cc0d1d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59b45a9538a54e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.231+0000 7f31bb7fe640 1 -- 172.21.15.103:0/2493827670 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc020040 con 0x55e2927aab60 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31bb7fe640 1 -- 172.21.15.103:0/2493827670 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc09c1f0 con 0x55e2927aab60 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31bb7fe640 1 -- 172.21.15.103:0/2493827670 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0cd1e0 con 0x55e2927aab60 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31d9fc9740 1 -- 172.21.15.103:0/2493827670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927aab60 msgr2=0x55e2927c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927aab60 0x55e2927c1c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0047e0 tx=0x7f31cc0d1d30 comp rx=0 tx=0).stop 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31d9fc9740 1 -- 172.21.15.103:0/2493827670 shutdown_connections 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927b5d40 0x55e2927cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927ab480 0x55e2927cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/2493827670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927aab60 0x55e2927c1c10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.232+0000 7f31d9fc9740 1 -- 172.21.15.103:0/2493827670 >> 172.21.15.103:0/2493827670 conn(0x55e2927a70f0 msgr2=0x55e2927b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:26.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.233+0000 7f31d9fc9740 1 -- 172.21.15.103:0/2493827670 shutdown_connections 2024-09-19T07:46:26.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.233+0000 7f31d9fc9740 1 -- 172.21.15.103:0/2493827670 wait complete. 2024-09-19T07:46:26.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.233+0000 7f31d9fc9740 1 -- start start 2024-09-19T07:46:26.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.234+0000 7f31d9fc9740 1 Processor -- start 2024-09-19T07:46:26.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.234+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927ab480 0x55e2927be4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d598c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d598c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927bdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:47788/0 (socket says 172.21.15.103:47788) 2024-09-19T07:46:26.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d9fc9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927b5d40 0x55e2927aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d598c640 1 -- 172.21.15.103:0/1621691973 learned_addr learned my addr 172.21.15.103:0/1621691973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:26.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01acb80 con 0x55e2927b5d40 2024-09-19T07:46:26.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d518b640 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927ab480 0x55e2927be4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01ac4b0 con 0x55e2927aab60 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01a9710 con 0x55e2927ab480 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d9fc9740 10 client.0 _ll_register_callbacks cb 0x7f31c800ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.235+0000 7f31d618d640 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927b5d40 0x55e2927aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31d618d640 1 -- 172.21.15.103:0/1621691973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927ab480 msgr2=0x55e2927be4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31d618d640 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927ab480 0x55e2927be4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31d618d640 1 -- 172.21.15.103:0/1621691973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 msgr2=0x55e2927bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31d618d640 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927bdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31d618d640 1 -- 172.21.15.103:0/1621691973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2927af530 con 0x55e2927b5d40 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31d618d640 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927b5d40 0x55e2927aed00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f31c8053c70 tx=0x7f31c8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a24da762a7288b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31b97fa640 1 -- 172.21.15.103:0/1621691973 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c805f020 con 0x55e2927b5d40 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.236+0000 7f31b97fa640 1 -- 172.21.15.103:0/1621691973 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f31c8053680 con 0x55e2927b5d40 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.237+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31cc09c1f0 con 0x55e2927b5d40 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.237+0000 7f31b97fa640 1 -- 172.21.15.103:0/1621691973 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f31c806a050 con 0x55e2927b5d40 2024-09-19T07:46:26.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.237+0000 7f31d9fc9740 10 client.17220 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.237+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e2927b0870 con 0x55e2927b5d40 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.237+0000 7f31b97fa640 1 -- 172.21.15.103:0/1621691973 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc09c1f0 con 0x55e2927b5d40 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.237+0000 7f31b97fa640 1 -- 172.21.15.103:0/1621691973 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f31c80046e0 con 0x55e2927b5d40 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.238+0000 7f31d9fc9740 10 client.17220 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.238+0000 7f31d9fc9740 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.238+0000 7f31d9fc9740 20 client.17220 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.238+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2928317d0 0x55e292833bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.238+0000 7f31d9fc9740 4 client.17220 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:46:26.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.238+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e292717d00 con 0x55e2928317d0 2024-09-19T07:46:26.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.238+0000 7f31d598c640 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2928317d0 0x55e292833bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:26.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.239+0000 7f31b97fa640 10 client.17220 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:26.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.239+0000 7f31d598c640 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2928317d0 0x55e292833bc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25693851a83ca5bb server_cookie=5ae3fae0ef490bde in_seq=0 out_seq=0 2024-09-19T07:46:26.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.240+0000 7f31b97fa640 1 -- 172.21.15.103:0/1621691973 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e292717d00 con 0x55e2928317d0 2024-09-19T07:46:26.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.240+0000 7f31b97fa640 10 client.17220 handle_command_reply: tid=0 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.240+0000 7f31d9fc9740 1 client.17220 shutdown 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.240+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927b5d40 msgr2=0x55e2927aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.240+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927b5d40 0x55e2927aed00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f31c8053c70 tx=0x7f31c8003c20 comp rx=0 tx=0).stop 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 shutdown_connections 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e2928317d0 0x55e292833bc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2927b5d40 0x55e2927aed00 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31b97fa640 0 client.17220 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:26.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2927ab480 0x55e2927be4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 --2- 172.21.15.103:0/1621691973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2927aab60 0x55e2927bdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 >> 172.21.15.103:0/1621691973 conn(0x55e2927a70f0 msgr2=0x55e2927a6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:26.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 shutdown_connections 2024-09-19T07:46:26.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 1 -- 172.21.15.103:0/1621691973 wait complete. 2024-09-19T07:46:26.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.241+0000 7f31d9fc9740 20 client.17220 trim_cache size 0 max 16384 2024-09-19T07:46:26.246 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:46:26.247 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:46:26.247 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "a9da0834-0e90-4067-8dfc-502b4a2c20f5", 2024-09-19T07:46:26.247 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:46:26.247 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:46:26.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.242+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f31bc0a3500 msgr2=0x7f31bc0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.242+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f31bc0a3500 0x7f31bc0a59c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f31c8005cc0 tx=0x7f31c800a6d0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31d0100430 msgr2=0x7f31d01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:26.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31d0100430 0x7f31d01983f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00d6d0 tx=0x7f31cc00db90 comp rx=0 tx=0).stop 2024-09-19T07:46:26.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 shutdown_connections 2024-09-19T07:46:26.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f31bc0a3500 0x7f31bc0a59c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0105560 0x7f31d019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d0100de0 0x7f31d0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 --2- 172.21.15.103:0/2949997270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31d0100430 0x7f31d01983f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:26.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 >> 172.21.15.103:0/2949997270 conn(0x7f31d0077230 msgr2=0x7f31d0103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:26.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 shutdown_connections 2024-09-19T07:46:26.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:26.243+0000 7f31d7bf0640 1 -- 172.21.15.103:0/2949997270 wait complete. 2024-09-19T07:46:26.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:25 smithi134 ceph-mon[35414]: pgmap v260: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 560 op/s 2024-09-19T07:46:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:25 smithi174 ceph-mon[35370]: pgmap v260: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 560 op/s 2024-09-19T07:46:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:46:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:46:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:46:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:46:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:46:27.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:26 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:46:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:46:27.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:46:27.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:46:27.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:46:27.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:26 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:46:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:46:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:46:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:46:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:46:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:46:27.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:26 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:27.848 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:46:28.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:27 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:46:28.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:27 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:28.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:27 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:46:28.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:27 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:28.208 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:27 smithi103 ceph-mon[31201]: pgmap v261: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 647 op/s 2024-09-19T07:46:28.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:27 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:46:28.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:27 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:28.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:27 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:46:28.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:27 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:28.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:27 smithi134 ceph-mon[35414]: pgmap v261: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 647 op/s 2024-09-19T07:46:28.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:27 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:46:28.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:27 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:28.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:27 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:46:28.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:27 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:46:28.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:27 smithi174 ceph-mon[35370]: pgmap v261: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 647 op/s 2024-09-19T07:46:28.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.434+0000 7ffae8cae640 1 -- 172.21.15.103:0/4142333829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4106a80 msgr2=0x7ffae4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.434+0000 7ffae8cae640 1 --2- 172.21.15.103:0/4142333829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4106a80 0x7ffae4106e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009900 tx=0x7ffad402a410 comp rx=0 tx=0).stop 2024-09-19T07:46:28.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.434+0000 7ffae8cae640 1 -- 172.21.15.103:0/4142333829 shutdown_connections 2024-09-19T07:46:28.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.434+0000 7ffae8cae640 1 --2- 172.21.15.103:0/4142333829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffae4101420 0x7ffae410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.434+0000 7ffae8cae640 1 --2- 172.21.15.103:0/4142333829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffae4100a80 0x7ffae4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.434+0000 7ffae8cae640 1 --2- 172.21.15.103:0/4142333829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4106a80 0x7ffae4106e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.434+0000 7ffae8cae640 1 -- 172.21.15.103:0/4142333829 >> 172.21.15.103:0/4142333829 conn(0x7ffae40fc7a0 msgr2=0x7ffae40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:28.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.435+0000 7ffae8cae640 1 -- 172.21.15.103:0/4142333829 shutdown_connections 2024-09-19T07:46:28.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.435+0000 7ffae8cae640 1 -- 172.21.15.103:0/4142333829 wait complete. 2024-09-19T07:46:28.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.435+0000 7ffae8cae640 1 Processor -- start 2024-09-19T07:46:28.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.435+0000 7ffae8cae640 1 -- start start 2024-09-19T07:46:28.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae8cae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffae4100a80 0x7ffae41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae8cae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4101420 0x7ffae41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae8cae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffae4106a80 0x7ffae419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae8cae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4110f10 con 0x7ffae4106a80 2024-09-19T07:46:28.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae8cae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4110d90 con 0x7ffae4101420 2024-09-19T07:46:28.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae8cae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4111090 con 0x7ffae4100a80 2024-09-19T07:46:28.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4101420 0x7ffae41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4101420 0x7ffae41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56874/0 (socket says 172.21.15.103:56874) 2024-09-19T07:46:28.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffae4106a80 0x7ffae419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 -- 172.21.15.103:0/1806529550 learned_addr learned my addr 172.21.15.103:0/1806529550 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:28.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffae259c640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffae4100a80 0x7ffae41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 -- 172.21.15.103:0/1806529550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffae4100a80 msgr2=0x7ffae41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffae4100a80 0x7ffae41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 -- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffae4106a80 msgr2=0x7ffae419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffae4106a80 0x7ffae419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.436+0000 7ffadbfff640 1 -- 172.21.15.103:0/1806529550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae419d450 con 0x7ffae4101420 2024-09-19T07:46:28.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.437+0000 7ffadbfff640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4101420 0x7ffae41989e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00e9f0 tx=0x7ffacc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:28.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.437+0000 7ffadb7fe640 1 -- 172.21.15.103:0/1806529550 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00cd80 con 0x7ffae4101420 2024-09-19T07:46:28.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.437+0000 7ffadb7fe640 1 -- 172.21.15.103:0/1806529550 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc010910 con 0x7ffae4101420 2024-09-19T07:46:28.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.437+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae419d740 con 0x7ffae4101420 2024-09-19T07:46:28.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.438+0000 7ffadb7fe640 1 -- 172.21.15.103:0/1806529550 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc018990 con 0x7ffae4101420 2024-09-19T07:46:28.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.438+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae41a5020 con 0x7ffae4101420 2024-09-19T07:46:28.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.440+0000 7ffadb7fe640 1 -- 172.21.15.103:0/1806529550 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffacc018b30 con 0x7ffae4101420 2024-09-19T07:46:28.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.441+0000 7ffadb7fe640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffad00a3390 0x7ffad00a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.442+0000 7ffae259c640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffad00a3390 0x7ffad00a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.442+0000 7ffadb7fe640 1 -- 172.21.15.103:0/1806529550 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ffacc0d00f0 con 0x7ffae4101420 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.442+0000 7ffaeb087740 1 Processor -- start 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.442+0000 7ffaeb087740 1 -- start start 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffae259c640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffad00a3390 0x7ffad00a5850 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffad4000c00 tx=0x7ffad4005a00 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb52880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffaeb087740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f02eab82e0 con 0x55f02eb524a0 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffaeb087740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f02ea5fce0 con 0x55f02eb5d680 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffaeb087740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f02eab42a0 con 0x55f02eb52dc0 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48022/0 (socket says 172.21.15.103:48022) 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 -- 172.21.15.103:0/1027145384 learned_addr learned my addr 172.21.15.103:0/1027145384 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 -- 172.21.15.103:0/1027145384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 msgr2=0x55f02eb5fa70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 --2- 172.21.15.103:0/1027145384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5fa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 -- 172.21.15.103:0/1027145384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 msgr2=0x55f02eb52880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 --2- 172.21.15.103:0/1027145384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb52880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.443+0000 7ffadbfff640 1 -- 172.21.15.103:0/1027145384 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f02ea44440 con 0x55f02eb52dc0 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.444+0000 7ffadbfff640 1 --2- 172.21.15.103:0/1027145384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5d140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0057f0 tx=0x7ffacc09cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de076ec5fbdb5c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.444+0000 7ffad8ff9640 1 -- 172.21.15.103:0/1027145384 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0989d0 con 0x55f02eb52dc0 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.444+0000 7ffad8ff9640 1 -- 172.21.15.103:0/1027145384 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc0cda50 con 0x55f02eb52dc0 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.444+0000 7ffad8ff9640 1 -- 172.21.15.103:0/1027145384 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0bc340 con 0x55f02eb52dc0 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 -- 172.21.15.103:0/1027145384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 msgr2=0x55f02eb5d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 --2- 172.21.15.103:0/1027145384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5d140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0057f0 tx=0x7ffacc09cba0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 -- 172.21.15.103:0/1027145384 shutdown_connections 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 --2- 172.21.15.103:0/1027145384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 --2- 172.21.15.103:0/1027145384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5d140 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 --2- 172.21.15.103:0/1027145384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb52880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 -- 172.21.15.103:0/1027145384 >> 172.21.15.103:0/1027145384 conn(0x55f02eb4ea30 msgr2=0x55f02eb4ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.445+0000 7ffaeb087740 1 -- 172.21.15.103:0/1027145384 shutdown_connections 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.446+0000 7ffaeb087740 1 -- 172.21.15.103:0/1027145384 wait complete. 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.446+0000 7ffaeb087740 1 -- start start 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.446+0000 7ffaeb087740 1 Processor -- start 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.446+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb5d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffaeb087740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f02eb60250 con 0x55f02eb524a0 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb5d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffaeb087740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f02ea73050 con 0x55f02eb5d680 2024-09-19T07:46:28.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffaeb087740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f02eb603d0 con 0x55f02eb52dc0 2024-09-19T07:46:28.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb5d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55074/0 (socket says 172.21.15.103:55074) 2024-09-19T07:46:28.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae259c640 1 -- 172.21.15.103:0/86832471 learned_addr learned my addr 172.21.15.103:0/86832471 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:28.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae2d9d640 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffaeb087740 10 client.0 _ll_register_callbacks cb 0x55f02eab3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:28.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae2d9d640 1 -- 172.21.15.103:0/86832471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 msgr2=0x55f02eb5a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:28.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae2d9d640 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae2d9d640 1 -- 172.21.15.103:0/86832471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 msgr2=0x55f02eb5d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:28.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae2d9d640 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb5d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.447+0000 7ffae2d9d640 1 -- 172.21.15.103:0/86832471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f02e8293c0 con 0x55f02eb5d680 2024-09-19T07:46:28.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffae2d9d640 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5a8e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffadc012b70 tx=0x7ffadc00a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3e8d3f185caf6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:28.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:46:28.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffac9ffb640 1 -- 172.21.15.103:0/86832471 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc01f060 con 0x55f02eb5d680 2024-09-19T07:46:28.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffac9ffb640 1 -- 172.21.15.103:0/86832471 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ffadc0076c0 con 0x55f02eb5d680 2024-09-19T07:46:28.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffacc0cda50 con 0x55f02eb5d680 2024-09-19T07:46:28.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffac9ffb640 1 -- 172.21.15.103:0/86832471 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffadc0078c0 con 0x55f02eb5d680 2024-09-19T07:46:28.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 10 client.26678 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:28.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f02eb6bce0 con 0x55f02eb5d680 2024-09-19T07:46:28.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffac9ffb640 1 -- 172.21.15.103:0/86832471 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc0cda50 con 0x55f02eb5d680 2024-09-19T07:46:28.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffac9ffb640 1 -- 172.21.15.103:0/86832471 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7ffadc022c30 con 0x55f02eb5d680 2024-09-19T07:46:28.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 10 client.26678 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 10 client.26678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 20 client.26678 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02eb6d1f0 0x55f02eb6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 4 client.26678 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.448+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f02eac5850 con 0x55f02eb6d1f0 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.449+0000 7ffae259c640 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02eb6d1f0 0x55f02eb6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.449+0000 7ffac9ffb640 10 client.26678 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.449+0000 7ffae259c640 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02eb6d1f0 0x55f02eb6f670 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f133532efc6ae222 server_cookie=f4c2973b53a6f6e6 in_seq=0 out_seq=0 2024-09-19T07:46:28.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.450+0000 7ffac9ffb640 1 -- 172.21.15.103:0/86832471 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f02eac5850 con 0x55f02eb6d1f0 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.450+0000 7ffac9ffb640 10 client.26678 handle_command_reply: tid=0 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.450+0000 7ffaeb087740 1 client.26678 shutdown 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 msgr2=0x55f02eb5a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5a8e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffadc012b70 tx=0x7ffadc00a7d0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 shutdown_connections 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02eb6d1f0 0x55f02eb6f670 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb5a8e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb5a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 --2- 172.21.15.103:0/86832471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb5d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 >> 172.21.15.103:0/86832471 conn(0x55f02eb4ea30 msgr2=0x55f02eb54760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:28.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffac9ffb640 0 client.26678 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:28.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 shutdown_connections 2024-09-19T07:46:28.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 1 -- 172.21.15.103:0/86832471 wait complete. 2024-09-19T07:46:28.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.451+0000 7ffaeb087740 20 client.26678 trim_cache size 0 max 16384 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 Processor -- start 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 -- start start 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb75480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41ab460 con 0x55f02eb524a0 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41acfa0 con 0x55f02eb5d680 2024-09-19T07:46:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffaeb087740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41abb30 con 0x55f02eb52dc0 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffae259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffae259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55082/0 (socket says 172.21.15.103:55082) 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffae259c640 1 -- 172.21.15.103:0/3063381085 learned_addr learned my addr 172.21.15.103:0/3063381085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffadbfff640 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffae259c640 1 -- 172.21.15.103:0/3063381085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 msgr2=0x55f02eb74f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffae259c640 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb74f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffae2d9d640 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb75480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.475+0000 7ffae259c640 1 -- 172.21.15.103:0/3063381085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 msgr2=0x55f02eb75480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.476+0000 7ffae259c640 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb75480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.476+0000 7ffae259c640 1 -- 172.21.15.103:0/3063381085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f02eb58840 con 0x55f02eb524a0 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.476+0000 7ffae2d9d640 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb75480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:46:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.476+0000 7ffae259c640 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb694b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ffad4006010 tx=0x7ffad403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afedae397adfd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:28.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.476+0000 7ffacbfff640 1 -- 172.21.15.103:0/3063381085 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4035050 con 0x55f02eb524a0 2024-09-19T07:46:28.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.476+0000 7ffacbfff640 1 -- 172.21.15.103:0/3063381085 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad4056070 con 0x55f02eb524a0 2024-09-19T07:46:28.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.476+0000 7ffacbfff640 1 -- 172.21.15.103:0/3063381085 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40035f0 con 0x55f02eb524a0 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.477+0000 7ffaeb087740 1 -- 172.21.15.103:0/3063381085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 msgr2=0x55f02eb694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.477+0000 7ffaeb087740 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb694b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ffad4006010 tx=0x7ffad403c000 comp rx=0 tx=0).stop 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.477+0000 7ffaeb087740 1 -- 172.21.15.103:0/3063381085 shutdown_connections 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.477+0000 7ffaeb087740 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb75480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.477+0000 7ffaeb087740 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb52dc0 0x55f02eb74f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.477+0000 7ffaeb087740 1 --2- 172.21.15.103:0/3063381085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb524a0 0x55f02eb694b0 secure :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ffad4006010 tx=0x7ffad403c000 comp rx=0 tx=0).stop 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.477+0000 7ffaeb087740 1 -- 172.21.15.103:0/3063381085 >> 172.21.15.103:0/3063381085 conn(0x55f02eb4ea30 msgr2=0x55f02eb4db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.478+0000 7ffaeb087740 1 -- 172.21.15.103:0/3063381085 shutdown_connections 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.478+0000 7ffaeb087740 1 -- 172.21.15.103:0/3063381085 wait complete. 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 -- start start 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 Processor -- start 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb52dc0 0x55f02eb65bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb633a0 0x55f02eb63820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41ac8d0 con 0x55f02eb52dc0 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41ac200 con 0x55f02eb5d680 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffaeb087740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffae41a9480 con 0x55f02eb633a0 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.479+0000 7ffae259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb52dc0 0x55f02eb65bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffae259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb52dc0 0x55f02eb65bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55094/0 (socket says 172.21.15.103:55094) 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffae259c640 1 -- 172.21.15.103:0/486157723 learned_addr learned my addr 172.21.15.103:0/486157723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffaeb087740 10 client.0 _ll_register_callbacks cb 0x7ffadc0099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffadbfff640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffae2d9d640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb633a0 0x55f02eb63820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffadbfff640 1 -- 172.21.15.103:0/486157723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb633a0 msgr2=0x55f02eb63820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffadbfff640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb633a0 0x55f02eb63820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffadbfff640 1 -- 172.21.15.103:0/486157723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb52dc0 msgr2=0x55f02eb65bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffadbfff640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb52dc0 0x55f02eb65bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffadbfff640 1 -- 172.21.15.103:0/486157723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f02ebd6f20 con 0x55f02eb5d680 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffae259c640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb52dc0 0x55f02eb65bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:46:28.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffadbfff640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb62e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55f02eb5b930 tx=0x7ffacc09cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e93b55b6c81a68f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:46:28.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffac9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:46:28.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffac9ffb640 1 -- 172.21.15.103:0/486157723 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0b5360 con 0x55f02eb5d680 2024-09-19T07:46:28.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffac9ffb640 1 -- 172.21.15.103:0/486157723 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ffacc09b7b0 con 0x55f02eb5d680 2024-09-19T07:46:28.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad4056070 con 0x55f02eb5d680 2024-09-19T07:46:28.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffac9ffb640 1 -- 172.21.15.103:0/486157723 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffacc0cdc60 con 0x55f02eb5d680 2024-09-19T07:46:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.480+0000 7ffaeb087740 10 client.26681 fetch_fsmap learned FSMap version 40 2024-09-19T07:46:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f02ebd80b0 con 0x55f02eb5d680 2024-09-19T07:46:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffac9ffb640 1 -- 172.21.15.103:0/486157723 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffacc09b9b0 con 0x55f02eb5d680 2024-09-19T07:46:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffac9ffb640 1 -- 172.21.15.103:0/486157723 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7ffacc0b0370 con 0x55f02eb5d680 2024-09-19T07:46:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffaeb087740 10 client.26681 fetch_fsmap finished waiting for FSMap version 40 2024-09-19T07:46:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffaeb087740 10 client.26681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:46:28.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffaeb087740 20 client.26681 populate_metadata read hostname 'smithi103' 2024-09-19T07:46:28.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffaeb087740 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02ebd8460 0x55f02ebda850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:46:28.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffaeb087740 4 client.26681 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:46:28.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.481+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ffad4056070 con 0x55f02ebd8460 2024-09-19T07:46:28.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.482+0000 7ffae259c640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02ebd8460 0x55f02ebda850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:46:28.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.482+0000 7ffae259c640 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02ebd8460 0x55f02ebda850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa1a17f1a620e0 server_cookie=aefa0c3e888e463d in_seq=0 out_seq=0 2024-09-19T07:46:28.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.482+0000 7ffac9ffb640 10 client.26681 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:28.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffac9ffb640 1 -- 172.21.15.103:0/486157723 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7ffad4056070 con 0x55f02ebd8460 2024-09-19T07:46:28.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffac9ffb640 10 client.26681 handle_command_reply: tid=0 2024-09-19T07:46:28.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 client.26681 shutdown 2024-09-19T07:46:28.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 msgr2=0x55f02eb62e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb62e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55f02eb5b930 tx=0x7ffacc09cba0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 shutdown_connections 2024-09-19T07:46:28.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f02ebd8460 0x55f02ebda850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f02eb633a0 0x55f02eb63820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f02eb5d680 0x55f02eb62e60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 --2- 172.21.15.103:0/486157723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f02eb52dc0 0x55f02eb65bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 >> 172.21.15.103:0/486157723 conn(0x55f02eb4ea30 msgr2=0x55f02eb5bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:28.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffac9ffb640 0 client.26681 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:46:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 shutdown_connections 2024-09-19T07:46:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 1 -- 172.21.15.103:0/486157723 wait complete. 2024-09-19T07:46:28.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.483+0000 7ffaeb087740 20 client.26681 trim_cache size 0 max 16384 2024-09-19T07:46:28.494 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:46:28.494 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:46:28.495 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:46:28.495 INFO:teuthology.orchestra.run.smithi103.stdout: "a9da0834-0e90-4067-8dfc-502b4a2c20f5": { 2024-09-19T07:46:28.495 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:46:28.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "a9da0834-0e90-4067-8dfc-502b4a2c20f5", 2024-09-19T07:46:28.495 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:46:28.496 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:46:28.496 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:46:28.496 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:46:28.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.484+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffad00a3390 msgr2=0x7ffad00a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.484+0000 7ffae8cae640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffad00a3390 0x7ffad00a5850 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffad4000c00 tx=0x7ffad4005a00 comp rx=0 tx=0).stop 2024-09-19T07:46:28.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.484+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4101420 msgr2=0x7ffae41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:46:28.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.484+0000 7ffae8cae640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4101420 0x7ffae41989e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00e9f0 tx=0x7ffacc00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.484+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 shutdown_connections 2024-09-19T07:46:28.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.484+0000 7ffae8cae640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffad00a3390 0x7ffad00a5850 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.485+0000 7ffae8cae640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffae4106a80 0x7ffae419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.485+0000 7ffae8cae640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae4101420 0x7ffae41989e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.485+0000 7ffae8cae640 1 --2- 172.21.15.103:0/1806529550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffae4100a80 0x7ffae41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:46:28.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.485+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 >> 172.21.15.103:0/1806529550 conn(0x7ffae40fc7a0 msgr2=0x7ffae4104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:46:28.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.485+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 shutdown_connections 2024-09-19T07:46:28.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:46:28.485+0000 7ffae8cae640 1 -- 172.21.15.103:0/1806529550 wait complete. 2024-09-19T07:46:29.351 INFO:tasks.cephfs.filesystem:scrub status for tag:a9da0834-0e90-4067-8dfc-502b4a2c20f5 - {'path': '/', 'tag': 'a9da0834-0e90-4067-8dfc-502b4a2c20f5', 'options': 'recursive,force'} 2024-09-19T07:46:30.217 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:29 smithi174 ceph-mon[35370]: pgmap v262: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 647 op/s 2024-09-19T07:46:30.219 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:29 smithi103 ceph-mon[31201]: pgmap v262: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 647 op/s 2024-09-19T07:46:30.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:29 smithi134 ceph-mon[35414]: pgmap v262: 97 pgs: 97 active+clean; 3.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 647 op/s 2024-09-19T07:46:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 475.0 s, 2269.6 tps, lat 43.931 ms stddev 74.353 2024-09-19T07:46:31.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:30 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:31.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:30 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:31.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:30 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:32.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:31 smithi103 ceph-mon[31201]: pgmap v263: 97 pgs: 97 active+clean; 3.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 1.92k op/s 2024-09-19T07:46:32.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:31 smithi134 ceph-mon[35414]: pgmap v263: 97 pgs: 97 active+clean; 3.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 1.92k op/s 2024-09-19T07:46:32.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:31 smithi174 ceph-mon[35370]: pgmap v263: 97 pgs: 97 active+clean; 3.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 1.92k op/s 2024-09-19T07:46:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:33 smithi103 ceph-mon[31201]: pgmap v264: 97 pgs: 97 active+clean; 3.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-09-19T07:46:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:33 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:46:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:33 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:46:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:33 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:46:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:33 smithi134 ceph-mon[35414]: pgmap v264: 97 pgs: 97 active+clean; 3.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-09-19T07:46:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:33 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:46:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:33 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:46:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:33 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:46:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:33 smithi174 ceph-mon[35370]: pgmap v264: 97 pgs: 97 active+clean; 3.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-09-19T07:46:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:33 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:46:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:33 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:46:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:33 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:46:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:34 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:46:35.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:34 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:46:35.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:34 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:46:35.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:34 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:46:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 480.0 s, 2811.8 tps, lat 35.795 ms stddev 37.400 2024-09-19T07:46:35.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:34 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:46:35.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:34 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:46:36.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:35 smithi134 ceph-mon[35414]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.81k op/s 2024-09-19T07:46:36.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:35 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:35 smithi174 ceph-mon[35370]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.81k op/s 2024-09-19T07:46:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:35 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:35 smithi103 ceph-mon[31201]: pgmap v265: 97 pgs: 97 active+clean; 3.9 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 MiB/s wr, 1.81k op/s 2024-09-19T07:46:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:35 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:38.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:37 smithi134 ceph-mon[35414]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-09-19T07:46:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:37 smithi174 ceph-mon[35370]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-09-19T07:46:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:37 smithi103 ceph-mon[31201]: pgmap v266: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.65k op/s 2024-09-19T07:46:40.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:39 smithi134 ceph-mon[35414]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-09-19T07:46:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:39 smithi174 ceph-mon[35370]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-09-19T07:46:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 485.0 s, 3038.8 tps, lat 32.833 ms stddev 35.837 2024-09-19T07:46:40.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:39 smithi103 ceph-mon[31201]: pgmap v267: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 1.56k op/s 2024-09-19T07:46:42.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:41 smithi134 ceph-mon[35414]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.86k op/s 2024-09-19T07:46:42.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:41 smithi174 ceph-mon[35370]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.86k op/s 2024-09-19T07:46:42.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:41 smithi103 ceph-mon[31201]: pgmap v268: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.86k op/s 2024-09-19T07:46:44.333 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:43 smithi134 ceph-mon[35414]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 589 op/s 2024-09-19T07:46:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:43 smithi174 ceph-mon[35370]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 589 op/s 2024-09-19T07:46:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:43 smithi103 ceph-mon[31201]: pgmap v269: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 589 op/s 2024-09-19T07:46:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 490.0 s, 2862.2 tps, lat 35.049 ms stddev 39.961 2024-09-19T07:46:46.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:46 smithi134 ceph-mon[35414]: pgmap v270: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.9 MiB/s wr, 818 op/s 2024-09-19T07:46:46.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:46 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:46 smithi174 ceph-mon[35370]: pgmap v270: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.9 MiB/s wr, 818 op/s 2024-09-19T07:46:46.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:46 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:46.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:46 smithi103 ceph-mon[31201]: pgmap v270: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.9 MiB/s wr, 818 op/s 2024-09-19T07:46:46.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:46 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:46:48.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:48 smithi134 ceph-mon[35414]: pgmap v271: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 648 op/s 2024-09-19T07:46:48.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:48 smithi174 ceph-mon[35370]: pgmap v271: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 648 op/s 2024-09-19T07:46:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:48 smithi103 ceph-mon[31201]: pgmap v271: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.2 MiB/s wr, 648 op/s 2024-09-19T07:46:50.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:50 smithi134 ceph-mon[35414]: pgmap v272: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 598 op/s 2024-09-19T07:46:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:50 smithi174 ceph-mon[35370]: pgmap v272: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 598 op/s 2024-09-19T07:46:50.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 495.0 s, 2827.4 tps, lat 35.214 ms stddev 38.748 2024-09-19T07:46:50.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:50 smithi103 ceph-mon[31201]: pgmap v272: 97 pgs: 97 active+clean; 3.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 598 op/s 2024-09-19T07:46:52.271 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:52 smithi103 ceph-mon[31201]: pgmap v273: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.9 MiB/s wr, 947 op/s 2024-09-19T07:46:52.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:52 smithi134 ceph-mon[35414]: pgmap v273: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.9 MiB/s wr, 947 op/s 2024-09-19T07:46:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:52 smithi174 ceph-mon[35370]: pgmap v273: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.9 MiB/s wr, 947 op/s 2024-09-19T07:46:54.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:54 smithi103 ceph-mon[31201]: pgmap v274: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.9 MiB/s wr, 641 op/s 2024-09-19T07:46:54.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:54 smithi134 ceph-mon[35414]: pgmap v274: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.9 MiB/s wr, 641 op/s 2024-09-19T07:46:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:54 smithi174 ceph-mon[35370]: pgmap v274: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.9 MiB/s wr, 641 op/s 2024-09-19T07:46:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 500.0 s, 2909.2 tps, lat 34.438 ms stddev 37.898 2024-09-19T07:46:56.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:56 smithi134 ceph-mon[35414]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.5 MiB/s wr, 892 op/s 2024-09-19T07:46:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:56 smithi174 ceph-mon[35370]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.5 MiB/s wr, 892 op/s 2024-09-19T07:46:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:56 smithi103 ceph-mon[31201]: pgmap v275: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.5 MiB/s wr, 892 op/s 2024-09-19T07:46:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:46:58 smithi134 ceph-mon[35414]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.6 MiB/s wr, 708 op/s 2024-09-19T07:46:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:46:58 smithi174 ceph-mon[35370]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.6 MiB/s wr, 708 op/s 2024-09-19T07:46:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:46:58 smithi103 ceph-mon[31201]: pgmap v276: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.6 MiB/s wr, 708 op/s 2024-09-19T07:46:59.353 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:46:59.958 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:00.277 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:00 smithi103 ceph-mon[31201]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.0 MiB/s wr, 644 op/s 2024-09-19T07:47:00.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:00 smithi134 ceph-mon[35414]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.0 MiB/s wr, 644 op/s 2024-09-19T07:47:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:00 smithi174 ceph-mon[35370]: pgmap v277: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.0 MiB/s wr, 644 op/s 2024-09-19T07:47:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 505.0 s, 3232.2 tps, lat 30.956 ms stddev 32.569 2024-09-19T07:47:00.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.525+0000 7f60e5f99640 1 -- 172.21.15.103:0/3680405192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102ef0 msgr2=0x7f60e010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.525+0000 7f60e5f99640 1 --2- 172.21.15.103:0/3680405192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102ef0 0x7f60e010da40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009c10 tx=0x7f60d802a7c0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 -- 172.21.15.103:0/3680405192 shutdown_connections 2024-09-19T07:47:00.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 --2- 172.21.15.103:0/3680405192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102ef0 0x7f60e010da40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 --2- 172.21.15.103:0/3680405192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e0102570 0x7f60e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 --2- 172.21.15.103:0/3680405192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0100e20 0x7f60e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 -- 172.21.15.103:0/3680405192 >> 172.21.15.103:0/3680405192 conn(0x7f60e00fc7d0 msgr2=0x7f60e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:00.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 -- 172.21.15.103:0/3680405192 shutdown_connections 2024-09-19T07:47:00.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 -- 172.21.15.103:0/3680405192 wait complete. 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.526+0000 7f60e5f99640 1 Processor -- start 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60e5f99640 1 -- start start 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60e5f99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e0100e20 0x7f60e01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60e5f99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102570 0x7f60e0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60e5f99640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102ef0 0x7f60e019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60e5f99640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0110fd0 con 0x7f60e0100e20 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60e5f99640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0110e50 con 0x7f60e0102ef0 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60e5f99640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0111150 con 0x7f60e0102570 2024-09-19T07:47:00.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60dffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102ef0 0x7f60e019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60dffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102ef0 0x7f60e019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39522/0 (socket says 172.21.15.103:39522) 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.527+0000 7f60dffff640 1 -- 172.21.15.103:0/1519850810 learned_addr learned my addr 172.21.15.103:0/1519850810 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60df7fe640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e0100e20 0x7f60e01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60dffff640 1 -- 172.21.15.103:0/1519850810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102570 msgr2=0x7f60e0198a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60dffff640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102570 0x7f60e0198a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60dffff640 1 -- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e0100e20 msgr2=0x7f60e01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60dffff640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e0100e20 0x7f60e01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60dffff640 1 -- 172.21.15.103:0/1519850810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e019d480 con 0x7f60e0102ef0 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60df7fe640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e0100e20 0x7f60e01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60dffff640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102ef0 0x7f60e019cda0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60d802adb0 tx=0x7f60d8031980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60dcff9640 1 -- 172.21.15.103:0/1519850810 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8042020 con 0x7f60e0102ef0 2024-09-19T07:47:00.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.528+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e019d710 con 0x7f60e0102ef0 2024-09-19T07:47:00.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.529+0000 7f60dcff9640 1 -- 172.21.15.103:0/1519850810 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d8032a00 con 0x7f60e0102ef0 2024-09-19T07:47:00.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.529+0000 7f60dcff9640 1 -- 172.21.15.103:0/1519850810 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d803c750 con 0x7f60e0102ef0 2024-09-19T07:47:00.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.529+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01a5050 con 0x7f60e0102ef0 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.531+0000 7f60dcff9640 1 -- 172.21.15.103:0/1519850810 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f60d8045020 con 0x7f60e0102ef0 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.531+0000 7f60e8372740 1 Processor -- start 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.531+0000 7f60e8372740 1 -- start start 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60e8372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eedc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60e8372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60e8372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60e8372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d21ee422e0 con 0x55d21eedc4a0 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60e8372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d21ede9ce0 con 0x55d21eee7680 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60e8372740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d21ee3e2a0 con 0x55d21eedcdc0 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60deffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eedc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60deffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eedc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49694/0 (socket says 172.21.15.103:49694) 2024-09-19T07:47:00.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60deffd640 1 -- 172.21.15.103:0/1071605435 learned_addr learned my addr 172.21.15.103:0/1071605435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:00.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60dcff9640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a3430 0x7f60c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60deffd640 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60df7fe640 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.532+0000 7f60dffff640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a3430 0x7f60c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.533+0000 7f60dcff9640 1 -- 172.21.15.103:0/1519850810 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f60d80f0e60 con 0x7f60e0102ef0 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.533+0000 7f60dffff640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a3430 0x7f60c80a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f60d80ee760 tx=0x7f60d80b8e40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.534+0000 7f60deffd640 1 -- 172.21.15.103:0/1071605435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 msgr2=0x55d21eee7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.534+0000 7f60deffd640 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.534+0000 7f60deffd640 1 -- 172.21.15.103:0/1071605435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 msgr2=0x55d21eedc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.534+0000 7f60deffd640 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eedc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.534+0000 7f60deffd640 1 -- 172.21.15.103:0/1071605435 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21edce440 con 0x55d21eee7680 2024-09-19T07:47:00.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.534+0000 7f60deffd640 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee9a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60d40137b0 tx=0x7f60d4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60d7fda8053df625 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:00.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.534+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1071605435 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4004050 con 0x55d21eee7680 2024-09-19T07:47:00.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.535+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1071605435 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4009e30 con 0x55d21eee7680 2024-09-19T07:47:00.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.535+0000 7f60c5ffb640 1 -- 172.21.15.103:0/1071605435 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4010ac0 con 0x55d21eee7680 2024-09-19T07:47:00.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 -- 172.21.15.103:0/1071605435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 msgr2=0x55d21eee9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee9a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60d40137b0 tx=0x7f60d4009260 comp rx=0 tx=0).stop 2024-09-19T07:47:00.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 -- 172.21.15.103:0/1071605435 shutdown_connections 2024-09-19T07:47:00.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee9a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 --2- 172.21.15.103:0/1071605435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eedc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 -- 172.21.15.103:0/1071605435 >> 172.21.15.103:0/1071605435 conn(0x55d21eed8a30 msgr2=0x55d21eed8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 -- 172.21.15.103:0/1071605435 shutdown_connections 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 -- 172.21.15.103:0/1071605435 wait complete. 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 -- start start 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.536+0000 7f60e8372740 1 Processor -- start 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60e8372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eee6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60e8372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60e8372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60e8372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d21eeee660 con 0x55d21eedc4a0 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60e8372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d21edfd050 con 0x55d21eee7680 2024-09-19T07:47:00.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60e8372740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d21eeee7e0 con 0x55d21eedcdc0 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60e8372740 10 client.0 _ll_register_callbacks cb 0x55d21ee3d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eee6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eee6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49698/0 (socket says 172.21.15.103:49698) 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 -- 172.21.15.103:0/908537013 learned_addr learned my addr 172.21.15.103:0/908537013 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46302/0 (socket says 172.21.15.103:46302) 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 -- 172.21.15.103:0/908537013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 msgr2=0x55d21eee4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 -- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 msgr2=0x55d21eee4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.537+0000 7f60deffd640 1 -- 172.21.15.103:0/908537013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d21ebb33c0 con 0x55d21eedc4a0 2024-09-19T07:47:00.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.538+0000 7f60deffd640 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eee6f60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009b00 tx=0x7f60d4004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f4cd5469d7a92d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:00.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.538+0000 7f60beffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:47:00.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.538+0000 7f60beffd640 1 -- 172.21.15.103:0/908537013 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4028020 con 0x55d21eedc4a0 2024-09-19T07:47:00.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.538+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d4009e30 con 0x55d21eedc4a0 2024-09-19T07:47:00.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.538+0000 7f60beffd640 1 -- 172.21.15.103:0/908537013 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f60d4005040 con 0x55d21eedc4a0 2024-09-19T07:47:00.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.539+0000 7f60beffd640 1 -- 172.21.15.103:0/908537013 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f60d4005240 con 0x55d21eedc4a0 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.539+0000 7f60e8372740 10 client.17247 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.539+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d21eefa000 con 0x55d21eedc4a0 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.539+0000 7f60beffd640 1 -- 172.21.15.103:0/908537013 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4009e30 con 0x55d21eedc4a0 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.539+0000 7f60beffd640 1 -- 172.21.15.103:0/908537013 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f60d4004730 con 0x55d21eedc4a0 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60e8372740 10 client.17247 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60e8372740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60e8372740 20 client.17247 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60e8372740 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21eefb7d0 0x55d21eefdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60e8372740 4 client.17247 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d21ee4f850 con 0x55d21eefb7d0 2024-09-19T07:47:00.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60df7fe640 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21eefb7d0 0x55d21eefdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.540+0000 7f60beffd640 10 client.17247 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.541+0000 7f60df7fe640 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21eefb7d0 0x55d21eefdc30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de3b49f99401c4b6 server_cookie=89f131534bf81407 in_seq=0 out_seq=0 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.541+0000 7f60beffd640 1 -- 172.21.15.103:0/908537013 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d21ee4f850 con 0x55d21eefb7d0 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.541+0000 7f60beffd640 10 client.17247 handle_command_reply: tid=0 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.541+0000 7f60e8372740 1 client.17247 shutdown 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 msgr2=0x55d21eee6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eee6f60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009b00 tx=0x7f60d4004510 comp rx=0 tx=0).stop 2024-09-19T07:47:00.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 shutdown_connections 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21eefb7d0 0x55d21eefdc30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21eee4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eee4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60beffd640 0 client.17247 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 --2- 172.21.15.103:0/908537013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eee6f60 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 >> 172.21.15.103:0/908537013 conn(0x55d21eed8a30 msgr2=0x55d21eede760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 shutdown_connections 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 1 -- 172.21.15.103:0/908537013 wait complete. 2024-09-19T07:47:00.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.542+0000 7f60e8372740 20 client.17247 trim_cache size 0 max 16384 2024-09-19T07:47:00.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.566+0000 7f60e8372740 1 Processor -- start 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.566+0000 7f60e8372740 1 -- start start 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60e8372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedc4a0 0x55d21eef78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60e8372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedcdc0 0x55d21eee2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60e8372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef03540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60e8372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ab790 con 0x55d21eedcdc0 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60e8372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ad2d0 con 0x55d21eee7680 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedc4a0 0x55d21eef78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedcdc0 0x55d21eee2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedc4a0 0x55d21eef78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46314/0 (socket says 172.21.15.103:46314) 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 -- 172.21.15.103:0/4129111662 learned_addr learned my addr 172.21.15.103:0/4129111662 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60e8372740 1 -- 172.21.15.103:0/4129111662 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01abe60 con 0x55d21eedc4a0 2024-09-19T07:47:00.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef03540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 -- 172.21.15.103:0/4129111662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 msgr2=0x55d21ef03540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef03540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 -- 172.21.15.103:0/4129111662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedcdc0 msgr2=0x55d21eee2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedcdc0 0x55d21eee2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.567+0000 7f60deffd640 1 -- 172.21.15.103:0/4129111662 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21eee31c0 con 0x55d21eedc4a0 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.568+0000 7f60deffd640 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedc4a0 0x55d21eef78f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60d4004ca0 tx=0x7f60d40099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d0e8d49b882c962 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.568+0000 7f60c57fa640 1 -- 172.21.15.103:0/4129111662 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4028040 con 0x55d21eedc4a0 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.568+0000 7f60c57fa640 1 -- 172.21.15.103:0/4129111662 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4005000 con 0x55d21eedc4a0 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.568+0000 7f60c57fa640 1 -- 172.21.15.103:0/4129111662 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4010a00 con 0x55d21eedc4a0 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.568+0000 7f60e8372740 1 -- 172.21.15.103:0/4129111662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedc4a0 msgr2=0x55d21eef78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.568+0000 7f60e8372740 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedc4a0 0x55d21eef78f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60d4004ca0 tx=0x7f60d40099e0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.569+0000 7f60e8372740 1 -- 172.21.15.103:0/4129111662 shutdown_connections 2024-09-19T07:47:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.569+0000 7f60e8372740 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef03540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.569+0000 7f60e8372740 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedcdc0 0x55d21eee2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.569+0000 7f60e8372740 1 --2- 172.21.15.103:0/4129111662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedc4a0 0x55d21eef78f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.569+0000 7f60e8372740 1 -- 172.21.15.103:0/4129111662 >> 172.21.15.103:0/4129111662 conn(0x55d21eed8a30 msgr2=0x55d21eedf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.570+0000 7f60e8372740 1 -- 172.21.15.103:0/4129111662 shutdown_connections 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.570+0000 7f60e8372740 1 -- 172.21.15.103:0/4129111662 wait complete. 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.571+0000 7f60e8372740 1 -- start start 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.571+0000 7f60e8372740 1 Processor -- start 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.571+0000 7f60e8372740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eeeefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.571+0000 7f60e8372740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eeef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60e8372740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef64500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60e8372740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01acc00 con 0x55d21eedc4a0 2024-09-19T07:47:00.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60e8372740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01ac530 con 0x55d21eee7680 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60e8372740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01a9790 con 0x55d21eedcdc0 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60e8372740 10 client.0 _ll_register_callbacks cb 0x7f60d4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eeef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eeef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46324/0 (socket says 172.21.15.103:46324) 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 -- 172.21.15.103:0/3602436627 learned_addr learned my addr 172.21.15.103:0/3602436627 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60deffd640 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eeeefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60deffd640 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef64500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 -- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 msgr2=0x55d21ef64500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef64500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 -- 172.21.15.103:0/3602436627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 msgr2=0x55d21eeeefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eeeefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 -- 172.21.15.103:0/3602436627 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d21ef64b10 con 0x55d21eedcdc0 2024-09-19T07:47:00.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60df7fe640 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eeef500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55d21eee3aa0 tx=0x7f60d000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d3b1161ad5396 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:00.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.572+0000 7f60beffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:00.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.573+0000 7f60beffd640 1 -- 172.21.15.103:0/3602436627 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004230 con 0x55d21eedcdc0 2024-09-19T07:47:00.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.573+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d4005000 con 0x55d21eedcdc0 2024-09-19T07:47:00.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.573+0000 7f60beffd640 1 -- 172.21.15.103:0/3602436627 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f60d00043d0 con 0x55d21eedcdc0 2024-09-19T07:47:00.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60beffd640 1 -- 172.21.15.103:0/3602436627 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f60d002b050 con 0x55d21eedcdc0 2024-09-19T07:47:00.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 10 client.26626 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:00.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d21ef65e70 con 0x55d21eedcdc0 2024-09-19T07:47:00.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60beffd640 1 -- 172.21.15.103:0/3602436627 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4005000 con 0x55d21eedcdc0 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60beffd640 1 -- 172.21.15.103:0/3602436627 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f60d001d7a0 con 0x55d21eedcdc0 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 10 client.26626 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 10 client.26626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 20 client.26626 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21ef66620 0x55d21ef68a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 4 client.26626 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.574+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d21ee49d60 con 0x55d21ef66620 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.575+0000 7f60deffd640 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21ef66620 0x55d21ef68a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.575+0000 7f60deffd640 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21ef66620 0x55d21ef68a40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8ae752c888b4c0 server_cookie=ebcc19b41a2a4e09 in_seq=0 out_seq=0 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.576+0000 7f60beffd640 10 client.26626 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.576+0000 7f60beffd640 1 -- 172.21.15.103:0/3602436627 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d21ee49d60 con 0x55d21ef66620 2024-09-19T07:47:00.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.576+0000 7f60beffd640 10 client.26626 handle_command_reply: tid=0 2024-09-19T07:47:00.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.576+0000 7f60e8372740 1 client.26626 shutdown 2024-09-19T07:47:00.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.576+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 msgr2=0x55d21eeef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.576+0000 7f60e8372740 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eeef500 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55d21eee3aa0 tx=0x7f60d000bfd0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 shutdown_connections 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d21ef66620 0x55d21ef68a40 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d21eee7680 0x55d21ef64500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d21eedcdc0 0x55d21eeef500 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 --2- 172.21.15.103:0/3602436627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d21eedc4a0 0x55d21eeeefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 >> 172.21.15.103:0/3602436627 conn(0x55d21eed8a30 msgr2=0x55d21eedf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60beffd640 0 client.26626 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 shutdown_connections 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.577+0000 7f60e8372740 1 -- 172.21.15.103:0/3602436627 wait complete. 2024-09-19T07:47:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.578+0000 7f60e8372740 20 client.26626 trim_cache size 0 max 16384 2024-09-19T07:47:00.580 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:47:00.580 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:47:00.580 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:47:00.580 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.578+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a3430 msgr2=0x7f60c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.578+0000 7f60e5f99640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a3430 0x7f60c80a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f60d80ee760 tx=0x7f60d80b8e40 comp rx=0 tx=0).stop 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102ef0 msgr2=0x7f60e019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102ef0 0x7f60e019cda0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60d802adb0 tx=0x7f60d8031980 comp rx=0 tx=0).stop 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 shutdown_connections 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60c80a3430 0x7f60c80a58f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e0102ef0 0x7f60e019cda0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60e0102570 0x7f60e0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 --2- 172.21.15.103:0/1519850810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60e0100e20 0x7f60e01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:00.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 >> 172.21.15.103:0/1519850810 conn(0x7f60e00fc7d0 msgr2=0x7f60e006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:00.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 shutdown_connections 2024-09-19T07:47:00.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:00.579+0000 7f60e5f99640 1 -- 172.21.15.103:0/1519850810 wait complete. 2024-09-19T07:47:01.492 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:47:01.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:47:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:02.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:02 smithi134 ceph-mon[35414]: pgmap v278: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 10 MiB/s wr, 953 op/s 2024-09-19T07:47:02.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:02 smithi103 ceph-mon[31201]: pgmap v278: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 10 MiB/s wr, 953 op/s 2024-09-19T07:47:02.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:02 smithi174 ceph-mon[35370]: pgmap v278: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 10 MiB/s wr, 953 op/s 2024-09-19T07:47:02.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.621+0000 7f835c736640 1 -- 172.21.15.103:0/622307265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 msgr2=0x7f835411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.621+0000 7f835c736640 1 --2- 172.21.15.103:0/622307265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f835411d530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f835002a3f0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.622+0000 7f835c736640 1 -- 172.21.15.103:0/622307265 shutdown_connections 2024-09-19T07:47:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.622+0000 7f835c736640 1 --2- 172.21.15.103:0/622307265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f835411d530 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.622+0000 7f835c736640 1 --2- 172.21.15.103:0/622307265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354109830 0x7f8354109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.622+0000 7f835c736640 1 --2- 172.21.15.103:0/622307265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354109450 0x7f8354110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.622+0000 7f835c736640 1 -- 172.21.15.103:0/622307265 >> 172.21.15.103:0/622307265 conn(0x7f835406e140 msgr2=0x7f835406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:02.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.623+0000 7f835c736640 1 -- 172.21.15.103:0/622307265 shutdown_connections 2024-09-19T07:47:02.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.623+0000 7f835c736640 1 -- 172.21.15.103:0/622307265 wait complete. 2024-09-19T07:47:02.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.623+0000 7f835c736640 1 Processor -- start 2024-09-19T07:47:02.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835c736640 1 -- start start 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835c736640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354109450 0x7f83541a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835c736640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354109830 0x7f83541a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835c736640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f83541aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835c736640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8354120da0 con 0x7f8354109450 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835c736640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8354120c20 con 0x7f835411b140 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835c736640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8354120f20 con 0x7f8354109830 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835acd3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f83541aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835acd3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f83541aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39588/0 (socket says 172.21.15.103:39588) 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.624+0000 7f835acd3640 1 -- 172.21.15.103:0/2574766939 learned_addr learned my addr 172.21.15.103:0/2574766939 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f8359cd1640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354109830 0x7f83541a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f835acd3640 1 -- 172.21.15.103:0/2574766939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354109830 msgr2=0x7f83541a76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:02.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f835a4d2640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354109450 0x7f83541a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f835acd3640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354109830 0x7f83541a76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f835acd3640 1 -- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354109450 msgr2=0x7f83541a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f835acd3640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354109450 0x7f83541a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f835acd3640 1 -- 172.21.15.103:0/2574766939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541ac170 con 0x7f835411b140 2024-09-19T07:47:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.625+0000 7f8359cd1640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354109830 0x7f83541a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.626+0000 7f835a4d2640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354109450 0x7f83541a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:02.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.626+0000 7f835acd3640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f83541aba70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f835002a960 tx=0x7f8350004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.626+0000 7f834b7fe640 1 -- 172.21.15.103:0/2574766939 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835003f030 con 0x7f835411b140 2024-09-19T07:47:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.626+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83541ac400 con 0x7f835411b140 2024-09-19T07:47:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.626+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8354110920 con 0x7f835411b140 2024-09-19T07:47:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.627+0000 7f834b7fe640 1 -- 172.21.15.103:0/2574766939 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8350003ad0 con 0x7f835411b140 2024-09-19T07:47:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.627+0000 7f834b7fe640 1 -- 172.21.15.103:0/2574766939 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350035070 con 0x7f835411b140 2024-09-19T07:47:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.628+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8338005290 con 0x7f835411b140 2024-09-19T07:47:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.630+0000 7f834b7fe640 1 -- 172.21.15.103:0/2574766939 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8350042030 con 0x7f835411b140 2024-09-19T07:47:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.631+0000 7f834b7fe640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a35c0 0x7f83400a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.631+0000 7f834b7fe640 1 -- 172.21.15.103:0/2574766939 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f83500efed0 con 0x7f835411b140 2024-09-19T07:47:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.632+0000 7f835a4d2640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a35c0 0x7f83400a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.633+0000 7f835a4d2640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a35c0 0x7f83400a5a80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f834c005fd0 tx=0x7f834c009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:02.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.637+0000 7f834b7fe640 1 -- 172.21.15.103:0/2574766939 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8350049030 con 0x7f835411b140 2024-09-19T07:47:02.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.977+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8338005850 con 0x7f835411b140 2024-09-19T07:47:02.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.982+0000 7f834b7fe640 1 -- 172.21.15.103:0/2574766939 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11991 (secure 0 0 0) 0x7f83500bc840 con 0x7f835411b140 2024-09-19T07:47:02.983 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:47:02.983 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":43,"btime":"2024-09-19T07:46:44:994665+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:46:44.857313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:47:02.983 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 43 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.987+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a35c0 msgr2=0x7f83400a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.987+0000 7f835c736640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a35c0 0x7f83400a5a80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f834c005fd0 tx=0x7f834c009500 comp rx=0 tx=0).stop 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.987+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 msgr2=0x7f83541aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.987+0000 7f835c736640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f83541aba70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f835002a960 tx=0x7f8350004610 comp rx=0 tx=0).stop 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.988+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 shutdown_connections 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.988+0000 7f835c736640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f83400a35c0 0x7f83400a5a80 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f834c005fd0 tx=0x7f834c009500 comp rx=0 tx=0).stop 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.988+0000 7f835c736640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f835411b140 0x7f83541aba70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.988+0000 7f835c736640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8354109830 0x7f83541a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.988+0000 7f835c736640 1 --2- 172.21.15.103:0/2574766939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8354109450 0x7f83541a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:02.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.988+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 >> 172.21.15.103:0/2574766939 conn(0x7f835406e140 msgr2=0x7f835411c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:02.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.990+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 shutdown_connections 2024-09-19T07:47:02.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:02.992+0000 7f835c736640 1 -- 172.21.15.103:0/2574766939 wait complete. 2024-09-19T07:47:03.267 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:03 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2574766939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:47:03.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:03 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2574766939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:47:03.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:03 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2574766939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:47:04.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:04 smithi134 ceph-mon[35414]: pgmap v279: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-19T07:47:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:04 smithi174 ceph-mon[35370]: pgmap v279: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-19T07:47:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:04 smithi103 ceph-mon[31201]: pgmap v279: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-19T07:47:04.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:47:05.106 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:05.383 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:05 smithi103 ceph-mon[31201]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T07:47:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:05 smithi174 ceph-mon[35370]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T07:47:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 510.0 s, 1382.8 tps, lat 72.066 ms stddev 138.880 2024-09-19T07:47:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:05 smithi134 ceph-mon[35414]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T07:47:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 -- 172.21.15.103:0/1705680892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420103760 msgr2=0x7f4420107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 --2- 172.21.15.103:0/1705680892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420103760 0x7f4420107ba0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4410009930 tx=0x7f441002a460 comp rx=0 tx=0).stop 2024-09-19T07:47:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 -- 172.21.15.103:0/1705680892 shutdown_connections 2024-09-19T07:47:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 --2- 172.21.15.103:0/1705680892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44201082e0 0x7f442010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 --2- 172.21.15.103:0/1705680892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420103760 0x7f4420107ba0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 --2- 172.21.15.103:0/1705680892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4420102db0 0x7f4420103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 -- 172.21.15.103:0/1705680892 >> 172.21.15.103:0/1705680892 conn(0x7f44200fc7d0 msgr2=0x7f44200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 -- 172.21.15.103:0/1705680892 shutdown_connections 2024-09-19T07:47:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.679+0000 7f4427aa9640 1 -- 172.21.15.103:0/1705680892 wait complete. 2024-09-19T07:47:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 Processor -- start 2024-09-19T07:47:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 -- start start 2024-09-19T07:47:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420102db0 0x7f4420103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4420103760 0x7f4420104170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44201082e0 0x7f44201046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4420112140 con 0x7f4420102db0 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4420111fc0 con 0x7f44201082e0 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4427aa9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44201122c0 con 0x7f4420103760 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.680+0000 7f4426046640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44201082e0 0x7f44201046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4426046640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44201082e0 0x7f44201046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39614/0 (socket says 172.21.15.103:39614) 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425845640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420102db0 0x7f4420103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4426046640 1 -- 172.21.15.103:0/3543161479 learned_addr learned my addr 172.21.15.103:0/3543161479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425044640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4420103760 0x7f4420104170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425845640 1 -- 172.21.15.103:0/3543161479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4420103760 msgr2=0x7f4420104170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425845640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4420103760 0x7f4420104170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425845640 1 -- 172.21.15.103:0/3543161479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44201082e0 msgr2=0x7f44201046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425845640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44201082e0 0x7f44201046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425845640 1 -- 172.21.15.103:0/3543161479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44201aaec0 con 0x7f4420102db0 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4426046640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44201082e0 0x7f44201046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.681+0000 7f4425845640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420102db0 0x7f4420103c30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f441c00ea20 tx=0x7f441c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.682+0000 7f4416ffd640 1 -- 172.21.15.103:0/3543161479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c00ccc0 con 0x7f4420102db0 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.682+0000 7f4416ffd640 1 -- 172.21.15.103:0/3543161479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441c00ce60 con 0x7f4420102db0 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.682+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44201ab1b0 con 0x7f4420102db0 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.682+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44201ab6c0 con 0x7f4420102db0 2024-09-19T07:47:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.682+0000 7f4416ffd640 1 -- 172.21.15.103:0/3543161479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0188b0 con 0x7f4420102db0 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.684+0000 7f4416ffd640 1 -- 172.21.15.103:0/3543161479 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f441c018a50 con 0x7f4420102db0 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 Processor -- start 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 -- start start 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x561014042090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610140425d0 0x561014042a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101404d370 0x56101404f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561013fa82f0 con 0x5610140425d0 2024-09-19T07:47:05.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561013f4fcf0 con 0x561014043b40 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4429e82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561013fa42b0 con 0x56101404d370 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4426046640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x561014042090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4426046640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x561014042090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39620/0 (socket says 172.21.15.103:39620) 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.685+0000 7f4426046640 1 -- 172.21.15.103:0/659182302 learned_addr learned my addr 172.21.15.103:0/659182302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4425044640 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101404d370 0x56101404f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4425845640 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610140425d0 0x561014042a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4426046640 1 -- 172.21.15.103:0/659182302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101404d370 msgr2=0x56101404f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4426046640 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101404d370 0x56101404f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4426046640 1 -- 172.21.15.103:0/659182302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610140425d0 msgr2=0x561014042a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4416ffd640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f440c0a33b0 0x7f440c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4425044640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f440c0a33b0 0x7f440c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4416ffd640 1 -- 172.21.15.103:0/3543161479 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f441c0cff60 con 0x7f4420102db0 2024-09-19T07:47:05.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4426046640 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610140425d0 0x561014042a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4426046640 1 -- 172.21.15.103:0/659182302 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561013f343a0 con 0x561014043b40 2024-09-19T07:47:05.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.686+0000 7f4425845640 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610140425d0 0x561014042a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:05.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.687+0000 7f4425044640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f440c0a33b0 0x7f440c0a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f441002a980 tx=0x7f4410005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:05.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.695+0000 7f4426046640 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x561014042090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f441800e580 tx=0x7f44180096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908942370491a8b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:05.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.696+0000 7f440bfff640 1 -- 172.21.15.103:0/659182302 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418017020 con 0x561014043b40 2024-09-19T07:47:05.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.696+0000 7f440bfff640 1 -- 172.21.15.103:0/659182302 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4418013db0 con 0x561014043b40 2024-09-19T07:47:05.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.696+0000 7f440bfff640 1 -- 172.21.15.103:0/659182302 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418007350 con 0x561014043b40 2024-09-19T07:47:05.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.697+0000 7f4429e82740 1 -- 172.21.15.103:0/659182302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 msgr2=0x561014042090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.697+0000 7f4429e82740 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x561014042090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f441800e580 tx=0x7f44180096f0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.698+0000 7f4429e82740 1 -- 172.21.15.103:0/659182302 shutdown_connections 2024-09-19T07:47:05.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.698+0000 7f4429e82740 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101404d370 0x56101404f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.698+0000 7f4429e82740 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610140425d0 0x561014042a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.698+0000 7f4429e82740 1 --2- 172.21.15.103:0/659182302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x561014042090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.698+0000 7f4429e82740 1 -- 172.21.15.103:0/659182302 >> 172.21.15.103:0/659182302 conn(0x56101400b310 msgr2=0x56101403ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:05.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.703+0000 7f4429e82740 1 -- 172.21.15.103:0/659182302 shutdown_connections 2024-09-19T07:47:05.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.706+0000 7f4429e82740 1 -- 172.21.15.103:0/659182302 wait complete. 2024-09-19T07:47:05.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.706+0000 7f4429e82740 1 -- start start 2024-09-19T07:47:05.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.706+0000 7f4429e82740 1 Processor -- start 2024-09-19T07:47:05.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4429e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x56101404cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4429e82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x56101404a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4429e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x56101404a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4429e82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561014050070 con 0x56101404d370 2024-09-19T07:47:05.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4429e82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561013f63060 con 0x561014043b40 2024-09-19T07:47:05.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4429e82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5610140501f0 con 0x5610140425d0 2024-09-19T07:47:05.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4429e82740 10 client.0 _ll_register_callbacks cb 0x561013cb68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4426046640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x56101404a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4426046640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x56101404a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49762/0 (socket says 172.21.15.103:49762) 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.708+0000 7f4426046640 1 -- 172.21.15.103:0/1954096499 learned_addr learned my addr 172.21.15.103:0/1954096499 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.709+0000 7f4426046640 1 -- 172.21.15.103:0/1954096499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 msgr2=0x56101404cd90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.709+0000 7f4426046640 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x56101404cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.709+0000 7f4426046640 1 -- 172.21.15.103:0/1954096499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 msgr2=0x56101404a040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.709+0000 7f4426046640 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x56101404a040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.709+0000 7f4426046640 1 -- 172.21.15.103:0/1954096499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561013d193c0 con 0x56101404d370 2024-09-19T07:47:05.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.709+0000 7f4426046640 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x56101404a650 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f44180099c0 tx=0x7f4418012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a652aa8c61c993 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:05.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.710+0000 7f44097fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:47:05.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.710+0000 7f44097fa640 1 -- 172.21.15.103:0/1954096499 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418017040 con 0x56101404d370 2024-09-19T07:47:05.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.710+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4418013db0 con 0x56101404d370 2024-09-19T07:47:05.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.710+0000 7f44097fa640 1 -- 172.21.15.103:0/1954096499 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f4418013ae0 con 0x56101404d370 2024-09-19T07:47:05.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.710+0000 7f44097fa640 1 -- 172.21.15.103:0/1954096499 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f441800e220 con 0x56101404d370 2024-09-19T07:47:05.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f4429e82740 10 client.17265 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:05.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56101405bab0 con 0x56101404d370 2024-09-19T07:47:05.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f44097fa640 1 -- 172.21.15.103:0/1954096499 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4418028960 con 0x56101404d370 2024-09-19T07:47:05.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f44097fa640 1 -- 172.21.15.103:0/1954096499 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f441801d810 con 0x56101404d370 2024-09-19T07:47:05.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f4429e82740 10 client.17265 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:05.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f4429e82740 10 client.17265 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:47:05.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f4429e82740 20 client.17265 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.711+0000 7f4429e82740 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56101405d240 0x56101405f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.712+0000 7f4429e82740 4 client.17265 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.712+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4418013db0 con 0x56101405d240 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.712+0000 7f4425845640 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56101405d240 0x56101405f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.713+0000 7f4425845640 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56101405d240 0x56101405f6d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=757a442b063b762d server_cookie=9fd3133fb5dfbffc in_seq=0 out_seq=0 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.713+0000 7f44097fa640 10 client.17265 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.714+0000 7f44097fa640 1 -- 172.21.15.103:0/1954096499 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4418013db0 con 0x56101405d240 2024-09-19T07:47:05.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.714+0000 7f44097fa640 10 client.17265 handle_command_reply: tid=0 2024-09-19T07:47:05.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.714+0000 7f4429e82740 1 client.17265 shutdown 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 msgr2=0x56101404a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x56101404a650 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f44180099c0 tx=0x7f4418012590 comp rx=0 tx=0).stop 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 shutdown_connections 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x56101405d240 0x56101405f6d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x56101404a650 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561014043b40 0x56101404a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 --2- 172.21.15.103:0/1954096499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x56101404cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 >> 172.21.15.103:0/1954096499 conn(0x56101400b310 msgr2=0x561014044930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.717+0000 7f44097fa640 0 client.17265 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.718+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 shutdown_connections 2024-09-19T07:47:05.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.718+0000 7f4429e82740 1 -- 172.21.15.103:0/1954096499 wait complete. 2024-09-19T07:47:05.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.718+0000 7f4429e82740 20 client.17265 trim_cache size 0 max 16384 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 Processor -- start 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 -- start start 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610140425d0 0x56101403ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561014043b40 0x561014059310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x561014049900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f442019cab0 con 0x56101404d370 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f442019e5f0 con 0x5610140425d0 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4429e82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f442019d180 con 0x561014043b40 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4426046640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x561014049900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4426046640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x561014049900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49770/0 (socket says 172.21.15.103:49770) 2024-09-19T07:47:05.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4426046640 1 -- 172.21.15.103:0/2991448145 learned_addr learned my addr 172.21.15.103:0/2991448145 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:05.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4425845640 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610140425d0 0x56101403ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4425044640 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561014043b40 0x561014059310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4425845640 1 -- 172.21.15.103:0/2991448145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561014043b40 msgr2=0x561014059310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4425845640 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561014043b40 0x561014059310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4425845640 1 -- 172.21.15.103:0/2991448145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 msgr2=0x561014049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4425845640 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x561014049900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4425845640 1 -- 172.21.15.103:0/2991448145 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561014048ff0 con 0x5610140425d0 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.737+0000 7f4426046640 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x561014049900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.738+0000 7f4425845640 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610140425d0 0x56101403ee10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f441c0cd670 tx=0x7f441c0d2470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25e0bf66f520107 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.738+0000 7f440b7fe640 1 -- 172.21.15.103:0/2991448145 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0a4e70 con 0x5610140425d0 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.738+0000 7f440b7fe640 1 -- 172.21.15.103:0/2991448145 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441c09a5b0 con 0x5610140425d0 2024-09-19T07:47:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.738+0000 7f440b7fe640 1 -- 172.21.15.103:0/2991448145 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0a49e0 con 0x5610140425d0 2024-09-19T07:47:05.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 -- 172.21.15.103:0/2991448145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610140425d0 msgr2=0x56101403ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610140425d0 0x56101403ee10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f441c0cd670 tx=0x7f441c0d2470 comp rx=0 tx=0).stop 2024-09-19T07:47:05.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 -- 172.21.15.103:0/2991448145 shutdown_connections 2024-09-19T07:47:05.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56101404d370 0x561014049900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561014043b40 0x561014059310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 --2- 172.21.15.103:0/2991448145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610140425d0 0x56101403ee10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 -- 172.21.15.103:0/2991448145 >> 172.21.15.103:0/2991448145 conn(0x56101400b310 msgr2=0x56101403e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:05.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 -- 172.21.15.103:0/2991448145 shutdown_connections 2024-09-19T07:47:05.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.739+0000 7f4429e82740 1 -- 172.21.15.103:0/2991448145 wait complete. 2024-09-19T07:47:05.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.740+0000 7f4429e82740 1 -- start start 2024-09-19T07:47:05.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.740+0000 7f4429e82740 1 Processor -- start 2024-09-19T07:47:05.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.740+0000 7f4429e82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x561014050440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4429e82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561014043b40 0x561014050980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4429e82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56101404d370 0x561014050ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x561014050440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x561014050440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46374/0 (socket says 172.21.15.103:46374) 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 -- 172.21.15.103:0/2886298677 learned_addr learned my addr 172.21.15.103:0/2886298677 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f442019df20 con 0x561014043b40 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f442019d850 con 0x56101404d370 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f442019aab0 con 0x5610140425d0 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4429e82740 10 client.0 _ll_register_callbacks cb 0x7f4410009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4426046640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56101404d370 0x561014050ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 -- 172.21.15.103:0/2886298677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56101404d370 msgr2=0x561014050ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425044640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561014043b40 0x561014050980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56101404d370 0x561014050ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 -- 172.21.15.103:0/2886298677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561014043b40 msgr2=0x561014050980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561014043b40 0x561014050980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425845640 1 -- 172.21.15.103:0/2886298677 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610140c5f70 con 0x5610140425d0 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.741+0000 7f4425044640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561014043b40 0x561014050980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f4425845640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x561014050440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f441c00d010 tx=0x7f441c010d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a25e118b13d81b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f44097fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f44097fa640 1 -- 172.21.15.103:0/2886298677 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0b5060 con 0x5610140425d0 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f44097fa640 1 -- 172.21.15.103:0/2886298677 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f441c098920 con 0x5610140425d0 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f441c09a5b0 con 0x5610140425d0 2024-09-19T07:47:05.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f44097fa640 1 -- 172.21.15.103:0/2886298677 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f441c0b5200 con 0x5610140425d0 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f4429e82740 10 client.26638 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.742+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5610140c7250 con 0x5610140425d0 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f44097fa640 1 -- 172.21.15.103:0/2886298677 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441c0a0170 con 0x5610140425d0 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f44097fa640 1 -- 172.21.15.103:0/2886298677 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f441c09b700 con 0x5610140425d0 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4429e82740 10 client.26638 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4429e82740 10 client.26638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4429e82740 20 client.26638 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4429e82740 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5610140c7870 0x5610140c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4429e82740 4 client.26638 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f441c09a5b0 con 0x5610140c7870 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4425044640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5610140c7870 0x5610140c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f44097fa640 10 client.26638 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:47:05.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.743+0000 7f4425044640 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5610140c7870 0x5610140c9c90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a129d208fca6fe2 server_cookie=d8e0d43c3d9bbdf1 in_seq=0 out_seq=0 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f44097fa640 1 -- 172.21.15.103:0/2886298677 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f441c09a5b0 con 0x5610140c7870 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f44097fa640 10 client.26638 handle_command_reply: tid=0 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 client.26638 shutdown 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 msgr2=0x561014050440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x561014050440 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f441c00d010 tx=0x7f441c010d70 comp rx=0 tx=0).stop 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 shutdown_connections 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x5610140c7870 0x5610140c9c90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56101404d370 0x561014050ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561014043b40 0x561014050980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 --2- 172.21.15.103:0/2886298677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5610140425d0 0x561014050440 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 >> 172.21.15.103:0/2886298677 conn(0x56101400b310 msgr2=0x56101404b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:05.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f44097fa640 0 client.26638 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:47:05.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 shutdown_connections 2024-09-19T07:47:05.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 1 -- 172.21.15.103:0/2886298677 wait complete. 2024-09-19T07:47:05.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.744+0000 7f4429e82740 20 client.26638 trim_cache size 0 max 16384 2024-09-19T07:47:05.749 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:47:05.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f440c0a33b0 msgr2=0x7f440c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f440c0a33b0 0x7f440c0a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f441002a980 tx=0x7f4410005a20 comp rx=0 tx=0).stop 2024-09-19T07:47:05.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420102db0 msgr2=0x7f4420103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420102db0 0x7f4420103c30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f441c00ea20 tx=0x7f441c00eee0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 shutdown_connections 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f440c0a33b0 0x7f440c0a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44201082e0 0x7f44201046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4420103760 0x7f4420104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 --2- 172.21.15.103:0/3543161479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4420102db0 0x7f4420103c30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 >> 172.21.15.103:0/3543161479 conn(0x7f44200fc7d0 msgr2=0x7f4420101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 shutdown_connections 2024-09-19T07:47:05.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:05.745+0000 7f4427aa9640 1 -- 172.21.15.103:0/3543161479 wait complete. 2024-09-19T07:47:06.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:06 smithi174 ceph-mon[35370]: pgmap v281: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-19T07:47:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:06 smithi103 ceph-mon[31201]: pgmap v281: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-19T07:47:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:06 smithi134 ceph-mon[35414]: pgmap v281: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-19T07:47:06.592 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:47:06.933 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:07.620 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:47:07.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.507+0000 7f20ecb79640 1 -- 172.21.15.103:0/454408060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8100e20 msgr2=0x7f20e8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.507+0000 7f20ecb79640 1 --2- 172.21.15.103:0/454408060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8100e20 0x7f20e8101200 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009900 tx=0x7f20d802a480 comp rx=0 tx=0).stop 2024-09-19T07:47:07.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 -- 172.21.15.103:0/454408060 shutdown_connections 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 --2- 172.21.15.103:0/454408060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20e8102ef0 0x7f20e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 --2- 172.21.15.103:0/454408060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20e8102570 0x7f20e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 --2- 172.21.15.103:0/454408060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8100e20 0x7f20e8101200 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 -- 172.21.15.103:0/454408060 >> 172.21.15.103:0/454408060 conn(0x7f20e80fc7d0 msgr2=0x7f20e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 -- 172.21.15.103:0/454408060 shutdown_connections 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 -- 172.21.15.103:0/454408060 wait complete. 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.508+0000 7f20ecb79640 1 Processor -- start 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20ecb79640 1 -- start start 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20ecb79640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20e8100e20 0x7f20e81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20ecb79640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8102570 0x7f20e8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20ecb79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20e8102ef0 0x7f20e819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20ecb79640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8110f80 con 0x7f20e8102570 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20ecb79640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8110e00 con 0x7f20e8100e20 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20ecb79640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8111100 con 0x7f20e8102ef0 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8102570 0x7f20e8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8102570 0x7f20e8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49792/0 (socket says 172.21.15.103:49792) 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e5d9b640 1 -- 172.21.15.103:0/1635203150 learned_addr learned my addr 172.21.15.103:0/1635203150 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20e8102ef0 0x7f20e819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e659c640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20e8100e20 0x7f20e81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e6d9d640 1 -- 172.21.15.103:0/1635203150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20e8100e20 msgr2=0x7f20e81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20e8100e20 0x7f20e81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e6d9d640 1 -- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8102570 msgr2=0x7f20e8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8102570 0x7f20e8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.509+0000 7f20e6d9d640 1 -- 172.21.15.103:0/1635203150 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e819d480 con 0x7f20e8102ef0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.510+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20e8102ef0 0x7f20e819cda0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009fd0 tx=0x7f20e000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.510+0000 7f20df7fe640 1 -- 172.21.15.103:0/1635203150 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000ce20 con 0x7f20e8102ef0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.511+0000 7f20df7fe640 1 -- 172.21.15.103:0/1635203150 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e00137d0 con 0x7f20e8102ef0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.511+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e819d770 con 0x7f20e8102ef0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.512+0000 7f20df7fe640 1 -- 172.21.15.103:0/1635203150 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00055e0 con 0x7f20e8102ef0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.512+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e8103680 con 0x7f20e8102ef0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.514+0000 7f20df7fe640 1 -- 172.21.15.103:0/1635203150 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f20e00058c0 con 0x7f20e8102ef0 2024-09-19T07:47:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.515+0000 7f20df7fe640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f20d40a3500 0x7f20d40a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.515+0000 7f20df7fe640 1 -- 172.21.15.103:0/1635203150 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f20e00d7170 con 0x7f20e8102ef0 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.515+0000 7f20e659c640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f20d40a3500 0x7f20d40a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.515+0000 7f20eef52740 1 Processor -- start 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.515+0000 7f20eef52740 1 -- start start 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20eef52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20eef52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbac45d0 0x5645bbac4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20eef52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbacf370 0x5645bbad1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20eef52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5645bba2a2f0 con 0x5645bbac5b40 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20eef52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5645bb9d1cf0 con 0x5645bbac45d0 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20eef52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645bba262b0 con 0x5645bbacf370 2024-09-19T07:47:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e659c640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f20d40a3500 0x7f20d40a59c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009230 tx=0x7f20d80058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49804/0 (socket says 172.21.15.103:49804) 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e659c640 1 -- 172.21.15.103:0/4283036118 learned_addr learned my addr 172.21.15.103:0/4283036118 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbacf370 0x5645bbad1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e5d9b640 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbac45d0 0x5645bbac4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e6d9d640 1 -- 172.21.15.103:0/4283036118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbac45d0 msgr2=0x5645bbac4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbac45d0 0x5645bbac4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e6d9d640 1 -- 172.21.15.103:0/4283036118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 msgr2=0x5645bbac4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e6d9d640 1 -- 172.21.15.103:0/4283036118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645bb9b63a0 con 0x5645bbacf370 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e659c640 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:07.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.516+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbacf370 0x5645bbad1760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f20e00d1390 tx=0x7f20e009fdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c486d571ce36e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.517+0000 7f20dcff9640 1 -- 172.21.15.103:0/4283036118 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00d8060 con 0x5645bbacf370 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.517+0000 7f20dcff9640 1 -- 172.21.15.103:0/4283036118 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e00a0620 con 0x5645bbacf370 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.517+0000 7f20dcff9640 1 -- 172.21.15.103:0/4283036118 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0013980 con 0x5645bbacf370 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.517+0000 7f20eef52740 1 -- 172.21.15.103:0/4283036118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbacf370 msgr2=0x5645bbad1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.517+0000 7f20eef52740 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbacf370 0x5645bbad1760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f20e00d1390 tx=0x7f20e009fdb0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.518+0000 7f20eef52740 1 -- 172.21.15.103:0/4283036118 shutdown_connections 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.518+0000 7f20eef52740 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbacf370 0x5645bbad1760 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.518+0000 7f20eef52740 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbac45d0 0x5645bbac4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.518+0000 7f20eef52740 1 --2- 172.21.15.103:0/4283036118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.518+0000 7f20eef52740 1 -- 172.21.15.103:0/4283036118 >> 172.21.15.103:0/4283036118 conn(0x5645bba8d310 msgr2=0x5645bbac0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:07.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.518+0000 7f20eef52740 1 -- 172.21.15.103:0/4283036118 shutdown_connections 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 -- 172.21.15.103:0/4283036118 wait complete. 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 -- start start 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 Processor -- start 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbaced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbacbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbacc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5645bbad20b0 con 0x5645bbac45d0 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5645bb9e5060 con 0x5645bbacf370 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645bbad2230 con 0x5645bbac5b40 2024-09-19T07:47:07.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.519+0000 7f20eef52740 10 client.0 _ll_register_callbacks cb 0x5645bb7388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbaced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbaced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49814/0 (socket says 172.21.15.103:49814) 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbacbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e659c640 1 -- 172.21.15.103:0/1716109650 learned_addr learned my addr 172.21.15.103:0/1716109650 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbacc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e5d9b640 1 -- 172.21.15.103:0/1716109650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 msgr2=0x5645bbacc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e5d9b640 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbacc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e5d9b640 1 -- 172.21.15.103:0/1716109650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 msgr2=0x5645bbaced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e5d9b640 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbaced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e5d9b640 1 -- 172.21.15.103:0/1716109650 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645bb79b3c0 con 0x5645bbac5b40 2024-09-19T07:47:07.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20e5d9b640 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbacbfe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009620 tx=0x7f20d0012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7fb81af02136735 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.520+0000 7f20cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.521+0000 7f20cdffb640 1 -- 172.21.15.103:0/1716109650 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0004050 con 0x5645bbac5b40 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.521+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e00a0620 con 0x5645bbac5b40 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.521+0000 7f20cdffb640 1 -- 172.21.15.103:0/1716109650 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f20d00041f0 con 0x5645bbac5b40 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.521+0000 7f20cdffb640 1 -- 172.21.15.103:0/1716109650 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f20d00043f0 con 0x5645bbac5b40 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.521+0000 7f20eef52740 10 client.26650 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.521+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5645bbaddc10 con 0x5645bbac5b40 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.523+0000 7f20cdffb640 1 -- 172.21.15.103:0/1716109650 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e00a0620 con 0x5645bbac5b40 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.523+0000 7f20cdffb640 1 -- 172.21.15.103:0/1716109650 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f20d0003cd0 con 0x5645bbac5b40 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.523+0000 7f20eef52740 10 client.26650 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.523+0000 7f20eef52740 10 client.26650 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:47:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.523+0000 7f20eef52740 20 client.26650 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.524+0000 7f20eef52740 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbadf490 0x5645bbae18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.524+0000 7f20eef52740 4 client.26650 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.524+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645bba37860 con 0x5645bbadf490 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.524+0000 7f20e659c640 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbadf490 0x5645bbae18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.524+0000 7f20e659c640 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbadf490 0x5645bbae18f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=745c2a124c2d6073 server_cookie=ca575c0c315e1f82 in_seq=0 out_seq=0 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.524+0000 7f20cdffb640 10 client.26650 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20cdffb640 1 -- 172.21.15.103:0/1716109650 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645bba37860 con 0x5645bbadf490 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20cdffb640 10 client.26650 handle_command_reply: tid=0 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 client.26650 shutdown 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 msgr2=0x5645bbacbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbacbfe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009620 tx=0x7f20d0012bf0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 shutdown_connections 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbadf490 0x5645bbae18f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbacc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbacbfe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 --2- 172.21.15.103:0/1716109650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbaced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20cdffb640 0 client.26650 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.525+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 >> 172.21.15.103:0/1716109650 conn(0x5645bba8d310 msgr2=0x5645bbac6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.526+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 shutdown_connections 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.526+0000 7f20eef52740 1 -- 172.21.15.103:0/1716109650 wait complete. 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.526+0000 7f20eef52740 20 client.26650 trim_cache size 0 max 16384 2024-09-19T07:47:07.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.542+0000 7f20eef52740 1 Processor -- start 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.542+0000 7f20eef52740 1 -- start start 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.542+0000 7f20eef52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac45d0 0x5645bbacb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.542+0000 7f20eef52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.542+0000 7f20eef52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbac7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.542+0000 7f20eef52740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81ab930 con 0x5645bbac5b40 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20eef52740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81ad470 con 0x5645bbacf370 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20eef52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81ac000 con 0x5645bbac45d0 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac45d0 0x5645bbacb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49824/0 (socket says 172.21.15.103:49824) 2024-09-19T07:47:07.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e5d9b640 1 -- 172.21.15.103:0/1548375336 learned_addr learned my addr 172.21.15.103:0/1548375336 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbac7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e659c640 1 -- 172.21.15.103:0/1548375336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 msgr2=0x5645bbac7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e659c640 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbac7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e659c640 1 -- 172.21.15.103:0/1548375336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 msgr2=0x5645bbac7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e659c640 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e659c640 1 -- 172.21.15.103:0/1548375336 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645bbacaf90 con 0x5645bbac45d0 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20e659c640 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac45d0 0x5645bbacb8a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5645bbae2170 tx=0x7f20d803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b3a01012d7391e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.543+0000 7f20cffff640 1 -- 172.21.15.103:0/1548375336 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8035050 con 0x5645bbac45d0 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20cffff640 1 -- 172.21.15.103:0/1548375336 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d8003e50 con 0x5645bbac45d0 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20cffff640 1 -- 172.21.15.103:0/1548375336 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8031540 con 0x5645bbac45d0 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 -- 172.21.15.103:0/1548375336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac45d0 msgr2=0x5645bbacb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac45d0 0x5645bbacb8a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5645bbae2170 tx=0x7f20d803e000 comp rx=0 tx=0).stop 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 -- 172.21.15.103:0/1548375336 shutdown_connections 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbac7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac5b40 0x5645bbac7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 --2- 172.21.15.103:0/1548375336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac45d0 0x5645bbacb8a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 -- 172.21.15.103:0/1548375336 >> 172.21.15.103:0/1548375336 conn(0x5645bba8d310 msgr2=0x5645bbace060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.544+0000 7f20eef52740 1 -- 172.21.15.103:0/1548375336 shutdown_connections 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.545+0000 7f20eef52740 1 -- 172.21.15.103:0/1548375336 wait complete. 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.545+0000 7f20eef52740 1 -- start start 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.545+0000 7f20eef52740 1 Processor -- start 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.546+0000 7f20eef52740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbad3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.546+0000 7f20eef52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbad3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.546+0000 7f20eef52740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbb491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20e5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbad3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20e5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbad3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46442/0 (socket says 172.21.15.103:46442) 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20e5d9b640 1 -- 172.21.15.103:0/174095672 learned_addr learned my addr 172.21.15.103:0/174095672 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81acda0 con 0x5645bbac45d0 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbb491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81ac6d0 con 0x5645bbacf370 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81a9930 con 0x5645bbac5b40 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20eef52740 10 client.0 _ll_register_callbacks cb 0x7f20d00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:07.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.547+0000 7f20e659c640 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbad3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20e6d9d640 1 -- 172.21.15.103:0/174095672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 msgr2=0x5645bbad3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbad3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20e6d9d640 1 -- 172.21.15.103:0/174095672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 msgr2=0x5645bbad3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbad3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20e6d9d640 1 -- 172.21.15.103:0/174095672 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645bbacc940 con 0x5645bbacf370 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20e6d9d640 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbb491d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20e009f9f0 tx=0x7f20e009fe20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ea55c7fda26e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20cdffb640 1 -- 172.21.15.103:0/174095672 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00ac020 con 0x5645bbacf370 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20cdffb640 1 -- 172.21.15.103:0/174095672 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f20e00beb60 con 0x5645bbacf370 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d8003e50 con 0x5645bbacf370 2024-09-19T07:47:07.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20cdffb640 1 -- 172.21.15.103:0/174095672 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f20e00b4050 con 0x5645bbacf370 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.548+0000 7f20eef52740 10 client.26738 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5645bbb4a7c0 con 0x5645bbacf370 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20cdffb640 1 -- 172.21.15.103:0/174095672 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d8003e50 con 0x5645bbacf370 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20cdffb640 1 -- 172.21.15.103:0/174095672 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f20e00a3070 con 0x5645bbacf370 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20eef52740 10 client.26738 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20eef52740 10 client.26738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20eef52740 20 client.26738 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20eef52740 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbb4aba0 0x5645bbb4cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20eef52740 4 client.26738 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645bba31d70 con 0x5645bbb4aba0 2024-09-19T07:47:07.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.549+0000 7f20e659c640 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbb4aba0 0x5645bbb4cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20e659c640 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbb4aba0 0x5645bbb4cf90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f2347200d999915 server_cookie=fef3559137240836 in_seq=0 out_seq=0 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20cdffb640 10 client.26738 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20cdffb640 1 -- 172.21.15.103:0/174095672 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645bba31d70 con 0x5645bbb4aba0 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20cdffb640 10 client.26738 handle_command_reply: tid=0 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 client.26738 shutdown 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 msgr2=0x5645bbb491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbb491d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20e009f9f0 tx=0x7f20e009fe20 comp rx=0 tx=0).stop 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 shutdown_connections 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5645bbb4aba0 0x5645bbb4cf90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645bbacf370 0x5645bbb491d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645bbac5b40 0x5645bbad3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20cdffb640 0 client.26738 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:47:07.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 --2- 172.21.15.103:0/174095672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5645bbac45d0 0x5645bbad3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 >> 172.21.15.103:0/174095672 conn(0x5645bba8d310 msgr2=0x5645bbabf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:07.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 shutdown_connections 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 1 -- 172.21.15.103:0/174095672 wait complete. 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.550+0000 7f20eef52740 20 client.26738 trim_cache size 0 max 16384 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f20d40a3500 msgr2=0x7f20d40a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f20d40a3500 0x7f20d40a59c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009230 tx=0x7f20d80058d0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20e8102ef0 msgr2=0x7f20e819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20e8102ef0 0x7f20e819cda0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009fd0 tx=0x7f20e000ef10 comp rx=0 tx=0).stop 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 shutdown_connections 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f20d40a3500 0x7f20d40a59c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20e8102ef0 0x7f20e819cda0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20e8102570 0x7f20e8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 --2- 172.21.15.103:0/1635203150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20e8100e20 0x7f20e81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 >> 172.21.15.103:0/1635203150 conn(0x7f20e80fc7d0 msgr2=0x7f20e806a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 shutdown_connections 2024-09-19T07:47:07.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:07.551+0000 7f20ecb79640 1 -- 172.21.15.103:0/1635203150 wait complete. 2024-09-19T07:47:08.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:08 smithi174 ceph-mon[35370]: pgmap v282: 97 pgs: 8 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 82 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-09-19T07:47:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:08 smithi103 ceph-mon[31201]: pgmap v282: 97 pgs: 8 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 82 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-09-19T07:47:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:08 smithi134 ceph-mon[35414]: pgmap v282: 97 pgs: 8 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 82 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-09-19T07:47:08.755 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:47:09.122 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:10 smithi174 ceph-mon[35370]: pgmap v283: 97 pgs: 8 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 82 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-09-19T07:47:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 515.0 s, 2092.8 tps, lat 47.833 ms stddev 51.629 2024-09-19T07:47:10.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:10 smithi103 ceph-mon[31201]: pgmap v283: 97 pgs: 8 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 82 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-09-19T07:47:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:10 smithi134 ceph-mon[35414]: pgmap v283: 97 pgs: 8 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 82 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-09-19T07:47:12.332 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:12 smithi103 ceph-mon[31201]: pgmap v284: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.91k op/s 2024-09-19T07:47:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:12 smithi174 ceph-mon[35370]: pgmap v284: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.91k op/s 2024-09-19T07:47:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 -- 172.21.15.103:0/820160740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8102780 msgr2=0x7fb0e81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 --2- 172.21.15.103:0/820160740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8102780 0x7fb0e81067d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009900 tx=0x7fb0d002a410 comp rx=0 tx=0).stop 2024-09-19T07:47:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 -- 172.21.15.103:0/820160740 shutdown_connections 2024-09-19T07:47:12.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 --2- 172.21.15.103:0/820160740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e8106f00 0x7fb0e810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 --2- 172.21.15.103:0/820160740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8102780 0x7fb0e81067d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 --2- 172.21.15.103:0/820160740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8101dd0 0x7fb0e81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 -- 172.21.15.103:0/820160740 >> 172.21.15.103:0/820160740 conn(0x7fb0e80fd8b0 msgr2=0x7fb0e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 -- 172.21.15.103:0/820160740 shutdown_connections 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.423+0000 7fb0ed697640 1 -- 172.21.15.103:0/820160740 wait complete. 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 Processor -- start 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 -- start start 2024-09-19T07:47:12.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8101dd0 0x7fb0e81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e8102780 0x7fb0e81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8106f00 0x7fb0e819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8111f60 con 0x7fb0e8106f00 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8111de0 con 0x7fb0e8102780 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0ed697640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81120e0 con 0x7fb0e8101dd0 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8106f00 0x7fb0e819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8101dd0 0x7fb0e81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8106f00 0x7fb0e819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33368/0 (socket says 172.21.15.103:33368) 2024-09-19T07:47:12.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3774731455 learned_addr learned my addr 172.21.15.103:0/3774731455 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0dffff640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e8102780 0x7fb0e81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3774731455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8101dd0 msgr2=0x7fb0e81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8101dd0 0x7fb0e81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3774731455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e8102780 msgr2=0x7fb0e81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e8102780 0x7fb0e81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.424+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3774731455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e819d350 con 0x7fb0e8106f00 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.425+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8106f00 0x7fb0e819cc70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000cce0 tx=0x7fb0e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.425+0000 7fb0e4ff9640 1 -- 172.21.15.103:0/3774731455 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000f450 con 0x7fb0e8106f00 2024-09-19T07:47:12.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.425+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e819d640 con 0x7fb0e8106f00 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.425+0000 7fb0e4ff9640 1 -- 172.21.15.103:0/3774731455 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e000fad0 con 0x7fb0e8106f00 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.425+0000 7fb0e4ff9640 1 -- 172.21.15.103:0/3774731455 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0018c10 con 0x7fb0e8106f00 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.425+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a4f20 con 0x7fb0e8106f00 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.428+0000 7fb0e4ff9640 1 -- 172.21.15.103:0/3774731455 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb0e000f5f0 con 0x7fb0e8106f00 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.428+0000 7fb0efa70740 1 Processor -- start 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.428+0000 7fb0efa70740 1 -- start start 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0e4ff9640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0d40a33b0 0x7fb0d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0e4ff9640 1 -- 172.21.15.103:0/3774731455 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fb0e00d02e0 con 0x7fb0e8106f00 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0dffff640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0d40a33b0 0x7fb0d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d691090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d691a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d69c370 0x55e05d69e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0efa70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e05d5f72f0 con 0x55e05d69c370 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0efa70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e05d59ecf0 con 0x55e05d692b40 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.430+0000 7fb0efa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e05d5f32b0 con 0x55e05d6915d0 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0dffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d691a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0e6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d691090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0dffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d691a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37458/0 (socket says 172.21.15.103:37458) 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0e6ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d691090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52384/0 (socket says 172.21.15.103:52384) 2024-09-19T07:47:12.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0dffff640 1 -- 172.21.15.103:0/121357564 learned_addr learned my addr 172.21.15.103:0/121357564 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0dffff640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0d40a33b0 0x7fb0d40a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009230 tx=0x7fb0d0005e30 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0e6ffd640 1 -- 172.21.15.103:0/121357564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 msgr2=0x55e05d691a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0e6ffd640 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d691a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0e6ffd640 1 -- 172.21.15.103:0/121357564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d69c370 msgr2=0x55e05d69e760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0e6ffd640 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d69c370 0x55e05d69e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0e6ffd640 1 -- 172.21.15.103:0/121357564 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05d5833a0 con 0x55e05d692b40 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.431+0000 7fb0dffff640 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d691a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.432+0000 7fb0e6ffd640 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d691090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009c00 tx=0x7fb0d8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee33e592697e1a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.432+0000 7fb0dd7fa640 1 -- 172.21.15.103:0/121357564 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8004230 con 0x55e05d692b40 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.432+0000 7fb0dd7fa640 1 -- 172.21.15.103:0/121357564 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d80043d0 con 0x55e05d692b40 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.432+0000 7fb0dd7fa640 1 -- 172.21.15.103:0/121357564 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8004d90 con 0x55e05d692b40 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 -- 172.21.15.103:0/121357564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 msgr2=0x55e05d691090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d691090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009c00 tx=0x7fb0d8011810 comp rx=0 tx=0).stop 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 -- 172.21.15.103:0/121357564 shutdown_connections 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d69c370 0x55e05d69e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d691a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 --2- 172.21.15.103:0/121357564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d691090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 -- 172.21.15.103:0/121357564 >> 172.21.15.103:0/121357564 conn(0x55e05d65a310 msgr2=0x55e05d68dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 -- 172.21.15.103:0/121357564 shutdown_connections 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 -- 172.21.15.103:0/121357564 wait complete. 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 -- start start 2024-09-19T07:47:12.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.433+0000 7fb0efa70740 1 Processor -- start 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d69bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d699030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d699640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0efa70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e05d69f070 con 0x55e05d692b40 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0efa70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e05d5b2060 con 0x55e05d69c370 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0efa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e05d69f1f0 con 0x55e05d6915d0 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0efa70740 10 client.0 _ll_register_callbacks cb 0x55e05d3058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d699030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d69bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d699640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d69bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37462/0 (socket says 172.21.15.103:37462) 2024-09-19T07:47:12.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e6ffd640 1 -- 172.21.15.103:0/1435699883 learned_addr learned my addr 172.21.15.103:0/1435699883 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d699030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33394/0 (socket says 172.21.15.103:33394) 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e77fe640 1 -- 172.21.15.103:0/1435699883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 msgr2=0x55e05d69bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d69bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e77fe640 1 -- 172.21.15.103:0/1435699883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 msgr2=0x55e05d699030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d699030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e77fe640 1 -- 172.21.15.103:0/1435699883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05d3683c0 con 0x55e05d69c370 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d699640 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00cd9f0 tx=0x7fb0e009c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33495dc15e59a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.434+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/1435699883 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e009c580 con 0x55e05d69c370 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d80043d0 con 0x55e05d69c370 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/1435699883 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fb0e000fc80 con 0x55e05d69c370 2024-09-19T07:47:12.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/1435699883 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb0e00a0930 con 0x55e05d69c370 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 10 client.26747 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e05d6aab90 con 0x55e05d69c370 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/1435699883 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d80043d0 con 0x55e05d69c370 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/1435699883 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb0e009b9d0 con 0x55e05d69c370 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 10 client.26747 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 10 client.26747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 20 client.26747 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6abfa0 0x55e05d6ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 4 client.26747 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.435+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e05d604860 con 0x55e05d6abfa0 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.436+0000 7fb0e6ffd640 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6abfa0 0x55e05d6ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.437+0000 7fb0e6ffd640 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6abfa0 0x55e05d6ae3b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfc6ea873985f393 server_cookie=d3634410e68acfe in_seq=0 out_seq=0 2024-09-19T07:47:12.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.437+0000 7fb0ce7fc640 10 client.26747 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.438+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/1435699883 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e05d604860 con 0x55e05d6abfa0 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.438+0000 7fb0ce7fc640 10 client.26747 handle_command_reply: tid=0 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.438+0000 7fb0efa70740 1 client.26747 shutdown 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 msgr2=0x55e05d699640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d699640 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00cd9f0 tx=0x7fb0e009c0c0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 shutdown_connections 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6abfa0 0x55e05d6ae3b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d699640 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d699030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 --2- 172.21.15.103:0/1435699883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d69bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 >> 172.21.15.103:0/1435699883 conn(0x55e05d65a310 msgr2=0x55e05d693960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0ce7fc640 0 client.26747 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:47:12.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 shutdown_connections 2024-09-19T07:47:12.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 1 -- 172.21.15.103:0/1435699883 wait complete. 2024-09-19T07:47:12.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.439+0000 7fb0efa70740 20 client.26747 trim_cache size 0 max 16384 2024-09-19T07:47:12.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.470+0000 7fb0efa70740 1 Processor -- start 2024-09-19T07:47:12.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.472+0000 7fb0efa70740 1 -- start start 2024-09-19T07:47:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.472+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d6915d0 0x55e05d6a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.472+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d6b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.472+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d69c370 0x55e05d6b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.472+0000 7fb0efa70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81ad7f0 con 0x55e05d6915d0 2024-09-19T07:47:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.472+0000 7fb0efa70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81af330 con 0x55e05d692b40 2024-09-19T07:47:12.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.472+0000 7fb0efa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81adec0 con 0x55e05d69c370 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d69c370 0x55e05d6b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d69c370 0x55e05d6b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37468/0 (socket says 172.21.15.103:37468) 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3012426811 learned_addr learned my addr 172.21.15.103:0/3012426811 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3012426811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 msgr2=0x55e05d6b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0dffff640 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d6b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d6b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3012426811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d6915d0 msgr2=0x55e05d6a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d6915d0 0x55e05d6a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.473+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3012426811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05d69c0f0 con 0x55e05d69c370 2024-09-19T07:47:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.474+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d69c370 0x55e05d6b41b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0098f50 tx=0x7fb0e000ff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=479cf1c28315b9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.474+0000 7fb0dcff9640 1 -- 172.21.15.103:0/3012426811 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e009bee0 con 0x55e05d69c370 2024-09-19T07:47:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.474+0000 7fb0dcff9640 1 -- 172.21.15.103:0/3012426811 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e009c580 con 0x55e05d69c370 2024-09-19T07:47:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.474+0000 7fb0dcff9640 1 -- 172.21.15.103:0/3012426811 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e009b9d0 con 0x55e05d69c370 2024-09-19T07:47:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.475+0000 7fb0efa70740 1 -- 172.21.15.103:0/3012426811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d69c370 msgr2=0x55e05d6b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.475+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d69c370 0x55e05d6b41b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0098f50 tx=0x7fb0e000ff60 comp rx=0 tx=0).stop 2024-09-19T07:47:12.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.475+0000 7fb0efa70740 1 -- 172.21.15.103:0/3012426811 shutdown_connections 2024-09-19T07:47:12.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.475+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d69c370 0x55e05d6b41b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.475+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d692b40 0x55e05d6b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.475+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3012426811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d6915d0 0x55e05d6a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.475+0000 7fb0efa70740 1 -- 172.21.15.103:0/3012426811 >> 172.21.15.103:0/3012426811 conn(0x55e05d65a310 msgr2=0x55e05d698120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:12.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.476+0000 7fb0efa70740 1 -- 172.21.15.103:0/3012426811 shutdown_connections 2024-09-19T07:47:12.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.476+0000 7fb0efa70740 1 -- 172.21.15.103:0/3012426811 wait complete. 2024-09-19T07:47:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.476+0000 7fb0efa70740 1 -- start start 2024-09-19T07:47:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.476+0000 7fb0efa70740 1 Processor -- start 2024-09-19T07:47:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d70f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d70f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0efa70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d70fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0efa70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81aec60 con 0x55e05d692b40 2024-09-19T07:47:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0efa70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81ae590 con 0x55e05d69c370 2024-09-19T07:47:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0efa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81ab7f0 con 0x55e05d6915d0 2024-09-19T07:47:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d70fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.477+0000 7fb0e77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d70fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52400/0 (socket says 172.21.15.103:52400) 2024-09-19T07:47:12.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.478+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3315885475 learned_addr learned my addr 172.21.15.103:0/3315885475 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.478+0000 7fb0efa70740 10 client.0 _ll_register_callbacks cb 0x7fb0e00185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.478+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3315885475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 msgr2=0x55e05d70f1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.478+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d70f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.478+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3315885475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 msgr2=0x55e05d70f6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.479+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d70f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.479+0000 7fb0e77fe640 1 -- 172.21.15.103:0/3315885475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05d6a36e0 con 0x55e05d69c370 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.479+0000 7fb0e77fe640 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d70fc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00a50c0 tx=0x7fb0e0098730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c76122e05a63c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.479+0000 7fb0ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.480+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/3315885475 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e009bee0 con 0x55e05d69c370 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.480+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/3315885475 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fb0e00a4d60 con 0x55e05d69c370 2024-09-19T07:47:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.480+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e009c580 con 0x55e05d69c370 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.480+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/3315885475 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb0e000f8e0 con 0x55e05d69c370 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.480+0000 7fb0efa70740 10 client.26750 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.480+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e05d6a49c0 con 0x55e05d69c370 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/3315885475 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0e009c580 con 0x55e05d69c370 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/3315885475 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fb0e00b0aa0 con 0x55e05d69c370 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0efa70740 10 client.26750 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0efa70740 10 client.26750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0efa70740 20 client.26750 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6a4e10 0x55e05d721850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0efa70740 4 client.26750 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:47:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e05d5fed70 con 0x55e05d6a4e10 2024-09-19T07:47:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.481+0000 7fb0e6ffd640 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6a4e10 0x55e05d721850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.482+0000 7fb0ce7fc640 10 client.26750 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:47:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.482+0000 7fb0e6ffd640 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6a4e10 0x55e05d721850 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d8c7d058e97acb4 server_cookie=12dc6a0bbdfb2078 in_seq=0 out_seq=0 2024-09-19T07:47:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.482+0000 7fb0ce7fc640 1 -- 172.21.15.103:0/3315885475 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e05d5fed70 con 0x55e05d6a4e10 2024-09-19T07:47:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.482+0000 7fb0ce7fc640 10 client.26750 handle_command_reply: tid=0 2024-09-19T07:47:12.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.482+0000 7fb0efa70740 1 client.26750 shutdown 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 msgr2=0x55e05d70fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d70fc20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00a50c0 tx=0x7fb0e0098730 comp rx=0 tx=0).stop 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 shutdown_connections 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e05d6a4e10 0x55e05d721850 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e05d69c370 0x55e05d70fc20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e05d692b40 0x55e05d70f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0ce7fc640 0 client.26750 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 --2- 172.21.15.103:0/3315885475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05d6915d0 0x55e05d70f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.483+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 >> 172.21.15.103:0/3315885475 conn(0x55e05d65a310 msgr2=0x55e05d68c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.484+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 shutdown_connections 2024-09-19T07:47:12.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.485+0000 7fb0efa70740 1 -- 172.21.15.103:0/3315885475 wait complete. 2024-09-19T07:47:12.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.485+0000 7fb0efa70740 20 client.26750 trim_cache size 0 max 16384 2024-09-19T07:47:12.487 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:47:12.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0d40a33b0 msgr2=0x7fb0d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0d40a33b0 0x7fb0d40a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009230 tx=0x7fb0d0005e30 comp rx=0 tx=0).stop 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8106f00 msgr2=0x7fb0e819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8106f00 0x7fb0e819cc70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000cce0 tx=0x7fb0e000a620 comp rx=0 tx=0).stop 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 shutdown_connections 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0d40a33b0 0x7fb0d40a5870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0e8106f00 0x7fb0e819cc70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e8102780 0x7fb0e81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 --2- 172.21.15.103:0/3774731455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8101dd0 0x7fb0e81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 >> 172.21.15.103:0/3774731455 conn(0x7fb0e80fd8b0 msgr2=0x7fb0e80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 shutdown_connections 2024-09-19T07:47:12.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:12.486+0000 7fb0ed697640 1 -- 172.21.15.103:0/3774731455 wait complete. 2024-09-19T07:47:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:12 smithi134 ceph-mon[35414]: pgmap v284: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.91k op/s 2024-09-19T07:47:14.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:47:14.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:14 smithi174 ceph-mon[35370]: pgmap v285: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.91k op/s 2024-09-19T07:47:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:14 smithi103 ceph-mon[31201]: pgmap v285: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.91k op/s 2024-09-19T07:47:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:14 smithi134 ceph-mon[35414]: pgmap v285: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.91k op/s 2024-09-19T07:47:14.641 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.239+0000 7ff68f59e640 1 -- 172.21.15.103:0/9411139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69006ea30 msgr2=0x7ff69006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.239+0000 7ff68f59e640 1 --2- 172.21.15.103:0/9411139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69006ea30 0x7ff69006ee70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff67801a910 tx=0x7ff67803b450 comp rx=0 tx=0).stop 2024-09-19T07:47:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.239+0000 7ff68f59e640 1 -- 172.21.15.103:0/9411139 shutdown_connections 2024-09-19T07:47:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.239+0000 7ff68f59e640 1 --2- 172.21.15.103:0/9411139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff69011b140 0x7ff69011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.239+0000 7ff68f59e640 1 --2- 172.21.15.103:0/9411139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff69006ea30 0x7ff69006ee70 secure :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff67801a910 tx=0x7ff67803b450 comp rx=0 tx=0).stop 2024-09-19T07:47:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.239+0000 7ff68f59e640 1 --2- 172.21.15.103:0/9411139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff690110190 0x7ff69006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.239+0000 7ff68f59e640 1 -- 172.21.15.103:0/9411139 >> 172.21.15.103:0/9411139 conn(0x7ff690073180 msgr2=0x7ff690073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:15.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.240+0000 7ff68f59e640 1 -- 172.21.15.103:0/9411139 shutdown_connections 2024-09-19T07:47:15.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.241+0000 7ff68f59e640 1 -- 172.21.15.103:0/9411139 wait complete. 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 Processor -- start 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 -- start start 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff690110190 0x7ff69011ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69011b140 0x7ff690116bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff690117110 0x7ff690117580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff690113300 con 0x7ff690110190 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff690113180 con 0x7ff69011b140 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68f59e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff690113480 con 0x7ff690117110 2024-09-19T07:47:15.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.243+0000 7ff68ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff690117110 0x7ff690117580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff690117110 0x7ff690117580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37476/0 (socket says 172.21.15.103:37476) 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 -- 172.21.15.103:0/2372034655 learned_addr learned my addr 172.21.15.103:0/2372034655 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff687fff640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69011b140 0x7ff690116bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 -- 172.21.15.103:0/2372034655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69011b140 msgr2=0x7ff690116bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69011b140 0x7ff690116bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 -- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff690110190 msgr2=0x7ff69011ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff690110190 0x7ff69011ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 -- 172.21.15.103:0/2372034655 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6901ab1c0 con 0x7ff690117110 2024-09-19T07:47:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.244+0000 7ff68ed9d640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff690117110 0x7ff690117580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff68800eab0 tx=0x7ff688009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.245+0000 7ff6877fe640 1 -- 172.21.15.103:0/2372034655 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68800ce40 con 0x7ff690117110 2024-09-19T07:47:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.245+0000 7ff6877fe640 1 -- 172.21.15.103:0/2372034655 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff688005020 con 0x7ff690117110 2024-09-19T07:47:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.245+0000 7ff6877fe640 1 -- 172.21.15.103:0/2372034655 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688010760 con 0x7ff690117110 2024-09-19T07:47:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.245+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6901ab4b0 con 0x7ff690117110 2024-09-19T07:47:15.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.245+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6901ab970 con 0x7ff690117110 2024-09-19T07:47:15.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.250+0000 7ff696e87740 1 Processor -- start 2024-09-19T07:47:15.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.251+0000 7ff6877fe640 1 -- 172.21.15.103:0/2372034655 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff688004a40 con 0x7ff690117110 2024-09-19T07:47:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.252+0000 7ff696e87740 1 -- start start 2024-09-19T07:47:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.252+0000 7ff696e87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d71a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.252+0000 7ff696e87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d71abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.252+0000 7ff6877fe640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff67c0a3540 0x7ff67c0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d71a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff6877fe640 1 -- 172.21.15.103:0/2372034655 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7ff6880d03c0 con 0x7ff690117110 2024-09-19T07:47:15.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d71a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52442/0 (socket says 172.21.15.103:52442) 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff696e87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 0x55a03d727940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff696e87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a03d6802f0 con 0x55a03d71a7b0 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff696e87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a03d627cf0 con 0x55a03d71bd20 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff696e87740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a03d67c2b0 con 0x55a03d725550 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 -- 172.21.15.103:0/3665806309 learned_addr learned my addr 172.21.15.103:0/3665806309 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 -- 172.21.15.103:0/3665806309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 msgr2=0x55a03d727940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 --2- 172.21.15.103:0/3665806309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 0x55a03d727940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 -- 172.21.15.103:0/3665806309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 msgr2=0x55a03d71abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 --2- 172.21.15.103:0/3665806309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d71abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 -- 172.21.15.103:0/3665806309 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03d60c3a0 con 0x55a03d71bd20 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.253+0000 7ff687fff640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff67c0a3540 0x7ff67c0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.254+0000 7ff687fff640 1 --2- 172.21.15.103:0/3665806309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d71a270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff67803ba40 tx=0x7ff67801a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca17d2f09bd76ace server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:15.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.254+0000 7ff684ff9640 1 -- 172.21.15.103:0/3665806309 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678004aa0 con 0x55a03d71bd20 2024-09-19T07:47:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.254+0000 7ff684ff9640 1 -- 172.21.15.103:0/3665806309 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff678004cc0 con 0x55a03d71bd20 2024-09-19T07:47:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff684ff9640 1 -- 172.21.15.103:0/3665806309 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6780056b0 con 0x55a03d71bd20 2024-09-19T07:47:15.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff696e87740 1 -- 172.21.15.103:0/3665806309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 msgr2=0x55a03d71a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff696e87740 1 --2- 172.21.15.103:0/3665806309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d71a270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff67803ba40 tx=0x7ff67801a0d0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff687fff640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff67c0a3540 0x7ff67c0a5a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff678005ab0 tx=0x7ff678003a70 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:15.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff696e87740 1 -- 172.21.15.103:0/3665806309 shutdown_connections 2024-09-19T07:47:15.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff696e87740 1 --2- 172.21.15.103:0/3665806309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 0x55a03d727940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff696e87740 1 --2- 172.21.15.103:0/3665806309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d71abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff696e87740 1 --2- 172.21.15.103:0/3665806309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d71a270 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.255+0000 7ff696e87740 1 -- 172.21.15.103:0/3665806309 >> 172.21.15.103:0/3665806309 conn(0x55a03d6e3310 msgr2=0x55a03d716e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.256+0000 7ff696e87740 1 -- 172.21.15.103:0/3665806309 shutdown_connections 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.256+0000 7ff696e87740 1 -- 172.21.15.103:0/3665806309 wait complete. 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.256+0000 7ff696e87740 1 -- start start 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.256+0000 7ff696e87740 1 Processor -- start 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.256+0000 7ff696e87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d724d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff696e87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d722030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff696e87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 0x55a03d722570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff696e87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a03d7282a0 con 0x55a03d71a7b0 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff696e87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a03d63b060 con 0x55a03d71bd20 2024-09-19T07:47:15.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff696e87740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a03d728420 con 0x55a03d725550 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff696e87740 10 client.0 _ll_register_callbacks cb 0x55a03d38e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff68e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d724d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff68e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d724d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33442/0 (socket says 172.21.15.103:33442) 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff68e59c640 1 -- 172.21.15.103:0/4042532106 learned_addr learned my addr 172.21.15.103:0/4042532106 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff687fff640 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d722030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff68ed9d640 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 0x55a03d722570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff687fff640 1 -- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 msgr2=0x55a03d722570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff687fff640 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 0x55a03d722570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff687fff640 1 -- 172.21.15.103:0/4042532106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 msgr2=0x55a03d724d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff687fff640 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d724d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.257+0000 7ff687fff640 1 -- 172.21.15.103:0/4042532106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03d3f13c0 con 0x55a03d71bd20 2024-09-19T07:47:15.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.258+0000 7ff687fff640 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d722030 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff67801a860 tx=0x7ff678047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e442e77537cb8318 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:15.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.258+0000 7ff675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:15.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.258+0000 7ff675ffb640 1 -- 172.21.15.103:0/4042532106 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67806c070 con 0x55a03d71bd20 2024-09-19T07:47:15.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.258+0000 7ff675ffb640 1 -- 172.21.15.103:0/4042532106 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7ff678004f20 con 0x55a03d71bd20 2024-09-19T07:47:15.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.258+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a03d68d860 con 0x55a03d71bd20 2024-09-19T07:47:15.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.258+0000 7ff675ffb640 1 -- 172.21.15.103:0/4042532106 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff678078050 con 0x55a03d71bd20 2024-09-19T07:47:15.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.259+0000 7ff696e87740 10 client.26762 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:15.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.259+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a03d7282a0 con 0x55a03d71bd20 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff675ffb640 1 -- 172.21.15.103:0/4042532106 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a03d68d860 con 0x55a03d71bd20 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff675ffb640 1 -- 172.21.15.103:0/4042532106 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff678057070 con 0x55a03d71bd20 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff696e87740 10 client.26762 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff696e87740 10 client.26762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff696e87740 20 client.26762 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff696e87740 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d734de0 0x55a03d7371d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff696e87740 4 client.26762 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:15.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.260+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a03d687d70 con 0x55a03d734de0 2024-09-19T07:47:15.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.263+0000 7ff68e59c640 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d734de0 0x55a03d7371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.265+0000 7ff68e59c640 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d734de0 0x55a03d7371d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a29582498108150 server_cookie=20ec3c276711fd57 in_seq=0 out_seq=0 2024-09-19T07:47:15.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.265+0000 7ff675ffb640 10 client.26762 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:47:15.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.266+0000 7ff675ffb640 1 -- 172.21.15.103:0/4042532106 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a03d687d70 con 0x55a03d734de0 2024-09-19T07:47:15.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.266+0000 7ff675ffb640 10 client.26762 handle_command_reply: tid=0 2024-09-19T07:47:15.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.266+0000 7ff696e87740 1 client.26762 shutdown 2024-09-19T07:47:15.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.266+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 msgr2=0x55a03d722030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.266+0000 7ff696e87740 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d722030 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff67801a860 tx=0x7ff678047000 comp rx=0 tx=0).stop 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 shutdown_connections 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d734de0 0x55a03d7371d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d725550 0x55a03d722570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71bd20 0x55a03d722030 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 --2- 172.21.15.103:0/4042532106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d71a7b0 0x55a03d724d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 >> 172.21.15.103:0/4042532106 conn(0x55a03d6e3310 msgr2=0x55a03d71cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff675ffb640 0 client.26762 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 shutdown_connections 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 1 -- 172.21.15.103:0/4042532106 wait complete. 2024-09-19T07:47:15.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.267+0000 7ff696e87740 20 client.26762 trim_cache size 0 max 16384 2024-09-19T07:47:15.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.293+0000 7ff696e87740 1 Processor -- start 2024-09-19T07:47:15.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.293+0000 7ff696e87740 1 -- start start 2024-09-19T07:47:15.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff696e87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d736df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff696e87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d716790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff696e87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d73c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff696e87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff67000e8c0 con 0x55a03d725550 2024-09-19T07:47:15.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff696e87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff670010400 con 0x55a03d71a7b0 2024-09-19T07:47:15.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff696e87740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff67000ef90 con 0x55a03d71bd20 2024-09-19T07:47:15.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff68ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d73c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d716790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d716790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37508/0 (socket says 172.21.15.103:37508) 2024-09-19T07:47:15.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff68ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d73c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33456/0 (socket says 172.21.15.103:33456) 2024-09-19T07:47:15.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 -- 172.21.15.103:0/795926909 learned_addr learned my addr 172.21.15.103:0/795926909 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:15.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 -- 172.21.15.103:0/795926909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 msgr2=0x55a03d736df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:47:15.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 --2- 172.21.15.103:0/795926909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d736df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 -- 172.21.15.103:0/795926909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 msgr2=0x55a03d73c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:15.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 --2- 172.21.15.103:0/795926909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d73c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.294+0000 7ff687fff640 1 -- 172.21.15.103:0/795926909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03d717380 con 0x55a03d71bd20 2024-09-19T07:47:15.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.295+0000 7ff687fff640 1 --2- 172.21.15.103:0/795926909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d716790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff678003ba0 tx=0x7ff678005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523d049dff018aa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:15.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.295+0000 7ff677fff640 1 -- 172.21.15.103:0/795926909 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67806c040 con 0x55a03d71bd20 2024-09-19T07:47:15.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.295+0000 7ff677fff640 1 -- 172.21.15.103:0/795926909 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff678040510 con 0x55a03d71bd20 2024-09-19T07:47:15.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.295+0000 7ff677fff640 1 -- 172.21.15.103:0/795926909 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678005340 con 0x55a03d71bd20 2024-09-19T07:47:15.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.296+0000 7ff696e87740 1 -- 172.21.15.103:0/795926909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 msgr2=0x55a03d716790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.296+0000 7ff696e87740 1 --2- 172.21.15.103:0/795926909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d716790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff678003ba0 tx=0x7ff678005fd0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.296+0000 7ff696e87740 1 -- 172.21.15.103:0/795926909 shutdown_connections 2024-09-19T07:47:15.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.296+0000 7ff696e87740 1 --2- 172.21.15.103:0/795926909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d73c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.296+0000 7ff696e87740 1 --2- 172.21.15.103:0/795926909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d716790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.296+0000 7ff696e87740 1 --2- 172.21.15.103:0/795926909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d736df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.296+0000 7ff696e87740 1 -- 172.21.15.103:0/795926909 >> 172.21.15.103:0/795926909 conn(0x55a03d6e3310 msgr2=0x55a03d7240b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.297+0000 7ff696e87740 1 -- 172.21.15.103:0/795926909 shutdown_connections 2024-09-19T07:47:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.298+0000 7ff696e87740 1 -- 172.21.15.103:0/795926909 wait complete. 2024-09-19T07:47:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.298+0000 7ff696e87740 1 -- start start 2024-09-19T07:47:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.298+0000 7ff696e87740 1 Processor -- start 2024-09-19T07:47:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.299+0000 7ff696e87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d72dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.299+0000 7ff696e87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d728fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.299+0000 7ff696e87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d729510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.299+0000 7ff68e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d72dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff687fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d728fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff687fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d728fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37524/0 (socket says 172.21.15.103:37524) 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff687fff640 1 -- 172.21.15.103:0/2163442132 learned_addr learned my addr 172.21.15.103:0/2163442132 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff67000fd30 con 0x55a03d725550 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff67000f660 con 0x55a03d71a7b0 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff67000c8c0 con 0x55a03d71bd20 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff696e87740 10 client.0 _ll_register_callbacks cb 0x7ff67801a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff68ed9d640 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d729510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff68e59c640 1 -- 172.21.15.103:0/2163442132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 msgr2=0x55a03d728fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:15.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff68e59c640 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d728fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff68e59c640 1 -- 172.21.15.103:0/2163442132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 msgr2=0x55a03d729510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff68e59c640 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d729510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.300+0000 7ff68e59c640 1 -- 172.21.15.103:0/2163442132 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03d717b40 con 0x55a03d71a7b0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.301+0000 7ff68e59c640 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d72dca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55a03d737880 tx=0x7ff680004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348f609e0fe380e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.301+0000 7ff675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.301+0000 7ff675ffb640 1 -- 172.21.15.103:0/2163442132 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff680003d20 con 0x55a03d71a7b0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.301+0000 7ff675ffb640 1 -- 172.21.15.103:0/2163442132 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7ff68001a3f0 con 0x55a03d71a7b0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.301+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff678040510 con 0x55a03d71a7b0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.302+0000 7ff675ffb640 1 -- 172.21.15.103:0/2163442132 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff68001a5f0 con 0x55a03d71a7b0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.302+0000 7ff696e87740 10 client.26765 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.302+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a03d79f570 con 0x55a03d71a7b0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.302+0000 7ff675ffb640 1 -- 172.21.15.103:0/2163442132 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff678040510 con 0x55a03d71a7b0 2024-09-19T07:47:15.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff675ffb640 1 -- 172.21.15.103:0/2163442132 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7ff6780035f0 con 0x55a03d71a7b0 2024-09-19T07:47:15.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff696e87740 10 client.26765 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:15.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff696e87740 10 client.26765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:47:15.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff696e87740 20 client.26765 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:15.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff696e87740 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d79fcc0 0x55a03d7a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:15.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff696e87740 4 client.26765 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:47:15.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a03d604170 con 0x55a03d79fcc0 2024-09-19T07:47:15.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.303+0000 7ff687fff640 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d79fcc0 0x55a03d7a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:15.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.304+0000 7ff687fff640 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d79fcc0 0x55a03d7a20e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e35d9d68a09fb510 server_cookie=7a224935d9619122 in_seq=0 out_seq=0 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.304+0000 7ff675ffb640 10 client.26765 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff675ffb640 1 -- 172.21.15.103:0/2163442132 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a03d604170 con 0x55a03d79fcc0 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff675ffb640 10 client.26765 handle_command_reply: tid=0 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 client.26765 shutdown 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 msgr2=0x55a03d72dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d72dca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55a03d737880 tx=0x7ff680004410 comp rx=0 tx=0).stop 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 shutdown_connections 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a03d79fcc0 0x55a03d7a20e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a03d725550 0x55a03d729510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03d71bd20 0x55a03d728fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 --2- 172.21.15.103:0/2163442132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a03d71a7b0 0x55a03d72dca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff675ffb640 0 client.26765 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:47:15.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 >> 172.21.15.103:0/2163442132 conn(0x55a03d6e3310 msgr2=0x55a03d723890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:15.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.305+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 shutdown_connections 2024-09-19T07:47:15.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.306+0000 7ff696e87740 1 -- 172.21.15.103:0/2163442132 wait complete. 2024-09-19T07:47:15.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.306+0000 7ff696e87740 20 client.26765 trim_cache size 0 max 16384 2024-09-19T07:47:15.310 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.309+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff67c0a3540 msgr2=0x7ff67c0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.309+0000 7ff68f59e640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff67c0a3540 0x7ff67c0a5a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff678005ab0 tx=0x7ff678003a70 comp rx=0 tx=0).stop 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.309+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff690117110 msgr2=0x7ff690117580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.309+0000 7ff68f59e640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff690117110 0x7ff690117580 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff68800eab0 tx=0x7ff688009f80 comp rx=0 tx=0).stop 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 shutdown_connections 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff67c0a3540 0x7ff67c0a5a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff690117110 0x7ff690117580 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff69011b140 0x7ff690116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 --2- 172.21.15.103:0/2372034655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff690110190 0x7ff69011ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:15.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 >> 172.21.15.103:0/2372034655 conn(0x7ff690073180 msgr2=0x7ff69011b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:15.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 shutdown_connections 2024-09-19T07:47:15.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:15.310+0000 7ff68f59e640 1 -- 172.21.15.103:0/2372034655 wait complete. 2024-09-19T07:47:15.326 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:15 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:47:15.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:15 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:47:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 520.0 s, 2104.6 tps, lat 47.638 ms stddev 56.435 2024-09-19T07:47:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:15 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:47:16.257 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:47:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:16 smithi174 ceph-mon[35370]: pgmap v286: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2024-09-19T07:47:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:16 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:47:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:16 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:47:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:16 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:47:16.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:16 smithi174 ceph-mon[35370]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T07:47:16.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:16 smithi103 ceph-mon[31201]: pgmap v286: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2024-09-19T07:47:16.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:16 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:47:16.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:16 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:47:16.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:16 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:47:16.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:16 smithi103 ceph-mon[31201]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T07:47:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:16 smithi134 ceph-mon[35414]: pgmap v286: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 MiB/s wr, 2.01k op/s 2024-09-19T07:47:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:16 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:47:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:16 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:47:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:16 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:47:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:16 smithi134 ceph-mon[35414]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T07:47:16.620 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:17.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.190+0000 7fcbc177f640 1 -- 172.21.15.103:0/1853438221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102570 msgr2=0x7fcbbc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.190+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1853438221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102570 0x7fcbbc1029b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009900 tx=0x7fcbb402a480 comp rx=0 tx=0).stop 2024-09-19T07:47:17.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.191+0000 7fcbc177f640 1 -- 172.21.15.103:0/1853438221 shutdown_connections 2024-09-19T07:47:17.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.191+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1853438221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbbc102ef0 0x7fcbbc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.191+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1853438221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102570 0x7fcbbc1029b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.191+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1853438221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 0x7fcbbc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.191+0000 7fcbc177f640 1 -- 172.21.15.103:0/1853438221 >> 172.21.15.103:0/1853438221 conn(0x7fcbbc0fc7d0 msgr2=0x7fcbbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:17.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.192+0000 7fcbc177f640 1 -- 172.21.15.103:0/1853438221 shutdown_connections 2024-09-19T07:47:17.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.192+0000 7fcbc177f640 1 -- 172.21.15.103:0/1853438221 wait complete. 2024-09-19T07:47:17.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.193+0000 7fcbc177f640 1 Processor -- start 2024-09-19T07:47:17.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.193+0000 7fcbc177f640 1 -- start start 2024-09-19T07:47:17.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbc177f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 0x7fcbbc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbc177f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbbc102570 0x7fcbbc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbc177f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102ef0 0x7fcbbc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 0x7fcbbc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 0x7fcbbc198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52504/0 (socket says 172.21.15.103:52504) 2024-09-19T07:47:17.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbbaffd640 1 -- 172.21.15.103:0/1221530981 learned_addr learned my addr 172.21.15.103:0/1221530981 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:17.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbbc102570 0x7fcbbc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc112090 con 0x7fcbbc102ef0 2024-09-19T07:47:17.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc111f10 con 0x7fcbbc100e20 2024-09-19T07:47:17.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc112210 con 0x7fcbbc102570 2024-09-19T07:47:17.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.194+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102ef0 0x7fcbbc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbba7fc640 1 -- 172.21.15.103:0/1221530981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 msgr2=0x7fcbbc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 0x7fcbbc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbba7fc640 1 -- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102ef0 msgr2=0x7fcbbc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102ef0 0x7fcbbc19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbba7fc640 1 -- 172.21.15.103:0/1221530981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc19d4c0 con 0x7fcbbc102570 2024-09-19T07:47:17.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 0x7fcbbc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:17.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102ef0 0x7fcbbc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:17.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbbc102570 0x7fcbbc198a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4005940 tx=0x7fcbb4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:17.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcba3fff640 1 -- 172.21.15.103:0/1221530981 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb403f070 con 0x7fcbbc102570 2024-09-19T07:47:17.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc19d750 con 0x7fcbbc102570 2024-09-19T07:47:17.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcba3fff640 1 -- 172.21.15.103:0/1221530981 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb4031c30 con 0x7fcbbc102570 2024-09-19T07:47:17.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc1a5090 con 0x7fcbbc102570 2024-09-19T07:47:17.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.195+0000 7fcba3fff640 1 -- 172.21.15.103:0/1221530981 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4035030 con 0x7fcbbc102570 2024-09-19T07:47:17.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.198+0000 7fcbc3b58740 1 Processor -- start 2024-09-19T07:47:17.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.198+0000 7fcbc3b58740 1 -- start start 2024-09-19T07:47:17.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.198+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f12b40 0x5603f1f11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.198+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f115d0 0x5603f1f11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.199+0000 7fcbbb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f12b40 0x5603f1f11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.199+0000 7fcbbb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f12b40 0x5603f1f11090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52508/0 (socket says 172.21.15.103:52508) 2024-09-19T07:47:17.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.199+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f115d0 0x5603f1f11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.199+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f115d0 0x5603f1f11a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33494/0 (socket says 172.21.15.103:33494) 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.200+0000 7fcba3fff640 1 -- 172.21.15.103:0/1221530981 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcbb4042070 con 0x7fcbbc102570 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.200+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f1e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.201+0000 7fcbbb7fe640 1 -- 172.21.15.103:0/4001766619 learned_addr learned my addr 172.21.15.103:0/4001766619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.201+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5603f1e772f0 con 0x5603f1f115d0 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.201+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603f1e1ecf0 con 0x5603f1f12b40 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.201+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5603f1e732b0 con 0x5603f1f1c370 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.201+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f1e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.201+0000 7fcba3fff640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcba40a34b0 0x7fcba40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcbbb7fe640 1 -- 172.21.15.103:0/4001766619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 msgr2=0x5603f1f1e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:17.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f1e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcbbb7fe640 1 -- 172.21.15.103:0/4001766619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f115d0 msgr2=0x5603f1f11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f115d0 0x5603f1f11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcbbb7fe640 1 -- 172.21.15.103:0/4001766619 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f1e033a0 con 0x5603f1f12b40 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcba3fff640 1 -- 172.21.15.103:0/1221530981 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fcbb40f06d0 con 0x7fcbbc102570 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f12b40 0x5603f1f11090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00e2f0 tx=0x7fcbac00e7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5d37a19a6c793a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcba40a34b0 0x7fcba40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.202+0000 7fcba17fa640 1 -- 172.21.15.103:0/4001766619 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac016070 con 0x5603f1f12b40 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.203+0000 7fcba17fa640 1 -- 172.21.15.103:0/4001766619 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbac011c90 con 0x5603f1f12b40 2024-09-19T07:47:17.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.203+0000 7fcba17fa640 1 -- 172.21.15.103:0/4001766619 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010430 con 0x5603f1f12b40 2024-09-19T07:47:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.203+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcba40a34b0 0x7fcba40a5970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbac018000 tx=0x7fcbac0106b0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f12b40 msgr2=0x5603f1f11090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f12b40 0x5603f1f11090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00e2f0 tx=0x7fcbac00e7b0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 shutdown_connections 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f1e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f115d0 0x5603f1f11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/4001766619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f12b40 0x5603f1f11090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 >> 172.21.15.103:0/4001766619 conn(0x5603f1eda310 msgr2=0x5603f1f0dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 shutdown_connections 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 -- 172.21.15.103:0/4001766619 wait complete. 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 -- start start 2024-09-19T07:47:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.204+0000 7fcbc3b58740 1 Processor -- start 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.205+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f1bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.205+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f19000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.205+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 0x5603f1f195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.205+0000 7fcbc3b58740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5603f1f1f070 con 0x5603f1f1c370 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.205+0000 7fcbc3b58740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603f1e32060 con 0x5603f1f115d0 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.205+0000 7fcbc3b58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5603f1f1f1f0 con 0x5603f1f12b40 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.205+0000 7fcbc3b58740 10 client.0 _ll_register_callbacks cb 0x5603f1b858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.206+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f1bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.206+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f1bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52510/0 (socket says 172.21.15.103:52510) 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.206+0000 7fcbbaffd640 1 -- 172.21.15.103:0/3810849370 learned_addr learned my addr 172.21.15.103:0/3810849370 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.206+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 0x5603f1f195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcbbaffd640 1 -- 172.21.15.103:0/3810849370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 msgr2=0x5603f1f19000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f19000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcbbaffd640 1 -- 172.21.15.103:0/3810849370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 msgr2=0x5603f1f195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 0x5603f1f195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcbbaffd640 1 -- 172.21.15.103:0/3810849370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f1be83c0 con 0x5603f1f115d0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f1bd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0012d20 tx=0x7fcbb000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c1ba7f446f108a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcb9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcb9eff5640 1 -- 172.21.15.103:0/3810849370 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0005020 con 0x5603f1f115d0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbac011c90 con 0x5603f1f115d0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.207+0000 7fcb9eff5640 1 -- 172.21.15.103:0/3810849370 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fcbb00051c0 con 0x5603f1f115d0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcb9eff5640 1 -- 172.21.15.103:0/3810849370 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcbb00053c0 con 0x5603f1f115d0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 10 client.26777 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5603f1f333b0 con 0x5603f1f115d0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcb9eff5640 1 -- 172.21.15.103:0/3810849370 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbac011c90 con 0x5603f1f115d0 2024-09-19T07:47:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcb9eff5640 1 -- 172.21.15.103:0/3810849370 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fcbb00104c0 con 0x5603f1f115d0 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 10 client.26777 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 10 client.26777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 20 client.26777 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f34ba0 0x5603f1f37020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 4 client.26777 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f1e84860 con 0x5603f1f34ba0 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.208+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f34ba0 0x5603f1f37020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.209+0000 7fcb9eff5640 10 client.26777 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.209+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f34ba0 0x5603f1f37020 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8852323f777b5c3b server_cookie=e7105fb9abbb5745 in_seq=0 out_seq=0 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcb9eff5640 1 -- 172.21.15.103:0/3810849370 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603f1e84860 con 0x5603f1f34ba0 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcb9eff5640 10 client.26777 handle_command_reply: tid=0 2024-09-19T07:47:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 client.26777 shutdown 2024-09-19T07:47:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 msgr2=0x5603f1f1bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f1bd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0012d20 tx=0x7fcbb000f790 comp rx=0 tx=0).stop 2024-09-19T07:47:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 shutdown_connections 2024-09-19T07:47:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f34ba0 0x5603f1f37020 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 0x5603f1f195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f19000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/3810849370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f1bd50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 >> 172.21.15.103:0/3810849370 conn(0x5603f1eda310 msgr2=0x5603f1f13960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcb9eff5640 0 client.26777 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 shutdown_connections 2024-09-19T07:47:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 1 -- 172.21.15.103:0/3810849370 wait complete. 2024-09-19T07:47:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.210+0000 7fcbc3b58740 20 client.26777 trim_cache size 0 max 16384 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.231+0000 7fcbc3b58740 1 Processor -- start 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbc3b58740 1 -- start start 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f12b40 0x5603f1f30ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbc3b58740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc1ab710 con 0x5603f1f12b40 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbc3b58740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc1ad250 con 0x5603f1f115d0 2024-09-19T07:47:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbc3b58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc1abde0 con 0x5603f1f1c370 2024-09-19T07:47:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f36f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52512/0 (socket says 172.21.15.103:52512) 2024-09-19T07:47:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbbaffd640 1 -- 172.21.15.103:0/2551996329 learned_addr learned my addr 172.21.15.103:0/2551996329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbbb7fe640 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.232+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f12b40 0x5603f1f30ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcbbaffd640 1 -- 172.21.15.103:0/2551996329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 msgr2=0x5603f1f14120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f14120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcbbaffd640 1 -- 172.21.15.103:0/2551996329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f12b40 msgr2=0x5603f1f30ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f12b40 0x5603f1f30ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcbbaffd640 1 -- 172.21.15.103:0/2551996329 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f1f1c080 con 0x5603f1f115d0 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f12b40 0x5603f1f30ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f36f90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00108e0 tx=0x7fcbb0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96da22313ab8219 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcba0ff9640 1 -- 172.21.15.103:0/2551996329 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0028070 con 0x5603f1f115d0 2024-09-19T07:47:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcba0ff9640 1 -- 172.21.15.103:0/2551996329 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb0009cf0 con 0x5603f1f115d0 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.233+0000 7fcba0ff9640 1 -- 172.21.15.103:0/2551996329 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb001cbb0 con 0x5603f1f115d0 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.234+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2551996329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 msgr2=0x5603f1f36f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.234+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f36f90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00108e0 tx=0x7fcbb0005770 comp rx=0 tx=0).stop 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.234+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2551996329 shutdown_connections 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.234+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f1c370 0x5603f1f14120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.234+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f12b40 0x5603f1f30ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.234+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2551996329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f36f90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.234+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2551996329 >> 172.21.15.103:0/2551996329 conn(0x5603f1eda310 msgr2=0x5603f1f08a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.235+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2551996329 shutdown_connections 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.235+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2551996329 wait complete. 2024-09-19T07:47:17.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.235+0000 7fcbc3b58740 1 -- start start 2024-09-19T07:47:17.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.235+0000 7fcbc3b58740 1 Processor -- start 2024-09-19T07:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f2cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f2d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbc3b58740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 0x5603f1f155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbba7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbba7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f2d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37572/0 (socket says 172.21.15.103:37572) 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f2cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f2cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:52514/0 (socket says 172.21.15.103:52514) 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbba7fc640 1 -- 172.21.15.103:0/2416670215 learned_addr learned my addr 172.21.15.103:0/2416670215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc1acb80 con 0x5603f1f1c370 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc1ac4b0 con 0x5603f1f115d0 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.236+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc1a9710 con 0x5603f1f12b40 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbba7fc640 1 -- 172.21.15.103:0/2416670215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 msgr2=0x5603f1f2cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f2cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbba7fc640 1 -- 172.21.15.103:0/2416670215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 msgr2=0x5603f1f155d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 0x5603f1f155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbba7fc640 1 -- 172.21.15.103:0/2416670215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f1f15c10 con 0x5603f1f12b40 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbc3b58740 10 client.0 _ll_register_callbacks cb 0x7fcbb0012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb0009cf0 con 0x5603f1f12b40 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcbba7fc640 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f2d250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4006010 tx=0x7fcbb403b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cc60ea4aafd5a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.237+0000 7fcb9eff5640 10 client.26701 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.238+0000 7fcb9eff5640 1 -- 172.21.15.103:0/2416670215 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40d0e60 con 0x5603f1f12b40 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.238+0000 7fcb9eff5640 1 -- 172.21.15.103:0/2416670215 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fcbb40ee030 con 0x5603f1f12b40 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.238+0000 7fcb9eff5640 1 -- 172.21.15.103:0/2416670215 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcbb4049070 con 0x5603f1f12b40 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.238+0000 7fcbc3b58740 10 client.26701 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:17.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.238+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=43}) -- 0x5603f1f9f050 con 0x5603f1f12b40 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcb9eff5640 1 -- 172.21.15.103:0/2416670215 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fcbb40b8fb0 con 0x5603f1f12b40 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcbc3b58740 10 client.26701 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcbc3b58740 10 client.26701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcbc3b58740 20 client.26701 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f9f670 0x5603f1fa1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcbc3b58740 4 client.26701 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603f1e7ed70 con 0x5603f1f9f670 2024-09-19T07:47:17.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.239+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f9f670 0x5603f1fa1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.242+0000 7fcb9eff5640 10 client.26701 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.242+0000 7fcbbaffd640 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f9f670 0x5603f1fa1b30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efbef2a48360191b server_cookie=5d565623290a771a in_seq=0 out_seq=0 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.242+0000 7fcb9eff5640 1 -- 172.21.15.103:0/2416670215 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603f1e7ed70 con 0x5603f1f9f670 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.242+0000 7fcb9eff5640 10 client.26701 handle_command_reply: tid=0 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.242+0000 7fcbc3b58740 1 client.26701 shutdown 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.243+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 msgr2=0x5603f1f2d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.243+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f2d250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4006010 tx=0x7fcbb403b6b0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.243+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 shutdown_connections 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.243+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5603f1f9f670 0x5603f1fa1b30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f1f1c370 0x5603f1f155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcb9eff5640 0 client.26701 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603f1f12b40 0x5603f1f2d250 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcbc3b58740 1 --2- 172.21.15.103:0/2416670215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603f1f115d0 0x5603f1f2cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 >> 172.21.15.103:0/2416670215 conn(0x5603f1eda310 msgr2=0x5603f1f0c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 shutdown_connections 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcbc3b58740 1 -- 172.21.15.103:0/2416670215 wait complete. 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.244+0000 7fcbc3b58740 20 client.26701 trim_cache size 0 max 16384 2024-09-19T07:47:17.246 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:47:17.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcba40a34b0 msgr2=0x7fcba40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcba40a34b0 0x7fcba40a5970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbac018000 tx=0x7fcbac0106b0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbbc102570 msgr2=0x7fcbbc198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbbc102570 0x7fcbbc198a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4005940 tx=0x7fcbb4003c10 comp rx=0 tx=0).stop 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 shutdown_connections 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcba40a34b0 0x7fcba40a5970 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbbc102ef0 0x7fcbbc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbbc102570 0x7fcbbc198a50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 --2- 172.21.15.103:0/1221530981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbbc100e20 0x7fcbbc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 >> 172.21.15.103:0/1221530981 conn(0x7fcbbc0fc7d0 msgr2=0x7fcbbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 shutdown_connections 2024-09-19T07:47:17.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:17.245+0000 7fcbc177f640 1 -- 172.21.15.103:0/1221530981 wait complete. 2024-09-19T07:47:18.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:18 smithi103 ceph-mon[31201]: pgmap v288: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-09-19T07:47:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:18 smithi174 ceph-mon[35370]: pgmap v288: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-09-19T07:47:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:18 smithi134 ceph-mon[35414]: pgmap v288: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-09-19T07:47:19.086 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:47:19.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:47:19.475 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 -- 172.21.15.103:0/542715641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f173c103ff0 msgr2=0x7f173c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 --2- 172.21.15.103:0/542715641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f173c103ff0 0x7f173c1043d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1724009860 tx=0x7f172402a340 comp rx=0 tx=0).stop 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 -- 172.21.15.103:0/542715641 shutdown_connections 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 --2- 172.21.15.103:0/542715641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f173c109120 0x7f173c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 --2- 172.21.15.103:0/542715641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c1049a0 0x7f173c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 --2- 172.21.15.103:0/542715641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f173c103ff0 0x7f173c1043d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 -- 172.21.15.103:0/542715641 >> 172.21.15.103:0/542715641 conn(0x7f173c0ffab0 msgr2=0x7f173c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 -- 172.21.15.103:0/542715641 shutdown_connections 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.071+0000 7f17429b3640 1 -- 172.21.15.103:0/542715641 wait complete. 2024-09-19T07:47:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.072+0000 7f17429b3640 1 Processor -- start 2024-09-19T07:47:20.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.072+0000 7f17429b3640 1 -- start start 2024-09-19T07:47:20.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.073+0000 7f17429b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f173c103ff0 0x7f173c073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.073+0000 7f17429b3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f173c1049a0 0x7f173c073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.073+0000 7f17429b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c109120 0x7f173c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.073+0000 7f17429b3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f173c114120 con 0x7f173c1049a0 2024-09-19T07:47:20.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.073+0000 7f17429b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f173c113fa0 con 0x7f173c109120 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.073+0000 7f17429b3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1142a0 con 0x7f173c103ff0 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f173c1049a0 0x7f173c073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c109120 0x7f173c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f173c1049a0 0x7f173c073ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49482/0 (socket says 172.21.15.103:49482) 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f173b7fe640 1 -- 172.21.15.103:0/2672451175 learned_addr learned my addr 172.21.15.103:0/2672451175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c109120 0x7f173c06cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43538/0 (socket says 172.21.15.103:43538) 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 -- 172.21.15.103:0/2672451175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f173c103ff0 msgr2=0x7f173c073960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:47:20.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f173c103ff0 0x7f173c073960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 -- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f173c1049a0 msgr2=0x7f173c073ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f173c1049a0 0x7f173c073ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 -- 172.21.15.103:0/2672451175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c06d490 con 0x7f173c109120 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.074+0000 7f1740f50640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c109120 0x7f173c06cbd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f172c00efc0 tx=0x7f172c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.075+0000 7f17397fa640 1 -- 172.21.15.103:0/2672451175 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c003970 con 0x7f173c109120 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.075+0000 7f17397fa640 1 -- 172.21.15.103:0/2672451175 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c004020 con 0x7f173c109120 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.075+0000 7f17397fa640 1 -- 172.21.15.103:0/2672451175 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c00b9b0 con 0x7f173c109120 2024-09-19T07:47:20.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.075+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173c06d780 con 0x7f173c109120 2024-09-19T07:47:20.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.075+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c071720 con 0x7f173c109120 2024-09-19T07:47:20.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.079+0000 7f1744d8c740 1 Processor -- start 2024-09-19T07:47:20.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f17397fa640 1 -- 172.21.15.103:0/2672451175 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f172c01d020 con 0x7f173c109120 2024-09-19T07:47:20.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f1744d8c740 1 -- start start 2024-09-19T07:47:20.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa303750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa30e550 0x5629fa310940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f1744d8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629fa260280 con 0x5629fa303c90 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f1744d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5629fa207b10 con 0x5629fa30e550 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.080+0000 7f1744d8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5629fa25c240 con 0x5629fa303370 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa303750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa303750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40876/0 (socket says 172.21.15.103:40876) 2024-09-19T07:47:20.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173b7fe640 1 -- 172.21.15.103:0/2122065916 learned_addr learned my addr 172.21.15.103:0/2122065916 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:20.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173b7fe640 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa30e550 0x5629fa310940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173bfff640 1 -- 172.21.15.103:0/2122065916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 msgr2=0x5629fa303750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173bfff640 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa303750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173bfff640 1 -- 172.21.15.103:0/2122065916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa30e550 msgr2=0x5629fa310940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173bfff640 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa30e550 0x5629fa310940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f173bfff640 1 -- 172.21.15.103:0/2122065916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629fa2141f0 con 0x5629fa303c90 2024-09-19T07:47:20.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.081+0000 7f17397fa640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17280a3500 0x7f17280a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.082+0000 7f173bfff640 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30e010 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1724009780 tx=0x7f1724005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec9aeab3ab1c580 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.082+0000 7f17397fa640 1 -- 172.21.15.103:0/2672451175 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f172c0cfae0 con 0x7f173c109120 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.082+0000 7f173b7fe640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17280a3500 0x7f17280a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.082+0000 7f17327fc640 1 -- 172.21.15.103:0/2122065916 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724037040 con 0x5629fa303c90 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.082+0000 7f17327fc640 1 -- 172.21.15.103:0/2122065916 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1724030c50 con 0x5629fa303c90 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.082+0000 7f17327fc640 1 -- 172.21.15.103:0/2122065916 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240035f0 con 0x5629fa303c90 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 -- 172.21.15.103:0/2122065916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 msgr2=0x5629fa30e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f173b7fe640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17280a3500 0x7f17280a59c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f173400afd0 tx=0x7f173400a380 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30e010 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1724009780 tx=0x7f1724005da0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 -- 172.21.15.103:0/2122065916 shutdown_connections 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa30e550 0x5629fa310940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30e010 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2122065916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa303750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 -- 172.21.15.103:0/2122065916 >> 172.21.15.103:0/2122065916 conn(0x5629fa2ff900 msgr2=0x5629fa2ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 -- 172.21.15.103:0/2122065916 shutdown_connections 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.083+0000 7f1744d8c740 1 -- 172.21.15.103:0/2122065916 wait complete. 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.084+0000 7f1744d8c740 1 -- start start 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.084+0000 7f1744d8c740 1 Processor -- start 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.084+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa30b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.084+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa30e550 0x5629fa30bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f1744d8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629fa311120 con 0x5629fa303c90 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f1744d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5629fa21af10 con 0x5629fa303370 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f1744d8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5629fa3112a0 con 0x5629fa30e550 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f1744d8c740 10 client.0 _ll_register_callbacks cb 0x5629fa1e3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa30b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa30b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43558/0 (socket says 172.21.15.103:43558) 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 -- 172.21.15.103:0/1812902443 learned_addr learned my addr 172.21.15.103:0/1812902443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 -- 172.21.15.103:0/1812902443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa30e550 msgr2=0x5629fa30bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa30e550 0x5629fa30bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 -- 172.21.15.103:0/1812902443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 msgr2=0x5629fa30b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 -- 172.21.15.103:0/1812902443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f9fd16a0 con 0x5629fa303370 2024-09-19T07:47:20.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f173bfff640 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa30b4b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1724009830 tx=0x7f1724005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43cd06db8405676 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:20.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.085+0000 7f171b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.086+0000 7f171b7fe640 1 -- 172.21.15.103:0/1812902443 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724037040 con 0x5629fa303370 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.087+0000 7f171b7fe640 1 -- 172.21.15.103:0/1812902443 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f1724046460 con 0x5629fa303370 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.087+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1724030c50 con 0x5629fa303370 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.090+0000 7f171b7fe640 1 -- 172.21.15.103:0/1812902443 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1724005850 con 0x5629fa303370 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.090+0000 7f1744d8c740 10 client.26792 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.090+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5629fa31cdd0 con 0x5629fa303370 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f171b7fe640 1 -- 172.21.15.103:0/1812902443 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1724030c50 con 0x5629fa303370 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f171b7fe640 1 -- 172.21.15.103:0/1812902443 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f172403a070 con 0x5629fa303370 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f1744d8c740 10 client.26792 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:20.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f1744d8c740 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:20.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f1744d8c740 20 client.26792 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:20.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa31e120 0x5629fa320510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f1744d8c740 4 client.26792 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:20.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629fa26d7f0 con 0x5629fa31e120 2024-09-19T07:47:20.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.091+0000 7f173b7fe640 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa31e120 0x5629fa320510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.092+0000 7f171b7fe640 10 client.26792 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.092+0000 7f173b7fe640 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa31e120 0x5629fa320510 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815e7a783dfd1eee server_cookie=4b6567dd2f7770d1 in_seq=0 out_seq=0 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.093+0000 7f171b7fe640 1 -- 172.21.15.103:0/1812902443 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629fa26d7f0 con 0x5629fa31e120 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.093+0000 7f171b7fe640 10 client.26792 handle_command_reply: tid=0 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.093+0000 7f1744d8c740 1 client.26792 shutdown 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.093+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 msgr2=0x5629fa30b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.093+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa30b4b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1724009830 tx=0x7f1724005780 comp rx=0 tx=0).stop 2024-09-19T07:47:20.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 shutdown_connections 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa31e120 0x5629fa320510 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa30e550 0x5629fa30bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa303c90 0x5629fa30b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1812902443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa30b4b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 >> 172.21.15.103:0/1812902443 conn(0x5629fa2ff900 msgr2=0x5629fa305630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f171b7fe640 0 client.26792 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 shutdown_connections 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 1 -- 172.21.15.103:0/1812902443 wait complete. 2024-09-19T07:47:20.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.094+0000 7f1744d8c740 20 client.26792 trim_cache size 0 max 16384 2024-09-19T07:47:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.122+0000 7f1744d8c740 1 Processor -- start 2024-09-19T07:47:20.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.122+0000 7f1744d8c740 1 -- start start 2024-09-19T07:47:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa325b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303c90 0x5629fa326080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa308730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f1744d8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1b3ed0 con 0x5629fa30e550 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f1744d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1b5a10 con 0x5629fa303370 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f1744d8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1b45a0 con 0x5629fa303c90 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303c90 0x5629fa326080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303c90 0x5629fa326080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40896/0 (socket says 172.21.15.103:40896) 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173b7fe640 1 -- 172.21.15.103:0/2185163240 learned_addr learned my addr 172.21.15.103:0/2185163240 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f1740f50640 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa308730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173bfff640 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa325b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173bfff640 1 -- 172.21.15.103:0/2185163240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303c90 msgr2=0x5629fa326080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173bfff640 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303c90 0x5629fa326080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173bfff640 1 -- 172.21.15.103:0/2185163240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 msgr2=0x5629fa308730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173bfff640 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa308730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.123+0000 7f173bfff640 1 -- 172.21.15.103:0/2185163240 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629fa2f1680 con 0x5629fa303370 2024-09-19T07:47:20.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.124+0000 7f173bfff640 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa325b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f17240059f0 tx=0x7f1724000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53349721f68b71c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:20.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.124+0000 7f1731ffb640 1 -- 172.21.15.103:0/2185163240 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724030c50 con 0x5629fa303370 2024-09-19T07:47:20.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.124+0000 7f1731ffb640 1 -- 172.21.15.103:0/2185163240 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1724032d70 con 0x5629fa303370 2024-09-19T07:47:20.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.124+0000 7f1731ffb640 1 -- 172.21.15.103:0/2185163240 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724004920 con 0x5629fa303370 2024-09-19T07:47:20.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 -- 172.21.15.103:0/2185163240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 msgr2=0x5629fa325b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa325b40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f17240059f0 tx=0x7f1724000bc0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 -- 172.21.15.103:0/2185163240 shutdown_connections 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa308730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303c90 0x5629fa326080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 --2- 172.21.15.103:0/2185163240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303370 0x5629fa325b40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 -- 172.21.15.103:0/2185163240 >> 172.21.15.103:0/2185163240 conn(0x5629fa2ff900 msgr2=0x5629fa2f6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 -- 172.21.15.103:0/2185163240 shutdown_connections 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.125+0000 7f1744d8c740 1 -- 172.21.15.103:0/2185163240 wait complete. 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.126+0000 7f1744d8c740 1 -- start start 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.126+0000 7f1744d8c740 1 Processor -- start 2024-09-19T07:47:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa316520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303c90 0x5629fa316a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1744d8c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa3134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1744d8c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1b5340 con 0x5629fa30e550 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1744d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1b4c70 con 0x5629fa303c90 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f173bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa316520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f173bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa316520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40898/0 (socket says 172.21.15.103:40898) 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f173bfff640 1 -- 172.21.15.103:0/1068871491 learned_addr learned my addr 172.21.15.103:0/1068871491 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f173b7fe640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303c90 0x5629fa316a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f173c07a7b0 con 0x5629fa303370 2024-09-19T07:47:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1744d8c740 10 client.0 _ll_register_callbacks cb 0x7f173400aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.127+0000 7f1740f50640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa3134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f173b7fe640 1 -- 172.21.15.103:0/1068871491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 msgr2=0x5629fa316520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f173b7fe640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa316520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f173b7fe640 1 -- 172.21.15.103:0/1068871491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 msgr2=0x5629fa3134f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f173b7fe640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa3134f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f173b7fe640 1 -- 172.21.15.103:0/1068871491 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629fa3171a0 con 0x5629fa303c90 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f1740f50640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa3134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f173bfff640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa316520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f173b7fe640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303c90 0x5629fa316a60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5629fa3043f0 tx=0x7f17340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c527e3cc672c7df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f171b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f171b7fe640 1 -- 172.21.15.103:0/1068871491 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734011050 con 0x5629fa303c90 2024-09-19T07:47:20.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f171b7fe640 1 -- 172.21.15.103:0/1068871491 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f17340111f0 con 0x5629fa303c90 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.128+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5629fa267d00 con 0x5629fa303c90 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.129+0000 7f171b7fe640 1 -- 172.21.15.103:0/1068871491 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1734034440 con 0x5629fa303c90 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.129+0000 7f1744d8c740 10 client.26798 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.129+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5629fa389210 con 0x5629fa303c90 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.129+0000 7f171b7fe640 1 -- 172.21.15.103:0/1068871491 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1734010910 con 0x5629fa303c90 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f171b7fe640 1 -- 172.21.15.103:0/1068871491 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f1734032020 con 0x5629fa303c90 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f1744d8c740 10 client.26798 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f1744d8c740 10 client.26798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f1744d8c740 20 client.26798 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa3898e0 0x5629fa38bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:20.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f1744d8c740 4 client.26798 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:47:20.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629fa267d00 con 0x5629fa3898e0 2024-09-19T07:47:20.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.130+0000 7f173bfff640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa3898e0 0x5629fa38bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:20.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.131+0000 7f173bfff640 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa3898e0 0x5629fa38bcd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16dab1de043c5d44 server_cookie=24201eb86101b18a in_seq=0 out_seq=0 2024-09-19T07:47:20.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.131+0000 7f171b7fe640 10 client.26798 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.132+0000 7f171b7fe640 1 -- 172.21.15.103:0/1068871491 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629fa267d00 con 0x5629fa3898e0 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.132+0000 7f171b7fe640 10 client.26798 handle_command_reply: tid=0 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.132+0000 7f1744d8c740 1 client.26798 shutdown 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303c90 msgr2=0x5629fa316a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303c90 0x5629fa316a60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5629fa3043f0 tx=0x7f17340046c0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 shutdown_connections 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5629fa3898e0 0x5629fa38bcd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629fa30e550 0x5629fa3134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629fa303c90 0x5629fa316a60 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5629fa3043f0 tx=0x7f17340046c0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 --2- 172.21.15.103:0/1068871491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629fa303370 0x5629fa316520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 >> 172.21.15.103:0/1068871491 conn(0x5629fa2ff900 msgr2=0x5629fa2f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.133+0000 7f171b7fe640 0 client.26798 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.134+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 shutdown_connections 2024-09-19T07:47:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.134+0000 7f1744d8c740 1 -- 172.21.15.103:0/1068871491 wait complete. 2024-09-19T07:47:20.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.134+0000 7f1744d8c740 20 client.26798 trim_cache size 0 max 16384 2024-09-19T07:47:20.139 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:47:20.139 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:47:20.139 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "573814b5-cd93-439e-9553-7c4377eb08a3", 2024-09-19T07:47:20.139 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:47:20.139 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:47:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.137+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17280a3500 msgr2=0x7f17280a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.137+0000 7f17429b3640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17280a3500 0x7f17280a59c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f173400afd0 tx=0x7f173400a380 comp rx=0 tx=0).stop 2024-09-19T07:47:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.137+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c109120 msgr2=0x7f173c06cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.137+0000 7f17429b3640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c109120 0x7f173c06cbd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f172c00efc0 tx=0x7f172c00c800 comp rx=0 tx=0).stop 2024-09-19T07:47:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 shutdown_connections 2024-09-19T07:47:20.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f17280a3500 0x7f17280a59c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f173c109120 0x7f173c06cbd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f173c1049a0 0x7f173c073ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 --2- 172.21.15.103:0/2672451175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f173c103ff0 0x7f173c073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 >> 172.21.15.103:0/2672451175 conn(0x7f173c0ffab0 msgr2=0x7f173c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 shutdown_connections 2024-09-19T07:47:20.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:20.138+0000 7f17429b3640 1 -- 172.21.15.103:0/2672451175 wait complete. 2024-09-19T07:47:20.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:20 smithi103 ceph-mon[31201]: pgmap v289: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-09-19T07:47:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:20 smithi174 ceph-mon[35370]: pgmap v289: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-09-19T07:47:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 525.0 s, 2838.0 tps, lat 35.279 ms stddev 39.798 2024-09-19T07:47:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:20 smithi134 ceph-mon[35414]: pgmap v289: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-09-19T07:47:21.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:47:21.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:47:21.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:47:21.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:47:21.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:47:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:47:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:47:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:47:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:47:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:47:21.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.576 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:47:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:47:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:47:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:47:21.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:47:21.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:47:21.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:47:21.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:21.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:47:22.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:22 smithi174 ceph-mon[35370]: pgmap v290: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 973 op/s 2024-09-19T07:47:22.415 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:22 smithi103 ceph-mon[31201]: pgmap v290: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 973 op/s 2024-09-19T07:47:22.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 -- 172.21.15.103:0/1116385892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9850109b10 msgr2=0x7f9850116960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 --2- 172.21.15.103:0/1116385892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9850109b10 0x7f9850116960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9840009900 tx=0x7f984002a410 comp rx=0 tx=0).stop 2024-09-19T07:47:22.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 -- 172.21.15.103:0/1116385892 shutdown_connections 2024-09-19T07:47:22.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 --2- 172.21.15.103:0/1116385892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9850116ea0 0x7f9850119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 --2- 172.21.15.103:0/1116385892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9850109b10 0x7f9850116960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 --2- 172.21.15.103:0/1116385892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98501091f0 0x7f98501095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 -- 172.21.15.103:0/1116385892 >> 172.21.15.103:0/1116385892 conn(0x7f985006e2d0 msgr2=0x7f985006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:22.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 -- 172.21.15.103:0/1116385892 shutdown_connections 2024-09-19T07:47:22.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.418+0000 7f98577cd640 1 -- 172.21.15.103:0/1116385892 wait complete. 2024-09-19T07:47:22.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.419+0000 7f98577cd640 1 Processor -- start 2024-09-19T07:47:22.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.420+0000 7f98577cd640 1 -- start start 2024-09-19T07:47:22.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.420+0000 7f98577cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98501091f0 0x7f985010c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.420+0000 7f98577cd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850109b10 0x7f985010cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.420+0000 7f98577cd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9850116ea0 0x7f985010d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.420+0000 7f98577cd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f985011c9d0 con 0x7f9850116ea0 2024-09-19T07:47:22.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.420+0000 7f98577cd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f985011c850 con 0x7f9850109b10 2024-09-19T07:47:22.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.420+0000 7f98577cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f985011cb50 con 0x7f98501091f0 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98501091f0 0x7f985010c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98501091f0 0x7f985010c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40908/0 (socket says 172.21.15.103:40908) 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 -- 172.21.15.103:0/3805030017 learned_addr learned my addr 172.21.15.103:0/3805030017 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9854d68640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850109b10 0x7f985010cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 -- 172.21.15.103:0/3805030017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850109b10 msgr2=0x7f985010cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850109b10 0x7f985010cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 -- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9850116ea0 msgr2=0x7f985010d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9850116ea0 0x7f985010d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9855569640 1 -- 172.21.15.103:0/3805030017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98501b3530 con 0x7f98501091f0 2024-09-19T07:47:22.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.421+0000 7f9854d68640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850109b10 0x7f985010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:22.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.422+0000 7f9855569640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98501091f0 0x7f985010c5e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f984c00ea20 tx=0x7f984c00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:22.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.422+0000 7f98467fc640 1 -- 172.21.15.103:0/3805030017 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c00cd80 con 0x7f98501091f0 2024-09-19T07:47:22.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.422+0000 7f98467fc640 1 -- 172.21.15.103:0/3805030017 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f984c005500 con 0x7f98501091f0 2024-09-19T07:47:22.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.422+0000 7f98467fc640 1 -- 172.21.15.103:0/3805030017 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c010690 con 0x7f98501091f0 2024-09-19T07:47:22.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.422+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98501b3820 con 0x7f98501091f0 2024-09-19T07:47:22.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.422+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98501b3d60 con 0x7f98501091f0 2024-09-19T07:47:22.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.425+0000 7f9859ba6740 1 Processor -- start 2024-09-19T07:47:22.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.425+0000 7f98467fc640 1 -- 172.21.15.103:0/3805030017 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f984c010830 con 0x7f98501091f0 2024-09-19T07:47:22.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.426+0000 7f9859ba6740 1 -- start start 2024-09-19T07:47:22.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.426+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a14a0 0x5612697a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.426+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a1dc0 0x5612697ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.426+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.426+0000 7f9859ba6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5612697072e0 con 0x5612697a1dc0 2024-09-19T07:47:22.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.426+0000 7f9859ba6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612696aece0 con 0x5612697a14a0 2024-09-19T07:47:22.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.426+0000 7f9859ba6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612697032a0 con 0x5612697ac680 2024-09-19T07:47:22.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9855569640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a1dc0 0x5612697ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9855d6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a14a0 0x5612697a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9855569640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a1dc0 0x5612697ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49540/0 (socket says 172.21.15.103:49540) 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9855569640 1 -- 172.21.15.103:0/3292892074 learned_addr learned my addr 172.21.15.103:0/3292892074 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9855d6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a14a0 0x5612697a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43602/0 (socket says 172.21.15.103:43602) 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9854d68640 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f98467fc640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f983c0a34b0 0x7f983c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9854d68640 1 -- 172.21.15.103:0/3292892074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a14a0 msgr2=0x5612697a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9854d68640 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a14a0 0x5612697a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9854d68640 1 -- 172.21.15.103:0/3292892074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a1dc0 msgr2=0x5612697ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9854d68640 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a1dc0 0x5612697ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9854d68640 1 -- 172.21.15.103:0/3292892074 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561269693440 con 0x5612697ac680 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.427+0000 7f9855d6a640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f983c0a34b0 0x7f983c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.428+0000 7f9854d68640 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697aea70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f985010dda0 tx=0x7f9840032c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245ee0be106ed987 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.428+0000 7f98467fc640 1 -- 172.21.15.103:0/3805030017 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f984c0d0210 con 0x7f98501091f0 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.428+0000 7f983b7fe640 1 -- 172.21.15.103:0/3292892074 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984003f070 con 0x5612697ac680 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.428+0000 7f983b7fe640 1 -- 172.21.15.103:0/3292892074 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9840032e20 con 0x5612697ac680 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.428+0000 7f983b7fe640 1 -- 172.21.15.103:0/3292892074 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840035070 con 0x5612697ac680 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 -- 172.21.15.103:0/3292892074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 msgr2=0x5612697aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697aea70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f985010dda0 tx=0x7f9840032c30 comp rx=0 tx=0).stop 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9855d6a640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f983c0a34b0 0x7f983c0a5970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9848009270 tx=0x7f9848004a40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 -- 172.21.15.103:0/3292892074 shutdown_connections 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697aea70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a1dc0 0x5612697ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 --2- 172.21.15.103:0/3292892074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a14a0 0x5612697a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 -- 172.21.15.103:0/3292892074 >> 172.21.15.103:0/3292892074 conn(0x56126979da30 msgr2=0x56126979de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:22.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 -- 172.21.15.103:0/3292892074 shutdown_connections 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 -- 172.21.15.103:0/3292892074 wait complete. 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 -- start start 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.429+0000 7f9859ba6740 1 Processor -- start 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9859ba6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5612697af250 con 0x5612697a14a0 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9859ba6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612696c2050 con 0x5612697a1dc0 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9859ba6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612697af3d0 con 0x5612697ac680 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9859ba6740 10 client.0 _ll_register_callbacks cb 0x561269702160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9854d68640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855569640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697a9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40922/0 (socket says 172.21.15.103:40922) 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 -- 172.21.15.103:0/2501399420 learned_addr learned my addr 172.21.15.103:0/2501399420 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 -- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 msgr2=0x5612697a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 -- 172.21.15.103:0/2501399420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 msgr2=0x5612697abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 -- 172.21.15.103:0/2501399420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612694783c0 con 0x5612697ac680 2024-09-19T07:47:22.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9854d68640 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9855d6a640 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697a9750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f98480049c0 tx=0x7f9848009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e66dac8abb1e301 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.430+0000 7f9838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9838ff9640 1 -- 172.21.15.103:0/2501399420 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984800ea70 con 0x5612697ac680 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9838ff9640 1 -- 172.21.15.103:0/2501399420 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f984800ec10 con 0x5612697ac680 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9840032e20 con 0x5612697ac680 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9838ff9640 1 -- 172.21.15.103:0/2501399420 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f984800ee10 con 0x5612697ac680 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 10 client.26725 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5612697dcc10 con 0x5612697ac680 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9838ff9640 1 -- 172.21.15.103:0/2501399420 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9840032e20 con 0x5612697ac680 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9838ff9640 1 -- 172.21.15.103:0/2501399420 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f9848042080 con 0x5612697ac680 2024-09-19T07:47:22.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 10 client.26725 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 20 client.26725 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5612697de0d0 0x5612697e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 4 client.26725 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.431+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561269714850 con 0x5612697de0d0 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.432+0000 7f9855569640 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5612697de0d0 0x5612697e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.433+0000 7f9855569640 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5612697de0d0 0x5612697e0590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe95784613eebd8 server_cookie=2569448f2e83a802 in_seq=0 out_seq=0 2024-09-19T07:47:22.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.433+0000 7f9838ff9640 10 client.26725 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:22.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9838ff9640 1 -- 172.21.15.103:0/2501399420 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561269714850 con 0x5612697de0d0 2024-09-19T07:47:22.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9838ff9640 10 client.26725 handle_command_reply: tid=0 2024-09-19T07:47:22.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 client.26725 shutdown 2024-09-19T07:47:22.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 msgr2=0x5612697a9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697a9750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f98480049c0 tx=0x7f9848009500 comp rx=0 tx=0).stop 2024-09-19T07:47:22.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 shutdown_connections 2024-09-19T07:47:22.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5612697de0d0 0x5612697e0590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697ac680 0x5612697a9750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2501399420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 >> 172.21.15.103:0/2501399420 conn(0x56126979da30 msgr2=0x5612697a3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:22.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.434+0000 7f9838ff9640 0 client.26725 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:22.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.435+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 shutdown_connections 2024-09-19T07:47:22.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.435+0000 7f9859ba6740 1 -- 172.21.15.103:0/2501399420 wait complete. 2024-09-19T07:47:22.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.435+0000 7f9859ba6740 20 client.26725 trim_cache size 0 max 16384 2024-09-19T07:47:22.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.466+0000 7f9859ba6740 1 Processor -- start 2024-09-19T07:47:22.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9859ba6740 1 -- start start 2024-09-19T07:47:22.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a1dc0 0x56126979de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9855569640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9855569640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49566/0 (socket says 172.21.15.103:49566) 2024-09-19T07:47:22.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697ac680 0x5612697a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9855569640 1 -- 172.21.15.103:0/903560787 learned_addr learned my addr 172.21.15.103:0/903560787 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:22.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9850074420 con 0x5612697a14a0 2024-09-19T07:47:22.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9850075f60 con 0x5612697ac680 2024-09-19T07:47:22.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.470+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9850074af0 con 0x5612697a1dc0 2024-09-19T07:47:22.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.472+0000 7f9855d6a640 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697ac680 0x5612697a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.473+0000 7f9854d68640 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a1dc0 0x56126979de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.474+0000 7f9855569640 1 -- 172.21.15.103:0/903560787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a1dc0 msgr2=0x56126979de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.474+0000 7f9855569640 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a1dc0 0x56126979de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.474+0000 7f9855569640 1 -- 172.21.15.103:0/903560787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697ac680 msgr2=0x5612697a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.474+0000 7f9855569640 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697ac680 0x5612697a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.474+0000 7f9855569640 1 -- 172.21.15.103:0/903560787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612697a1bf0 con 0x5612697a14a0 2024-09-19T07:47:22.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.475+0000 7f9855569640 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697b84b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f984c0988e0 tx=0x7f984c0d25e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80ef1b8fa06e82d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:22.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.476+0000 7f983affd640 1 -- 172.21.15.103:0/903560787 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c09bfc0 con 0x5612697a14a0 2024-09-19T07:47:22.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.476+0000 7f983affd640 1 -- 172.21.15.103:0/903560787 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f984c09cb80 con 0x5612697a14a0 2024-09-19T07:47:22.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.476+0000 7f983affd640 1 -- 172.21.15.103:0/903560787 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c09bde0 con 0x5612697a14a0 2024-09-19T07:47:22.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.477+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 msgr2=0x5612697b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.477+0000 7f9859ba6740 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697b84b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f984c0988e0 tx=0x7f984c0d25e0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.481+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 shutdown_connections 2024-09-19T07:47:22.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.481+0000 7f9859ba6740 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697ac680 0x5612697a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.481+0000 7f9859ba6740 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a1dc0 0x56126979de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.481+0000 7f9859ba6740 1 --2- 172.21.15.103:0/903560787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697a14a0 0x5612697b84b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.481+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 >> 172.21.15.103:0/903560787 conn(0x56126979da30 msgr2=0x5612697a4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.484+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 shutdown_connections 2024-09-19T07:47:22.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.487+0000 7f9859ba6740 1 -- 172.21.15.103:0/903560787 wait complete. 2024-09-19T07:47:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.488+0000 7f9859ba6740 1 -- start start 2024-09-19T07:47:22.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.488+0000 7f9859ba6740 1 Processor -- start 2024-09-19T07:47:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.489+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a14a0 0x5612697b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.489+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.489+0000 7f9859ba6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697ac680 0x5612697b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9855569640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a14a0 0x5612697b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9855569640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a14a0 0x5612697b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40946/0 (socket says 172.21.15.103:40946) 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9855569640 1 -- 172.21.15.103:0/2139501647 learned_addr learned my addr 172.21.15.103:0/2139501647 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9850075890 con 0x5612697ac680 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98500751c0 con 0x5612697a1dc0 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98501a9330 con 0x5612697a14a0 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9854d68640 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.490+0000 7f9859ba6740 10 client.0 _ll_register_callbacks cb 0x7f9848009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:22.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.492+0000 7f9855d6a640 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697ac680 0x5612697b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.492+0000 7f9855569640 1 -- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 msgr2=0x5612697affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.492+0000 7f9855569640 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.492+0000 7f9855569640 1 -- 172.21.15.103:0/2139501647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697ac680 msgr2=0x5612697b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:22.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.492+0000 7f9855569640 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697ac680 0x5612697b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.492+0000 7f9855569640 1 -- 172.21.15.103:0/2139501647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612697b0d30 con 0x5612697a14a0 2024-09-19T07:47:22.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.493+0000 7f9855569640 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a14a0 0x5612697b4c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f984c0a4bc0 tx=0x7f984c09a990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3caa3f00a7aef2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:22.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.493+0000 7f9838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:22.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.493+0000 7f9838ff9640 1 -- 172.21.15.103:0/2139501647 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c09bfc0 con 0x5612697a14a0 2024-09-19T07:47:22.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.493+0000 7f9838ff9640 1 -- 172.21.15.103:0/2139501647 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f984c0cdb70 con 0x5612697a14a0 2024-09-19T07:47:22.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.494+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f984c09cb80 con 0x5612697a14a0 2024-09-19T07:47:22.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.494+0000 7f9838ff9640 1 -- 172.21.15.103:0/2139501647 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f984c09c480 con 0x5612697a14a0 2024-09-19T07:47:22.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.495+0000 7f9859ba6740 10 client.26731 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:22.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.495+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561269848da0 con 0x5612697a14a0 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.496+0000 7f9838ff9640 1 -- 172.21.15.103:0/2139501647 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f984c0a0830 con 0x5612697a14a0 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.496+0000 7f9838ff9640 1 -- 172.21.15.103:0/2139501647 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f984c0a4c60 con 0x5612697a14a0 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.497+0000 7f9859ba6740 10 client.26731 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.497+0000 7f9859ba6740 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.497+0000 7f9859ba6740 20 client.26731 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.497+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561269849320 0x56126984b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.497+0000 7f9859ba6740 4 client.26731 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.497+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f984c09cb80 con 0x561269849320 2024-09-19T07:47:22.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.497+0000 7f9854d68640 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561269849320 0x56126984b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:22.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.498+0000 7f9854d68640 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561269849320 0x56126984b710 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2397190e9cdc4f server_cookie=cf80807abce7383e in_seq=0 out_seq=0 2024-09-19T07:47:22.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.498+0000 7f9838ff9640 10 client.26731 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:22.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.498+0000 7f9838ff9640 1 -- 172.21.15.103:0/2139501647 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f984c09cb80 con 0x561269849320 2024-09-19T07:47:22.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.498+0000 7f9838ff9640 10 client.26731 handle_command_reply: tid=0 2024-09-19T07:47:22.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.499+0000 7f9859ba6740 1 client.26731 shutdown 2024-09-19T07:47:22.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.499+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a14a0 msgr2=0x5612697b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.500+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a14a0 0x5612697b4c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f984c0a4bc0 tx=0x7f984c09a990 comp rx=0 tx=0).stop 2024-09-19T07:47:22.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.504+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 shutdown_connections 2024-09-19T07:47:22.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.504+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561269849320 0x56126984b710 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.504+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5612697ac680 0x5612697b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.504+0000 7f9838ff9640 0 client.26731 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:22.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.504+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612697a1dc0 0x5612697affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.504+0000 7f9859ba6740 1 --2- 172.21.15.103:0/2139501647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612697a14a0 0x5612697b4c90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.504+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 >> 172.21.15.103:0/2139501647 conn(0x56126979da30 msgr2=0x5612697a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:22.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.505+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 shutdown_connections 2024-09-19T07:47:22.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.505+0000 7f9859ba6740 1 -- 172.21.15.103:0/2139501647 wait complete. 2024-09-19T07:47:22.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.505+0000 7f9859ba6740 20 client.26731 trim_cache size 0 max 16384 2024-09-19T07:47:22.506 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:47:22.506 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:47:22.506 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:47:22.507 INFO:teuthology.orchestra.run.smithi103.stdout: "573814b5-cd93-439e-9553-7c4377eb08a3": { 2024-09-19T07:47:22.507 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:47:22.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "573814b5-cd93-439e-9553-7c4377eb08a3", 2024-09-19T07:47:22.507 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:47:22.507 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:47:22.507 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:47:22.507 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:47:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f983c0a34b0 msgr2=0x7f983c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f983c0a34b0 0x7f983c0a5970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9848009270 tx=0x7f9848004a40 comp rx=0 tx=0).stop 2024-09-19T07:47:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98501091f0 msgr2=0x7f985010c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98501091f0 0x7f985010c5e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f984c00ea20 tx=0x7f984c00ee20 comp rx=0 tx=0).stop 2024-09-19T07:47:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 shutdown_connections 2024-09-19T07:47:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f983c0a34b0 0x7f983c0a5970 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9850116ea0 0x7f985010d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850109b10 0x7f985010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 --2- 172.21.15.103:0/3805030017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98501091f0 0x7f985010c5e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:22.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.507+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 >> 172.21.15.103:0/3805030017 conn(0x7f985006e2d0 msgr2=0x7f9850117420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:22.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.510+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 shutdown_connections 2024-09-19T07:47:22.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:22.510+0000 7f98577cd640 1 -- 172.21.15.103:0/3805030017 wait complete. 2024-09-19T07:47:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:22 smithi134 ceph-mon[35414]: pgmap v290: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 973 op/s 2024-09-19T07:47:23.290 INFO:tasks.cephfs.filesystem:scrub status for tag:573814b5-cd93-439e-9553-7c4377eb08a3 - {'path': '/', 'tag': '573814b5-cd93-439e-9553-7c4377eb08a3', 'options': 'recursive,force'} 2024-09-19T07:47:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:24 smithi174 ceph-mon[35370]: pgmap v291: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 973 op/s 2024-09-19T07:47:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:24 smithi103 ceph-mon[31201]: pgmap v291: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 973 op/s 2024-09-19T07:47:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:24 smithi134 ceph-mon[35414]: pgmap v291: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 973 op/s 2024-09-19T07:47:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 530.0 s, 2566.8 tps, lat 38.604 ms stddev 43.715 2024-09-19T07:47:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:26 smithi174 ceph-mon[35370]: pgmap v292: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 26 MiB/s wr, 967 op/s 2024-09-19T07:47:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:26 smithi103 ceph-mon[31201]: pgmap v292: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 26 MiB/s wr, 967 op/s 2024-09-19T07:47:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:26 smithi134 ceph-mon[35414]: pgmap v292: 97 pgs: 97 active+clean; 3.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 26 MiB/s wr, 967 op/s 2024-09-19T07:47:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:28 smithi174 ceph-mon[35370]: pgmap v293: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 852 op/s 2024-09-19T07:47:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:28 smithi103 ceph-mon[31201]: pgmap v293: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 852 op/s 2024-09-19T07:47:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:28 smithi134 ceph-mon[35414]: pgmap v293: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 852 op/s 2024-09-19T07:47:29.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:47:29.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:47:29.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:47:29.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:47:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:47:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:47:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:47:29.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:47:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:47:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:47:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:47:29.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:47:30.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:30 smithi174 ceph-mon[35370]: pgmap v294: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 747 op/s 2024-09-19T07:47:30.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:30 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:47:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 535.0 s, 2845.8 tps, lat 35.487 ms stddev 41.673 2024-09-19T07:47:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:30 smithi103 ceph-mon[31201]: pgmap v294: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 747 op/s 2024-09-19T07:47:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:30 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:47:30.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:30 smithi134 ceph-mon[35414]: pgmap v294: 97 pgs: 97 active+clean; 4.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 747 op/s 2024-09-19T07:47:30.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:30 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:47:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:32 smithi103 ceph-mon[31201]: pgmap v295: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 1.04k op/s 2024-09-19T07:47:32.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:32 smithi134 ceph-mon[35414]: pgmap v295: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 1.04k op/s 2024-09-19T07:47:32.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:32 smithi174 ceph-mon[35370]: pgmap v295: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 1.04k op/s 2024-09-19T07:47:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:34 smithi103 ceph-mon[31201]: pgmap v296: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 592 op/s 2024-09-19T07:47:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:34 smithi134 ceph-mon[35414]: pgmap v296: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 592 op/s 2024-09-19T07:47:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:34 smithi174 ceph-mon[35370]: pgmap v296: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 592 op/s 2024-09-19T07:47:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 540.0 s, 2510.8 tps, lat 39.302 ms stddev 80.650 2024-09-19T07:47:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:36 smithi103 ceph-mon[31201]: pgmap v297: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2024-09-19T07:47:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:36 smithi134 ceph-mon[35414]: pgmap v297: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2024-09-19T07:47:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:36 smithi174 ceph-mon[35370]: pgmap v297: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 24 MiB/s wr, 1.01k op/s 2024-09-19T07:47:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:38 smithi103 ceph-mon[31201]: pgmap v298: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.80k op/s 2024-09-19T07:47:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:38 smithi134 ceph-mon[35414]: pgmap v298: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.80k op/s 2024-09-19T07:47:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:38 smithi174 ceph-mon[35370]: pgmap v298: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 1.80k op/s 2024-09-19T07:47:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 545.0 s, 2553.6 tps, lat 39.492 ms stddev 58.129 2024-09-19T07:47:40.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:40 smithi103 ceph-mon[31201]: pgmap v299: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.65k op/s 2024-09-19T07:47:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:40 smithi134 ceph-mon[35414]: pgmap v299: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.65k op/s 2024-09-19T07:47:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:40 smithi174 ceph-mon[35370]: pgmap v299: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 33 MiB/s wr, 1.65k op/s 2024-09-19T07:47:42.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:42 smithi103 ceph-mon[31201]: pgmap v300: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 47 MiB/s wr, 2.60k op/s 2024-09-19T07:47:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:42 smithi134 ceph-mon[35414]: pgmap v300: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 47 MiB/s wr, 2.60k op/s 2024-09-19T07:47:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:42 smithi174 ceph-mon[35370]: pgmap v300: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 47 MiB/s wr, 2.60k op/s 2024-09-19T07:47:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:44 smithi103 ceph-mon[31201]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 40 MiB/s wr, 2.31k op/s 2024-09-19T07:47:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:44 smithi134 ceph-mon[35414]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 40 MiB/s wr, 2.31k op/s 2024-09-19T07:47:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:44 smithi174 ceph-mon[35370]: pgmap v301: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 40 MiB/s wr, 2.31k op/s 2024-09-19T07:47:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 550.0 s, 3117.0 tps, lat 32.139 ms stddev 36.567 2024-09-19T07:47:46.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:46 smithi103 ceph-mon[31201]: pgmap v302: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 41 MiB/s wr, 2.41k op/s 2024-09-19T07:47:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:46 smithi134 ceph-mon[35414]: pgmap v302: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 41 MiB/s wr, 2.41k op/s 2024-09-19T07:47:46.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:46 smithi174 ceph-mon[35370]: pgmap v302: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 41 MiB/s wr, 2.41k op/s 2024-09-19T07:47:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:48 smithi103 ceph-mon[31201]: pgmap v303: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 36 MiB/s wr, 2.19k op/s 2024-09-19T07:47:48.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:48 smithi134 ceph-mon[35414]: pgmap v303: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 36 MiB/s wr, 2.19k op/s 2024-09-19T07:47:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:48 smithi174 ceph-mon[35370]: pgmap v303: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 36 MiB/s wr, 2.19k op/s 2024-09-19T07:47:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 555.0 s, 2730.4 tps, lat 36.283 ms stddev 39.699 2024-09-19T07:47:50.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:50 smithi103 ceph-mon[31201]: pgmap v304: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 19 MiB/s wr, 1.26k op/s 2024-09-19T07:47:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:50 smithi134 ceph-mon[35414]: pgmap v304: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 19 MiB/s wr, 1.26k op/s 2024-09-19T07:47:50.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:50 smithi174 ceph-mon[35370]: pgmap v304: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 19 MiB/s wr, 1.26k op/s 2024-09-19T07:47:52.148 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:47:52.148 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T07:47:52.149 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:47:52.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:52 smithi103 ceph-mon[31201]: pgmap v305: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2024-09-19T07:47:52.514 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:52.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:52 smithi134 ceph-mon[35414]: pgmap v305: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2024-09-19T07:47:52.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:52 smithi174 ceph-mon[35370]: pgmap v305: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 24 MiB/s wr, 1.65k op/s 2024-09-19T07:47:53.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.087+0000 7f68df8ee640 1 -- 172.21.15.103:0/2145440378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8100e20 msgr2=0x7f68d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:53.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.087+0000 7f68df8ee640 1 --2- 172.21.15.103:0/2145440378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8100e20 0x7f68d8101200 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009900 tx=0x7f68d402a480 comp rx=0 tx=0).stop 2024-09-19T07:47:53.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.088+0000 7f68df8ee640 1 -- 172.21.15.103:0/2145440378 shutdown_connections 2024-09-19T07:47:53.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.088+0000 7f68df8ee640 1 --2- 172.21.15.103:0/2145440378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102ef0 0x7f68d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.088+0000 7f68df8ee640 1 --2- 172.21.15.103:0/2145440378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d8102570 0x7f68d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.088+0000 7f68df8ee640 1 --2- 172.21.15.103:0/2145440378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8100e20 0x7f68d8101200 secure :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009900 tx=0x7f68d402a480 comp rx=0 tx=0).stop 2024-09-19T07:47:53.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.088+0000 7f68df8ee640 1 -- 172.21.15.103:0/2145440378 >> 172.21.15.103:0/2145440378 conn(0x7f68d80fc7d0 msgr2=0x7f68d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:53.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.089+0000 7f68df8ee640 1 -- 172.21.15.103:0/2145440378 shutdown_connections 2024-09-19T07:47:53.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.089+0000 7f68df8ee640 1 -- 172.21.15.103:0/2145440378 wait complete. 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.090+0000 7f68df8ee640 1 Processor -- start 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68df8ee640 1 -- start start 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68df8ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102570 0x7f68d81a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68df8ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8102ef0 0x7f68d81a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68df8ee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d819a6a0 0x7f68d819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68df8ee640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8112070 con 0x7f68d8102ef0 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68df8ee640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8111ef0 con 0x7f68d819a6a0 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68df8ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81121f0 con 0x7f68d8102570 2024-09-19T07:47:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68dce89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8102ef0 0x7f68d81a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68dd68a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102570 0x7f68d81a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68dd68a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102570 0x7f68d81a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46144/0 (socket says 172.21.15.103:46144) 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68dd68a640 1 -- 172.21.15.103:0/1052716704 learned_addr learned my addr 172.21.15.103:0/1052716704 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.091+0000 7f68dde8b640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d819a6a0 0x7f68d819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68dd68a640 1 -- 172.21.15.103:0/1052716704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d819a6a0 msgr2=0x7f68d819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68dd68a640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d819a6a0 0x7f68d819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68dd68a640 1 -- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8102ef0 msgr2=0x7f68d81a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68dd68a640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8102ef0 0x7f68d81a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68dd68a640 1 -- 172.21.15.103:0/1052716704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d819b250 con 0x7f68d8102570 2024-09-19T07:47:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68dd68a640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102570 0x7f68d81a05d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68d4004770 tx=0x7f68d4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68ce7fc640 1 -- 172.21.15.103:0/1052716704 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d403f020 con 0x7f68d8102570 2024-09-19T07:47:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d819b4e0 con 0x7f68d8102570 2024-09-19T07:47:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68ce7fc640 1 -- 172.21.15.103:0/1052716704 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d4031c30 con 0x7f68d8102570 2024-09-19T07:47:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68ce7fc640 1 -- 172.21.15.103:0/1052716704 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4035070 con 0x7f68d8102570 2024-09-19T07:47:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.092+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d81a7660 con 0x7f68d8102570 2024-09-19T07:47:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.093+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d81021b0 con 0x7f68d8102570 2024-09-19T07:47:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.094+0000 7f68ce7fc640 1 -- 172.21.15.103:0/1052716704 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f68d4042020 con 0x7f68d8102570 2024-09-19T07:47:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.095+0000 7f68ce7fc640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f68c40a3430 0x7f68c40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.095+0000 7f68ce7fc640 1 -- 172.21.15.103:0/1052716704 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f68d40f0c80 con 0x7f68d8102570 2024-09-19T07:47:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.095+0000 7f68dce89640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f68c40a3430 0x7f68c40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.096+0000 7f68dce89640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f68c40a3430 0x7f68c40a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68d819bef0 tx=0x7f68c800a2d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:53.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.098+0000 7f68ce7fc640 1 -- 172.21.15.103:0/1052716704 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f68d4049020 con 0x7f68d8102570 2024-09-19T07:47:53.291 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:47:53.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.433+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68d8106df0 con 0x7f68d8102570 2024-09-19T07:47:53.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.435+0000 7f68ce7fc640 1 -- 172.21.15.103:0/1052716704 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11991 (secure 0 0 0) 0x7f68d4031de0 con 0x7f68d8102570 2024-09-19T07:47:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:47:53.435 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":43,"btime":"2024-09-19T07:46:44:994665+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:46:44.857313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:47:53.436 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 43 2024-09-19T07:47:53.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f68c40a3430 msgr2=0x7f68c40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f68c40a3430 0x7f68c40a58f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68d819bef0 tx=0x7f68c800a2d0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102570 msgr2=0x7f68d81a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102570 0x7f68d81a05d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68d4004770 tx=0x7f68d4003c10 comp rx=0 tx=0).stop 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 shutdown_connections 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f68c40a3430 0x7f68c40a58f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68d819a6a0 0x7f68d819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68d8102ef0 0x7f68d81a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 --2- 172.21.15.103:0/1052716704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8102570 0x7f68d81a05d0 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68d4004770 tx=0x7f68d4003c10 comp rx=0 tx=0).stop 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.445+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 >> 172.21.15.103:0/1052716704 conn(0x7f68d80fc7d0 msgr2=0x7f68d8108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:53.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.447+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 shutdown_connections 2024-09-19T07:47:53.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:53.447+0000 7f68df8ee640 1 -- 172.21.15.103:0/1052716704 wait complete. 2024-09-19T07:47:54.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:54 smithi103 ceph-mon[31201]: pgmap v306: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-09-19T07:47:54.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:54 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1052716704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:47:54.279 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T07:47:54.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T07:47:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:54 smithi134 ceph-mon[35414]: pgmap v306: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-09-19T07:47:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:54 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1052716704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:47:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:54 smithi174 ceph-mon[35370]: pgmap v306: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 690 op/s 2024-09-19T07:47:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:54 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1052716704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:47:54.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.957+0000 7fc9635cc640 1 -- 172.21.15.103:0/3589717119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109830 msgr2=0x7fc95c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.957+0000 7fc9635cc640 1 --2- 172.21.15.103:0/3589717119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109830 0x7fc95c109c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009860 tx=0x7fc94c02a380 comp rx=0 tx=0).stop 2024-09-19T07:47:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.958+0000 7fc9635cc640 1 -- 172.21.15.103:0/3589717119 shutdown_connections 2024-09-19T07:47:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.958+0000 7fc9635cc640 1 --2- 172.21.15.103:0/3589717119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 0x7fc95c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.958+0000 7fc9635cc640 1 --2- 172.21.15.103:0/3589717119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109830 0x7fc95c109c90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.958+0000 7fc9635cc640 1 --2- 172.21.15.103:0/3589717119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc95c109450 0x7fc95c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.958+0000 7fc9635cc640 1 -- 172.21.15.103:0/3589717119 >> 172.21.15.103:0/3589717119 conn(0x7fc95c06e140 msgr2=0x7fc95c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.964+0000 7fc9635cc640 1 -- 172.21.15.103:0/3589717119 shutdown_connections 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.964+0000 7fc9635cc640 1 -- 172.21.15.103:0/3589717119 wait complete. 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 Processor -- start 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 -- start start 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109450 0x7fc95c118730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc95c109830 0x7fc95c1117c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 0x7fc95c111d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c120da0 con 0x7fc95c109830 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c120c20 con 0x7fc95c109450 2024-09-19T07:47:54.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc9635cc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c120f20 con 0x7fc95c11b140 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc961b69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 0x7fc95c111d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc961368640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109450 0x7fc95c118730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc961b69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 0x7fc95c111d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46164/0 (socket says 172.21.15.103:46164) 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc961368640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109450 0x7fc95c118730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42546/0 (socket says 172.21.15.103:42546) 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc961b69640 1 -- 172.21.15.103:0/432366374 learned_addr learned my addr 172.21.15.103:0/432366374 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.965+0000 7fc960b67640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc95c109830 0x7fc95c1117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.966+0000 7fc961b69640 1 -- 172.21.15.103:0/432366374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109450 msgr2=0x7fc95c118730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.966+0000 7fc961b69640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109450 0x7fc95c118730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.966+0000 7fc961b69640 1 -- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc95c109830 msgr2=0x7fc95c1117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.966+0000 7fc961b69640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc95c109830 0x7fc95c1117c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.966+0000 7fc961b69640 1 -- 172.21.15.103:0/432366374 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95c1122f0 con 0x7fc95c11b140 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.966+0000 7fc961b69640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 0x7fc95c111d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc95400ebc0 tx=0x7fc95400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.967+0000 7fc9527fc640 1 -- 172.21.15.103:0/432366374 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95400cdf0 con 0x7fc95c11b140 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.967+0000 7fc9527fc640 1 -- 172.21.15.103:0/432366374 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc954005020 con 0x7fc95c11b140 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.967+0000 7fc9527fc640 1 -- 172.21.15.103:0/432366374 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954010670 con 0x7fc95c11b140 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.967+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95c1125e0 con 0x7fc95c11b140 2024-09-19T07:47:54.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.967+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95c116550 con 0x7fc95c11b140 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.971+0000 7fc9659a5740 1 Processor -- start 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.972+0000 7fc9527fc640 1 -- 172.21.15.103:0/432366374 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc954010810 con 0x7fc95c11b140 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.973+0000 7fc9527fc640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9480a34b0 0x7fc9480a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.973+0000 7fc9527fc640 1 -- 172.21.15.103:0/432366374 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fc9540d0300 con 0x7fc95c11b140 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.973+0000 7fc960b67640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9480a34b0 0x7fc9480a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc9659a5740 1 -- start start 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224f4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224f4dc0 0x555c224ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224ff680 0x555c22501a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc9659a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c2245a2e0 con 0x555c224ff680 2024-09-19T07:47:54.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc9659a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555c22401ce0 con 0x555c224f44a0 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc9659a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c224562a0 con 0x555c224f4dc0 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc961368640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc961368640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42556/0 (socket says 172.21.15.103:42556) 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc961368640 1 -- 172.21.15.103:0/2126090555 learned_addr learned my addr 172.21.15.103:0/2126090555 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc960b67640 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224f4dc0 0x555c224ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.974+0000 7fc960b67640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9480a34b0 0x7fc9480a5970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02a970 tx=0x7fc94c030d00 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.975+0000 7fc960b67640 1 -- 172.21.15.103:0/2126090555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 msgr2=0x555c224f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.975+0000 7fc960b67640 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.975+0000 7fc960b67640 1 -- 172.21.15.103:0/2126090555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224ff680 msgr2=0x555c22501a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.975+0000 7fc960b67640 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224ff680 0x555c22501a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.975+0000 7fc960b67640 1 -- 172.21.15.103:0/2126090555 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c223e6440 con 0x555c224f4dc0 2024-09-19T07:47:54.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.976+0000 7fc960b67640 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224f4dc0 0x555c224ff140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc94c035040 tx=0x7fc94c043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdca646aede697cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:54.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.976+0000 7fc9477fe640 1 -- 172.21.15.103:0/2126090555 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c045040 con 0x555c224f4dc0 2024-09-19T07:47:54.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.976+0000 7fc9477fe640 1 -- 172.21.15.103:0/2126090555 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c005040 con 0x555c224f4dc0 2024-09-19T07:47:54.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.977+0000 7fc9659a5740 1 -- 172.21.15.103:0/2126090555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224f4dc0 msgr2=0x555c224ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.977+0000 7fc9659a5740 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224f4dc0 0x555c224ff140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc94c035040 tx=0x7fc94c043000 comp rx=0 tx=0).stop 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.978+0000 7fc9659a5740 1 -- 172.21.15.103:0/2126090555 shutdown_connections 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.978+0000 7fc9659a5740 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224ff680 0x555c22501a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.978+0000 7fc9659a5740 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224f4dc0 0x555c224ff140 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc94c035040 tx=0x7fc94c043000 comp rx=0 tx=0).stop 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.978+0000 7fc9659a5740 1 --2- 172.21.15.103:0/2126090555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.978+0000 7fc9659a5740 1 -- 172.21.15.103:0/2126090555 >> 172.21.15.103:0/2126090555 conn(0x555c224f0a30 msgr2=0x555c224f0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.978+0000 7fc9659a5740 1 -- 172.21.15.103:0/2126090555 shutdown_connections 2024-09-19T07:47:54.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.978+0000 7fc9659a5740 1 -- 172.21.15.103:0/2126090555 wait complete. 2024-09-19T07:47:54.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 -- start start 2024-09-19T07:47:54.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 Processor -- start 2024-09-19T07:47:54.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224fc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224ff680 0x555c224fca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224fcf80 0x555c2250b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c225022a0 con 0x555c224fcf80 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555c22415050 con 0x555c224f44a0 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c22502420 con 0x555c224ff680 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.979+0000 7fc9659a5740 10 client.0 _ll_register_callbacks cb 0x555c22455160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.980+0000 7fc960b67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224ff680 0x555c224fca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.980+0000 7fc960b67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224ff680 0x555c224fca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46182/0 (socket says 172.21.15.103:46182) 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.980+0000 7fc960b67640 1 -- 172.21.15.103:0/3349593555 learned_addr learned my addr 172.21.15.103:0/3349593555 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:54.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.980+0000 7fc961b69640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224fcf80 0x555c2250b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.980+0000 7fc961368640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.980+0000 7fc960b67640 1 -- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 msgr2=0x555c224fc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.981+0000 7fc960b67640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224fc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.981+0000 7fc960b67640 1 -- 172.21.15.103:0/3349593555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224fcf80 msgr2=0x555c2250b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.981+0000 7fc960b67640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224fcf80 0x555c2250b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.981+0000 7fc960b67640 1 -- 172.21.15.103:0/3349593555 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c221cb3c0 con 0x555c224ff680 2024-09-19T07:47:54.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.981+0000 7fc961b69640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224fcf80 0x555c2250b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:47:54.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.982+0000 7fc960b67640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224ff680 0x555c224fca40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02a8a0 tx=0x7fc94c030470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7cd6c8c5ff061e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:54.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.982+0000 7fc944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:54.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.983+0000 7fc944ff9640 1 -- 172.21.15.103:0/3349593555 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0051f0 con 0x555c224ff680 2024-09-19T07:47:54.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.983+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94c005040 con 0x555c224ff680 2024-09-19T07:47:54.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.983+0000 7fc944ff9640 1 -- 172.21.15.103:0/3349593555 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fc94c005390 con 0x555c224ff680 2024-09-19T07:47:54.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.984+0000 7fc944ff9640 1 -- 172.21.15.103:0/3349593555 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc94c05b050 con 0x555c224ff680 2024-09-19T07:47:54.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.984+0000 7fc9659a5740 10 client.26749 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:54.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.984+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c225022a0 con 0x555c224ff680 2024-09-19T07:47:54.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.984+0000 7fc944ff9640 1 -- 172.21.15.103:0/3349593555 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c005040 con 0x555c224ff680 2024-09-19T07:47:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.985+0000 7fc944ff9640 1 -- 172.21.15.103:0/3349593555 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fc94c03d070 con 0x555c224ff680 2024-09-19T07:47:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.986+0000 7fc9659a5740 10 client.26749 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.986+0000 7fc9659a5740 10 client.26749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.986+0000 7fc9659a5740 20 client.26749 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.986+0000 7fc9659a5740 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2250f8b0 0x555c22511cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:54.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.986+0000 7fc9659a5740 4 client.26749 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.986+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c22467850 con 0x555c2250f8b0 2024-09-19T07:47:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.986+0000 7fc961368640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2250f8b0 0x555c22511cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.987+0000 7fc961368640 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2250f8b0 0x555c22511cd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a501b18710581c78 server_cookie=c231385a5213afb2 in_seq=0 out_seq=0 2024-09-19T07:47:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.987+0000 7fc944ff9640 10 client.26749 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc944ff9640 1 -- 172.21.15.103:0/3349593555 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c22467850 con 0x555c2250f8b0 2024-09-19T07:47:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc944ff9640 10 client.26749 handle_command_reply: tid=0 2024-09-19T07:47:54.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 client.26749 shutdown 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224ff680 msgr2=0x555c224fca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224ff680 0x555c224fca40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02a8a0 tx=0x7fc94c030470 comp rx=0 tx=0).stop 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 shutdown_connections 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2250f8b0 0x555c22511cd0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224fcf80 0x555c2250b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224ff680 0x555c224fca40 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02a8a0 tx=0x7fc94c030470 comp rx=0 tx=0).stop 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 --2- 172.21.15.103:0/3349593555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c224fc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 >> 172.21.15.103:0/3349593555 conn(0x555c224f0a30 msgr2=0x555c224f6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc944ff9640 0 client.26749 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 shutdown_connections 2024-09-19T07:47:54.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.988+0000 7fc9659a5740 1 -- 172.21.15.103:0/3349593555 wait complete. 2024-09-19T07:47:54.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:54.989+0000 7fc9659a5740 20 client.26749 trim_cache size 0 max 16384 2024-09-19T07:47:55.005 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:55.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.018+0000 7fc9659a5740 1 Processor -- start 2024-09-19T07:47:55.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.018+0000 7fc9659a5740 1 -- start start 2024-09-19T07:47:55.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.019+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c224f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.019+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c2250b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.019+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c2250ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.019+0000 7fc961368640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c224f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.019+0000 7fc961368640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c224f69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58612/0 (socket says 172.21.15.103:58612) 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.020+0000 7fc960b67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c2250b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.020+0000 7fc960b67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c2250b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42570/0 (socket says 172.21.15.103:42570) 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.020+0000 7fc961368640 1 -- 172.21.15.103:0/699565271 learned_addr learned my addr 172.21.15.103:0/699565271 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.020+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1c5000 con 0x555c224f0a30 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.020+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1c6b40 con 0x555c224f44a0 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.020+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1c56d0 con 0x555c224fcf80 2024-09-19T07:47:55.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.020+0000 7fc961b69640 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c2250ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.021+0000 7fc961b69640 1 -- 172.21.15.103:0/699565271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 msgr2=0x555c2250b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.021+0000 7fc961b69640 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c2250b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.021+0000 7fc961b69640 1 -- 172.21.15.103:0/699565271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 msgr2=0x555c224f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.021+0000 7fc961b69640 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c224f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.021+0000 7fc961b69640 1 -- 172.21.15.103:0/699565271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c2250bfd0 con 0x555c224fcf80 2024-09-19T07:47:55.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.021+0000 7fc961b69640 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c2250ba90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc9540cd9e0 tx=0x7fc954098940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8801bf68b57624c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.021+0000 7fc946ffd640 1 -- 172.21.15.103:0/699565271 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9540ba070 con 0x555c224fcf80 2024-09-19T07:47:55.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.022+0000 7fc946ffd640 1 -- 172.21.15.103:0/699565271 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9540a0920 con 0x555c224fcf80 2024-09-19T07:47:55.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.022+0000 7fc946ffd640 1 -- 172.21.15.103:0/699565271 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954098d70 con 0x555c224fcf80 2024-09-19T07:47:55.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 msgr2=0x555c2250ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c2250ba90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc9540cd9e0 tx=0x7fc954098940 comp rx=0 tx=0).stop 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 shutdown_connections 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c2250ba90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c2250b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 --2- 172.21.15.103:0/699565271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c224f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 >> 172.21.15.103:0/699565271 conn(0x555c224f7e50 msgr2=0x555c224fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 shutdown_connections 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 -- 172.21.15.103:0/699565271 wait complete. 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 -- start start 2024-09-19T07:47:55.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.023+0000 7fc9659a5740 1 Processor -- start 2024-09-19T07:47:55.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.024+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c22503580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.024+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c22503ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.024+0000 7fc9659a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c22504000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc960b67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c22503ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc960b67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c22503ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42574/0 (socket says 172.21.15.103:42574) 2024-09-19T07:47:55.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc960b67640 1 -- 172.21.15.103:0/980925021 learned_addr learned my addr 172.21.15.103:0/980925021 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:55.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1c6470 con 0x555c224f0a30 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc961b69640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c22504000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1c5da0 con 0x555c224f44a0 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc95c1c3000 con 0x555c224fcf80 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.025+0000 7fc9659a5740 10 client.0 _ll_register_callbacks cb 0x7fc9580097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961368640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c22503580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961b69640 1 -- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 msgr2=0x555c22503ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961b69640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c22503ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961b69640 1 -- 172.21.15.103:0/980925021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 msgr2=0x555c22503580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961b69640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c22503580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961b69640 1 -- 172.21.15.103:0/980925021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c225796a0 con 0x555c224fcf80 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc960b67640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c22503ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:55.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961368640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c22503580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:47:55.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc961b69640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c22504000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x555c22415050 tx=0x7fc95409aa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8af2ec4e79614c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.026+0000 7fc944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:47:55.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.027+0000 7fc944ff9640 1 -- 172.21.15.103:0/980925021 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9540ba040 con 0x555c224fcf80 2024-09-19T07:47:55.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.027+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9540a0920 con 0x555c224fcf80 2024-09-19T07:47:55.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.027+0000 7fc944ff9640 1 -- 172.21.15.103:0/980925021 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fc9540a4d50 con 0x555c224fcf80 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.027+0000 7fc944ff9640 1 -- 172.21.15.103:0/980925021 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc9540cde60 con 0x555c224fcf80 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 10 client.26755 fetch_fsmap learned FSMap version 43 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555c2257a970 con 0x555c224fcf80 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc944ff9640 1 -- 172.21.15.103:0/980925021 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9540a4f90 con 0x555c224fcf80 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc944ff9640 1 -- 172.21.15.103:0/980925021 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fc9540982c0 con 0x555c224fcf80 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 10 client.26755 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 20 client.26755 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2257afd0 0x555c2257d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 4 client.26755 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc9540a0920 con 0x555c2257afd0 2024-09-19T07:47:55.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.028+0000 7fc961368640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2257afd0 0x555c2257d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.029+0000 7fc961368640 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2257afd0 0x555c2257d3c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be6478ae72d2d73 server_cookie=ac3d2f6418f29cc5 in_seq=0 out_seq=0 2024-09-19T07:47:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.029+0000 7fc944ff9640 10 client.26755 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.029+0000 7fc944ff9640 1 -- 172.21.15.103:0/980925021 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc9540a0920 con 0x555c2257afd0 2024-09-19T07:47:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.029+0000 7fc944ff9640 10 client.26755 handle_command_reply: tid=0 2024-09-19T07:47:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.029+0000 7fc9659a5740 1 client.26755 shutdown 2024-09-19T07:47:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.030+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 msgr2=0x555c22504000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.030+0000 7fc9659a5740 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c22504000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x555c22415050 tx=0x7fc95409aa80 comp rx=0 tx=0).stop 2024-09-19T07:47:55.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 shutdown_connections 2024-09-19T07:47:55.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc9659a5740 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x555c2257afd0 0x555c2257d3c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc9659a5740 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c224fcf80 0x555c22504000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc944ff9640 0 client.26755 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:55.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc9659a5740 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c224f44a0 0x555c22503ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc9659a5740 1 --2- 172.21.15.103:0/980925021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c224f0a30 0x555c22503580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 >> 172.21.15.103:0/980925021 conn(0x555c224f7e50 msgr2=0x555c224fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:55.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.031+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 shutdown_connections 2024-09-19T07:47:55.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.032+0000 7fc9659a5740 1 -- 172.21.15.103:0/980925021 wait complete. 2024-09-19T07:47:55.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.032+0000 7fc9659a5740 20 client.26755 trim_cache size 0 max 16384 2024-09-19T07:47:55.032 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:47:55.033 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:47:55.033 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:47:55.033 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:47:55.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9480a34b0 msgr2=0x7fc9480a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9480a34b0 0x7fc9480a5970 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc94c02a970 tx=0x7fc94c030d00 comp rx=0 tx=0).stop 2024-09-19T07:47:55.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 msgr2=0x7fc95c111d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 0x7fc95c111d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc95400ebc0 tx=0x7fc95400c2d0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 shutdown_connections 2024-09-19T07:47:55.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc9480a34b0 0x7fc9480a5970 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c11b140 0x7fc95c111d00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc95c109830 0x7fc95c1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 --2- 172.21.15.103:0/432366374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc95c109450 0x7fc95c118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 >> 172.21.15.103:0/432366374 conn(0x7fc95c06e140 msgr2=0x7fc95c11c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:55.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.034+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 shutdown_connections 2024-09-19T07:47:55.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.035+0000 7fc9635cc640 1 -- 172.21.15.103:0/432366374 wait complete. 2024-09-19T07:47:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 560.0 s, 2233.0 tps, lat 45.273 ms stddev 52.601 2024-09-19T07:47:55.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 -- 172.21.15.103:0/2828063400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100430 msgr2=0x7f4398100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 --2- 172.21.15.103:0/2828063400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100430 0x7f4398100810 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f438c009900 tx=0x7f438c02a410 comp rx=0 tx=0).stop 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 -- 172.21.15.103:0/2828063400 shutdown_connections 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 --2- 172.21.15.103:0/2828063400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4398105560 0x7f439810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 --2- 172.21.15.103:0/2828063400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4398100de0 0x7f4398104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 --2- 172.21.15.103:0/2828063400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100430 0x7f4398100810 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 -- 172.21.15.103:0/2828063400 >> 172.21.15.103:0/2828063400 conn(0x7f4398077230 msgr2=0x7f43980fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 -- 172.21.15.103:0/2828063400 shutdown_connections 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.863+0000 7f439ec2f640 1 -- 172.21.15.103:0/2828063400 wait complete. 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 Processor -- start 2024-09-19T07:47:55.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 -- start start 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4398100430 0x7f43981a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100de0 0x7f43981a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4398105560 0x7f439819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f439810fd40 con 0x7f4398100de0 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f439810fbc0 con 0x7f4398105560 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439ec2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f439810fec0 con 0x7f4398100430 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100de0 0x7f43981a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439c9cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4398100430 0x7f43981a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100de0 0x7f43981a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58628/0 (socket says 172.21.15.103:58628) 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439c9cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4398100430 0x7f43981a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46218/0 (socket says 172.21.15.103:46218) 2024-09-19T07:47:55.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 -- 172.21.15.103:0/501056451 learned_addr learned my addr 172.21.15.103:0/501056451 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f439d1cc640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4398105560 0x7f439819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 -- 172.21.15.103:0/501056451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4398100430 msgr2=0x7f43981a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4398100430 0x7f43981a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 -- 172.21.15.103:0/501056451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4398105560 msgr2=0x7f439819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4398105560 0x7f439819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.864+0000 7f4397fff640 1 -- 172.21.15.103:0/501056451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439819f050 con 0x7f4398100de0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.865+0000 7f4397fff640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100de0 0x7f43981a4ca0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f439000eb90 tx=0x7f439000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.865+0000 7f4395ffb640 1 -- 172.21.15.103:0/501056451 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390004d20 con 0x7f4398100de0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.865+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439819f340 con 0x7f4398100de0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.865+0000 7f4395ffb640 1 -- 172.21.15.103:0/501056451 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f439000ee90 con 0x7f4398100de0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.865+0000 7f4395ffb640 1 -- 172.21.15.103:0/501056451 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390010830 con 0x7f4398100de0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.865+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43981ab630 con 0x7f4398100de0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.869+0000 7f4395ffb640 1 -- 172.21.15.103:0/501056451 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4390005020 con 0x7f4398100de0 2024-09-19T07:47:55.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.869+0000 7f43a1008740 1 Processor -- start 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.869+0000 7f43a1008740 1 -- start start 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f43a1008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c096760 0x56224c096b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f43a1008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c0a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f43a1008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f43a1008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56224bffc2a0 con 0x56224c097080 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f43a1008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56224bfa3b10 con 0x56224c096760 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f43a1008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56224bff8260 con 0x56224c0a1940 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f4395ffb640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f43800a33b0 0x7f43800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439c9cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c0a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c096760 0x56224c096b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439c9cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c0a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58642/0 (socket says 172.21.15.103:58642) 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439c9cb640 1 -- 172.21.15.103:0/3319869919 learned_addr learned my addr 172.21.15.103:0/3319869919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:55.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c096760 0x56224c096b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42600/0 (socket says 172.21.15.103:42600) 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f4395ffb640 1 -- 172.21.15.103:0/501056451 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f43900d0370 con 0x7f4398100de0 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f43800a33b0 0x7f43800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f4397fff640 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 -- 172.21.15.103:0/3319869919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 msgr2=0x56224c0a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 -- 172.21.15.103:0/3319869919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 msgr2=0x56224c0a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c0a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.870+0000 7f439d1cc640 1 -- 172.21.15.103:0/3319869919 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56224bd6db90 con 0x56224c096760 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.871+0000 7f439d1cc640 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c096760 0x56224c096b40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f438800e550 tx=0x7f438800ea10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13cae06889b087e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.871+0000 7f439d1cc640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f43800a33b0 0x7f43800a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4388009000 tx=0x7f438801e000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.871+0000 7f437effd640 1 -- 172.21.15.103:0/3319869919 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438801c7e0 con 0x56224c096760 2024-09-19T07:47:55.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.871+0000 7f437effd640 1 -- 172.21.15.103:0/3319869919 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4388020070 con 0x56224c096760 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.871+0000 7f437effd640 1 -- 172.21.15.103:0/3319869919 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388025b10 con 0x56224c096760 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 -- 172.21.15.103:0/3319869919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c096760 msgr2=0x56224c096b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c096760 0x56224c096b40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f438800e550 tx=0x7f438800ea10 comp rx=0 tx=0).stop 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 -- 172.21.15.103:0/3319869919 shutdown_connections 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c0a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 --2- 172.21.15.103:0/3319869919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c096760 0x56224c096b40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 -- 172.21.15.103:0/3319869919 >> 172.21.15.103:0/3319869919 conn(0x56224c092cf0 msgr2=0x56224c0930d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 -- 172.21.15.103:0/3319869919 shutdown_connections 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.872+0000 7f43a1008740 1 -- 172.21.15.103:0/3319869919 wait complete. 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 -- start start 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 Processor -- start 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c09e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c09ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c09f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56224c0a4510 con 0x56224c097080 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56224bf846e0 con 0x56224c0a1940 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56224c0a4690 con 0x56224c096760 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f43a1008740 10 client.0 _ll_register_callbacks cb 0x56224bfa8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.873+0000 7f439d1cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c09f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c09ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c09ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58646/0 (socket says 172.21.15.103:58646) 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f439d1cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c09f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42612/0 (socket says 172.21.15.103:42612) 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 -- 172.21.15.103:0/110067345 learned_addr learned my addr 172.21.15.103:0/110067345 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 -- 172.21.15.103:0/110067345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 msgr2=0x56224c09e840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:47:55.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c09e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 -- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 msgr2=0x56224c09f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c09f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 -- 172.21.15.103:0/110067345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56224c0ada30 con 0x56224c097080 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f439d1cc640 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c09f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4377fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.874+0000 7f4397fff640 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c09ed80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f43900049d0 tx=0x7f4390009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadbee130e6a5df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.876+0000 7f4377fff640 1 -- 172.21.15.103:0/110067345 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900b4070 con 0x56224c097080 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.876+0000 7f4377fff640 1 -- 172.21.15.103:0/110067345 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f4390098cf0 con 0x56224c097080 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.876+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4388020070 con 0x56224c097080 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.876+0000 7f4377fff640 1 -- 172.21.15.103:0/110067345 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f439009c640 con 0x56224c097080 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.876+0000 7f43a1008740 10 client.17379 fetch_fsmap learned FSMap version 44 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.876+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56224c0b03c0 con 0x56224c097080 2024-09-19T07:47:55.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f4377fff640 1 -- 172.21.15.103:0/110067345 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4388020070 con 0x56224c097080 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f4377fff640 1 -- 172.21.15.103:0/110067345 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f43900a4e20 con 0x56224c097080 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f43a1008740 10 client.17379 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f43a1008740 10 client.17379 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f43a1008740 20 client.17379 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f43a1008740 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c0b1880 0x56224c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f43a1008740 4 client.17379 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.877+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56224c0099d0 con 0x56224c0b1880 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.878+0000 7f439c9cb640 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c0b1880 0x56224c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.878+0000 7f439c9cb640 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c0b1880 0x56224c0b3c90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1df4bf72ea8544 server_cookie=2a0d25a0e21205d in_seq=0 out_seq=0 2024-09-19T07:47:55.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.878+0000 7f4377fff640 10 client.17379 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f4377fff640 1 -- 172.21.15.103:0/110067345 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56224c0099d0 con 0x56224c0b1880 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f4377fff640 10 client.17379 handle_command_reply: tid=0 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 client.17379 shutdown 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 msgr2=0x56224c09ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c09ed80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f43900049d0 tx=0x7f4390009aa0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 shutdown_connections 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c0b1880 0x56224c0b3c90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f4377fff640 0 client.17379 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c09f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c097080 0x56224c09ed80 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 --2- 172.21.15.103:0/110067345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c09e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 >> 172.21.15.103:0/110067345 conn(0x56224c092cf0 msgr2=0x56224c098a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:55.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 shutdown_connections 2024-09-19T07:47:55.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.879+0000 7f43a1008740 1 -- 172.21.15.103:0/110067345 wait complete. 2024-09-19T07:47:55.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.880+0000 7f43a1008740 20 client.17379 trim_cache size 0 max 16384 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 Processor -- start 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 -- start start 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c096760 0x56224c0991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c097080 0x56224c091d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43981b3f10 con 0x56224c096760 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43981b5a50 con 0x56224c097080 2024-09-19T07:47:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f43a1008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43981b45e0 con 0x56224c0a1940 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f439c9cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c096760 0x56224c0991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f4397fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c097080 0x56224c091d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f4397fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c097080 0x56224c091d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42614/0 (socket says 172.21.15.103:42614) 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f4397fff640 1 -- 172.21.15.103:0/2513986599 learned_addr learned my addr 172.21.15.103:0/2513986599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.906+0000 7f439c9cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c096760 0x56224c0991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58656/0 (socket says 172.21.15.103:58656) 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f439d1cc640 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f4397fff640 1 -- 172.21.15.103:0/2513986599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 msgr2=0x56224c0b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f4397fff640 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f4397fff640 1 -- 172.21.15.103:0/2513986599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c096760 msgr2=0x56224c0991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f4397fff640 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c096760 0x56224c0991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f4397fff640 1 -- 172.21.15.103:0/2513986599 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56224c08e260 con 0x56224c097080 2024-09-19T07:47:55.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f439c9cb640 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c096760 0x56224c0991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:47:55.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f4397fff640 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c097080 0x56224c091d60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43900a4d80 tx=0x7f439009ab50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2cc3dcfd2ae64c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f437e7fc640 1 -- 172.21.15.103:0/2513986599 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390083070 con 0x56224c097080 2024-09-19T07:47:55.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f437e7fc640 1 -- 172.21.15.103:0/2513986599 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43900a52b0 con 0x56224c097080 2024-09-19T07:47:55.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.907+0000 7f437e7fc640 1 -- 172.21.15.103:0/2513986599 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900b4070 con 0x56224c097080 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.908+0000 7f43a1008740 1 -- 172.21.15.103:0/2513986599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c097080 msgr2=0x56224c091d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.908+0000 7f43a1008740 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c097080 0x56224c091d60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43900a4d80 tx=0x7f439009ab50 comp rx=0 tx=0).stop 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.908+0000 7f43a1008740 1 -- 172.21.15.103:0/2513986599 shutdown_connections 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.908+0000 7f43a1008740 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c0a1940 0x56224c0b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.908+0000 7f43a1008740 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c097080 0x56224c091d60 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f43900a4d80 tx=0x7f439009ab50 comp rx=0 tx=0).stop 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.908+0000 7f43a1008740 1 --2- 172.21.15.103:0/2513986599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c096760 0x56224c0991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.908+0000 7f43a1008740 1 -- 172.21.15.103:0/2513986599 >> 172.21.15.103:0/2513986599 conn(0x56224c092cf0 msgr2=0x56224c091650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.909+0000 7f43a1008740 1 -- 172.21.15.103:0/2513986599 shutdown_connections 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.909+0000 7f43a1008740 1 -- 172.21.15.103:0/2513986599 wait complete. 2024-09-19T07:47:55.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.909+0000 7f43a1008740 1 -- start start 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.910+0000 7f43a1008740 1 Processor -- start 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.910+0000 7f43a1008740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.910+0000 7f43a1008740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c0a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.910+0000 7f43a1008740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c0b1880 0x56224c0a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.910+0000 7f43a1008740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43981b5380 con 0x56224c0b1880 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.910+0000 7f43a1008740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43981b4cb0 con 0x56224c0a1940 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.910+0000 7f43a1008740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43981b1f10 con 0x56224c096760 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f43a1008740 10 client.0 _ll_register_callbacks cb 0x56224bfa8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c0a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c0a9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:42620/0 (socket says 172.21.15.103:42620) 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 -- 172.21.15.103:0/1856660313 learned_addr learned my addr 172.21.15.103:0/1856660313 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f439c9cb640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f439d1cc640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c0b1880 0x56224c0a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 -- 172.21.15.103:0/1856660313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 msgr2=0x56224c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c0a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 -- 172.21.15.103:0/1856660313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c0b1880 msgr2=0x56224c0a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c0b1880 0x56224c0a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 -- 172.21.15.103:0/1856660313 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56224c0a6df0 con 0x56224c0a1940 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f439c9cb640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4397fff640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c0a9e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x56224c0a4210 tx=0x7f4390009aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc78506e8bfbd55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4377fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4377fff640 1 -- 172.21.15.103:0/1856660313 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390083070 con 0x56224c0a1940 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f4377fff640 1 -- 172.21.15.103:0/1856660313 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f439009bd10 con 0x56224c0a1940 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.911+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43900a52b0 con 0x56224c0a1940 2024-09-19T07:47:55.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.912+0000 7f4377fff640 1 -- 172.21.15.103:0/1856660313 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f43900b4070 con 0x56224c0a1940 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.912+0000 7f43a1008740 10 client.26852 fetch_fsmap learned FSMap version 44 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.912+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56224c11cb70 con 0x56224c0a1940 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.912+0000 7f4377fff640 1 -- 172.21.15.103:0/1856660313 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43900a09f0 con 0x56224c0a1940 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.912+0000 7f4377fff640 1 -- 172.21.15.103:0/1856660313 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f43900a1c10 con 0x56224c0a1940 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.913+0000 7f43a1008740 10 client.26852 fetch_fsmap finished waiting for FSMap version 44 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.913+0000 7f43a1008740 10 client.26852 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.913+0000 7f43a1008740 20 client.26852 populate_metadata read hostname 'smithi103' 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.913+0000 7f43a1008740 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c11d090 0x56224c11f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.913+0000 7f43a1008740 4 client.26852 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T07:47:55.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.913+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f43900a52b0 con 0x56224c11d090 2024-09-19T07:47:55.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.931+0000 7f439c9cb640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c11d090 0x56224c11f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:55.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.936+0000 7f439c9cb640 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c11d090 0x56224c11f4b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0578ee507cd2f server_cookie=d854ffed078bafe6 in_seq=0 out_seq=0 2024-09-19T07:47:55.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:55.936+0000 7f4377fff640 10 client.26852 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:56.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f4377fff640 1 -- 172.21.15.103:0/1856660313 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f43900a52b0 con 0x56224c11d090 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f4377fff640 10 client.26852 handle_command_reply: tid=0 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 client.26852 shutdown 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 msgr2=0x56224c0a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c0a9e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x56224c0a4210 tx=0x7f4390009aa0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 shutdown_connections 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56224c11d090 0x56224c11f4b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56224c0b1880 0x56224c0a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56224c0a1940 0x56224c0a9e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 --2- 172.21.15.103:0/1856660313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56224c096760 0x56224c0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f4377fff640 0 client.26852 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 >> 172.21.15.103:0/1856660313 conn(0x56224c092cf0 msgr2=0x56224c09f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:56.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.181+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 shutdown_connections 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f43a1008740 1 -- 172.21.15.103:0/1856660313 wait complete. 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f43a1008740 20 client.26852 trim_cache size 0 max 16384 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 44, 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 34, 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "a7fefb2f": { 2024-09-19T07:47:56.183 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 34, 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:47:56.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:47:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:47:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:47:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:47:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:47:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:47:56.185 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:47:56.185 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f43800a33b0 msgr2=0x7f43800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f43800a33b0 0x7f43800a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4388009000 tx=0x7f438801e000 comp rx=0 tx=0).stop 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100de0 msgr2=0x7f43981a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100de0 0x7f43981a4ca0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f439000eb90 tx=0x7f439000c6a0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 shutdown_connections 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f43800a33b0 0x7f43800a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4398105560 0x7f439819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4398100de0 0x7f43981a4ca0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 --2- 172.21.15.103:0/501056451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4398100430 0x7f43981a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 >> 172.21.15.103:0/501056451 conn(0x7f4398077230 msgr2=0x7f43980fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 shutdown_connections 2024-09-19T07:47:56.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:56.182+0000 7f439ec2f640 1 -- 172.21.15.103:0/501056451 wait complete. 2024-09-19T07:47:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:56 smithi103 ceph-mon[31201]: pgmap v307: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 815 op/s 2024-09-19T07:47:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:56 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:47:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:56 smithi134 ceph-mon[35414]: pgmap v307: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 815 op/s 2024-09-19T07:47:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:56 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:47:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:56 smithi174 ceph-mon[35370]: pgmap v307: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 13 MiB/s wr, 815 op/s 2024-09-19T07:47:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:56 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:47:57.135 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:47:57.135 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:47:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:58 smithi103 ceph-mon[31201]: pgmap v308: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 13 MiB/s wr, 873 op/s 2024-09-19T07:47:58.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:58 smithi134 ceph-mon[35414]: pgmap v308: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 13 MiB/s wr, 873 op/s 2024-09-19T07:47:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:58 smithi174 ceph-mon[35370]: pgmap v308: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 13 MiB/s wr, 873 op/s 2024-09-19T07:47:58.699 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a7fefb2f', quiesce took 6.6 seconds. Will release after: 23.4 2024-09-19T07:47:58.715 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:47:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 -- 172.21.15.103:0/774004584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef410c170 msgr2=0x7faef410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 --2- 172.21.15.103:0/774004584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef410c170 0x7faef410e610 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7faeec009900 tx=0x7faeec02a480 comp rx=0 tx=0).stop 2024-09-19T07:47:59.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 -- 172.21.15.103:0/774004584 shutdown_connections 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 --2- 172.21.15.103:0/774004584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef410c170 0x7faef410e610 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 --2- 172.21.15.103:0/774004584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faef4103660 0x7faef410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 --2- 172.21.15.103:0/774004584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef4102d40 0x7faef4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 -- 172.21.15.103:0/774004584 >> 172.21.15.103:0/774004584 conn(0x7faef40fc7a0 msgr2=0x7faef40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 -- 172.21.15.103:0/774004584 shutdown_connections 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 -- 172.21.15.103:0/774004584 wait complete. 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.192+0000 7faef90fa640 1 Processor -- start 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef90fa640 1 -- start start 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef90fa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef4102d40 0x7faef4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef90fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef4103660 0x7faef4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef90fa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faef410c170 0x7faef419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:59.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef90fa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faef4111f30 con 0x7faef4103660 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef90fa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faef4111db0 con 0x7faef4102d40 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef90fa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faef41120b0 con 0x7faef410c170 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef4103660 0x7faef4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef4103660 0x7faef4198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45660/0 (socket says 172.21.15.103:45660) 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 -- 172.21.15.103:0/3211085376 learned_addr learned my addr 172.21.15.103:0/3211085376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef2d9d640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef4102d40 0x7faef4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef359e640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faef410c170 0x7faef419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 -- 172.21.15.103:0/3211085376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faef410c170 msgr2=0x7faef419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faef410c170 0x7faef419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 -- 172.21.15.103:0/3211085376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef4102d40 msgr2=0x7faef4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef4102d40 0x7faef4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef259c640 1 -- 172.21.15.103:0/3211085376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef419d3f0 con 0x7faef4103660 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.193+0000 7faef2d9d640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef4102d40 0x7faef4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:47:59.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.194+0000 7faef259c640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef4103660 0x7faef4198980 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faee400ea00 tx=0x7faee400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.194+0000 7faedbfff640 1 -- 172.21.15.103:0/3211085376 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee400cd80 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.194+0000 7faedbfff640 1 -- 172.21.15.103:0/3211085376 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee4011950 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.194+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef419d6e0 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.194+0000 7faedbfff640 1 -- 172.21.15.103:0/3211085376 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee401aa30 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.194+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef41a4fc0 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.195+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef4103120 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.196+0000 7faedbfff640 1 -- 172.21.15.103:0/3211085376 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7faee4011b00 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.196+0000 7faedbfff640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7faedc0a33b0 0x7faedc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.197+0000 7faef2d9d640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7faedc0a33b0 0x7faedc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.197+0000 7faedbfff640 1 -- 172.21.15.103:0/3211085376 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7faee40d0240 con 0x7faef4103660 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.197+0000 7faef2d9d640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7faedc0a33b0 0x7faedc0a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faee80058c0 tx=0x7faee8005ec0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:47:59.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.201+0000 7faedbfff640 1 -- 172.21.15.103:0/3211085376 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faee4016030 con 0x7faef4103660 2024-09-19T07:47:59.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.408+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faef4106c50 con 0x7faef4103660 2024-09-19T07:47:59.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.408+0000 7faedbfff640 1 -- 172.21.15.103:0/3211085376 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11995 (secure 0 0 0) 0x7faee409cc10 con 0x7faef4103660 2024-09-19T07:47:59.409 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:47:59.409 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":45,"btime":"2024-09-19T07:47:58:208607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:47:57.308042+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:47:59.410 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 45 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7faedc0a33b0 msgr2=0x7faedc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7faedc0a33b0 0x7faedc0a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faee80058c0 tx=0x7faee8005ec0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef4103660 msgr2=0x7faef4198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef4103660 0x7faef4198980 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faee400ea00 tx=0x7faee400eec0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 shutdown_connections 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7faedc0a33b0 0x7faedc0a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faef410c170 0x7faef419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef4103660 0x7faef4198980 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 --2- 172.21.15.103:0/3211085376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faef4102d40 0x7faef4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 >> 172.21.15.103:0/3211085376 conn(0x7faef40fc7a0 msgr2=0x7faef40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:47:59.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 shutdown_connections 2024-09-19T07:47:59.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:47:59.411+0000 7faef90fa640 1 -- 172.21.15.103:0/3211085376 wait complete. 2024-09-19T07:47:59.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:47:59 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:47:59.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:47:59 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:47:59.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:47:59 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:48:00.313 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:48:00.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 565.0 s, 277.8 tps, lat 43.163 ms stddev 41.744 2024-09-19T07:48:00.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:00 smithi134 ceph-mon[35414]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-19T07:48:00.480 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:00 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3211085376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:00.566 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:00 smithi174 ceph-mon[35370]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-19T07:48:00.567 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:00 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3211085376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:00.614 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:00.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:00 smithi103 ceph-mon[31201]: pgmap v309: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-19T07:48:00.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:00 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3211085376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 -- 172.21.15.103:0/2856999788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0106f10 msgr2=0x7fa1d010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2856999788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0106f10 0x7fa1d010ea90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009ca0 tx=0x7fa1c002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 -- 172.21.15.103:0/2856999788 shutdown_connections 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2856999788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0106f10 0x7fa1d010ea90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2856999788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1d0102790 0x7fa1d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2856999788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0101de0 0x7fa1d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 -- 172.21.15.103:0/2856999788 >> 172.21.15.103:0/2856999788 conn(0x7fa1d00fd8c0 msgr2=0x7fa1d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:01.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 -- 172.21.15.103:0/2856999788 shutdown_connections 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.071+0000 7fa1d5118640 1 -- 172.21.15.103:0/2856999788 wait complete. 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 Processor -- start 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 -- start start 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0101de0 0x7fa1d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1d0102790 0x7fa1d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0106f10 0x7fa1d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0111f70 con 0x7fa1d0101de0 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0111df0 con 0x7fa1d0106f10 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1d5118640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01120f0 con 0x7fa1d0102790 2024-09-19T07:48:01.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0101de0 0x7fa1d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0101de0 0x7fa1d00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45682/0 (socket says 172.21.15.103:45682) 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2696418609 learned_addr learned my addr 172.21.15.103:0/2696418609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1d0102790 0x7fa1d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.072+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0106f10 0x7fa1d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2696418609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1d0102790 msgr2=0x7fa1d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1d0102790 0x7fa1d0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2696418609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0106f10 msgr2=0x7fa1d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0106f10 0x7fa1d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2696418609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01a68d0 con 0x7fa1d0101de0 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0106f10 0x7fa1d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0101de0 0x7fa1d00744f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800d950 tx=0x7fa1c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:01.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1b7fff640 1 -- 172.21.15.103:0/2696418609 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8017070 con 0x7fa1d0101de0 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01a6bc0 con 0x7fa1d0101de0 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1b7fff640 1 -- 172.21.15.103:0/2696418609 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c8009c30 con 0x7fa1d0101de0 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1b7fff640 1 -- 172.21.15.103:0/2696418609 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800f6b0 con 0x7fa1d0101de0 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.073+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a7150 con 0x7fa1d0101de0 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.075+0000 7fa1b7fff640 1 -- 172.21.15.103:0/2696418609 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa1c80109d0 con 0x7fa1d0101de0 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.075+0000 7fa1d74f1740 1 Processor -- start 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.075+0000 7fa1d74f1740 1 -- start start 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914db40 0x56076914c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914c5d0 0x56076914ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769157370 0x560769159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607690b22f0 con 0x560769157370 2024-09-19T07:48:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560769059cf0 con 0x56076914db40 2024-09-19T07:48:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607690ae2b0 con 0x56076914c5d0 2024-09-19T07:48:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769157370 0x560769159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769157370 0x560769159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45694/0 (socket says 172.21.15.103:45694) 2024-09-19T07:48:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1602549436 learned_addr learned my addr 172.21.15.103:0/1602549436 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1b7fff640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa1b80a33b0 0x7fa1b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914db40 0x56076914c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1602549436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914c5d0 msgr2=0x56076914ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914c5d0 0x56076914ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914c5d0 0x56076914ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1602549436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914db40 msgr2=0x56076914c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914db40 0x56076914c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1602549436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076903e3a0 con 0x560769157370 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914db40 0x56076914c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914c5d0 0x56076914ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa1b80a33b0 0x7fa1b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1b7fff640 1 -- 172.21.15.103:0/2696418609 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa1c80cf940 con 0x7fa1d0101de0 2024-09-19T07:48:01.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769157370 0x560769159760 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b690 tx=0x7fa1c400bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b9c83c452b2ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.076+0000 7fa1b57fa640 1 -- 172.21.15.103:0/1602549436 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4003bd0 con 0x560769157370 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1b57fa640 1 -- 172.21.15.103:0/1602549436 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c4015070 con 0x560769157370 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1b57fa640 1 -- 172.21.15.103:0/1602549436 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4004d50 con 0x560769157370 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1602549436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769157370 msgr2=0x560769159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa1b80a33b0 0x7fa1b80a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005cc0 tx=0x7fa1c0005da0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769157370 0x560769159760 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b690 tx=0x7fa1c400bb50 comp rx=0 tx=0).stop 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1602549436 shutdown_connections 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769157370 0x560769159760 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914c5d0 0x56076914ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1602549436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914db40 0x56076914c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1602549436 >> 172.21.15.103:0/1602549436 conn(0x560769115310 msgr2=0x560769148c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1602549436 shutdown_connections 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.077+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1602549436 wait complete. 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 -- start start 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 Processor -- start 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914db40 0x560769153f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560769157370 0x560769154450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56076915a070 con 0x56076914db40 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56076906d060 con 0x56076914c5d0 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56076915a1f0 con 0x560769157370 2024-09-19T07:48:01.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1d74f1740 10 client.0 _ll_register_callbacks cb 0x560768dc08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914db40 0x560769153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914db40 0x560769153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45708/0 (socket says 172.21.15.103:45708) 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1087652180 learned_addr learned my addr 172.21.15.103:0/1087652180 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560769157370 0x560769154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560769157370 msgr2=0x560769154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560769157370 0x560769154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1087652180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 msgr2=0x560769156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.078+0000 7fa1ce59c640 1 -- 172.21.15.103:0/1087652180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560768e233c0 con 0x56076914db40 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914db40 0x560769153f10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40048f0 tx=0x7fa1c400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6813dc164264c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/1087652180 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4028060 con 0x56076914db40 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c4015070 con 0x56076914db40 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/1087652180 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa1c4004d30 con 0x56076914db40 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/1087652180 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa1c4004f30 con 0x56076914db40 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 10 client.17403 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5607691659f0 con 0x56076914db40 2024-09-19T07:48:01.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/1087652180 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c4015070 con 0x56076914db40 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/1087652180 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fa1c4022030 con 0x56076914db40 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 10 client.17403 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 10 client.17403 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 20 client.17403 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769166df0 0x5607691691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 4 client.17403 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.079+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607690bf860 con 0x560769166df0 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.080+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769166df0 0x5607691691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.080+0000 7fa1b2ff5640 10 client.17403 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.080+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769166df0 0x5607691691e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=226eeea1382e0428 server_cookie=d3fbc97f1ee7a5a4 in_seq=0 out_seq=0 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/1087652180 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607690bf860 con 0x560769166df0 2024-09-19T07:48:01.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1b2ff5640 10 client.17403 handle_command_reply: tid=0 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 client.17403 shutdown 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914db40 msgr2=0x560769153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914db40 0x560769153f10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40048f0 tx=0x7fa1c400bf20 comp rx=0 tx=0).stop 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 shutdown_connections 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769166df0 0x5607691691e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560769157370 0x560769154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914db40 0x560769153f10 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1b2ff5640 0 client.17403 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/1087652180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 >> 172.21.15.103:0/1087652180 conn(0x560769115310 msgr2=0x56076914e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 shutdown_connections 2024-09-19T07:48:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 1 -- 172.21.15.103:0/1087652180 wait complete. 2024-09-19T07:48:01.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.081+0000 7fa1d74f1740 20 client.17403 trim_cache size 0 max 16384 2024-09-19T07:48:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 Processor -- start 2024-09-19T07:48:01.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 -- start start 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769115310 0x5607691537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769155e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076914f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01afb10 con 0x560769115310 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b1680 con 0x56076914c5d0 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.094+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b0210 con 0x56076914db40 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769115310 0x5607691537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769115310 0x5607691537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45716/0 (socket says 172.21.15.103:45716) 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2896906175 learned_addr learned my addr 172.21.15.103:0/2896906175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:01.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076914f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2896906175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 msgr2=0x56076914f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076914f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2896906175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 msgr2=0x560769155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 -- 172.21.15.103:0/2896906175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560769147310 con 0x560769115310 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1cf59e640 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076914f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769115310 0x5607691537d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8098070 tx=0x7fa1c806ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75367ecdc4a5d206 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1b4ff9640 1 -- 172.21.15.103:0/2896906175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8020040 con 0x560769115310 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1b4ff9640 1 -- 172.21.15.103:0/2896906175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c809ffc0 con 0x560769115310 2024-09-19T07:48:01.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.095+0000 7fa1b4ff9640 1 -- 172.21.15.103:0/2896906175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8098280 con 0x560769115310 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 -- 172.21.15.103:0/2896906175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769115310 msgr2=0x5607691537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769115310 0x5607691537d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8098070 tx=0x7fa1c806ea40 comp rx=0 tx=0).stop 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 -- 172.21.15.103:0/2896906175 shutdown_connections 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076914f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56076914c5d0 0x560769155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/2896906175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560769115310 0x5607691537d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 -- 172.21.15.103:0/2896906175 >> 172.21.15.103:0/2896906175 conn(0x5607691545b0 msgr2=0x560769155770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 -- 172.21.15.103:0/2896906175 shutdown_connections 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 -- 172.21.15.103:0/2896906175 wait complete. 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 -- start start 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.096+0000 7fa1d74f1740 1 Processor -- start 2024-09-19T07:48:01.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560769115310 0x56076915fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914c5d0 0x56076915b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076915b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b0fb0 con 0x56076914c5d0 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01b08e0 con 0x560769115310 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01adb10 con 0x56076914db40 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 10 client.0 _ll_register_callbacks cb 0x7fa1c40044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914c5d0 0x56076915b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914c5d0 0x56076915b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45730/0 (socket says 172.21.15.103:45730) 2024-09-19T07:48:01.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 -- 172.21.15.103:0/941162020 learned_addr learned my addr 172.21.15.103:0/941162020 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:01.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560769115310 0x56076915fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 -- 172.21.15.103:0/941162020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 msgr2=0x56076915b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076915b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 -- 172.21.15.103:0/941162020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560769115310 msgr2=0x56076915fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560769115310 0x56076915fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 -- 172.21.15.103:0/941162020 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607691d0750 con 0x56076914c5d0 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1ce59c640 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914c5d0 0x56076915b140 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4004580 tx=0x7fa1c4003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f027eddfda50a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/941162020 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4028070 con 0x56076914c5d0 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/941162020 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa1c40054e0 con 0x56076914c5d0 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.097+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c809ffc0 con 0x56076914c5d0 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/941162020 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa1c4011440 con 0x56076914c5d0 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 10 client.17409 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:01.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5607691d1930 con 0x56076914c5d0 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/941162020 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c809ffc0 con 0x56076914c5d0 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/941162020 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fa1c4022070 con 0x56076914c5d0 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 10 client.17409 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 10 client.17409 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 20 client.17409 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769157370 0x5607691d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 4 client.17409 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607690b9d70 con 0x560769157370 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.098+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769157370 0x5607691d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1b2ff5640 10 client.17409 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1ced9d640 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769157370 0x5607691d4070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b1dd282995a4d86 server_cookie=c95b883d8b30394f in_seq=0 out_seq=0 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1b2ff5640 1 -- 172.21.15.103:0/941162020 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607690b9d70 con 0x560769157370 2024-09-19T07:48:01.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1b2ff5640 10 client.17409 handle_command_reply: tid=0 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 client.17409 shutdown 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914c5d0 msgr2=0x56076915b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914c5d0 0x56076915b140 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4004580 tx=0x7fa1c4003c40 comp rx=0 tx=0).stop 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 shutdown_connections 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x560769157370 0x5607691d4070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56076914db40 0x56076915b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56076914c5d0 0x56076915b140 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 --2- 172.21.15.103:0/941162020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560769115310 0x56076915fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 >> 172.21.15.103:0/941162020 conn(0x5607691545b0 msgr2=0x560769155f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1b2ff5640 0 client.17409 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 shutdown_connections 2024-09-19T07:48:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 1 -- 172.21.15.103:0/941162020 wait complete. 2024-09-19T07:48:01.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.099+0000 7fa1d74f1740 20 client.17409 trim_cache size 0 max 16384 2024-09-19T07:48:01.117 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa1b80a33b0 msgr2=0x7fa1b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa1b80a33b0 0x7fa1b80a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005cc0 tx=0x7fa1c0005da0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0101de0 msgr2=0x7fa1d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0101de0 0x7fa1d00744f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800d950 tx=0x7fa1c800de10 comp rx=0 tx=0).stop 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 shutdown_connections 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa1b80a33b0 0x7fa1b80a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0106f10 0x7fa1d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1d0102790 0x7fa1d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 --2- 172.21.15.103:0/2696418609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1d0101de0 0x7fa1d00744f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 >> 172.21.15.103:0/2696418609 conn(0x7fa1d00fd8c0 msgr2=0x7fa1d00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 shutdown_connections 2024-09-19T07:48:01.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:01.100+0000 7fa1d5118640 1 -- 172.21.15.103:0/2696418609 wait complete. 2024-09-19T07:48:01.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:01 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:48:01.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:01 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:48:01.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:01 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:48:01.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:48:02.124 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:02.559 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:02 smithi103 ceph-mon[31201]: pgmap v310: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-09-19T07:48:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:02 smithi134 ceph-mon[35414]: pgmap v310: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 -- 172.21.15.103:0/438749504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c1027a0 msgr2=0x7f0b7c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 --2- 172.21.15.103:0/438749504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c1027a0 0x7f0b7c1067f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009900 tx=0x7f0b7002a410 comp rx=0 tx=0).stop 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 -- 172.21.15.103:0/438749504 shutdown_connections 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 --2- 172.21.15.103:0/438749504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c106f20 0x7f0b7c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 --2- 172.21.15.103:0/438749504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c1027a0 0x7f0b7c1067f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 --2- 172.21.15.103:0/438749504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7c101df0 0x7f0b7c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 -- 172.21.15.103:0/438749504 >> 172.21.15.103:0/438749504 conn(0x7f0b7c0fd8b0 msgr2=0x7f0b7c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 -- 172.21.15.103:0/438749504 shutdown_connections 2024-09-19T07:48:02.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.640+0000 7f0b829ac640 1 -- 172.21.15.103:0/438749504 wait complete. 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 Processor -- start 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 -- start start 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7c101df0 0x7f0b7c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c1027a0 0x7f0b7c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c106f20 0x7f0b7c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c111f80 con 0x7f0b7c106f20 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c111e00 con 0x7f0b7c101df0 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b829ac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c112100 con 0x7f0b7c1027a0 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b80f49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c106f20 0x7f0b7c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b80f49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c106f20 0x7f0b7c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45754/0 (socket says 172.21.15.103:45754) 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b80f49640 1 -- 172.21.15.103:0/3815601280 learned_addr learned my addr 172.21.15.103:0/3815601280 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c1027a0 0x7f0b7c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.641+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7c101df0 0x7f0b7c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b80f49640 1 -- 172.21.15.103:0/3815601280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c1027a0 msgr2=0x7f0b7c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b80f49640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c1027a0 0x7f0b7c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b80f49640 1 -- 172.21.15.103:0/3815601280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7c101df0 msgr2=0x7f0b7c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b80f49640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7c101df0 0x7f0b7c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b80f49640 1 -- 172.21.15.103:0/3815601280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c19d390 con 0x7f0b7c106f20 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b80f49640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c106f20 0x7f0b7c19ccb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400eaa0 tx=0x7f0b7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b797fa640 1 -- 172.21.15.103:0/3815601280 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400cdb0 con 0x7f0b7c106f20 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c19d680 con 0x7f0b7c106f20 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b797fa640 1 -- 172.21.15.103:0/3815601280 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b74005020 con 0x7f0b7c106f20 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b797fa640 1 -- 172.21.15.103:0/3815601280 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b740116d0 con 0x7f0b7c106f20 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.642+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1a4f60 con 0x7f0b7c106f20 2024-09-19T07:48:02.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.644+0000 7f0b797fa640 1 -- 172.21.15.103:0/3815601280 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0b740054d0 con 0x7f0b7c106f20 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.644+0000 7f0b84d85740 1 Processor -- start 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.644+0000 7f0b84d85740 1 -- start start 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe3902b40 0x561fe3901090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe39015d0 0x561fe3901a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe390e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b84d85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fe38672f0 con 0x561fe39015d0 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b84d85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fe380ecf0 con 0x561fe390c370 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b84d85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fe38632b0 con 0x561fe3902b40 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe39015d0 0x561fe3901a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe39015d0 0x561fe3901a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45760/0 (socket says 172.21.15.103:45760) 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 -- 172.21.15.103:0/851569964 learned_addr learned my addr 172.21.15.103:0/851569964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:02.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe3902b40 0x561fe3901090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b797fa640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0b640a33b0 0x7f0b640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe390e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 -- 172.21.15.103:0/851569964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe3902b40 msgr2=0x561fe3901090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe3902b40 0x561fe3901090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 -- 172.21.15.103:0/851569964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 msgr2=0x561fe390e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe390e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 -- 172.21.15.103:0/851569964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe37f33a0 con 0x561fe39015d0 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe390e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b80f49640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0b640a33b0 0x7f0b640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe39015d0 0x561fe3901a10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00bcb0 tx=0x7f0b6c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f036c1117652e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b797fa640 1 -- 172.21.15.103:0/3815601280 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f0b740cff30 con 0x7f0b7c106f20 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.645+0000 7f0b627fc640 1 -- 172.21.15.103:0/851569964 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c015070 con 0x561fe39015d0 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b627fc640 1 -- 172.21.15.103:0/851569964 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c011920 con 0x561fe39015d0 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b627fc640 1 -- 172.21.15.103:0/851569964 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c010930 con 0x561fe39015d0 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 -- 172.21.15.103:0/851569964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe39015d0 msgr2=0x561fe3901a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b80f49640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0b640a33b0 0x7f0b640a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b740cd720 tx=0x7f0b7406d040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe39015d0 0x561fe3901a10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00bcb0 tx=0x7f0b6c00f730 comp rx=0 tx=0).stop 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 -- 172.21.15.103:0/851569964 shutdown_connections 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe390e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe39015d0 0x561fe3901a10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 --2- 172.21.15.103:0/851569964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe3902b40 0x561fe3901090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 -- 172.21.15.103:0/851569964 >> 172.21.15.103:0/851569964 conn(0x561fe38ca310 msgr2=0x561fe38fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 -- 172.21.15.103:0/851569964 shutdown_connections 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.646+0000 7f0b84d85740 1 -- 172.21.15.103:0/851569964 wait complete. 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 -- start start 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 Processor -- start 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe390bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3909000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe39095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fe390f070 con 0x561fe3902b40 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fe3822060 con 0x561fe390c370 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fe390f1f0 con 0x561fe39015d0 2024-09-19T07:48:02.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b84d85740 10 client.0 _ll_register_callbacks cb 0x561fe35758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3909000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3909000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45770/0 (socket says 172.21.15.103:45770) 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7bfff640 1 -- 172.21.15.103:0/995959161 learned_addr learned my addr 172.21.15.103:0/995959161 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe390bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe39095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7bfff640 1 -- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 msgr2=0x561fe390bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe390bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.647+0000 7f0b7bfff640 1 -- 172.21.15.103:0/995959161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 msgr2=0x561fe39095d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe39095d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b7bfff640 1 -- 172.21.15.103:0/995959161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe35d83c0 con 0x561fe3902b40 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe39095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:02.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3909000 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b9f0 tx=0x7f0b6c00b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9869914513dcb4d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/995959161 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c028060 con 0x561fe3902b40 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/995959161 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f0b6c011ad0 con 0x561fe3902b40 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c011920 con 0x561fe3902b40 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/995959161 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c015070 con 0x561fe3902b40 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 10 client.17421 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561fe391aa60 con 0x561fe3902b40 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/995959161 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c011920 con 0x561fe3902b40 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/995959161 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f0b6c004a20 con 0x561fe3902b40 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 10 client.17421 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 10 client.17421 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:48:02.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 20 client.17421 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe391be60 0x561fe391e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 4 client.17421 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.648+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe3874860 con 0x561fe391be60 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.649+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe391be60 0x561fe391e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.649+0000 7f0b5b7fe640 10 client.17421 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.649+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe391be60 0x561fe391e250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d44449a54d706490 server_cookie=295a595c2fd8964f in_seq=0 out_seq=0 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/995959161 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fe3874860 con 0x561fe391be60 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b5b7fe640 10 client.17421 handle_command_reply: tid=0 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 client.17421 shutdown 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 msgr2=0x561fe3909000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3909000 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b9f0 tx=0x7f0b6c00b3a0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 shutdown_connections 2024-09-19T07:48:02.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe391be60 0x561fe391e250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe39095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3909000 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 --2- 172.21.15.103:0/995959161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe390bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b5b7fe640 0 client.17421 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 >> 172.21.15.103:0/995959161 conn(0x561fe38ca310 msgr2=0x561fe3903960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 shutdown_connections 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 1 -- 172.21.15.103:0/995959161 wait complete. 2024-09-19T07:48:02.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.650+0000 7f0b84d85740 20 client.17421 trim_cache size 0 max 16384 2024-09-19T07:48:02.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.663+0000 7f0b84d85740 1 Processor -- start 2024-09-19T07:48:02.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.663+0000 7f0b84d85740 1 -- start start 2024-09-19T07:48:02.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe39015d0 0x561fe38fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3918310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe390c370 0x561fe39238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b84d85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1ad970 con 0x561fe3902b40 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b84d85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1af4e0 con 0x561fe39015d0 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b84d85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1ae070 con 0x561fe390c370 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3918310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3918310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45774/0 (socket says 172.21.15.103:45774) 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 -- 172.21.15.103:0/492395093 learned_addr learned my addr 172.21.15.103:0/492395093 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe390c370 0x561fe39238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 -- 172.21.15.103:0/492395093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe390c370 msgr2=0x561fe39238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe390c370 0x561fe39238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe39015d0 0x561fe38fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 -- 172.21.15.103:0/492395093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe39015d0 msgr2=0x561fe38fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe39015d0 0x561fe38fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 -- 172.21.15.103:0/492395093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe38fc310 con 0x561fe3902b40 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe39015d0 0x561fe38fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3918310 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c004e70 tx=0x7f0b6c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26d463b3e8a2809 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b61ffb640 1 -- 172.21.15.103:0/492395093 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c028040 con 0x561fe3902b40 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b61ffb640 1 -- 172.21.15.103:0/492395093 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c00fc30 con 0x561fe3902b40 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.664+0000 7f0b61ffb640 1 -- 172.21.15.103:0/492395093 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c004fe0 con 0x561fe3902b40 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 -- 172.21.15.103:0/492395093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 msgr2=0x561fe3918310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3918310 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c004e70 tx=0x7f0b6c00f730 comp rx=0 tx=0).stop 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 -- 172.21.15.103:0/492395093 shutdown_connections 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe390c370 0x561fe39238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3918310 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 --2- 172.21.15.103:0/492395093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe39015d0 0x561fe38fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 -- 172.21.15.103:0/492395093 >> 172.21.15.103:0/492395093 conn(0x561fe38ca310 msgr2=0x561fe38fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:02.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 -- 172.21.15.103:0/492395093 shutdown_connections 2024-09-19T07:48:02.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 -- 172.21.15.103:0/492395093 wait complete. 2024-09-19T07:48:02.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.665+0000 7f0b84d85740 1 -- start start 2024-09-19T07:48:02.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 1 Processor -- start 2024-09-19T07:48:02.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:02 smithi174 ceph-mon[35370]: pgmap v310: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 11 MiB/s wr, 778 op/s 2024-09-19T07:48:02.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe3906230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3906770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe3906cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1aee10 con 0x561fe3902b40 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1ae740 con 0x561fe390c370 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1ab970 con 0x561fe39015d0 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b84d85740 10 client.0 _ll_register_callbacks cb 0x7f0b6c00bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3906770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3906770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45778/0 (socket says 172.21.15.103:45778) 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 -- 172.21.15.103:0/151717929 learned_addr learned my addr 172.21.15.103:0/151717929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe3906230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe3906cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 -- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 msgr2=0x561fe3906230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe3906230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 -- 172.21.15.103:0/151717929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 msgr2=0x561fe3906cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe3906cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.666+0000 7f0b7bfff640 1 -- 172.21.15.103:0/151717929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe38fe960 con 0x561fe3902b40 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b7bfff640 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3906770 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00bbd0 tx=0x7f0b6c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71706b9cd1922efd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/151717929 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c028070 con 0x561fe3902b40 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/151717929 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f0b6c010a80 con 0x561fe3902b40 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c00fc30 con 0x561fe3902b40 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/151717929 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c004bd0 con 0x561fe3902b40 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 10 client.17427 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561fe3986af0 con 0x561fe3902b40 2024-09-19T07:48:02.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/151717929 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c00fc30 con 0x561fe3902b40 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/151717929 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f0b6c005360 con 0x561fe3902b40 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 10 client.17427 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 10 client.17427 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 20 client.17427 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe39871b0 0x561fe39895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 4 client.17427 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.667+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fe386ed70 con 0x561fe39871b0 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe39871b0 0x561fe39895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b7b7fe640 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe39871b0 0x561fe39895d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9cd79634a830f6bf server_cookie=e25a55d7280d8f77 in_seq=0 out_seq=0 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b5b7fe640 10 client.17427 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b5b7fe640 1 -- 172.21.15.103:0/151717929 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fe386ed70 con 0x561fe39871b0 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b5b7fe640 10 client.17427 handle_command_reply: tid=0 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 client.17427 shutdown 2024-09-19T07:48:02.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 msgr2=0x561fe3906770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3906770 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00bbd0 tx=0x7f0b6c00bfd0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 shutdown_connections 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x561fe39871b0 0x561fe39895d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fe390c370 0x561fe3906cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fe3902b40 0x561fe3906770 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 --2- 172.21.15.103:0/151717929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe39015d0 0x561fe3906230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 >> 172.21.15.103:0/151717929 conn(0x561fe38ca310 msgr2=0x561fe38fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.668+0000 7f0b5b7fe640 0 client.17427 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 shutdown_connections 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b84d85740 1 -- 172.21.15.103:0/151717929 wait complete. 2024-09-19T07:48:02.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b84d85740 20 client.17427 trim_cache size 0 max 16384 2024-09-19T07:48:02.677 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:02.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0b640a33b0 msgr2=0x7f0b640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0b640a33b0 0x7f0b640a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0b740cd720 tx=0x7f0b7406d040 comp rx=0 tx=0).stop 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c106f20 msgr2=0x7f0b7c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c106f20 0x7f0b7c19ccb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400eaa0 tx=0x7f0b7400ef60 comp rx=0 tx=0).stop 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 shutdown_connections 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f0b640a33b0 0x7f0b640a5870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b7c106f20 0x7f0b7c19ccb0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c1027a0 0x7f0b7c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 --2- 172.21.15.103:0/3815601280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b7c101df0 0x7f0b7c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 >> 172.21.15.103:0/3815601280 conn(0x7f0b7c0fd8b0 msgr2=0x7f0b7c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 shutdown_connections 2024-09-19T07:48:02.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:02.669+0000 7f0b829ac640 1 -- 172.21.15.103:0/3815601280 wait complete. 2024-09-19T07:48:03.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:48:03.940 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:04.316 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:04 smithi103 ceph-mon[31201]: pgmap v311: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.1 MiB/s wr, 386 op/s 2024-09-19T07:48:04.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.450+0000 7f628b132640 1 -- 172.21.15.103:0/3697270541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 msgr2=0x7f628406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.450+0000 7f628b132640 1 --2- 172.21.15.103:0/3697270541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f628406e4f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6274009860 tx=0x7f627402a340 comp rx=0 tx=0).stop 2024-09-19T07:48:04.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.453+0000 7f628b132640 1 -- 172.21.15.103:0/3697270541 shutdown_connections 2024-09-19T07:48:04.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.453+0000 7f628b132640 1 --2- 172.21.15.103:0/3697270541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f628411b140 0x7f628411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.453+0000 7f628b132640 1 --2- 172.21.15.103:0/3697270541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f628406ea30 0x7f628406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.453+0000 7f628b132640 1 --2- 172.21.15.103:0/3697270541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f628406e4f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.453+0000 7f628b132640 1 -- 172.21.15.103:0/3697270541 >> 172.21.15.103:0/3697270541 conn(0x7f6284073210 msgr2=0x7f6284073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:04.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.453+0000 7f628b132640 1 -- 172.21.15.103:0/3697270541 shutdown_connections 2024-09-19T07:48:04.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.453+0000 7f628b132640 1 -- 172.21.15.103:0/3697270541 wait complete. 2024-09-19T07:48:04.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.454+0000 7f628b132640 1 Processor -- start 2024-09-19T07:48:04.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.455+0000 7f628b132640 1 -- start start 2024-09-19T07:48:04.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.455+0000 7f628b132640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f628406ea30 0x7f628411a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.455+0000 7f628b132640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f6284114920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.455+0000 7f628b132640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f628411b140 0x7f6284114e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.455+0000 7f628b132640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6284111310 con 0x7f628406ea30 2024-09-19T07:48:04.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f628b132640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6284111190 con 0x7f628411b140 2024-09-19T07:48:04.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f628b132640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6284111490 con 0x7f6284110220 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f6284114920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f6284114920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:39508/0 (socket says 172.21.15.103:39508) 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 -- 172.21.15.103:0/2855438635 learned_addr learned my addr 172.21.15.103:0/2855438635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f62896cf640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f628411b140 0x7f6284114e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 -- 172.21.15.103:0/2855438635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f628411b140 msgr2=0x7f6284114e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f628411b140 0x7f6284114e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 -- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f628406ea30 msgr2=0x7f628411a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f628406ea30 0x7f628411a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.456+0000 7f6283fff640 1 -- 172.21.15.103:0/2855438635 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6284115590 con 0x7f6284110220 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.457+0000 7f6283fff640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f6284114920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f627800e9f0 tx=0x7f627800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:04.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.457+0000 7f6281ffb640 1 -- 172.21.15.103:0/2855438635 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627800cd80 con 0x7f6284110220 2024-09-19T07:48:04.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.457+0000 7f6281ffb640 1 -- 172.21.15.103:0/2855438635 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6278005500 con 0x7f6284110220 2024-09-19T07:48:04.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.457+0000 7f6281ffb640 1 -- 172.21.15.103:0/2855438635 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278010690 con 0x7f6284110220 2024-09-19T07:48:04.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.457+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62841bc060 con 0x7f6284110220 2024-09-19T07:48:04.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.458+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62841bc460 con 0x7f6284110220 2024-09-19T07:48:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.463+0000 7f6281ffb640 1 -- 172.21.15.103:0/2855438635 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6278010830 con 0x7f6284110220 2024-09-19T07:48:04.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.463+0000 7f628d50b740 1 Processor -- start 2024-09-19T07:48:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.464+0000 7f6281ffb640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f626c0a3540 0x7f626c0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.464+0000 7f6281ffb640 1 -- 172.21.15.103:0/2855438635 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f62780d11e0 con 0x7f6284110220 2024-09-19T07:48:04.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.464+0000 7f62896cf640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f626c0a3540 0x7f626c0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f628d50b740 1 -- start start 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f628d50b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edecf0 0x55d530edd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f628d50b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edd780 0x55d530eddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f628d50b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530eea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f628d50b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d530e432f0 con 0x55d530edecf0 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f628d50b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d530deacf0 con 0x55d530edd780 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f628d50b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d530e3f2b0 con 0x55d530ee8520 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f62896cf640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f626c0a3540 0x7f626c0a5a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f627c005fd0 tx=0x7f627c0095c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:04.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f62896cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530eea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f62896cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530eea910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:39512/0 (socket says 172.21.15.103:39512) 2024-09-19T07:48:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f62896cf640 1 -- 172.21.15.103:0/1718970443 learned_addr learned my addr 172.21.15.103:0/1718970443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.465+0000 7f6283fff640 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edd780 0x55d530eddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f6288ece640 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edecf0 0x55d530edd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f62896cf640 1 -- 172.21.15.103:0/1718970443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edd780 msgr2=0x55d530eddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f62896cf640 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edd780 0x55d530eddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f62896cf640 1 -- 172.21.15.103:0/1718970443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edecf0 msgr2=0x55d530edd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f62896cf640 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edecf0 0x55d530edd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f62896cf640 1 -- 172.21.15.103:0/1718970443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d530dcf3a0 con 0x55d530ee8520 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f62896cf640 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530eea910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f627c01b770 tx=0x7f627c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3d0cb0c526ad0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.466+0000 7f626affd640 1 -- 172.21.15.103:0/1718970443 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c02b070 con 0x55d530ee8520 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.467+0000 7f626affd640 1 -- 172.21.15.103:0/1718970443 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627c01bd60 con 0x55d530ee8520 2024-09-19T07:48:04.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.467+0000 7f626affd640 1 -- 172.21.15.103:0/1718970443 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c019590 con 0x55d530ee8520 2024-09-19T07:48:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 -- 172.21.15.103:0/1718970443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 msgr2=0x55d530eea910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530eea910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f627c01b770 tx=0x7f627c014000 comp rx=0 tx=0).stop 2024-09-19T07:48:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 -- 172.21.15.103:0/1718970443 shutdown_connections 2024-09-19T07:48:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530eea910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edd780 0x55d530eddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 --2- 172.21.15.103:0/1718970443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edecf0 0x55d530edd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 -- 172.21.15.103:0/1718970443 >> 172.21.15.103:0/1718970443 conn(0x55d530ea6310 msgr2=0x55d530ed9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:04.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.469+0000 7f628d50b740 1 -- 172.21.15.103:0/1718970443 shutdown_connections 2024-09-19T07:48:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.473+0000 7f628d50b740 1 -- 172.21.15.103:0/1718970443 wait complete. 2024-09-19T07:48:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.473+0000 7f628d50b740 1 -- start start 2024-09-19T07:48:04.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.473+0000 7f628d50b740 1 Processor -- start 2024-09-19T07:48:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.475+0000 7f628d50b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ee7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.475+0000 7f628d50b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.475+0000 7f628d50b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.475+0000 7f628d50b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d530eeb220 con 0x55d530ee8520 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.475+0000 7f628d50b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d530dfe060 con 0x55d530edecf0 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.475+0000 7f628d50b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d530eeb3a0 con 0x55d530edd780 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.475+0000 7f628d50b740 10 client.0 _ll_register_callbacks cb 0x55d530b518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f62896cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ee7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ee7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:39518/0 (socket says 172.21.15.103:39518) 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 -- 172.21.15.103:0/4065448693 learned_addr learned my addr 172.21.15.103:0/4065448693 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 -- 172.21.15.103:0/4065448693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 msgr2=0x55d530ee5110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:48:04.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 -- 172.21.15.103:0/4065448693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 msgr2=0x55d530ee5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.476+0000 7f6288ece640 1 -- 172.21.15.103:0/4065448693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d530bb43c0 con 0x55d530edd780 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.477+0000 7f6288ece640 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ee7e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f627402a8b0 tx=0x7f6274004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7141650b35c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.477+0000 7f6263fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.477+0000 7f6263fff640 1 -- 172.21.15.103:0/4065448693 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274040020 con 0x55d530edd780 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.477+0000 7f6263fff640 1 -- 172.21.15.103:0/4065448693 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f6274030550 con 0x55d530edd780 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.477+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f627c01bd60 con 0x55d530edd780 2024-09-19T07:48:04.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.478+0000 7f6263fff640 1 -- 172.21.15.103:0/4065448693 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6274037040 con 0x55d530edd780 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.478+0000 7f628d50b740 10 client.26815 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.478+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d530ef6c70 con 0x55d530edd780 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.478+0000 7f6263fff640 1 -- 172.21.15.103:0/4065448693 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627c01bd60 con 0x55d530edd780 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.479+0000 7f6263fff640 1 -- 172.21.15.103:0/4065448693 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f62740047a0 con 0x55d530edd780 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.479+0000 7f628d50b740 10 client.26815 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.479+0000 7f628d50b740 10 client.26815 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.479+0000 7f628d50b740 20 client.26815 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.479+0000 7f628d50b740 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530ef80d0 0x55d530efa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.479+0000 7f628d50b740 4 client.26815 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.479+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d530e50860 con 0x55d530ef80d0 2024-09-19T07:48:04.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.480+0000 7f6283fff640 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530ef80d0 0x55d530efa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.480+0000 7f6283fff640 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530ef80d0 0x55d530efa540 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16371cc11fa56b3a server_cookie=a1462ab48625684b in_seq=0 out_seq=0 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.480+0000 7f6263fff640 10 client.26815 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.482+0000 7f6263fff640 1 -- 172.21.15.103:0/4065448693 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d530e50860 con 0x55d530ef80d0 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.482+0000 7f6263fff640 10 client.26815 handle_command_reply: tid=0 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.482+0000 7f628d50b740 1 client.26815 shutdown 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.483+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 msgr2=0x55d530ee7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.483+0000 7f628d50b740 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ee7e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f627402a8b0 tx=0x7f6274004250 comp rx=0 tx=0).stop 2024-09-19T07:48:04.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 shutdown_connections 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530ef80d0 0x55d530efa540 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 --2- 172.21.15.103:0/4065448693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ee7e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 >> 172.21.15.103:0/4065448693 conn(0x55d530ea6310 msgr2=0x55d530edfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f6263fff640 0 client.26815 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 shutdown_connections 2024-09-19T07:48:04.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.484+0000 7f628d50b740 1 -- 172.21.15.103:0/4065448693 wait complete. 2024-09-19T07:48:04.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.485+0000 7f628d50b740 20 client.26815 trim_cache size 0 max 16384 2024-09-19T07:48:04.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.509+0000 7f628d50b740 1 Processor -- start 2024-09-19T07:48:04.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.509+0000 7f628d50b740 1 -- start start 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.509+0000 7f628d50b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edd780 0x55d530ee7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.509+0000 7f628d50b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.509+0000 7f6288ece640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edd780 0x55d530ee7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.509+0000 7f6288ece640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edd780 0x55d530ee7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45826/0 (socket says 172.21.15.103:45826) 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6288ece640 1 -- 172.21.15.103:0/3857310106 learned_addr learned my addr 172.21.15.103:0/3857310106 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f628d50b740 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530ee0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f627000e870 con 0x55d530edd780 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62700103e0 con 0x55d530edecf0 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f627000ef70 con 0x55d530ee8520 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6283fff640 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f62896cf640 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530ee0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6283fff640 1 -- 172.21.15.103:0/3857310106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 msgr2=0x55d530ee0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6283fff640 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530ee0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6283fff640 1 -- 172.21.15.103:0/3857310106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edd780 msgr2=0x55d530ee7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6283fff640 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edd780 0x55d530ee7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6283fff640 1 -- 172.21.15.103:0/3857310106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d530efa8b0 con 0x55d530edecf0 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.510+0000 7f6283fff640 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee02d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62780ce890 tx=0x7f6278009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f17cff40c75517 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.512+0000 7f626a7fc640 1 -- 172.21.15.103:0/3857310106 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62780b6070 con 0x55d530edecf0 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.512+0000 7f626a7fc640 1 -- 172.21.15.103:0/3857310106 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627809d420 con 0x55d530edecf0 2024-09-19T07:48:04.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.512+0000 7f626a7fc640 1 -- 172.21.15.103:0/3857310106 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278099a90 con 0x55d530edecf0 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.513+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 msgr2=0x55d530ee02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.513+0000 7f628d50b740 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee02d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f62780ce890 tx=0x7f6278009310 comp rx=0 tx=0).stop 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.513+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 shutdown_connections 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.513+0000 7f628d50b740 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530ee8520 0x55d530ee0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.513+0000 7f628d50b740 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee02d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.513+0000 7f628d50b740 1 --2- 172.21.15.103:0/3857310106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530edd780 0x55d530ee7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.513+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 >> 172.21.15.103:0/3857310106 conn(0x55d530ea6310 msgr2=0x55d530ee6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.515+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 shutdown_connections 2024-09-19T07:48:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.515+0000 7f628d50b740 1 -- 172.21.15.103:0/3857310106 wait complete. 2024-09-19T07:48:04.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.515+0000 7f628d50b740 1 -- start start 2024-09-19T07:48:04.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.515+0000 7f628d50b740 1 Processor -- start 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.516+0000 7f628d50b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ef0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.516+0000 7f628d50b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.516+0000 7f628d50b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.516+0000 7f628d50b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f627000fd10 con 0x55d530ee8520 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.516+0000 7f628d50b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f627000f640 con 0x55d530edecf0 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.516+0000 7f628d50b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f627000c870 con 0x55d530edd780 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.516+0000 7f628d50b740 10 client.0 _ll_register_callbacks cb 0x7f6274035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:04.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.517+0000 7f62896cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.517+0000 7f62896cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45836/0 (socket says 172.21.15.103:45836) 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.517+0000 7f62896cf640 1 -- 172.21.15.103:0/2445187653 learned_addr learned my addr 172.21.15.103:0/2445187653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.517+0000 7f6288ece640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ef0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.517+0000 7f6283fff640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.517+0000 7f62896cf640 1 -- 172.21.15.103:0/2445187653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 msgr2=0x55d530ef0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f62896cf640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ef0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f62896cf640 1 -- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 msgr2=0x55d530ee1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f62896cf640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f62896cf640 1 -- 172.21.15.103:0/2445187653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d530ee2a60 con 0x55d530ee8520 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f6283fff640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f62896cf640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee2320 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x55d530ede220 tx=0x7f627c01bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460d0ebeee534eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:04.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f6263fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f6263fff640 1 -- 172.21.15.103:0/2445187653 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c02b040 con 0x55d530ee8520 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.518+0000 7f6263fff640 1 -- 172.21.15.103:0/2445187653 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f627c004a00 con 0x55d530ee8520 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.519+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f627809d420 con 0x55d530ee8520 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.519+0000 7f6263fff640 1 -- 172.21.15.103:0/2445187653 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f627c004c40 con 0x55d530ee8520 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.519+0000 7f628d50b740 10 client.17442 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.519+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d530f62d50 con 0x55d530ee8520 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f6263fff640 1 -- 172.21.15.103:0/2445187653 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f627c004de0 con 0x55d530ee8520 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f6263fff640 1 -- 172.21.15.103:0/2445187653 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f627c00b040 con 0x55d530ee8520 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f628d50b740 10 client.17442 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f628d50b740 10 client.17442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:48:04.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f628d50b740 20 client.17442 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f628d50b740 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530f630e0 0x55d530f654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f628d50b740 4 client.17442 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.520+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f627809d420 con 0x55d530f630e0 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.521+0000 7f6288ece640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530f630e0 0x55d530f654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.521+0000 7f6263fff640 10 client.17442 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.521+0000 7f6288ece640 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530f630e0 0x55d530f654d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20d86f953faa250f server_cookie=cedb887e2253e3a in_seq=0 out_seq=0 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f6263fff640 1 -- 172.21.15.103:0/2445187653 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f627809d420 con 0x55d530f630e0 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f6263fff640 10 client.17442 handle_command_reply: tid=0 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 client.17442 shutdown 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 msgr2=0x55d530ee2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee2320 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x55d530ede220 tx=0x7f627c01bdf0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 shutdown_connections 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d530f630e0 0x55d530f654d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d530ee8520 0x55d530ee2320 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d530edecf0 0x55d530ee1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 --2- 172.21.15.103:0/2445187653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d530edd780 0x55d530ef0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 >> 172.21.15.103:0/2445187653 conn(0x55d530ea6310 msgr2=0x55d530ed8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f6263fff640 0 client.17442 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.522+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 shutdown_connections 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.523+0000 7f628d50b740 1 -- 172.21.15.103:0/2445187653 wait complete. 2024-09-19T07:48:04.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.523+0000 7f628d50b740 20 client.17442 trim_cache size 0 max 16384 2024-09-19T07:48:04.527 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f626c0a3540 msgr2=0x7f626c0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f626c0a3540 0x7f626c0a5a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f627c005fd0 tx=0x7f627c0095c0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 msgr2=0x7f6284114920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f6284114920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f627800e9f0 tx=0x7f627800edf0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 shutdown_connections 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f626c0a3540 0x7f626c0a5a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f628411b140 0x7f6284114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6284110220 0x7f6284114920 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 --2- 172.21.15.103:0/2855438635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f628406ea30 0x7f628411a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:04.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 >> 172.21.15.103:0/2855438635 conn(0x7f6284073210 msgr2=0x7f628411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:04.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.524+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 shutdown_connections 2024-09-19T07:48:04.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:04.525+0000 7f628b132640 1 -- 172.21.15.103:0/2855438635 wait complete. 2024-09-19T07:48:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:04 smithi134 ceph-mon[35414]: pgmap v311: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.1 MiB/s wr, 386 op/s 2024-09-19T07:48:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:04 smithi174 ceph-mon[35370]: pgmap v311: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.1 MiB/s wr, 386 op/s 2024-09-19T07:48:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:05 smithi103 ceph-mon[31201]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:48:05.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 570.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:05.557 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:48:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:05 smithi134 ceph-mon[35414]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:48:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:05 smithi174 ceph-mon[35370]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T07:48:05.813 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 -- 172.21.15.103:0/4063289933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0102790 msgr2=0x7f3bf01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4063289933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0102790 0x7f3bf01067e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009900 tx=0x7f3bd802a410 comp rx=0 tx=0).stop 2024-09-19T07:48:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 -- 172.21.15.103:0/4063289933 shutdown_connections 2024-09-19T07:48:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4063289933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf0106f10 0x7f3bf010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4063289933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0102790 0x7f3bf01067e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4063289933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bf0101de0 0x7f3bf01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 -- 172.21.15.103:0/4063289933 >> 172.21.15.103:0/4063289933 conn(0x7f3bf00fd8c0 msgr2=0x7f3bf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 -- 172.21.15.103:0/4063289933 shutdown_connections 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.321+0000 7f3bf6987640 1 -- 172.21.15.103:0/4063289933 wait complete. 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 Processor -- start 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 -- start start 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bf0101de0 0x7f3bf0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf0102790 0x7f3bf0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0106f10 0x7f3bf019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0111f70 con 0x7f3bf0106f10 2024-09-19T07:48:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0111df0 con 0x7f3bf0102790 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf6987640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01120f0 con 0x7f3bf0101de0 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf4f24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0106f10 0x7f3bf019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf4f24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0106f10 0x7f3bf019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45844/0 (socket says 172.21.15.103:45844) 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3bf4f24640 1 -- 172.21.15.103:0/4238872202 learned_addr learned my addr 172.21.15.103:0/4238872202 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3beffff640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bf0101de0 0x7f3bf0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.322+0000 7f3be7fff640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf0102790 0x7f3bf0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf4f24640 1 -- 172.21.15.103:0/4238872202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bf0101de0 msgr2=0x7f3bf0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf4f24640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bf0101de0 0x7f3bf0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf4f24640 1 -- 172.21.15.103:0/4238872202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf0102790 msgr2=0x7f3bf0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf4f24640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf0102790 0x7f3bf0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf4f24640 1 -- 172.21.15.103:0/4238872202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf019d4c0 con 0x7f3bf0106f10 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf4f24640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0106f10 0x7f3bf019cde0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009fd0 tx=0x7f3be800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bedffb640 1 -- 172.21.15.103:0/4238872202 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8017070 con 0x7f3bf0106f10 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bedffb640 1 -- 172.21.15.103:0/4238872202 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be8003e50 con 0x7f3bf0106f10 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf019d7b0 con 0x7f3bf0106f10 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bedffb640 1 -- 172.21.15.103:0/4238872202 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be80106c0 con 0x7f3bf0106f10 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.323+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf01a5090 con 0x7f3bf0106f10 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.325+0000 7f3bedffb640 1 -- 172.21.15.103:0/4238872202 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3be8003970 con 0x7f3bf0106f10 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.325+0000 7f3bf8d60740 1 Processor -- start 2024-09-19T07:48:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf8d60740 1 -- start start 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fdb40 0x55ee347fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee347fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee3480db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ee347622f0 con 0x55ee3480b780 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ee34709cf0 con 0x55ee347fc5d0 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee3475e2b0 con 0x55ee347fdb40 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bedffb640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3bdc0a33b0 0x7f3bdc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee3480db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee3480db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45856/0 (socket says 172.21.15.103:45856) 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 -- 172.21.15.103:0/445618479 learned_addr learned my addr 172.21.15.103:0/445618479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bf4f24640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3bdc0a33b0 0x7f3bdc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3beffff640 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee347fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3bedffb640 1 -- 172.21.15.103:0/4238872202 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f3be80cfad0 con 0x7f3bf0106f10 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fdb40 0x55ee347fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 -- 172.21.15.103:0/445618479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fdb40 msgr2=0x55ee347fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fdb40 0x55ee347fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 -- 172.21.15.103:0/445618479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 msgr2=0x55ee347fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee347fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3be7fff640 1 -- 172.21.15.103:0/445618479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee346ee3a0 con 0x55ee3480b780 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.326+0000 7f3beffff640 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee347fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf4f24640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3bdc0a33b0 0x7f3bdc0a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3be809bd00 tx=0x7f3be8004020 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3be7fff640 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee3480db70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8031550 tx=0x7f3bd8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190e9b04b78cb9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3be67fc640 1 -- 172.21.15.103:0/445618479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8035040 con 0x55ee3480b780 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3be67fc640 1 -- 172.21.15.103:0/445618479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8004030 con 0x55ee3480b780 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3be67fc640 1 -- 172.21.15.103:0/445618479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8003b60 con 0x55ee3480b780 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf8d60740 1 -- 172.21.15.103:0/445618479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 msgr2=0x55ee3480db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee3480db70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8031550 tx=0x7f3bd8005b50 comp rx=0 tx=0).stop 2024-09-19T07:48:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf8d60740 1 -- 172.21.15.103:0/445618479 shutdown_connections 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee3480db70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee347fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/445618479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fdb40 0x55ee347fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.327+0000 7f3bf8d60740 1 -- 172.21.15.103:0/445618479 >> 172.21.15.103:0/445618479 conn(0x55ee347c5310 msgr2=0x55ee347f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 -- 172.21.15.103:0/445618479 shutdown_connections 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 -- 172.21.15.103:0/445618479 wait complete. 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 -- start start 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 Processor -- start 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee3480b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34808430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee3480b780 0x55ee34808a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ee3480e420 con 0x55ee347fdb40 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ee3471d060 con 0x55ee3480b780 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee3480e5a0 con 0x55ee347fc5d0 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3bf8d60740 10 client.0 _ll_register_callbacks cb 0x55ee344708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34808430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34808430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45862/0 (socket says 172.21.15.103:45862) 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.328+0000 7f3beffff640 1 -- 172.21.15.103:0/2721446080 learned_addr learned my addr 172.21.15.103:0/2721446080 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3be7fff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee3480b780 0x55ee34808a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3be7fff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee3480b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3beffff640 1 -- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 msgr2=0x55ee3480b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3beffff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee3480b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3beffff640 1 -- 172.21.15.103:0/2721446080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee3480b780 msgr2=0x55ee34808a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3beffff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee3480b780 0x55ee34808a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3beffff640 1 -- 172.21.15.103:0/2721446080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee344d33c0 con 0x55ee347fdb40 2024-09-19T07:48:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3be7fff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee3480b780 0x55ee34808a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3beffff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34808430 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ba10 tx=0x7f3be000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc233140e486f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/2721446080 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0003e80 con 0x55ee347fdb40 2024-09-19T07:48:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/2721446080 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f3be0011440 con 0x55ee347fdb40 2024-09-19T07:48:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd8004030 con 0x55ee347fdb40 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/2721446080 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3be0011640 con 0x55ee347fdb40 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bf8d60740 10 client.17454 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ee34819f70 con 0x55ee347fdb40 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.329+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/2721446080 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8004030 con 0x55ee347fdb40 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/2721446080 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f3be0021030 con 0x55ee347fdb40 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bf8d60740 10 client.17454 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bf8d60740 10 client.17454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bf8d60740 20 client.17454 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee3481b4d0 0x55ee3481d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bf8d60740 4 client.17454 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:06.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee3476f860 con 0x55ee3481b4d0 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3be7fff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee3481b4d0 0x55ee3481d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3bcf7fe640 10 client.17454 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.330+0000 7f3be7fff640 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee3481b4d0 0x55ee3481d8f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=416682a35c7c75cc server_cookie=33ee7fc00e50e611 in_seq=0 out_seq=0 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/2721446080 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee3476f860 con 0x55ee3481b4d0 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bcf7fe640 10 client.17454 handle_command_reply: tid=0 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 client.17454 shutdown 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 msgr2=0x55ee34808430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34808430 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ba10 tx=0x7f3be000bed0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 shutdown_connections 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee3481b4d0 0x55ee3481d8f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee3480b780 0x55ee34808a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bcf7fe640 0 client.17454 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34808430 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/2721446080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee3480b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 >> 172.21.15.103:0/2721446080 conn(0x55ee347c5310 msgr2=0x55ee347fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:06.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.331+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 shutdown_connections 2024-09-19T07:48:06.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.332+0000 7f3bf8d60740 1 -- 172.21.15.103:0/2721446080 wait complete. 2024-09-19T07:48:06.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.332+0000 7f3bf8d60740 20 client.17454 trim_cache size 0 max 16384 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.344+0000 7f3bf8d60740 1 Processor -- start 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3bf8d60740 1 -- start start 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee347f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fdb40 0x55ee348176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee34807430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01ab700 con 0x55ee3480b780 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01ad270 con 0x55ee347fdb40 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01abe00 con 0x55ee347fc5d0 2024-09-19T07:48:06.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee34807430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee34807430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45868/0 (socket says 172.21.15.103:45868) 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 -- 172.21.15.103:0/1772904339 learned_addr learned my addr 172.21.15.103:0/1772904339 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee347f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 -- 172.21.15.103:0/1772904339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 msgr2=0x55ee347f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee347f8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3beffff640 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fdb40 0x55ee348176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 -- 172.21.15.103:0/1772904339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fdb40 msgr2=0x55ee348176c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fdb40 0x55ee348176c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 -- 172.21.15.103:0/1772904339 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee347f9960 con 0x55ee3480b780 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3beffff640 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fdb40 0x55ee348176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be7fff640 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee34807430 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8003770 tx=0x7f3bd804a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b4fa9d9690bca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:06.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.345+0000 7f3be5ffb640 1 -- 172.21.15.103:0/1772904339 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8058030 con 0x55ee3480b780 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3be5ffb640 1 -- 172.21.15.103:0/1772904339 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8003db0 con 0x55ee3480b780 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3be5ffb640 1 -- 172.21.15.103:0/1772904339 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8035040 con 0x55ee3480b780 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 -- 172.21.15.103:0/1772904339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 msgr2=0x55ee34807430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee34807430 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8003770 tx=0x7f3bd804a040 comp rx=0 tx=0).stop 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 -- 172.21.15.103:0/1772904339 shutdown_connections 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee3480b780 0x55ee34807430 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fdb40 0x55ee348176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/1772904339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee347fc5d0 0x55ee347f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 -- 172.21.15.103:0/1772904339 >> 172.21.15.103:0/1772904339 conn(0x55ee347c5310 msgr2=0x55ee34806d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 -- 172.21.15.103:0/1772904339 shutdown_connections 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.346+0000 7f3bf8d60740 1 -- 172.21.15.103:0/1772904339 wait complete. 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 -- start start 2024-09-19T07:48:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 Processor -- start 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee348139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34813f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee3480b780 0x55ee34804a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01acba0 con 0x55ee347fdb40 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01ac4d0 con 0x55ee347fc5d0 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01a9700 con 0x55ee3480b780 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3bf8d60740 10 client.0 _ll_register_callbacks cb 0x7f3be00042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34813f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34813f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45870/0 (socket says 172.21.15.103:45870) 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 -- 172.21.15.103:0/3822213319 learned_addr learned my addr 172.21.15.103:0/3822213319 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3be7fff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee348139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3be7fff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee3480b780 0x55ee34804a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 -- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee3480b780 msgr2=0x55ee34804a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee3480b780 0x55ee34804a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 -- 172.21.15.103:0/3822213319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 msgr2=0x55ee348139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee348139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3beffff640 1 -- 172.21.15.103:0/3822213319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee348050c0 con 0x55ee347fdb40 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.347+0000 7f3be7fff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee3480b780 0x55ee34804a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3beffff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34813f20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3be0004280 tx=0x7f3be001b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16c1f63fd54f845 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/3822213319 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0003e80 con 0x55ee347fdb40 2024-09-19T07:48:06.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/3822213319 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f3be0011440 con 0x55ee347fdb40 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd8003db0 con 0x55ee347fdb40 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/3822213319 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3be0005030 con 0x55ee347fdb40 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 10 client.17460 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ee34885a20 con 0x55ee347fdb40 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/3822213319 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8003db0 con 0x55ee347fdb40 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/3822213319 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f3be0004680 con 0x55ee347fdb40 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 10 client.17460 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 10 client.17460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 20 client.17460 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee34886170 0x55ee34888590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 4 client.17460 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.348+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee34769d70 con 0x55ee34886170 2024-09-19T07:48:06.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3be7fff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee34886170 0x55ee34888590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3be7fff640 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee34886170 0x55ee34888590 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7887e5a6620a5b4 server_cookie=5d345b823bf7a16d in_seq=0 out_seq=0 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bcf7fe640 10 client.17460 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bcf7fe640 1 -- 172.21.15.103:0/3822213319 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee34769d70 con 0x55ee34886170 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bcf7fe640 10 client.17460 handle_command_reply: tid=0 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bf8d60740 1 client.17460 shutdown 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 msgr2=0x55ee34813f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34813f20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3be0004280 tx=0x7f3be001b9f0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 shutdown_connections 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55ee34886170 0x55ee34888590 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee3480b780 0x55ee34804a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.349+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ee347fdb40 0x55ee34813f20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf8d60740 1 --2- 172.21.15.103:0/3822213319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ee347fc5d0 0x55ee348139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bcf7fe640 0 client.17460 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:06.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 >> 172.21.15.103:0/3822213319 conn(0x55ee347c5310 msgr2=0x55ee34807540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:06.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 shutdown_connections 2024-09-19T07:48:06.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf8d60740 1 -- 172.21.15.103:0/3822213319 wait complete. 2024-09-19T07:48:06.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf8d60740 20 client.17460 trim_cache size 0 max 16384 2024-09-19T07:48:06.365 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3bdc0a33b0 msgr2=0x7f3bdc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3bdc0a33b0 0x7f3bdc0a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3be809bd00 tx=0x7f3be8004020 comp rx=0 tx=0).stop 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0106f10 msgr2=0x7f3bf019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0106f10 0x7f3bf019cde0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009fd0 tx=0x7f3be800b260 comp rx=0 tx=0).stop 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 shutdown_connections 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3bdc0a33b0 0x7f3bdc0a5870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3bf0106f10 0x7f3bf019cde0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf0102790 0x7f3bf0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 --2- 172.21.15.103:0/4238872202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bf0101de0 0x7f3bf0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:06.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 >> 172.21.15.103:0/4238872202 conn(0x7f3bf00fd8c0 msgr2=0x7f3bf00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:06.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 shutdown_connections 2024-09-19T07:48:06.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:06.350+0000 7f3bf6987640 1 -- 172.21.15.103:0/4238872202 wait complete. 2024-09-19T07:48:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:06 smithi134 ceph-mon[35414]: pgmap v313: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.9 MiB/s wr, 340 op/s 2024-09-19T07:48:06.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:06 smithi103 ceph-mon[31201]: pgmap v313: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.9 MiB/s wr, 340 op/s 2024-09-19T07:48:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:06 smithi174 ceph-mon[35370]: pgmap v313: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.9 MiB/s wr, 340 op/s 2024-09-19T07:48:07.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:48:07.400 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:07.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 -- 172.21.15.103:0/3883116553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 msgr2=0x7f3c28100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 --2- 172.21.15.103:0/3883116553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c28100ed0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009930 tx=0x7f3c1802a420 comp rx=0 tx=0).stop 2024-09-19T07:48:07.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 -- 172.21.15.103:0/3883116553 shutdown_connections 2024-09-19T07:48:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 --2- 172.21.15.103:0/3883116553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c28101410 0x7f3c2810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 --2- 172.21.15.103:0/3883116553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c28100ed0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 --2- 172.21.15.103:0/3883116553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c28106a70 0x7f3c28106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 -- 172.21.15.103:0/3883116553 >> 172.21.15.103:0/3883116553 conn(0x7f3c280fc7d0 msgr2=0x7f3c280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 -- 172.21.15.103:0/3883116553 shutdown_connections 2024-09-19T07:48:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.865+0000 7f3c30367640 1 -- 172.21.15.103:0/3883116553 wait complete. 2024-09-19T07:48:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 Processor -- start 2024-09-19T07:48:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 -- start start 2024-09-19T07:48:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c281983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c28101410 0x7f3c28198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c28106a70 0x7f3c2819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28111f90 con 0x7f3c28100a70 2024-09-19T07:48:07.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28111e10 con 0x7f3c28106a70 2024-09-19T07:48:07.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c30367640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28112110 con 0x7f3c28101410 2024-09-19T07:48:07.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c281983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c281983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45884/0 (socket says 172.21.15.103:45884) 2024-09-19T07:48:07.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 -- 172.21.15.103:0/2326085467 learned_addr learned my addr 172.21.15.103:0/2326085467 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:07.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2d902640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c28101410 0x7f3c28198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e904640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c28106a70 0x7f3c2819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 -- 172.21.15.103:0/2326085467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c28101410 msgr2=0x7f3c28198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c28101410 0x7f3c28198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 -- 172.21.15.103:0/2326085467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c28106a70 msgr2=0x7f3c2819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c28106a70 0x7f3c2819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2e103640 1 -- 172.21.15.103:0/2326085467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2819d370 con 0x7f3c28100a70 2024-09-19T07:48:07.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.866+0000 7f3c2d902640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c28101410 0x7f3c28198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:07.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.867+0000 7f3c2e103640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c281983c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c920 tx=0x7f3c2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:07.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.867+0000 7f3c1f7fe640 1 -- 172.21.15.103:0/2326085467 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24007c70 con 0x7f3c28100a70 2024-09-19T07:48:07.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.867+0000 7f3c1f7fe640 1 -- 172.21.15.103:0/2326085467 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c24007e10 con 0x7f3c28100a70 2024-09-19T07:48:07.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.867+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2819d660 con 0x7f3c28100a70 2024-09-19T07:48:07.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.867+0000 7f3c1f7fe640 1 -- 172.21.15.103:0/2326085467 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24018890 con 0x7f3c28100a70 2024-09-19T07:48:07.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.867+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c281a4f40 con 0x7f3c28100a70 2024-09-19T07:48:07.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.869+0000 7f3c1f7fe640 1 -- 172.21.15.103:0/2326085467 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3c24018a70 con 0x7f3c28100a70 2024-09-19T07:48:07.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.869+0000 7f3c32740740 1 Processor -- start 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c32740740 1 -- start start 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c32740740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c88090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c32740740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c885d0 0x561fa2c88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c32740740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c32740740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fa2bee2f0 con 0x561fa2c93370 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c32740740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fa2b95cf0 con 0x561fa2c89b40 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c32740740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fa2bea2b0 con 0x561fa2c885d0 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c2d902640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c2e904640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c88090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c2d902640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c95760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45886/0 (socket says 172.21.15.103:45886) 2024-09-19T07:48:07.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c2d902640 1 -- 172.21.15.103:0/38294044 learned_addr learned my addr 172.21.15.103:0/38294044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:07.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c2e103640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c885d0 0x561fa2c88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c1f7fe640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3c140a33b0 0x7f3c140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c2d902640 1 -- 172.21.15.103:0/38294044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c885d0 msgr2=0x561fa2c88a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.870+0000 7f3c2d902640 1 --2- 172.21.15.103:0/38294044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c885d0 0x561fa2c88a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c2d902640 1 -- 172.21.15.103:0/38294044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 msgr2=0x561fa2c88090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c2d902640 1 --2- 172.21.15.103:0/38294044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c88090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c2d902640 1 -- 172.21.15.103:0/38294044 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa2b7a3a0 con 0x561fa2c93370 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c2e904640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3c140a33b0 0x7f3c140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c1f7fe640 1 -- 172.21.15.103:0/2326085467 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f3c240cfee0 con 0x7f3c28100a70 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c2d902640 1 --2- 172.21.15.103:0/38294044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c95760 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009930 tx=0x7f3c18032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7ca3436d95fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c1cdea640 1 -- 172.21.15.103:0/38294044 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18035050 con 0x561fa2c93370 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c1cdea640 1 -- 172.21.15.103:0/38294044 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c18004030 con 0x561fa2c93370 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c1cdea640 1 -- 172.21.15.103:0/38294044 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180055a0 con 0x561fa2c93370 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.871+0000 7f3c2e904640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3c140a33b0 0x7f3c140a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c20005fd0 tx=0x7f3c20004a40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 -- 172.21.15.103:0/38294044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 msgr2=0x561fa2c95760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 --2- 172.21.15.103:0/38294044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c95760 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009930 tx=0x7f3c18032ce0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 -- 172.21.15.103:0/38294044 shutdown_connections 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 --2- 172.21.15.103:0/38294044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c95760 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 --2- 172.21.15.103:0/38294044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c885d0 0x561fa2c88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 --2- 172.21.15.103:0/38294044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 -- 172.21.15.103:0/38294044 >> 172.21.15.103:0/38294044 conn(0x561fa2c51310 msgr2=0x561fa2c84c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 -- 172.21.15.103:0/38294044 shutdown_connections 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 -- 172.21.15.103:0/38294044 wait complete. 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 -- start start 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.872+0000 7f3c32740740 1 Processor -- start 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c885d0 0x561fa2c92d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fa2c95fb0 con 0x561fa2c93370 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561fa2ba9060 con 0x561fa2c885d0 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fa2c96130 con 0x561fa2c89b40 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 10 client.0 _ll_register_callbacks cb 0x561fa28fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c905d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45890/0 (socket says 172.21.15.103:45890) 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 -- 172.21.15.103:0/4125614265 learned_addr learned my addr 172.21.15.103:0/4125614265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e103640 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c885d0 0x561fa2c92d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2d902640 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 -- 172.21.15.103:0/4125614265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 msgr2=0x561fa2c8ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c8ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 -- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c885d0 msgr2=0x561fa2c92d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c885d0 0x561fa2c92d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 -- 172.21.15.103:0/4125614265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa295f3c0 con 0x561fa2c93370 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c2e904640 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c905d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3c20013a00 tx=0x7f3c20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42484ab7a722ca21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c11ffb640 1 -- 172.21.15.103:0/4125614265 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001ed70 con 0x561fa2c93370 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c11ffb640 1 -- 172.21.15.103:0/4125614265 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f3c2001f450 con 0x561fa2c93370 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.873+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c18004030 con 0x561fa2c93370 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c11ffb640 1 -- 172.21.15.103:0/4125614265 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3c2001f650 con 0x561fa2c93370 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 10 client.17472 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561fa2ca1a70 con 0x561fa2c93370 2024-09-19T07:48:07.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c11ffb640 1 -- 172.21.15.103:0/4125614265 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c18004030 con 0x561fa2c93370 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c11ffb640 1 -- 172.21.15.103:0/4125614265 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f3c2002e020 con 0x561fa2c93370 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 10 client.17472 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 20 client.17472 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2ca2f30 0x561fa2ca5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 4 client.17472 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fa2bfb860 con 0x561fa2ca2f30 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.874+0000 7f3c2e103640 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2ca2f30 0x561fa2ca5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.875+0000 7f3c2e103640 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2ca2f30 0x561fa2ca5360 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d104449e9a5c5834 server_cookie=9577b9baf8a8c5ea in_seq=0 out_seq=0 2024-09-19T07:48:07.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.875+0000 7f3c11ffb640 10 client.17472 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c11ffb640 1 -- 172.21.15.103:0/4125614265 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fa2bfb860 con 0x561fa2ca2f30 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c11ffb640 10 client.17472 handle_command_reply: tid=0 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 client.17472 shutdown 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 msgr2=0x561fa2c905d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c905d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3c20013a00 tx=0x7f3c20004ab0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 shutdown_connections 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2ca2f30 0x561fa2ca5360 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c93370 0x561fa2c905d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c8ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c11ffb640 0 client.17472 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 --2- 172.21.15.103:0/4125614265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c885d0 0x561fa2c92d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.876+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 >> 172.21.15.103:0/4125614265 conn(0x561fa2c51310 msgr2=0x561fa2c8a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:07.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.877+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 shutdown_connections 2024-09-19T07:48:07.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.877+0000 7f3c32740740 1 -- 172.21.15.103:0/4125614265 wait complete. 2024-09-19T07:48:07.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.877+0000 7f3c32740740 20 client.17472 trim_cache size 0 max 16384 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 Processor -- start 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 -- start start 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c94760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c93370 0x561fa2c94ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281ab730 con 0x561fa2c885d0 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281ad2a0 con 0x561fa2c89b40 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c32740740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281abe30 con 0x561fa2c93370 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c2e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c2e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45902/0 (socket says 172.21.15.103:45902) 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c2e103640 1 -- 172.21.15.103:0/862637690 learned_addr learned my addr 172.21.15.103:0/862637690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:07.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.890+0000 7f3c2d902640 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c94760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c2e904640 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c93370 0x561fa2c94ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c2e103640 1 -- 172.21.15.103:0/862637690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c93370 msgr2=0x561fa2c94ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c2e103640 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c93370 0x561fa2c94ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c2e103640 1 -- 172.21.15.103:0/862637690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 msgr2=0x561fa2c94760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c2e103640 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c94760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c2e103640 1 -- 172.21.15.103:0/862637690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa2c84ed0 con 0x561fa2c885d0 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c2e103640 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9f250 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c8f0 tx=0x7f3c240d3a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961fb2e79ded669 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c13fff640 1 -- 172.21.15.103:0/862637690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2409c880 con 0x561fa2c885d0 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c13fff640 1 -- 172.21.15.103:0/862637690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2409c180 con 0x561fa2c885d0 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.891+0000 7f3c13fff640 1 -- 172.21.15.103:0/862637690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240cd870 con 0x561fa2c885d0 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 -- 172.21.15.103:0/862637690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 msgr2=0x561fa2c9f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9f250 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c8f0 tx=0x7f3c240d3a30 comp rx=0 tx=0).stop 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 -- 172.21.15.103:0/862637690 shutdown_connections 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c93370 0x561fa2c94ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c89b40 0x561fa2c94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 --2- 172.21.15.103:0/862637690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9f250 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 -- 172.21.15.103:0/862637690 >> 172.21.15.103:0/862637690 conn(0x561fa2c51310 msgr2=0x561fa2c8ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 -- 172.21.15.103:0/862637690 shutdown_connections 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 -- 172.21.15.103:0/862637690 wait complete. 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 -- start start 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.892+0000 7f3c32740740 1 Processor -- start 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c32740740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c32740740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c32740740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c93370 0x561fa2c96380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c32740740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281acbd0 con 0x561fa2c885d0 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c32740740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281ac500 con 0x561fa2c93370 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c32740740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281a9730 con 0x561fa2c89b40 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c32740740 10 client.0 _ll_register_callbacks cb 0x7f3c20009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45912/0 (socket says 172.21.15.103:45912) 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 -- 172.21.15.103:0/2423382952 learned_addr learned my addr 172.21.15.103:0/2423382952 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2d902640 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e904640 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c93370 0x561fa2c96380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 -- 172.21.15.103:0/2423382952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 msgr2=0x561fa2c9b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c9b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 -- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c93370 msgr2=0x561fa2c96380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:07.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c93370 0x561fa2c96380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 -- 172.21.15.103:0/2423382952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa2c9c000 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c2e103640 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9b3d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3c24007590 tx=0x7f3c24098420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9442e50716b5fff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.893+0000 7f3c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c11ffb640 1 -- 172.21.15.103:0/2423382952 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2409c880 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c11ffb640 1 -- 172.21.15.103:0/2423382952 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f3c240cd870 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2409c180 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c11ffb640 1 -- 172.21.15.103:0/2423382952 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3c24018d60 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 10 client.17478 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561fa2d0e1f0 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c11ffb640 1 -- 172.21.15.103:0/2423382952 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2409c180 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c11ffb640 1 -- 172.21.15.103:0/2423382952 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f3c240bc0b0 con 0x561fa2c885d0 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 10 client.17478 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:07.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 20 client.17478 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2d0e810 0x561fa2d10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 4 client.17478 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fa2bf5d70 con 0x561fa2d0e810 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.894+0000 7f3c2d902640 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2d0e810 0x561fa2d10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c2d902640 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2d0e810 0x561fa2d10c30 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e375597ace660d server_cookie=4b6636c9999646af in_seq=0 out_seq=0 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c11ffb640 10 client.17478 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c11ffb640 1 -- 172.21.15.103:0/2423382952 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fa2bf5d70 con 0x561fa2d0e810 2024-09-19T07:48:07.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c11ffb640 10 client.17478 handle_command_reply: tid=0 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 client.17478 shutdown 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 msgr2=0x561fa2c9b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9b3d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3c24007590 tx=0x7f3c24098420 comp rx=0 tx=0).stop 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 shutdown_connections 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561fa2d0e810 0x561fa2d10c30 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561fa2c93370 0x561fa2c96380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fa2c89b40 0x561fa2c9b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 --2- 172.21.15.103:0/2423382952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fa2c885d0 0x561fa2c9b3d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 >> 172.21.15.103:0/2423382952 conn(0x561fa2c51310 msgr2=0x561fa2c91840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:07.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c11ffb640 0 client.17478 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:07.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 shutdown_connections 2024-09-19T07:48:07.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 1 -- 172.21.15.103:0/2423382952 wait complete. 2024-09-19T07:48:07.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.895+0000 7f3c32740740 20 client.17478 trim_cache size 0 max 16384 2024-09-19T07:48:07.906 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:07.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3c140a33b0 msgr2=0x7f3c140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3c140a33b0 0x7f3c140a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3c20005fd0 tx=0x7f3c20004a40 comp rx=0 tx=0).stop 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 msgr2=0x7f3c281983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c281983c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c920 tx=0x7f3c2400cde0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 shutdown_connections 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3c140a33b0 0x7f3c140a5870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c28106a70 0x7f3c2819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c28101410 0x7f3c28198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 --2- 172.21.15.103:0/2326085467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28100a70 0x7f3c281983c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:07.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 >> 172.21.15.103:0/2326085467 conn(0x7f3c280fc7d0 msgr2=0x7f3c280fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:07.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 shutdown_connections 2024-09-19T07:48:07.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:07.896+0000 7f3c30367640 1 -- 172.21.15.103:0/2326085467 wait complete. 2024-09-19T07:48:08.534 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:08 smithi103 ceph-mon[31201]: pgmap v314: 97 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 77 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T07:48:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:08 smithi134 ceph-mon[35414]: pgmap v314: 97 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 77 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T07:48:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:08 smithi174 ceph-mon[35370]: pgmap v314: 97 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 77 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T07:48:09.934 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:48:09.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:48:10.186 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:10.469 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 575.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:10 smithi134 ceph-mon[35414]: pgmap v315: 97 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 77 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T07:48:10.595 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:10 smithi103 ceph-mon[31201]: pgmap v315: 97 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 77 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T07:48:10.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:10 smithi174 ceph-mon[35370]: pgmap v315: 97 pgs: 7 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 77 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.679+0000 7f9eefb5e640 1 -- 172.21.15.103:0/3961683238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8102d40 msgr2=0x7f9ee8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.679+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/3961683238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8102d40 0x7f9ee8103120 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009900 tx=0x7f9ee402a480 comp rx=0 tx=0).stop 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.679+0000 7f9eefb5e640 1 -- 172.21.15.103:0/3961683238 shutdown_connections 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.679+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/3961683238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ee810c170 0x7f9ee810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.679+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/3961683238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee8103660 0x7f9ee810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.679+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/3961683238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8102d40 0x7f9ee8103120 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.679+0000 7f9eefb5e640 1 -- 172.21.15.103:0/3961683238 >> 172.21.15.103:0/3961683238 conn(0x7f9ee80fc7a0 msgr2=0x7f9ee80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 -- 172.21.15.103:0/3961683238 shutdown_connections 2024-09-19T07:48:10.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 -- 172.21.15.103:0/3961683238 wait complete. 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 Processor -- start 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 -- start start 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ee8102d40 0x7f9ee81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8103660 0x7f9ee8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee810c170 0x7f9ee8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8111f70 con 0x7f9ee8103660 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8111df0 con 0x7f9ee810c170 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eefb5e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81120f0 con 0x7f9ee8102d40 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eed0f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8103660 0x7f9ee8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eed0f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8103660 0x7f9ee8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34264/0 (socket says 172.21.15.103:34264) 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.680+0000 7f9eed0f9640 1 -- 172.21.15.103:0/417503574 learned_addr learned my addr 172.21.15.103:0/417503574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:10.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ee8102d40 0x7f9ee81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee810c170 0x7f9ee8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eed0f9640 1 -- 172.21.15.103:0/417503574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ee8102d40 msgr2=0x7f9ee81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ee8102d40 0x7f9ee81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eed0f9640 1 -- 172.21.15.103:0/417503574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee810c170 msgr2=0x7f9ee8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee810c170 0x7f9ee8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eed0f9640 1 -- 172.21.15.103:0/417503574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee81a68d0 con 0x7f9ee8103660 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8103660 0x7f9ee8100ee0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800c940 tx=0x7f9ed800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9edeffd640 1 -- 172.21.15.103:0/417503574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8004220 con 0x7f9ee8103660 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9edeffd640 1 -- 172.21.15.103:0/417503574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed80043c0 con 0x7f9ee8103660 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee81a6bc0 con 0x7f9ee8103660 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9edeffd640 1 -- 172.21.15.103:0/417503574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8004a00 con 0x7f9ee8103660 2024-09-19T07:48:10.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.681+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee81a7100 con 0x7f9ee8103660 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.684+0000 7f9ef1f37740 1 Processor -- start 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.684+0000 7f9ef1f37740 1 -- start start 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.684+0000 7f9edeffd640 1 -- 172.21.15.103:0/417503574 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f9ed80079b0 con 0x7f9ee8103660 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.684+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2249f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.684+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce2254800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce2257130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562ce21af280 con 0x562ce224a480 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562ce2156b10 con 0x562ce2249b60 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ce21ab240 con 0x562ce2254d40 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce2254800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce2254800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34266/0 (socket says 172.21.15.103:34266) 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 -- 172.21.15.103:0/652935773 learned_addr learned my addr 172.21.15.103:0/652935773 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:10.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2249f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce2257130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 -- 172.21.15.103:0/652935773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 msgr2=0x562ce2257130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce2257130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 -- 172.21.15.103:0/652935773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 msgr2=0x562ce2249f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2249f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 -- 172.21.15.103:0/652935773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce21631f0 con 0x562ce224a480 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce2257130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9edeffd640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ed40a33b0 0x7f9ed40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ed40a33b0 0x7f9ed40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9edeffd640 1 -- 172.21.15.103:0/417503574 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f9ed80cfd60 con 0x7f9ee8103660 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce2254800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009900 tx=0x7f9ee4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b24afbefe8adde3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9ed3fff640 1 -- 172.21.15.103:0/652935773 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4040070 con 0x562ce224a480 2024-09-19T07:48:10.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.685+0000 7f9ed3fff640 1 -- 172.21.15.103:0/652935773 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4005020 con 0x562ce224a480 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ed3fff640 1 -- 172.21.15.103:0/652935773 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4035040 con 0x562ce224a480 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 -- 172.21.15.103:0/652935773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 msgr2=0x562ce2254800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ed40a33b0 0x7f9ed40a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005fd0 tx=0x7f9ee00096e0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce2254800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009900 tx=0x7f9ee4031630 comp rx=0 tx=0).stop 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 -- 172.21.15.103:0/652935773 shutdown_connections 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce2257130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce2254800 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/652935773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2249f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 -- 172.21.15.103:0/652935773 >> 172.21.15.103:0/652935773 conn(0x562ce22460f0 msgr2=0x562ce22464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 -- 172.21.15.103:0/652935773 shutdown_connections 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.686+0000 7f9ef1f37740 1 -- 172.21.15.103:0/652935773 wait complete. 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 -- start start 2024-09-19T07:48:10.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 Processor -- start 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2251b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce22520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce22525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562ce2257910 con 0x562ce224a480 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562ce2169f10 con 0x562ce2249b60 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ce2257a90 con 0x562ce2254d40 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9ef1f37740 10 client.0 _ll_register_callbacks cb 0x562ce2132a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9eed0f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce22520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9eed0f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce22520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34276/0 (socket says 172.21.15.103:34276) 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9eed0f9640 1 -- 172.21.15.103:0/1233245861 learned_addr learned my addr 172.21.15.103:0/1233245861 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.687+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2251b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce22525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eed0f9640 1 -- 172.21.15.103:0/1233245861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 msgr2=0x562ce22525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce22525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eed0f9640 1 -- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 msgr2=0x562ce2251b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2251b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eed0f9640 1 -- 172.21.15.103:0/1233245861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce1f206a0 con 0x562ce224a480 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2251b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce22520a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed809bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bc432014c78dd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ed17fa640 1 -- 172.21.15.103:0/1233245861 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed801d040 con 0x562ce224a480 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ed17fa640 1 -- 172.21.15.103:0/1233245861 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f9ed8098670 con 0x562ce224a480 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee4005020 con 0x562ce224a480 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ed17fa640 1 -- 172.21.15.103:0/1233245861 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9ed80a03b0 con 0x562ce224a480 2024-09-19T07:48:10.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ef1f37740 10 client.17490 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562ce22633d0 con 0x562ce224a480 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.688+0000 7f9ed17fa640 1 -- 172.21.15.103:0/1233245861 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4005020 con 0x562ce224a480 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ed17fa640 1 -- 172.21.15.103:0/1233245861 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f9ed80a47e0 con 0x562ce224a480 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ef1f37740 10 client.17490 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ef1f37740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ef1f37740 20 client.17490 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22647e0 0x562ce2266c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ef1f37740 4 client.17490 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce21bc7f0 con 0x562ce22647e0 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22647e0 0x562ce2266c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9ed17fa640 10 client.17490 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:10.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.689+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22647e0 0x562ce2266c50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd4db55afc47d344 server_cookie=8e32622862cd769c in_seq=0 out_seq=0 2024-09-19T07:48:10.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ed17fa640 1 -- 172.21.15.103:0/1233245861 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ce21bc7f0 con 0x562ce22647e0 2024-09-19T07:48:10.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ed17fa640 10 client.17490 handle_command_reply: tid=0 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 client.17490 shutdown 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 msgr2=0x562ce22520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce22520a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed809bbc0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 shutdown_connections 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22647e0 0x562ce2266c50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2254d40 0x562ce22525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce22520a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/1233245861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce2251b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ed17fa640 0 client.17490 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 >> 172.21.15.103:0/1233245861 conn(0x562ce22460f0 msgr2=0x562ce224be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 shutdown_connections 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 1 -- 172.21.15.103:0/1233245861 wait complete. 2024-09-19T07:48:10.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.690+0000 7f9ef1f37740 20 client.17490 trim_cache size 0 max 16384 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.707+0000 7f9ef1f37740 1 Processor -- start 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.707+0000 7f9ef1f37740 1 -- start start 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2249b60 0x562ce224bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce226c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2254d40 0x562ce226ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81afb50 con 0x562ce224a480 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81b16c0 con 0x562ce2254d40 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81b0250 con 0x562ce2249b60 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce226c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce226c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34286/0 (socket says 172.21.15.103:34286) 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 -- 172.21.15.103:0/2987130093 learned_addr learned my addr 172.21.15.103:0/2987130093 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2249b60 0x562ce224bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 -- 172.21.15.103:0/2987130093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2249b60 msgr2=0x562ce224bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2249b60 0x562ce224bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 -- 172.21.15.103:0/2987130093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2254d40 msgr2=0x562ce226ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2254d40 0x562ce226ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 -- 172.21.15.103:0/2987130093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce224a2b0 con 0x562ce224a480 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2249b60 0x562ce224bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce226c4f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005fd0 tx=0x7f9ed80053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeabf888fde6b2f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ed37fe640 1 -- 172.21.15.103:0/2987130093 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed807f030 con 0x562ce224a480 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ed37fe640 1 -- 172.21.15.103:0/2987130093 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8007ca0 con 0x562ce224a480 2024-09-19T07:48:10.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.708+0000 7f9ed37fe640 1 -- 172.21.15.103:0/2987130093 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed801d030 con 0x562ce224a480 2024-09-19T07:48:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2987130093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 msgr2=0x562ce226c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce226c4f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8005fd0 tx=0x7f9ed80053f0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2987130093 shutdown_connections 2024-09-19T07:48:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2254d40 0x562ce226ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce224a480 0x562ce226c4f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2987130093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce2249b60 0x562ce224bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2987130093 >> 172.21.15.103:0/2987130093 conn(0x562ce22460f0 msgr2=0x562ce224b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2987130093 shutdown_connections 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2987130093 wait complete. 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 -- start start 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.709+0000 7f9ef1f37740 1 Processor -- start 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce224ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce224a480 0x562ce224f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ef1f37740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce2254d40 0x562ce224f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81b0ff0 con 0x562ce2254d40 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81b0920 con 0x562ce2249b60 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ef1f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81adb50 con 0x562ce224a480 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ef1f37740 10 client.0 _ll_register_callbacks cb 0x7f9ed8004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce2254d40 0x562ce224f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce2254d40 0x562ce224f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34302/0 (socket says 172.21.15.103:34302) 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 -- 172.21.15.103:0/2250747603 learned_addr learned my addr 172.21.15.103:0/2250747603 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce224a480 0x562ce224f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce224ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 -- 172.21.15.103:0/2250747603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce224a480 msgr2=0x562ce224f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce224a480 0x562ce224f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 -- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 msgr2=0x562ce224ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:10.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce224ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 -- 172.21.15.103:0/2250747603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce22ce620 con 0x562ce2254d40 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eed0f9640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce224a480 0x562ce224f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce224ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9eee0fb640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce2254d40 0x562ce224f700 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9ee001f770 tx=0x7f9ee0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50c5a0387a58c98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.710+0000 7f9ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ed17fa640 1 -- 172.21.15.103:0/2250747603 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee001d840 con 0x562ce2254d40 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ed17fa640 1 -- 172.21.15.103:0/2250747603 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f9ee001d9e0 con 0x562ce2254d40 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed8007ca0 con 0x562ce2254d40 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ed17fa640 1 -- 172.21.15.103:0/2250747603 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9ee001dbe0 con 0x562ce2254d40 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 10 client.17496 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562ce22cf8d0 con 0x562ce2254d40 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ed17fa640 1 -- 172.21.15.103:0/2250747603 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8007ca0 con 0x562ce2254d40 2024-09-19T07:48:10.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ed17fa640 1 -- 172.21.15.103:0/2250747603 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f9ee002e030 con 0x562ce2254d40 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 10 client.17496 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 20 client.17496 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22d00a0 0x562ce22d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 4 client.17496 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ce21b6d00 con 0x562ce22d00a0 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.711+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22d00a0 0x562ce22d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.712+0000 7f9eed8fa640 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22d00a0 0x562ce22d24c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441cd7faf2dd2c05 server_cookie=fb909cf33821cd11 in_seq=0 out_seq=0 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.712+0000 7f9ed17fa640 10 client.17496 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ed17fa640 1 -- 172.21.15.103:0/2250747603 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ce21b6d00 con 0x562ce22d00a0 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ed17fa640 10 client.17496 handle_command_reply: tid=0 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 client.17496 shutdown 2024-09-19T07:48:10.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce2254d40 msgr2=0x562ce224f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce2254d40 0x562ce224f700 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9ee001f770 tx=0x7f9ee0004ab0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 shutdown_connections 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562ce22d00a0 0x562ce22d24c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ce2254d40 0x562ce224f700 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ce224a480 0x562ce224f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ed17fa640 0 client.17496 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 --2- 172.21.15.103:0/2250747603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2249b60 0x562ce224ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 >> 172.21.15.103:0/2250747603 conn(0x562ce22460f0 msgr2=0x562ce224be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 shutdown_connections 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 1 -- 172.21.15.103:0/2250747603 wait complete. 2024-09-19T07:48:10.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.713+0000 7f9ef1f37740 20 client.17496 trim_cache size 0 max 16384 2024-09-19T07:48:10.720 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:48:10.720 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:48:10.720 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "dc914197-b643-458f-b13d-c052109054e4", 2024-09-19T07:48:10.720 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:48:10.720 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ed40a33b0 msgr2=0x7f9ed40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ed40a33b0 0x7f9ed40a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005fd0 tx=0x7f9ee00096e0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8103660 msgr2=0x7f9ee8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8103660 0x7f9ee8100ee0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800c940 tx=0x7f9ed800ce00 comp rx=0 tx=0).stop 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 shutdown_connections 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f9ed40a33b0 0x7f9ed40a5870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee810c170 0x7f9ee8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee8103660 0x7f9ee8100ee0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 --2- 172.21.15.103:0/417503574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ee8102d40 0x7f9ee81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 >> 172.21.15.103:0/417503574 conn(0x7f9ee80fc7a0 msgr2=0x7f9ee80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:10.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 shutdown_connections 2024-09-19T07:48:10.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:10.714+0000 7f9eefb5e640 1 -- 172.21.15.103:0/417503574 wait complete. 2024-09-19T07:48:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:11 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:11 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:11 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:11 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:11 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:11.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:11 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:48:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:11 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:11 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:11 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:11.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:11 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:11 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:11.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:11 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:11 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:11 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:11 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:11.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:11 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:11 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:11.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:11 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:11.696 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 -- 172.21.15.103:0/3780897126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c41002f0 msgr2=0x7f19c41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 --2- 172.21.15.103:0/3780897126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c41002f0 0x7f19c41006d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009930 tx=0x7f19b802a420 comp rx=0 tx=0).stop 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 -- 172.21.15.103:0/3780897126 shutdown_connections 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 --2- 172.21.15.103:0/3780897126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c4106f50 0x7f19c410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 --2- 172.21.15.103:0/3780897126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19c4100ca0 0x7f19c4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 --2- 172.21.15.103:0/3780897126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c41002f0 0x7f19c41006d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 -- 172.21.15.103:0/3780897126 >> 172.21.15.103:0/3780897126 conn(0x7f19c4077210 msgr2=0x7f19c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 -- 172.21.15.103:0/3780897126 shutdown_connections 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 -- 172.21.15.103:0/3780897126 wait complete. 2024-09-19T07:48:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 Processor -- start 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.204+0000 7f19cacea640 1 -- start start 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19cacea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19c41002f0 0x7f19c4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19cacea640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c4100ca0 0x7f19c4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19cacea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c4106f50 0x7f19c419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19cacea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4112080 con 0x7f19c4106f50 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19cacea640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4111f00 con 0x7f19c4100ca0 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19cacea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4112200 con 0x7f19c41002f0 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c4106f50 0x7f19c419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c4106f50 0x7f19c419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34320/0 (socket says 172.21.15.103:34320) 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 -- 172.21.15.103:0/3495161361 learned_addr learned my addr 172.21.15.103:0/3495161361 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c8a86640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19c41002f0 0x7f19c4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 -- 172.21.15.103:0/3495161361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19c41002f0 msgr2=0x7f19c4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19c41002f0 0x7f19c4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c4100ca0 0x7f19c4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 -- 172.21.15.103:0/3495161361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c4100ca0 msgr2=0x7f19c4198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c4100ca0 0x7f19c4198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 -- 172.21.15.103:0/3495161361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c419d5f0 con 0x7f19c4106f50 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c8a86640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19c41002f0 0x7f19c4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c4100ca0 0x7f19c4198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c9287640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c4106f50 0x7f19c419cf10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00b970 tx=0x7f19bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c1ffb640 1 -- 172.21.15.103:0/3495161361 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc019070 con 0x7f19c4106f50 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c1ffb640 1 -- 172.21.15.103:0/3495161361 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19bc00cb40 con 0x7f19c4106f50 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c419d8e0 con 0x7f19c4106f50 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.205+0000 7f19c1ffb640 1 -- 172.21.15.103:0/3495161361 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc007650 con 0x7f19c4106f50 2024-09-19T07:48:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.206+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c41a51c0 con 0x7f19c4106f50 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.208+0000 7f19c1ffb640 1 -- 172.21.15.103:0/3495161361 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f19bc01c070 con 0x7f19c4106f50 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.208+0000 7f19cd0c3740 1 Processor -- start 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.208+0000 7f19cd0c3740 1 -- start start 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bcea8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bceb3680 0x55d8bceb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d8bce0e2e0 con 0x55d8bcea84a0 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d8bcdb5ce0 con 0x55d8bceb3680 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8bce0a2a0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bcea8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c1ffb640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19b40a33b0 0x7f19b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bcea8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34332/0 (socket says 172.21.15.103:34332) 2024-09-19T07:48:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 -- 172.21.15.103:0/3803676428 learned_addr learned my addr 172.21.15.103:0/3803676428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bceb3680 0x55d8bceb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c9287640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19b40a33b0 0x7f19b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c8a86640 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c1ffb640 1 -- 172.21.15.103:0/3495161361 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f19bc0cf2f0 con 0x7f19c4106f50 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 -- 172.21.15.103:0/3803676428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bcea8dc0 msgr2=0x55d8bceb3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 -- 172.21.15.103:0/3803676428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bceb3680 msgr2=0x55d8bceb5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bceb3680 0x55d8bceb5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 -- 172.21.15.103:0/3803676428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bcd9a440 con 0x55d8bcea84a0 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bcea8880 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f19b000af70 tx=0x7f19b000a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c34bfef457778ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19c9287640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19b40a33b0 0x7f19b40a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0ccbc0 tx=0x7f19bc06d420 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19aeffd640 1 -- 172.21.15.103:0/3803676428 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b001b070 con 0x55d8bcea84a0 2024-09-19T07:48:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.209+0000 7f19aeffd640 1 -- 172.21.15.103:0/3803676428 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0016ad0 con 0x55d8bcea84a0 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19aeffd640 1 -- 172.21.15.103:0/3803676428 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b000f040 con 0x55d8bcea84a0 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3803676428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 msgr2=0x55d8bcea8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bcea8880 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f19b000af70 tx=0x7f19b000a1b0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3803676428 shutdown_connections 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bceb3680 0x55d8bceb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3803676428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bcea8880 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3803676428 >> 172.21.15.103:0/3803676428 conn(0x55d8bcea4a30 msgr2=0x55d8bcea4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3803676428 shutdown_connections 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.210+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3803676428 wait complete. 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 -- start start 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 Processor -- start 2024-09-19T07:48:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bceb2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bceb07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d8bceb6250 con 0x55d8bcea8dc0 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d8bcdc9050 con 0x55d8bcea84a0 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8bceb63d0 con 0x55d8bceb3680 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19cd0c3740 10 client.0 _ll_register_callbacks cb 0x55d8bce09160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19c8a86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19c8a86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34334/0 (socket says 172.21.15.103:34334) 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19c8a86640 1 -- 172.21.15.103:0/1321466639 learned_addr learned my addr 172.21.15.103:0/1321466639 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19c3fff640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bceb07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.211+0000 7f19c3fff640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bceb2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19c8a86640 1 -- 172.21.15.103:0/1321466639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 msgr2=0x55d8bceb07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19c8a86640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bceb07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19c8a86640 1 -- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 msgr2=0x55d8bceb2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19c8a86640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bceb2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19c8a86640 1 -- 172.21.15.103:0/1321466639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bcb7f3c0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19c3fff640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bceb2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19c8a86640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb0270 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f19b8031550 tx=0x7f19b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf4a2678dd4969b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19a3fff640 1 -- 172.21.15.103:0/1321466639 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8035040 con 0x55d8bcea8dc0 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19a3fff640 1 -- 172.21.15.103:0/1321466639 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f19b8005020 con 0x55d8bcea8dc0 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b0016ad0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19a3fff640 1 -- 172.21.15.103:0/1321466639 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f19b8005220 con 0x55d8bcea8dc0 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19cd0c3740 10 client.17508 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:12.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.212+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d8bcec1c00 con 0x55d8bcea8dc0 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19a3fff640 1 -- 172.21.15.103:0/1321466639 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0016ad0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19a3fff640 1 -- 172.21.15.103:0/1321466639 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f19b8003730 con 0x55d8bcea8dc0 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19cd0c3740 10 client.17508 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19cd0c3740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19cd0c3740 20 client.17508 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcec30c0 0x55d8bcec5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19cd0c3740 4 client.17508 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8bce1b850 con 0x55d8bcec30c0 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.213+0000 7f19c3fff640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcec30c0 0x55d8bcec5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.214+0000 7f19c3fff640 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcec30c0 0x55d8bcec5520 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c86ae144a3c6e6 server_cookie=d5553ed4c9ce93e3 in_seq=0 out_seq=0 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.214+0000 7f19a3fff640 10 client.17508 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.214+0000 7f19a3fff640 1 -- 172.21.15.103:0/1321466639 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8bce1b850 con 0x55d8bcec30c0 2024-09-19T07:48:12.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.214+0000 7f19a3fff640 10 client.17508 handle_command_reply: tid=0 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.214+0000 7f19cd0c3740 1 client.17508 shutdown 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 msgr2=0x55d8bceb0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb0270 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f19b8031550 tx=0x7f19b8004270 comp rx=0 tx=0).stop 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 shutdown_connections 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcec30c0 0x55d8bcec5520 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bceb07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bceb0270 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19a3fff640 0 client.17508 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/1321466639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bceb2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 >> 172.21.15.103:0/1321466639 conn(0x55d8bcea4a30 msgr2=0x55d8bceaa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 shutdown_connections 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 1 -- 172.21.15.103:0/1321466639 wait complete. 2024-09-19T07:48:12.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.215+0000 7f19cd0c3740 20 client.17508 trim_cache size 0 max 16384 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 Processor -- start 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 -- start start 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bceb21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea8dc0 0x55d8bcebba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bcebbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19c41ada80 con 0x55d8bcea84a0 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19c41af5f0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19c41ae180 con 0x55d8bceb3680 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bceb21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bceb21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34338/0 (socket says 172.21.15.103:34338) 2024-09-19T07:48:12.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 -- 172.21.15.103:0/2571617748 learned_addr learned my addr 172.21.15.103:0/2571617748 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c8a86640 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea8dc0 0x55d8bcebba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 -- 172.21.15.103:0/2571617748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 msgr2=0x55d8bcebbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bcebbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 -- 172.21.15.103:0/2571617748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea8dc0 msgr2=0x55d8bcebba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea8dc0 0x55d8bcebba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 -- 172.21.15.103:0/2571617748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bcec5ba0 con 0x55d8bcea84a0 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c8a86640 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea8dc0 0x55d8bcebba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.232+0000 7f19c3fff640 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bceb21e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f19b000af70 tx=0x7f19b000a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18b004cd357ee81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19ae7fc640 1 -- 172.21.15.103:0/2571617748 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b001b040 con 0x55d8bcea84a0 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19ae7fc640 1 -- 172.21.15.103:0/2571617748 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0016ad0 con 0x55d8bcea84a0 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19ae7fc640 1 -- 172.21.15.103:0/2571617748 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0024450 con 0x55d8bcea84a0 2024-09-19T07:48:12.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 -- 172.21.15.103:0/2571617748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 msgr2=0x55d8bceb21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bceb21e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f19b000af70 tx=0x7f19b000a1b0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 -- 172.21.15.103:0/2571617748 shutdown_connections 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bcebbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea8dc0 0x55d8bcebba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/2571617748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea84a0 0x55d8bceb21e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 -- 172.21.15.103:0/2571617748 >> 172.21.15.103:0/2571617748 conn(0x55d8bcea4a30 msgr2=0x55d8bceb1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 -- 172.21.15.103:0/2571617748 shutdown_connections 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.233+0000 7f19cd0c3740 1 -- 172.21.15.103:0/2571617748 wait complete. 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 -- start start 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 Processor -- start 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bcf24c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bcf251a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bcf29270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19c41aef20 con 0x55d8bcea8dc0 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19c41ae850 con 0x55d8bcea84a0 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19c41aba80 con 0x55d8bceb3680 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19cd0c3740 10 client.0 _ll_register_callbacks cb 0x7f19b8031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19c8a86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bcf251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19c8a86640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bcf251a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34350/0 (socket says 172.21.15.103:34350) 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19c8a86640 1 -- 172.21.15.103:0/3759746754 learned_addr learned my addr 172.21.15.103:0/3759746754 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.234+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bcf24c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c8a86640 1 -- 172.21.15.103:0/3759746754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 msgr2=0x55d8bcf29270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bcf29270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c8a86640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bcf29270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c8a86640 1 -- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 msgr2=0x55d8bcf24c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c8a86640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bcf24c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c8a86640 1 -- 172.21.15.103:0/3759746754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bcf29910 con 0x55d8bcea8dc0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bcf24c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19c8a86640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bcf251a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f19b8003b50 tx=0x7f19b8003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24515a4020b37f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19a3fff640 1 -- 172.21.15.103:0/3759746754 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b805a060 con 0x55d8bcea8dc0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b0016ad0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19a3fff640 1 -- 172.21.15.103:0/3759746754 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8586+0+0 (secure 0 0 0) 0x7f19b80055c0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19a3fff640 1 -- 172.21.15.103:0/3759746754 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f19b8035040 con 0x55d8bcea8dc0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19cd0c3740 10 client.17514 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d8bcf31e80 con 0x55d8bcea8dc0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19a3fff640 1 -- 172.21.15.103:0/3759746754 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b0016ad0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.235+0000 7f19a3fff640 1 -- 172.21.15.103:0/3759746754 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f19b8045ab0 con 0x55d8bcea8dc0 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19cd0c3740 10 client.17514 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19cd0c3740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19cd0c3740 20 client.17514 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcf32600 0x55d8bcf34a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19cd0c3740 4 client.17514 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8bce15d60 con 0x55d8bcf32600 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcf32600 0x55d8bcf34a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19a3fff640 10 client.17514 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.236+0000 7f19c3fff640 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcf32600 0x55d8bcf34a20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a4e9512f13f531 server_cookie=ea84cddbbfbe96a4 in_seq=0 out_seq=0 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19a3fff640 1 -- 172.21.15.103:0/3759746754 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d8bce15d60 con 0x55d8bcf32600 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19a3fff640 10 client.17514 handle_command_reply: tid=0 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 client.17514 shutdown 2024-09-19T07:48:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 msgr2=0x55d8bcf251a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bcf251a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f19b8003b50 tx=0x7f19b8003fd0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 shutdown_connections 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d8bcf32600 0x55d8bcf34a20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bceb3680 0x55d8bcf29270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8bcea8dc0 0x55d8bcf251a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 --2- 172.21.15.103:0/3759746754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d8bcea84a0 0x55d8bcf24c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 >> 172.21.15.103:0/3759746754 conn(0x55d8bcea4a30 msgr2=0x55d8bceb22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19a3fff640 0 client.17514 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 shutdown_connections 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 1 -- 172.21.15.103:0/3759746754 wait complete. 2024-09-19T07:48:12.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.237+0000 7f19cd0c3740 20 client.17514 trim_cache size 0 max 16384 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: "dc914197-b643-458f-b13d-c052109054e4": { 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "dc914197-b643-458f-b13d-c052109054e4", 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:12.245 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:12.246 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:48:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19b40a33b0 msgr2=0x7f19b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19b40a33b0 0x7f19b40a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0ccbc0 tx=0x7f19bc06d420 comp rx=0 tx=0).stop 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c4106f50 msgr2=0x7f19c419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c4106f50 0x7f19c419cf10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00b970 tx=0x7f19bc00be30 comp rx=0 tx=0).stop 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 shutdown_connections 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f19b40a33b0 0x7f19b40a5870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c4106f50 0x7f19c419cf10 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c4100ca0 0x7f19c4198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 --2- 172.21.15.103:0/3495161361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19c41002f0 0x7f19c4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 >> 172.21.15.103:0/3495161361 conn(0x7f19c4077210 msgr2=0x7f19c4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 shutdown_connections 2024-09-19T07:48:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:12.238+0000 7f19cacea640 1 -- 172.21.15.103:0/3495161361 wait complete. 2024-09-19T07:48:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: pgmap v316: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-19T07:48:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:48:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:12.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:12.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:12 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: pgmap v316: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-19T07:48:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:48:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:12.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:12.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:12 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: pgmap v316: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-19T07:48:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:48:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:12 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:12.957 INFO:tasks.cephfs.filesystem:scrub status for tag:dc914197-b643-458f-b13d-c052109054e4 - {'path': '/', 'tag': 'dc914197-b643-458f-b13d-c052109054e4', 'options': 'recursive,force'} 2024-09-19T07:48:14.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:14 smithi134 ceph-mon[35414]: pgmap v317: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-19T07:48:14.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:14 smithi174 ceph-mon[35370]: pgmap v317: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-19T07:48:14.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:14 smithi103 ceph-mon[31201]: pgmap v317: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-19T07:48:15.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 580.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:16.544 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:16 smithi103 ceph-mon[31201]: pgmap v318: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.8 MiB/s wr, 229 op/s 2024-09-19T07:48:16.544 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:16 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:48:16.544 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:16 smithi103 ceph-mon[31201]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:48:16.571 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:16 smithi174 ceph-mon[35370]: pgmap v318: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.8 MiB/s wr, 229 op/s 2024-09-19T07:48:16.571 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:16 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:48:16.571 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:16 smithi174 ceph-mon[35370]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:48:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:16 smithi134 ceph-mon[35414]: pgmap v318: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.8 MiB/s wr, 229 op/s 2024-09-19T07:48:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:16 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:48:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:16 smithi134 ceph-mon[35414]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T07:48:18.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: pgmap v320: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.2 MiB/s wr, 383 op/s 2024-09-19T07:48:18.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:48:18.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:18 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: pgmap v320: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.2 MiB/s wr, 383 op/s 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:48:18.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:48:18.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:18 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: pgmap v320: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.2 MiB/s wr, 383 op/s 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:48:18.582 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:18 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:48:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T07:48:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T07:48:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T07:48:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-19T07:48:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T07:48:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-19T07:48:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:48:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:48:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:48:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:19 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:48:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T07:48:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T07:48:19.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T07:48:19.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-19T07:48:19.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T07:48:19.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-19T07:48:19.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:48:19.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:48:19.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:48:19.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:19 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:48:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: Unable to set osd_memory_target on smithi103 to 184581734: error parsing value: Value '184581734' is below minimum 939524096 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: Adjusting osd_memory_target on smithi134 to 176.0M 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: Unable to set osd_memory_target on smithi134 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:48:19.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:19 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:48:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:20 smithi174 ceph-mon[35370]: pgmap v321: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.2 MiB/s wr, 383 op/s 2024-09-19T07:48:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:20 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:48:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:20 smithi103 ceph-mon[31201]: pgmap v321: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.2 MiB/s wr, 383 op/s 2024-09-19T07:48:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:20 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:48:20.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 585.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:20 smithi134 ceph-mon[35414]: pgmap v321: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.2 MiB/s wr, 383 op/s 2024-09-19T07:48:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:20 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:48:22.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:48:22.361 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:22 smithi103 ceph-mon[31201]: pgmap v322: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.7 MiB/s wr, 275 op/s 2024-09-19T07:48:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:22 smithi134 ceph-mon[35414]: pgmap v322: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.7 MiB/s wr, 275 op/s 2024-09-19T07:48:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:22 smithi174 ceph-mon[35370]: pgmap v322: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.7 MiB/s wr, 275 op/s 2024-09-19T07:48:23.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 -- 172.21.15.103:0/2897830832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 msgr2=0x7f7444101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:23.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 --2- 172.21.15.103:0/2897830832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444101200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7434009900 tx=0x7f743402a480 comp rx=0 tx=0).stop 2024-09-19T07:48:23.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 -- 172.21.15.103:0/2897830832 shutdown_connections 2024-09-19T07:48:23.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 --2- 172.21.15.103:0/2897830832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7444102ef0 0x7f744410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 --2- 172.21.15.103:0/2897830832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7444102570 0x7f74441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 --2- 172.21.15.103:0/2897830832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444101200 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 -- 172.21.15.103:0/2897830832 >> 172.21.15.103:0/2897830832 conn(0x7f74440fc7d0 msgr2=0x7f74440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 -- 172.21.15.103:0/2897830832 shutdown_connections 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 -- 172.21.15.103:0/2897830832 wait complete. 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 Processor -- start 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.827+0000 7f744a059640 1 -- start start 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f744a059640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f744a059640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7444102570 0x7f7444198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f744a059640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7444102ef0 0x7f744419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f744a059640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7444110f80 con 0x7f7444100e20 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f744a059640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7444110e00 con 0x7f7444102ef0 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f744a059640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7444111100 con 0x7f7444102570 2024-09-19T07:48:23.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:23.219 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:48:23.219 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":46,"btime":"2024-09-19T07:48:00:225048+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:47:59.861225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:48:23.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56448/0 (socket says 172.21.15.103:56448) 2024-09-19T07:48:23.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 -- 172.21.15.103:0/2032441387 learned_addr learned my addr 172.21.15.103:0/2032441387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f7442ffd640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7444102570 0x7f7444198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f7443fff640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7444102ef0 0x7f744419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 -- 172.21.15.103:0/2032441387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7444102570 msgr2=0x7f7444198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7444102570 0x7f7444198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 -- 172.21.15.103:0/2032441387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7444102ef0 msgr2=0x7f744419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7444102ef0 0x7f744419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 -- 172.21.15.103:0/2032441387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744419d4b0 con 0x7f7444100e20 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f7443fff640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7444102ef0 0x7f744419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f74437fe640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444198500 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7434006010 tx=0x7f7434003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f7440ff9640 1 -- 172.21.15.103:0/2032441387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743403f070 con 0x7f7444100e20 2024-09-19T07:48:23.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f7440ff9640 1 -- 172.21.15.103:0/2032441387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7434031c30 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.828+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744419d740 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.829+0000 7f7440ff9640 1 -- 172.21.15.103:0/2032441387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434035030 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.829+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74441a5080 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.829+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74441021b0 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.831+0000 7f7440ff9640 1 -- 172.21.15.103:0/2032441387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7434042070 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.831+0000 7f7440ff9640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f74300a33b0 0x7f74300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.831+0000 7f7442ffd640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f74300a33b0 0x7f74300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.832+0000 7f7440ff9640 1 -- 172.21.15.103:0/2032441387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f74340f0590 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.832+0000 7f7442ffd640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f74300a33b0 0x7f74300a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74440fff50 tx=0x7f742c00a2d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:22.835+0000 7f7440ff9640 1 -- 172.21.15.103:0/2032441387 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7434031de0 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.036+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f744410a0a0 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.036+0000 7f7440ff9640 1 -- 172.21.15.103:0/2032441387 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7f74340bcf60 con 0x7f7444100e20 2024-09-19T07:48:23.221 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 46 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f74300a33b0 msgr2=0x7f74300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f74300a33b0 0x7f74300a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f74440fff50 tx=0x7f742c00a2d0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 msgr2=0x7f7444198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444198500 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7434006010 tx=0x7f7434003c10 comp rx=0 tx=0).stop 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 shutdown_connections 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f74300a33b0 0x7f74300a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7444102ef0 0x7f744419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7444102570 0x7f7444198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 --2- 172.21.15.103:0/2032441387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7444100e20 0x7f7444198500 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 >> 172.21.15.103:0/2032441387 conn(0x7f74440fc7d0 msgr2=0x7f744406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 shutdown_connections 2024-09-19T07:48:23.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:23.039+0000 7f744a059640 1 -- 172.21.15.103:0/2032441387 wait complete. 2024-09-19T07:48:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:23 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2032441387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:23 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2032441387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:23.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:23 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2032441387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:23.939 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id a7fefb2f --release --await' 2024-09-19T07:48:23.939 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id a7fefb2f --release --await 2024-09-19T07:48:24.187 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:24 smithi103 ceph-mon[31201]: pgmap v323: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.7 MiB/s wr, 275 op/s 2024-09-19T07:48:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:24 smithi134 ceph-mon[35414]: pgmap v323: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.7 MiB/s wr, 275 op/s 2024-09-19T07:48:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:24 smithi174 ceph-mon[35370]: pgmap v323: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 3.7 MiB/s wr, 275 op/s 2024-09-19T07:48:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 -- 172.21.15.103:0/1878877001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8101280 msgr2=0x7f55f810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/1878877001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8101280 0x7f55f810df40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009900 tx=0x7f55e802a410 comp rx=0 tx=0).stop 2024-09-19T07:48:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 -- 172.21.15.103:0/1878877001 shutdown_connections 2024-09-19T07:48:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/1878877001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f810e480 0x7f55f8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/1878877001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f8101280 0x7f55f810df40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/1878877001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f81008d0 0x7f55f8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 -- 172.21.15.103:0/1878877001 >> 172.21.15.103:0/1878877001 conn(0x7f55f80fc7a0 msgr2=0x7f55f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.893+0000 7f55ffe6f640 1 -- 172.21.15.103:0/1878877001 shutdown_connections 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 -- 172.21.15.103:0/1878877001 wait complete. 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 Processor -- start 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 -- start start 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f81008d0 0x7f55f819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8101280 0x7f55f819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f810e480 0x7f55f819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81141a0 con 0x7f55f81008d0 2024-09-19T07:48:24.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8114020 con 0x7f55f8101280 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55ffe6f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8114320 con 0x7f55f810e480 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55fdc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f81008d0 0x7f55f819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55fdc0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f81008d0 0x7f55f819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56470/0 (socket says 172.21.15.103:56470) 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.894+0000 7f55fdc0b640 1 -- 172.21.15.103:0/3005600722 learned_addr learned my addr 172.21.15.103:0/3005600722 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fe40c640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f810e480 0x7f55f819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fd40a640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8101280 0x7f55f819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fdc0b640 1 -- 172.21.15.103:0/3005600722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f810e480 msgr2=0x7f55f819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f810e480 0x7f55f819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fdc0b640 1 -- 172.21.15.103:0/3005600722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8101280 msgr2=0x7f55f819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8101280 0x7f55f819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fdc0b640 1 -- 172.21.15.103:0/3005600722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819f650 con 0x7f55f81008d0 2024-09-19T07:48:24.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f81008d0 0x7f55f819a6a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c890 tx=0x7f55f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55eeffd640 1 -- 172.21.15.103:0/3005600722 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4004220 con 0x7f55f81008d0 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55eeffd640 1 -- 172.21.15.103:0/3005600722 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f40043c0 con 0x7f55f81008d0 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819f940 con 0x7f55f81008d0 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55eeffd640 1 -- 172.21.15.103:0/3005600722 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4004a00 con 0x7f55f81008d0 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.895+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a7220 con 0x7f55f81008d0 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.897+0000 7f55eeffd640 1 -- 172.21.15.103:0/3005600722 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f55f40079b0 con 0x7f55f81008d0 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.898+0000 7f5602248740 1 Processor -- start 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.898+0000 7f5602248740 1 -- start start 2024-09-19T07:48:24.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.898+0000 7f5602248740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89bf6060 0x559f89bf6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f5602248740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89c00d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f5602248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89c01240 0x559f89c03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f5602248740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f89b5b620 con 0x559f89bf6060 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f5602248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f89b02e50 con 0x559f89c01240 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f5602248740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559f89b575e0 con 0x559f89bf6980 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89bf6060 0x559f89bf6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55eeffd640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f55e40a33b0 0x7f55e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89bf6060 0x559f89bf6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56472/0 (socket says 172.21.15.103:56472) 2024-09-19T07:48:24.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 -- 172.21.15.103:0/2903589423 learned_addr learned my addr 172.21.15.103:0/2903589423 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fd40a640 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89c01240 0x559f89c03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89c00d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f55e40a33b0 0x7f55e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 -- 172.21.15.103:0/2903589423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 msgr2=0x559f89c00d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89c00d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55eeffd640 1 -- 172.21.15.103:0/3005600722 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f55f40cfa90 con 0x7f55f81008d0 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 -- 172.21.15.103:0/2903589423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89c01240 msgr2=0x559f89c03630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.899+0000 7f55fe40c640 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89c01240 0x559f89c03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.900+0000 7f55fe40c640 1 -- 172.21.15.103:0/2903589423 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f89af3d40 con 0x559f89bf6060 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.900+0000 7f55fd40a640 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89c01240 0x559f89c03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:24.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.900+0000 7f55fe40c640 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89bf6060 0x559f89bf6440 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f55f00049d0 tx=0x7f55f0014eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e70ca3bc424bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.900+0000 7f55e3fff640 1 -- 172.21.15.103:0/2903589423 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f001ea80 con 0x559f89bf6060 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.900+0000 7f55e3fff640 1 -- 172.21.15.103:0/2903589423 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0023070 con 0x559f89bf6060 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.900+0000 7f55e3fff640 1 -- 172.21.15.103:0/2903589423 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f001dc50 con 0x559f89bf6060 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 -- 172.21.15.103:0/2903589423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89bf6060 msgr2=0x559f89bf6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89bf6060 0x559f89bf6440 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f55f00049d0 tx=0x7f55f0014eb0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f55fe40c640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f55e40a33b0 0x7f55e40a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f55f0022000 tx=0x7f55f0020000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 -- 172.21.15.103:0/2903589423 shutdown_connections 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89c01240 0x559f89c03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89c00d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 --2- 172.21.15.103:0/2903589423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89bf6060 0x559f89bf6440 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 -- 172.21.15.103:0/2903589423 >> 172.21.15.103:0/2903589423 conn(0x559f89bf25f0 msgr2=0x559f89bf29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 -- 172.21.15.103:0/2903589423 shutdown_connections 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 -- 172.21.15.103:0/2903589423 wait complete. 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 -- start start 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.901+0000 7f5602248740 1 Processor -- start 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f5602248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bfe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f5602248740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bfe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f5602248740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89bfec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f5602248740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f89c03e10 con 0x559f89c01240 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f5602248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f89ae3f10 con 0x559f89bf6060 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f5602248740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559f89c03f90 con 0x559f89bf6980 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f5602248740 10 client.0 _ll_register_callbacks cb 0x559f89b07db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89bfec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89bfec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56482/0 (socket says 172.21.15.103:56482) 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 -- 172.21.15.103:0/378702466 learned_addr learned my addr 172.21.15.103:0/378702466 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bfe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fd40a640 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bfe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 -- 172.21.15.103:0/378702466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 msgr2=0x559f89bfe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bfe6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 -- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 msgr2=0x559f89bfe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bfe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 -- 172.21.15.103:0/378702466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8986a580 con 0x559f89c01240 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55fe40c640 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89bfec30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f55f00049a0 tx=0x7f55f00144d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b081b531e539b8b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.902+0000 7f55e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f55e17fa640 1 -- 172.21.15.103:0/378702466 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f001d350 con 0x559f89c01240 2024-09-19T07:48:24.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f55e17fa640 1 -- 172.21.15.103:0/378702466 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f55f001d4f0 con 0x559f89c01240 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f0023070 con 0x559f89c01240 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f55e17fa640 1 -- 172.21.15.103:0/378702466 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f55f003f050 con 0x559f89c01240 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 10 client.17532 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559f89934960 con 0x559f89c01240 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f55e17fa640 1 -- 172.21.15.103:0/378702466 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0023070 con 0x559f89c01240 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f55e17fa640 1 -- 172.21.15.103:0/378702466 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f55f000f040 con 0x559f89c01240 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 10 client.17532 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 10 client.17532 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 20 client.17532 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c10dd0 0x559f89c131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 4 client.17532 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.903+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f89b68b90 con 0x559f89c10dd0 2024-09-19T07:48:24.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.904+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c10dd0 0x559f89c131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.904+0000 7f55e17fa640 10 client.17532 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.904+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c10dd0 0x559f89c131c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488dc479848648a4 server_cookie=c1f0d0ab4a6729da in_seq=0 out_seq=0 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f55e17fa640 1 -- 172.21.15.103:0/378702466 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f89b68b90 con 0x559f89c10dd0 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f55e17fa640 10 client.17532 handle_command_reply: tid=0 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 client.17532 shutdown 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 msgr2=0x559f89bfec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89bfec30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f55f00049a0 tx=0x7f55f00144d0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 shutdown_connections 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c10dd0 0x559f89c131c0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89bfec30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bfe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 --2- 172.21.15.103:0/378702466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bfe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f55e17fa640 0 client.17532 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:24.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 >> 172.21.15.103:0/378702466 conn(0x559f89bf25f0 msgr2=0x559f89bf82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:24.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 shutdown_connections 2024-09-19T07:48:24.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 1 -- 172.21.15.103:0/378702466 wait complete. 2024-09-19T07:48:24.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.905+0000 7f5602248740 20 client.17532 trim_cache size 0 max 16384 2024-09-19T07:48:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 Processor -- start 2024-09-19T07:48:24.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 -- start start 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bf8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bf1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81afbd0 con 0x559f89c01240 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81b1740 con 0x559f89bf6060 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f5602248740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81b02d0 con 0x559f89bf6980 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c18820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56498/0 (socket says 172.21.15.103:56498) 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 -- 172.21.15.103:0/176906052 learned_addr learned my addr 172.21.15.103:0/176906052 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bf8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fd40a640 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bf1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 -- 172.21.15.103:0/176906052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 msgr2=0x559f89bf1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bf1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 -- 172.21.15.103:0/176906052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 msgr2=0x559f89bf8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bf8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 -- 172.21.15.103:0/176906052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f89bf5e90 con 0x559f89c01240 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bf8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.922+0000 7f55fe40c640 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c18820 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f55f0006010 tx=0x7f55f001df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36b1bd0ac3480ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f55e37fe640 1 -- 172.21.15.103:0/176906052 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0047020 con 0x559f89c01240 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f55e37fe640 1 -- 172.21.15.103:0/176906052 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0023040 con 0x559f89c01240 2024-09-19T07:48:24.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f55e37fe640 1 -- 172.21.15.103:0/176906052 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f001d680 con 0x559f89c01240 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f5602248740 1 -- 172.21.15.103:0/176906052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 msgr2=0x559f89c18820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f5602248740 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c18820 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f55f0006010 tx=0x7f55f001df90 comp rx=0 tx=0).stop 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f5602248740 1 -- 172.21.15.103:0/176906052 shutdown_connections 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f5602248740 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c18820 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f5602248740 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6980 0x559f89bf1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f5602248740 1 --2- 172.21.15.103:0/176906052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6060 0x559f89bf8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.923+0000 7f5602248740 1 -- 172.21.15.103:0/176906052 >> 172.21.15.103:0/176906052 conn(0x559f89bf25f0 msgr2=0x559f89bf0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.924+0000 7f5602248740 1 -- 172.21.15.103:0/176906052 shutdown_connections 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.924+0000 7f5602248740 1 -- 172.21.15.103:0/176906052 wait complete. 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.924+0000 7f5602248740 1 -- start start 2024-09-19T07:48:24.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.924+0000 7f5602248740 1 Processor -- start 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.924+0000 7f5602248740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6060 0x559f89c055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.924+0000 7f5602248740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6980 0x559f89c05b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.924+0000 7f5602248740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c83000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f5602248740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81b1070 con 0x559f89c01240 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f5602248740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81b09a0 con 0x559f89bf6980 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f5602248740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81adbd0 con 0x559f89bf6060 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c83000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f5602248740 10 client.0 _ll_register_callbacks cb 0x7f55f001ee40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fd40a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6980 0x559f89c05b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fe40c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c83000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56500/0 (socket says 172.21.15.103:56500) 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fdc0b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6060 0x559f89c055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fe40c640 1 -- 172.21.15.103:0/3606826498 learned_addr learned my addr 172.21.15.103:0/3606826498 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fd40a640 1 -- 172.21.15.103:0/3606826498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6060 msgr2=0x559f89c055d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fd40a640 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6060 0x559f89c055d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fd40a640 1 -- 172.21.15.103:0/3606826498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 msgr2=0x559f89c83000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fd40a640 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c83000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fd40a640 1 -- 172.21.15.103:0/3606826498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f89c83540 con 0x559f89bf6980 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fe40c640 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55fd40a640 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6980 0x559f89c05b10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009230 tx=0x7f55e8031920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=667924f41cef3be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.925+0000 7f55e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.926+0000 7f55e17fa640 1 -- 172.21.15.103:0/3606826498 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8035050 con 0x559f89bf6980 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.926+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f0023040 con 0x559f89bf6980 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.926+0000 7f55e17fa640 1 -- 172.21.15.103:0/3606826498 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f55e8004250 con 0x559f89bf6980 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.926+0000 7f55e17fa640 1 -- 172.21.15.103:0/3606826498 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f55e8004470 con 0x559f89bf6980 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.926+0000 7f5602248740 10 client.26951 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:24.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.926+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x559f89c84830 con 0x559f89bf6980 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.926+0000 7f55e17fa640 1 -- 172.21.15.103:0/3606826498 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0023040 con 0x559f89bf6980 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f55e17fa640 1 -- 172.21.15.103:0/3606826498 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f55e8004ae0 con 0x559f89bf6980 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f5602248740 10 client.26951 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f5602248740 10 client.26951 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f5602248740 20 client.26951 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f5602248740 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c84fb0 0x559f89c873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f5602248740 4 client.26951 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a7fefb2f", "release": true, "await": true}] 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a7fefb2f", "release": true, "await": true}) -- 0x559f89b630a0 con 0x559f89c84fb0 2024-09-19T07:48:24.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c84fb0 0x559f89c873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f55fdc0b640 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c84fb0 0x559f89c873d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694bdd54c231ec2d server_cookie=f1b20ddc7a93a14b in_seq=0 out_seq=0 2024-09-19T07:48:24.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.927+0000 7f55e17fa640 10 client.26951 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:24.976 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 46, 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 36, 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "a7fefb2f": { 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 36, 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:48:24.977 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:48:24.978 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T07:48:24.978 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:24.981 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:48:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f55e17fa640 1 -- 172.21.15.103:0/3606826498 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559f89b630a0 con 0x559f89c84fb0 2024-09-19T07:48:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f55e17fa640 10 client.26951 handle_command_reply: tid=0 2024-09-19T07:48:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 client.26951 shutdown 2024-09-19T07:48:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6980 msgr2=0x559f89c05b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6980 0x559f89c05b10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009230 tx=0x7f55e8031920 comp rx=0 tx=0).stop 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 shutdown_connections 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559f89c84fb0 0x559f89c873d0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f89c01240 0x559f89c83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f89bf6980 0x559f89c05b10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f55e17fa640 0 client.26951 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 --2- 172.21.15.103:0/3606826498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f89bf6060 0x559f89c055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 >> 172.21.15.103:0/3606826498 conn(0x559f89bf25f0 msgr2=0x559f89bfe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 shutdown_connections 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 1 -- 172.21.15.103:0/3606826498 wait complete. 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.975+0000 7f5602248740 20 client.26951 trim_cache size 0 max 16384 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f55e40a33b0 msgr2=0x7f55e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f55e40a33b0 0x7f55e40a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f55f0022000 tx=0x7f55f0020000 comp rx=0 tx=0).stop 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f81008d0 msgr2=0x7f55f819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f81008d0 0x7f55f819a6a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c890 tx=0x7f55f400cd50 comp rx=0 tx=0).stop 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 shutdown_connections 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f55e40a33b0 0x7f55e40a5870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55f810e480 0x7f55f819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f8101280 0x7f55f819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 --2- 172.21.15.103:0/3005600722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55f81008d0 0x7f55f819a6a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 >> 172.21.15.103:0/3005600722 conn(0x7f55f80fc7a0 msgr2=0x7f55f80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 shutdown_connections 2024-09-19T07:48:24.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:24.976+0000 7f55ffe6f640 1 -- 172.21.15.103:0/3005600722 wait complete. 2024-09-19T07:48:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 590.0 s, 140.7 tps, lat 4180.797 ms stddev 10137.145 2024-09-19T07:48:25.696 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a7fefb2f', total seconds elapsed: 33.5 2024-09-19T07:48:25.696 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T07:48:25.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:48:26.083 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:26 smithi103 ceph-mon[31201]: pgmap v324: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 506 op/s 2024-09-19T07:48:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:26 smithi134 ceph-mon[35414]: pgmap v324: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 506 op/s 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 -- 172.21.15.103:0/3302956442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c109450 msgr2=0x7f666c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 --2- 172.21.15.103:0/3302956442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c109450 0x7f666c110250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f665c01a910 tx=0x7f665c03b450 comp rx=0 tx=0).stop 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 -- 172.21.15.103:0/3302956442 shutdown_connections 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 --2- 172.21.15.103:0/3302956442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c11b140 0x7f666c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 --2- 172.21.15.103:0/3302956442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c109830 0x7f666c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 --2- 172.21.15.103:0/3302956442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c109450 0x7f666c110250 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 -- 172.21.15.103:0/3302956442 >> 172.21.15.103:0/3302956442 conn(0x7f666c06e140 msgr2=0x7f666c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 -- 172.21.15.103:0/3302956442 shutdown_connections 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.639+0000 7f6673c43640 1 -- 172.21.15.103:0/3302956442 wait complete. 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 Processor -- start 2024-09-19T07:48:26.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 -- start start 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c109450 0x7f666c11a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c109830 0x7f666c116820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c11b140 0x7f666c116d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f666c113320 con 0x7f666c109450 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f666c1131a0 con 0x7f666c109830 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f6673c43640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f666c1134a0 con 0x7f666c11b140 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.640+0000 7f66719df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c109450 0x7f666c11a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.641+0000 7f66719df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c109450 0x7f666c11a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56526/0 (socket says 172.21.15.103:56526) 2024-09-19T07:48:26.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.641+0000 7f66719df640 1 -- 172.21.15.103:0/2226141957 learned_addr learned my addr 172.21.15.103:0/2226141957 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.643+0000 7f66721e0640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c11b140 0x7f666c116d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.643+0000 7f66719df640 1 -- 172.21.15.103:0/2226141957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c11b140 msgr2=0x7f666c116d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.643+0000 7f66719df640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c11b140 0x7f666c116d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.643+0000 7f66719df640 1 -- 172.21.15.103:0/2226141957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c109830 msgr2=0x7f666c116820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.643+0000 7f66719df640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c109830 0x7f666c116820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.643+0000 7f66719df640 1 -- 172.21.15.103:0/2226141957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c117320 con 0x7f666c109450 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.643+0000 7f66721e0640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c11b140 0x7f666c116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:26.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.646+0000 7f66719df640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c109450 0x7f666c11a780 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f665c006010 tx=0x7f665c0056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:26.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.650+0000 7f6662ffd640 1 -- 172.21.15.103:0/2226141957 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c04f020 con 0x7f666c109450 2024-09-19T07:48:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.650+0000 7f6662ffd640 1 -- 172.21.15.103:0/2226141957 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c004ca0 con 0x7f666c109450 2024-09-19T07:48:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.650+0000 7f6662ffd640 1 -- 172.21.15.103:0/2226141957 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c046030 con 0x7f666c109450 2024-09-19T07:48:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.650+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c1175b0 con 0x7f666c109450 2024-09-19T07:48:26.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.650+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c07c6f0 con 0x7f666c109450 2024-09-19T07:48:26.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.652+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66500052c0 con 0x7f666c109450 2024-09-19T07:48:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.656+0000 7f6662ffd640 1 -- 172.21.15.103:0/2226141957 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f665c052020 con 0x7f666c109450 2024-09-19T07:48:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.657+0000 7f6662ffd640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66580a35c0 0x7f66580a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:26.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.658+0000 7f6662ffd640 1 -- 172.21.15.103:0/2226141957 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f665c101320 con 0x7f666c109450 2024-09-19T07:48:26.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:26 smithi174 ceph-mon[35370]: pgmap v324: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 506 op/s 2024-09-19T07:48:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.662+0000 7f66711de640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66580a35c0 0x7f66580a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:26.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.664+0000 7f6662ffd640 1 -- 172.21.15.103:0/2226141957 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f665c059020 con 0x7f666c109450 2024-09-19T07:48:26.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.668+0000 7f66711de640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66580a35c0 0x7f66580a5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f666c117fe0 tx=0x7f6664009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:26.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.998+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6650005880 con 0x7f666c109450 2024-09-19T07:48:27.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:26.999+0000 7f6662ffd640 1 -- 172.21.15.103:0/2226141957 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7f665c0054d0 con 0x7f666c109450 2024-09-19T07:48:27.000 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:48:27.000 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":46,"btime":"2024-09-19T07:48:00:225048+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:47:59.861225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:48:27.001 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 46 2024-09-19T07:48:27.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66580a35c0 msgr2=0x7f66580a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:27.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66580a35c0 0x7f66580a5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f666c117fe0 tx=0x7f6664009500 comp rx=0 tx=0).stop 2024-09-19T07:48:27.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c109450 msgr2=0x7f666c11a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:27.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c109450 0x7f666c11a780 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f665c006010 tx=0x7f665c0056b0 comp rx=0 tx=0).stop 2024-09-19T07:48:27.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 shutdown_connections 2024-09-19T07:48:27.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f66580a35c0 0x7f66580a5a80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:27.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f666c11b140 0x7f666c116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:27.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f666c109830 0x7f666c116820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:27.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 --2- 172.21.15.103:0/2226141957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f666c109450 0x7f666c11a780 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:27.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 >> 172.21.15.103:0/2226141957 conn(0x7f666c06e140 msgr2=0x7f666c070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:27.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 shutdown_connections 2024-09-19T07:48:27.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:27.001+0000 7f6673c43640 1 -- 172.21.15.103:0/2226141957 wait complete. 2024-09-19T07:48:27.342 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:27 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2226141957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:27.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:27 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2226141957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:27.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:27 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2226141957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:28.049 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T07:48:28.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T07:48:28.424 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:28 smithi103 ceph-mon[31201]: pgmap v325: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 945 op/s 2024-09-19T07:48:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:28 smithi134 ceph-mon[35414]: pgmap v325: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 945 op/s 2024-09-19T07:48:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:28 smithi174 ceph-mon[35370]: pgmap v325: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 945 op/s 2024-09-19T07:48:29.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.308+0000 7fbea9abb640 1 -- 172.21.15.103:0/2720166490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102570 msgr2=0x7fbea41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.308+0000 7fbea9abb640 1 --2- 172.21.15.103:0/2720166490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102570 0x7fbea41029b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009890 tx=0x7fbe8c02a3f0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 -- 172.21.15.103:0/2720166490 shutdown_connections 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 --2- 172.21.15.103:0/2720166490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbea4102ef0 0x7fbea410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 --2- 172.21.15.103:0/2720166490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102570 0x7fbea41029b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 --2- 172.21.15.103:0/2720166490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 0x7fbea4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 -- 172.21.15.103:0/2720166490 >> 172.21.15.103:0/2720166490 conn(0x7fbea40fc7d0 msgr2=0x7fbea40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 -- 172.21.15.103:0/2720166490 shutdown_connections 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 -- 172.21.15.103:0/2720166490 wait complete. 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 Processor -- start 2024-09-19T07:48:29.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.309+0000 7fbea9abb640 1 -- start start 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea9abb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 0x7fbea4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea9abb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbea4102570 0x7fbea41998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea9abb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102ef0 0x7fbea41a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea9abb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4110f80 con 0x7fbea4102570 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea9abb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4110e00 con 0x7fbea4102ef0 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea9abb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4111100 con 0x7fbea4100e20 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 0x7fbea4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 0x7fbea4199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40700/0 (socket says 172.21.15.103:40700) 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 -- 172.21.15.103:0/3316851659 learned_addr learned my addr 172.21.15.103:0/3316851659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea3fff640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102ef0 0x7fbea41a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbea4102570 0x7fbea41998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 -- 172.21.15.103:0/3316851659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102ef0 msgr2=0x7fbea41a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102ef0 0x7fbea41a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 -- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbea4102570 msgr2=0x7fbea41998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbea4102570 0x7fbea41998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 -- 172.21.15.103:0/3316851659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea41a4930 con 0x7fbea4100e20 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea37fe640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 0x7fbea4199370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400d950 tx=0x7fbe9400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.310+0000 7fbea17fa640 1 -- 172.21.15.103:0/3316851659 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94017070 con 0x7fbea4100e20 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.311+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea41a4b00 con 0x7fbea4100e20 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.311+0000 7fbea17fa640 1 -- 172.21.15.103:0/3316851659 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe94009c30 con 0x7fbea4100e20 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.311+0000 7fbea17fa640 1 -- 172.21.15.103:0/3316851659 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400f6b0 con 0x7fbea4100e20 2024-09-19T07:48:29.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.311+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea41a5010 con 0x7fbea4100e20 2024-09-19T07:48:29.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.314+0000 7fbea17fa640 1 -- 172.21.15.103:0/3316851659 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fbe940109d0 con 0x7fbea4100e20 2024-09-19T07:48:29.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.315+0000 7fbea17fa640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe900a3430 0x7fbe900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.315+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe900a3430 0x7fbe900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.315+0000 7fbea17fa640 1 -- 172.21.15.103:0/3316851659 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fbe940cfbe0 con 0x7fbea4100e20 2024-09-19T07:48:29.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.315+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe900a3430 0x7fbe900a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c035040 tx=0x7fbe8c0056d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:29.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 Processor -- start 2024-09-19T07:48:29.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 -- start start 2024-09-19T07:48:29.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38680 0x564c3fe38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 0x564c3fe47810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe4a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c3fd9e2a0 con 0x564c3fe38680 2024-09-19T07:48:29.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564c3fd45b10 con 0x564c3fe47d50 2024-09-19T07:48:29.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.316+0000 7fbeabe94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c3fd9a260 con 0x564c3fe38fa0 2024-09-19T07:48:29.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.317+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38680 0x564c3fe38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.317+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38680 0x564c3fe38a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47636/0 (socket says 172.21.15.103:47636) 2024-09-19T07:48:29.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.317+0000 7fbea37fe640 1 -- 172.21.15.103:0/1028434108 learned_addr learned my addr 172.21.15.103:0/1028434108 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:29.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea3fff640 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe4a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea37fe640 1 -- 172.21.15.103:0/1028434108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 msgr2=0x564c3fe47810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:48:29.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 0x564c3fe47810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea37fe640 1 -- 172.21.15.103:0/1028434108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 msgr2=0x564c3fe4a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:29.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe4a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea37fe640 1 -- 172.21.15.103:0/1028434108 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3fb0fb90 con 0x564c3fe38680 2024-09-19T07:48:29.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea3fff640 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:29.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38680 0x564c3fe38a60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbe94010d20 tx=0x7fbe9409bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1b2bfc875d4af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:29.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbe99ffb640 1 -- 172.21.15.103:0/1028434108 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940a4ac0 con 0x564c3fe38680 2024-09-19T07:48:29.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbe99ffb640 1 -- 172.21.15.103:0/1028434108 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe9409e030 con 0x564c3fe38680 2024-09-19T07:48:29.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.318+0000 7fbe99ffb640 1 -- 172.21.15.103:0/1028434108 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9409b550 con 0x564c3fe38680 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.320+0000 7fbeabe94740 1 -- 172.21.15.103:0/1028434108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38680 msgr2=0x564c3fe38a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.320+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38680 0x564c3fe38a60 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbe94010d20 tx=0x7fbe9409bfc0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 -- 172.21.15.103:0/1028434108 shutdown_connections 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 0x564c3fe47810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1028434108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38680 0x564c3fe38a60 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 -- 172.21.15.103:0/1028434108 >> 172.21.15.103:0/1028434108 conn(0x564c3fe34cf0 msgr2=0x564c3fe350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 -- 172.21.15.103:0/1028434108 shutdown_connections 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 -- 172.21.15.103:0/1028434108 wait complete. 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.321+0000 7fbeabe94740 1 -- start start 2024-09-19T07:48:29.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.322+0000 7fbeabe94740 1 Processor -- start 2024-09-19T07:48:29.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.322+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38680 0x564c3fe47270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.322+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.322+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe43760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.322+0000 7fbeabe94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c3fe3b580 con 0x564c3fe38fa0 2024-09-19T07:48:29.328 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.322+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38680 0x564c3fe47270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbeabe94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564c3fd266e0 con 0x564c3fe47d50 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbeabe94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c3fe3b700 con 0x564c3fe38680 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38680 0x564c3fe47270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40722/0 (socket says 172.21.15.103:40722) 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 -- 172.21.15.103:0/1666654074 learned_addr learned my addr 172.21.15.103:0/1666654074 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbeabe94740 10 client.0 _ll_register_callbacks cb 0x564c3fd4aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea3fff640 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe43760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 -- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 msgr2=0x564c3fe43760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe43760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 -- 172.21.15.103:0/1666654074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 msgr2=0x564c3fe477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:29.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 -- 172.21.15.103:0/1666654074 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c3fe43e90 con 0x564c3fe38680 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38680 0x564c3fe47270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbe9409be40 tx=0x7fbe940a4b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e716604d40e1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.323+0000 7fbe82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.324+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1666654074 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940cd540 con 0x564c3fe38680 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.324+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1666654074 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fbe9409b350 con 0x564c3fe38680 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.324+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe9409e030 con 0x564c3fe38680 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.324+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1666654074 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbe940ab050 con 0x564c3fe38680 2024-09-19T07:48:29.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.324+0000 7fbeabe94740 10 client.26908 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:29.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564c3fe3b580 con 0x564c3fe38680 2024-09-19T07:48:29.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1666654074 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe9409e030 con 0x564c3fe38680 2024-09-19T07:48:29.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1666654074 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fbe94084070 con 0x564c3fe38680 2024-09-19T07:48:29.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbeabe94740 10 client.26908 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:29.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbeabe94740 10 client.26908 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:48:29.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbeabe94740 20 client.26908 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:29.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fe645e0 0x564c3fe669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.325+0000 7fbeabe94740 4 client.26908 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:29.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.326+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c3fdab9d0 con 0x564c3fe645e0 2024-09-19T07:48:29.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.326+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fe645e0 0x564c3fe669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.326+0000 7fbe82ffd640 10 client.26908 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.327+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fe645e0 0x564c3fe669d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e32c7d416ef832f2 server_cookie=ceb7e9fdddea01f4 in_seq=0 out_seq=0 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.327+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1666654074 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c3fdab9d0 con 0x564c3fe645e0 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.327+0000 7fbe82ffd640 10 client.26908 handle_command_reply: tid=0 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.327+0000 7fbeabe94740 1 client.26908 shutdown 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38680 msgr2=0x564c3fe47270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38680 0x564c3fe47270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbe9409be40 tx=0x7fbe940a4b50 comp rx=0 tx=0).stop 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 shutdown_connections 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fe645e0 0x564c3fe669d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe47d50 0x564c3fe43760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1666654074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38680 0x564c3fe47270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 >> 172.21.15.103:0/1666654074 conn(0x564c3fe34cf0 msgr2=0x564c3fe3a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:29.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbe82ffd640 0 client.26908 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:29.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 shutdown_connections 2024-09-19T07:48:29.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 1 -- 172.21.15.103:0/1666654074 wait complete. 2024-09-19T07:48:29.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.328+0000 7fbeabe94740 20 client.26908 trim_cache size 0 max 16384 2024-09-19T07:48:29.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 Processor -- start 2024-09-19T07:48:29.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 -- start start 2024-09-19T07:48:29.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe42a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 0x564c3fe3a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe47d50 0x564c3fe3ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbea40721f0 con 0x564c3fe47d50 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4073d60 con 0x564c3fe38680 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.357+0000 7fbeabe94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbea40728f0 con 0x564c3fe38fa0 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe42a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 0x564c3fe3a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe47d50 0x564c3fe3ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe42a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50076/0 (socket says 172.21.15.103:50076) 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 -- 172.21.15.103:0/560012652 learned_addr learned my addr 172.21.15.103:0/560012652 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 -- 172.21.15.103:0/560012652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 msgr2=0x564c3fe3a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:29.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 --2- 172.21.15.103:0/560012652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 0x564c3fe3a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 -- 172.21.15.103:0/560012652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe47d50 msgr2=0x564c3fe3ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:29.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 --2- 172.21.15.103:0/560012652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe47d50 0x564c3fe3ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 -- 172.21.15.103:0/560012652 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3fe30260 con 0x564c3fe38680 2024-09-19T07:48:29.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbea37fe640 1 --2- 172.21.15.103:0/560012652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe42a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbe940a03c0 tx=0x7fbe940a4c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=925304ea4785ccec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:29.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbe997fa640 1 -- 172.21.15.103:0/560012652 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940cd540 con 0x564c3fe38680 2024-09-19T07:48:29.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbe997fa640 1 -- 172.21.15.103:0/560012652 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe9409e070 con 0x564c3fe38680 2024-09-19T07:48:29.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.358+0000 7fbe997fa640 1 -- 172.21.15.103:0/560012652 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9409adc0 con 0x564c3fe38680 2024-09-19T07:48:29.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.359+0000 7fbeabe94740 1 -- 172.21.15.103:0/560012652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 msgr2=0x564c3fe42a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.359+0000 7fbeabe94740 1 --2- 172.21.15.103:0/560012652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe42a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbe940a03c0 tx=0x7fbe940a4c40 comp rx=0 tx=0).stop 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.360+0000 7fbeabe94740 1 -- 172.21.15.103:0/560012652 shutdown_connections 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.360+0000 7fbeabe94740 1 --2- 172.21.15.103:0/560012652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe47d50 0x564c3fe3ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.360+0000 7fbeabe94740 1 --2- 172.21.15.103:0/560012652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe38fa0 0x564c3fe3a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.360+0000 7fbeabe94740 1 --2- 172.21.15.103:0/560012652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe42a90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.360+0000 7fbeabe94740 1 -- 172.21.15.103:0/560012652 >> 172.21.15.103:0/560012652 conn(0x564c3fe34cf0 msgr2=0x564c3fe3a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.360+0000 7fbeabe94740 1 -- 172.21.15.103:0/560012652 shutdown_connections 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.361+0000 7fbeabe94740 1 -- 172.21.15.103:0/560012652 wait complete. 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.361+0000 7fbeabe94740 1 -- start start 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.361+0000 7fbeabe94740 1 Processor -- start 2024-09-19T07:48:29.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.362+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe4b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.362+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe4b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.362+0000 7fbeabe94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe47d50 0x564c3fe4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.362+0000 7fbeabe94740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4073690 con 0x564c3fe38fa0 2024-09-19T07:48:29.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.362+0000 7fbeabe94740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4072fc0 con 0x564c3fe38680 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.362+0000 7fbeabe94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbea40701f0 con 0x564c3fe47d50 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.362+0000 7fbeabe94740 10 client.0 _ll_register_callbacks cb 0x7fbe940a46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe4b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47662/0 (socket says 172.21.15.103:47662) 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 -- 172.21.15.103:0/1279746539 learned_addr learned my addr 172.21.15.103:0/1279746539 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe4b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbea3fff640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe47d50 0x564c3fe4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 -- 172.21.15.103:0/1279746539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe47d50 msgr2=0x564c3fe4bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe47d50 0x564c3fe4bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 -- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 msgr2=0x564c3fe4b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe4b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.363+0000 7fbe9bfff640 1 -- 172.21.15.103:0/1279746539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c3fece2a0 con 0x564c3fe38fa0 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.364+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe4b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:29.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.364+0000 7fbea3fff640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe47d50 0x564c3fe4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.364+0000 7fbe9bfff640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe4b600 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c02a910 tx=0x7fbe8c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e1b974772794ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.364+0000 7fbe82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:29.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.366+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1279746539 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c037040 con 0x564c3fe38fa0 2024-09-19T07:48:29.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.366+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1279746539 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fbe8c003d00 con 0x564c3fe38fa0 2024-09-19T07:48:29.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.366+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe9409e070 con 0x564c3fe38fa0 2024-09-19T07:48:29.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.367+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1279746539 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbe8c0305c0 con 0x564c3fe38fa0 2024-09-19T07:48:29.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.367+0000 7fbeabe94740 10 client.17556 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:29.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.367+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564c3fecf550 con 0x564c3fe38fa0 2024-09-19T07:48:29.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.368+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1279746539 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe9409e070 con 0x564c3fe38fa0 2024-09-19T07:48:29.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.368+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1279746539 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fbe8c0498d0 con 0x564c3fe38fa0 2024-09-19T07:48:29.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.368+0000 7fbeabe94740 10 client.17556 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:29.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.368+0000 7fbeabe94740 10 client.17556 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:48:29.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.368+0000 7fbeabe94740 20 client.17556 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:29.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.369+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fecfcd0 0x564c3fed20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:29.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.369+0000 7fbeabe94740 4 client.17556 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T07:48:29.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.369+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x564c3fda5d20 con 0x564c3fecfcd0 2024-09-19T07:48:29.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.369+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fecfcd0 0x564c3fed20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:29.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.370+0000 7fbea37fe640 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fecfcd0 0x564c3fed20f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ac84fe49eec328 server_cookie=cf76f21dd64abbf8 in_seq=0 out_seq=0 2024-09-19T07:48:29.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.370+0000 7fbe82ffd640 10 client.17556 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:29.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbe82ffd640 1 -- 172.21.15.103:0/1279746539 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564c3fda5d20 con 0x564c3fecfcd0 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbe82ffd640 10 client.17556 handle_command_reply: tid=0 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 client.17556 shutdown 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 msgr2=0x564c3fe4b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe4b600 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c02a910 tx=0x7fbe8c004470 comp rx=0 tx=0).stop 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 shutdown_connections 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564c3fecfcd0 0x564c3fed20f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c3fe47d50 0x564c3fe4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c3fe38fa0 0x564c3fe4b600 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 --2- 172.21.15.103:0/1279746539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c3fe38680 0x564c3fe4b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 >> 172.21.15.103:0/1279746539 conn(0x564c3fe34cf0 msgr2=0x564c3fe3a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:29.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbe82ffd640 0 client.17556 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 shutdown_connections 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.573+0000 7fbeabe94740 1 -- 172.21.15.103:0/1279746539 wait complete. 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbeabe94740 20 client.17556 trim_cache size 0 max 16384 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 46, 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 38, 2024-09-19T07:48:29.576 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "8ac5c642": { 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 38, 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:48:29.577 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:29.578 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe900a3430 msgr2=0x7fbe900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe900a3430 0x7fbe900a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c035040 tx=0x7fbe8c0056d0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 msgr2=0x7fbea4199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 0x7fbea4199370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400d950 tx=0x7fbe9400de10 comp rx=0 tx=0).stop 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 shutdown_connections 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fbe900a3430 0x7fbe900a58f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbea4102ef0 0x7fbea41a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbea4102570 0x7fbea41998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 --2- 172.21.15.103:0/3316851659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea4100e20 0x7fbea4199370 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:29.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 >> 172.21.15.103:0/3316851659 conn(0x7fbea40fc7d0 msgr2=0x7fbea41048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:29.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 shutdown_connections 2024-09-19T07:48:29.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:29.574+0000 7fbea9abb640 1 -- 172.21.15.103:0/3316851659 wait complete. 2024-09-19T07:48:30.283 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8ac5c642', quiesce took 4.6 seconds. Will release after: 25.4 2024-09-19T07:48:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 595.0 s, 2325.8 tps, lat 33.701 ms stddev 39.467 2024-09-19T07:48:30.467 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:30 smithi174 ceph-mon[35370]: pgmap v326: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 830 op/s 2024-09-19T07:48:30.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:30 smithi103 ceph-mon[31201]: pgmap v326: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 830 op/s 2024-09-19T07:48:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:30 smithi134 ceph-mon[35414]: pgmap v326: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 830 op/s 2024-09-19T07:48:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:32 smithi103 ceph-mon[31201]: pgmap v327: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-19T07:48:32.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:32 smithi134 ceph-mon[35414]: pgmap v327: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-19T07:48:32.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:32 smithi174 ceph-mon[35370]: pgmap v327: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-19T07:48:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:34 smithi103 ceph-mon[31201]: pgmap v328: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-19T07:48:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:34 smithi134 ceph-mon[35414]: pgmap v328: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-19T07:48:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:34 smithi174 ceph-mon[35370]: pgmap v328: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-19T07:48:35.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 600.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:36 smithi103 ceph-mon[31201]: pgmap v329: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-19T07:48:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:36 smithi134 ceph-mon[35414]: pgmap v329: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-19T07:48:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:36 smithi174 ceph-mon[35370]: pgmap v329: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-19T07:48:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:38 smithi103 ceph-mon[31201]: pgmap v330: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 824 op/s 2024-09-19T07:48:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:38 smithi134 ceph-mon[35414]: pgmap v330: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 824 op/s 2024-09-19T07:48:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:38 smithi174 ceph-mon[35370]: pgmap v330: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 824 op/s 2024-09-19T07:48:40.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:40 smithi103 ceph-mon[31201]: pgmap v331: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-09-19T07:48:40.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 605.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:40 smithi134 ceph-mon[35414]: pgmap v331: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-09-19T07:48:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:40 smithi174 ceph-mon[35370]: pgmap v331: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-09-19T07:48:42.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:42 smithi103 ceph-mon[31201]: pgmap v332: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-19T07:48:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:42 smithi134 ceph-mon[35414]: pgmap v332: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-19T07:48:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:42 smithi174 ceph-mon[35370]: pgmap v332: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-19T07:48:42.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:48:43.201 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:43.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 -- 172.21.15.103:0/3697582165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc106a80 msgr2=0x7f60fc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3697582165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc106a80 0x7f60fc106e60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009930 tx=0x7f60ec02a420 comp rx=0 tx=0).stop 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 -- 172.21.15.103:0/3697582165 shutdown_connections 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3697582165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60fc101420 0x7f60fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3697582165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60fc100a80 0x7f60fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3697582165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc106a80 0x7f60fc106e60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 -- 172.21.15.103:0/3697582165 >> 172.21.15.103:0/3697582165 conn(0x7f60fc0fc7a0 msgr2=0x7f60fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 -- 172.21.15.103:0/3697582165 shutdown_connections 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.725+0000 7f6100ec1640 1 -- 172.21.15.103:0/3697582165 wait complete. 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 Processor -- start 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 -- start start 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60fc100a80 0x7f60fc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc101420 0x7f60fc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60fc106a80 0x7f60fc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc111fa0 con 0x7f60fc101420 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc111e20 con 0x7f60fc106a80 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f6100ec1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc112120 con 0x7f60fc100a80 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc101420 0x7f60fc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc101420 0x7f60fc1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40706/0 (socket says 172.21.15.103:40706) 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3947496410 learned_addr learned my addr 172.21.15.103:0/3947496410 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60fa59c640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60fc100a80 0x7f60fc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60fc106a80 0x7f60fc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3947496410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60fc100a80 msgr2=0x7f60fc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60fc100a80 0x7f60fc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3947496410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60fc106a80 msgr2=0x7f60fc19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60fc106a80 0x7f60fc19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3947496410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc19d450 con 0x7f60fc101420 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.726+0000 7f60fa59c640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60fc100a80 0x7f60fc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.727+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc101420 0x7f60fc1989e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d770 tx=0x7f60e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.727+0000 7f60f37fe640 1 -- 172.21.15.103:0/3947496410 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4017070 con 0x7f60fc101420 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.727+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc19d740 con 0x7f60fc101420 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.727+0000 7f60f37fe640 1 -- 172.21.15.103:0/3947496410 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e4005500 con 0x7f60fc101420 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.727+0000 7f60f37fe640 1 -- 172.21.15.103:0/3947496410 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4004e00 con 0x7f60fc101420 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.727+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a5020 con 0x7f60fc101420 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.729+0000 7f60f37fe640 1 -- 172.21.15.103:0/3947496410 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f60e4004fa0 con 0x7f60fc101420 2024-09-19T07:48:43.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.729+0000 7f610329a740 1 Processor -- start 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f610329a740 1 -- start start 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f37fe640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60e80a33b0 0x7f60e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f610329a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x559078325880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f610329a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x559078330140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f610329a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x559078332a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f610329a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55907828b2e0 con 0x559078325dc0 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f610329a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559078232ce0 con 0x5590783254a0 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f610329a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590782872a0 con 0x559078330680 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f37fe640 1 -- 172.21.15.103:0/3947496410 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f60e40cf860 con 0x7f60fc101420 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x559078330140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60e80a33b0 0x7f60e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x559078330140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40718/0 (socket says 172.21.15.103:40718) 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3240302500 learned_addr learned my addr 172.21.15.103:0/3240302500 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60fa59c640 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x559078325880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x559078332a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3240302500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 msgr2=0x559078332a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x559078332a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3240302500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 msgr2=0x559078325880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x559078325880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 -- 172.21.15.103:0/3240302500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559078217440 con 0x559078325dc0 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x559078332a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x559078330140 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f60e4098500 tx=0x7f60e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539af13b743c8a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.730+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60e80a33b0 0x7f60e80a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x559078322db0 tx=0x7f60f4008230 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f60f0ff9640 1 -- 172.21.15.103:0/3240302500 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e409ce60 con 0x559078325dc0 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f60f0ff9640 1 -- 172.21.15.103:0/3240302500 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e409fee0 con 0x559078325dc0 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f60f0ff9640 1 -- 172.21.15.103:0/3240302500 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4020040 con 0x559078325dc0 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 -- 172.21.15.103:0/3240302500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 msgr2=0x559078330140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x559078330140 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f60e4098500 tx=0x7f60e40046c0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 -- 172.21.15.103:0/3240302500 shutdown_connections 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x559078332a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x559078330140 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 --2- 172.21.15.103:0/3240302500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x559078325880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 -- 172.21.15.103:0/3240302500 >> 172.21.15.103:0/3240302500 conn(0x559078321a30 msgr2=0x559078321e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 -- 172.21.15.103:0/3240302500 shutdown_connections 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.731+0000 7f610329a740 1 -- 172.21.15.103:0/3240302500 wait complete. 2024-09-19T07:48:43.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 -- start start 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 Processor -- start 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907832ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x55907832d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x55907832d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5590783331f0 con 0x559078325dc0 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559078246050 con 0x5590783254a0 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559078333370 con 0x559078330680 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f610329a740 10 client.0 _ll_register_callbacks cb 0x559078286160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x55907832d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f60fa59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907832ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f60f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x55907832d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40732/0 (socket says 172.21.15.103:40732) 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f60f9d9b640 1 -- 172.21.15.103:0/4221562942 learned_addr learned my addr 172.21.15.103:0/4221562942 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:43.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f60fa59c640 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907832ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:47032/0 (socket says 172.21.15.103:47032) 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.732+0000 7f60fad9d640 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x55907832d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60f9d9b640 1 -- 172.21.15.103:0/4221562942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 msgr2=0x55907832d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x55907832d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60f9d9b640 1 -- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 msgr2=0x55907832ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907832ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60f9d9b640 1 -- 172.21.15.103:0/4221562942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559077ffc3c0 con 0x559078325dc0 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x55907832d1f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f60e40a4270 tx=0x7f60e40a47a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63a35f3f673433c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60e1ffb640 1 -- 172.21.15.103:0/4221562942 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4096020 con 0x559078325dc0 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60e1ffb640 1 -- 172.21.15.103:0/4221562942 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f60e40981a0 con 0x559078325dc0 2024-09-19T07:48:43.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e409fee0 con 0x559078325dc0 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60e1ffb640 1 -- 172.21.15.103:0/4221562942 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f60e409ce60 con 0x559078325dc0 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 10 client.17568 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55907833eb40 con 0x559078325dc0 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60e1ffb640 1 -- 172.21.15.103:0/4221562942 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e409fee0 con 0x559078325dc0 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f60e1ffb640 1 -- 172.21.15.103:0/4221562942 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f60e4096250 con 0x559078325dc0 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 10 client.17568 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 20 client.17568 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55907833fe90 0x559078342280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 4 client.17568 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.733+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559078298850 con 0x55907833fe90 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.734+0000 7f60fa59c640 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55907833fe90 0x559078342280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.734+0000 7f60e1ffb640 10 client.17568 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.734+0000 7f60fa59c640 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55907833fe90 0x559078342280 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694d39ac74d7a59f server_cookie=b4b8c05b47d93384 in_seq=0 out_seq=0 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f60e1ffb640 1 -- 172.21.15.103:0/4221562942 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559078298850 con 0x55907833fe90 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f60e1ffb640 10 client.17568 handle_command_reply: tid=0 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 client.17568 shutdown 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 msgr2=0x55907832d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x55907832d1f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f60e40a4270 tx=0x7f60e40a47a0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 shutdown_connections 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55907833fe90 0x559078342280 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078330680 0x55907832d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078325dc0 0x55907832d1f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 --2- 172.21.15.103:0/4221562942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907832ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 >> 172.21.15.103:0/4221562942 conn(0x559078321a30 msgr2=0x559078327760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f60e1ffb640 0 client.17568 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 shutdown_connections 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 1 -- 172.21.15.103:0/4221562942 wait complete. 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.735+0000 7f610329a740 20 client.17568 trim_cache size 0 max 16384 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 Processor -- start 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 -- start start 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907833c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x559078347880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078347dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1ad990 con 0x559078330680 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1af500 con 0x5590783254a0 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.752+0000 7f610329a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1ae090 con 0x559078325dc0 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078347dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078347dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40744/0 (socket says 172.21.15.103:40744) 2024-09-19T07:48:43.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 -- 172.21.15.103:0/3580467527 learned_addr learned my addr 172.21.15.103:0/3580467527 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fa59c640 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907833c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x559078347880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 -- 172.21.15.103:0/3580467527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 msgr2=0x559078347880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x559078347880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 -- 172.21.15.103:0/3580467527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 msgr2=0x55907833c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907833c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 -- 172.21.15.103:0/3580467527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559078321860 con 0x559078330680 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x559078347880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fa59c640 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907833c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60fad9d640 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078347dc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005fd0 tx=0x7f60f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df70fa5b66def61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60e3fff640 1 -- 172.21.15.103:0/3580467527 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f400fc00 con 0x559078330680 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60e3fff640 1 -- 172.21.15.103:0/3580467527 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f400fda0 con 0x559078330680 2024-09-19T07:48:43.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.753+0000 7f60e3fff640 1 -- 172.21.15.103:0/3580467527 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f401b790 con 0x559078330680 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 -- 172.21.15.103:0/3580467527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 msgr2=0x559078347dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078347dc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005fd0 tx=0x7f60f4004ab0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 -- 172.21.15.103:0/3580467527 shutdown_connections 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078347dc0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x559078347880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 --2- 172.21.15.103:0/3580467527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x55907833c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 -- 172.21.15.103:0/3580467527 >> 172.21.15.103:0/3580467527 conn(0x559078321a30 msgr2=0x55907832c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 -- 172.21.15.103:0/3580467527 shutdown_connections 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 -- 172.21.15.103:0/3580467527 wait complete. 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 -- start start 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.754+0000 7f610329a740 1 Processor -- start 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f610329a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x5590783a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f610329a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x5590783a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f610329a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078335b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f610329a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1aee30 con 0x559078330680 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f610329a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1ae760 con 0x5590783254a0 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f610329a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1ab990 con 0x559078325dc0 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f610329a740 10 client.0 _ll_register_callbacks cb 0x7f60e409bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078335b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078335b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40756/0 (socket says 172.21.15.103:40756) 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 -- 172.21.15.103:0/2749691687 learned_addr learned my addr 172.21.15.103:0/2749691687 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fa59c640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x5590783a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x5590783a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 -- 172.21.15.103:0/2749691687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 msgr2=0x5590783a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x5590783a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 -- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 msgr2=0x5590783a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:43.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x5590783a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 -- 172.21.15.103:0/2749691687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590783363a0 con 0x559078330680 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60f9d9b640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x5590783a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60fad9d640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078335b70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005e10 tx=0x7f60f4027e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625d3a3323bc98e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.755+0000 7f60e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f60e1ffb640 1 -- 172.21.15.103:0/2749691687 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4034060 con 0x559078330680 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f60e1ffb640 1 -- 172.21.15.103:0/2749691687 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f60f40083e0 con 0x559078330680 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f400fda0 con 0x559078330680 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f60e1ffb640 1 -- 172.21.15.103:0/2749691687 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f60f4027650 con 0x559078330680 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 10 client.17574 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5590783aa3a0 con 0x559078330680 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f60e1ffb640 1 -- 172.21.15.103:0/2749691687 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f400fda0 con 0x559078330680 2024-09-19T07:48:43.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f60e1ffb640 1 -- 172.21.15.103:0/2749691687 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f60f4025020 con 0x559078330680 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 10 client.17574 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 20 client.17574 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5590783aaa10 0x5590783ace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 4 client.17574 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559078292d60 con 0x5590783aaa10 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.756+0000 7f60fa59c640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5590783aaa10 0x5590783ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f60e1ffb640 10 client.17574 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f60fa59c640 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5590783aaa10 0x5590783ace00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f722c07638aef5a server_cookie=aead9f6456c340eb in_seq=0 out_seq=0 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f60e1ffb640 1 -- 172.21.15.103:0/2749691687 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559078292d60 con 0x5590783aaa10 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f60e1ffb640 10 client.17574 handle_command_reply: tid=0 2024-09-19T07:48:43.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 client.17574 shutdown 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 msgr2=0x559078335b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078335b70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005e10 tx=0x7f60f4027e80 comp rx=0 tx=0).stop 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 shutdown_connections 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5590783aaa10 0x5590783ace00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559078330680 0x559078335b70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559078325dc0 0x5590783a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 --2- 172.21.15.103:0/2749691687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590783254a0 0x5590783a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 >> 172.21.15.103:0/2749691687 conn(0x559078321a30 msgr2=0x55907832ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f60e1ffb640 0 client.17574 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 shutdown_connections 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 1 -- 172.21.15.103:0/2749691687 wait complete. 2024-09-19T07:48:43.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.757+0000 7f610329a740 20 client.17574 trim_cache size 0 max 16384 2024-09-19T07:48:43.775 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:48:43.775 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:48:43.775 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:48:43.775 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:48:43.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60e80a33b0 msgr2=0x7f60e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60e80a33b0 0x7f60e80a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x559078322db0 tx=0x7f60f4008230 comp rx=0 tx=0).stop 2024-09-19T07:48:43.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc101420 msgr2=0x7f60fc1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:43.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc101420 0x7f60fc1989e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d770 tx=0x7f60e400dc30 comp rx=0 tx=0).stop 2024-09-19T07:48:43.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 shutdown_connections 2024-09-19T07:48:43.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f60e80a33b0 0x7f60e80a5870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60fc106a80 0x7f60fc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc101420 0x7f60fc1989e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 --2- 172.21.15.103:0/3947496410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60fc100a80 0x7f60fc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:43.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 >> 172.21.15.103:0/3947496410 conn(0x7f60fc0fc7a0 msgr2=0x7f60fc0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:43.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 shutdown_connections 2024-09-19T07:48:43.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:43.758+0000 7f6100ec1640 1 -- 172.21.15.103:0/3947496410 wait complete. 2024-09-19T07:48:44.517 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:44 smithi103 ceph-mon[31201]: pgmap v333: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-19T07:48:44.546 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:48:44.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:48:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:44 smithi134 ceph-mon[35414]: pgmap v333: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-19T07:48:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:44 smithi174 ceph-mon[35370]: pgmap v333: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-09-19T07:48:44.793 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1796837629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c4103660 msgr2=0x7fe3c410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1796837629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c4103660 0x7fe3c410bb50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009900 tx=0x7fe3b402a480 comp rx=0 tx=0).stop 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1796837629 shutdown_connections 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1796837629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3c410c170 0x7fe3c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1796837629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c4103660 0x7fe3c410bb50 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1796837629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c4102d40 0x7fe3c4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1796837629 >> 172.21.15.103:0/1796837629 conn(0x7fe3c40fc7a0 msgr2=0x7fe3c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1796837629 shutdown_connections 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1796837629 wait complete. 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 Processor -- start 2024-09-19T07:48:45.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.277+0000 7fe3cc0e4640 1 -- start start 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3cc0e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c4102d40 0x7fe3c419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3cc0e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3c4103660 0x7fe3c419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3cc0e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c410c170 0x7fe3c419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3cc0e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4111f70 con 0x7fe3c410c170 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3cc0e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4111df0 con 0x7fe3c4102d40 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3cc0e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41120f0 con 0x7fe3c4103660 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c410c170 0x7fe3c419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c410c170 0x7fe3c419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40776/0 (socket says 172.21.15.103:40776) 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 -- 172.21.15.103:0/1797656976 learned_addr learned my addr 172.21.15.103:0/1797656976 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:45.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3c9e80640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c4102d40 0x7fe3c419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3c967f640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3c4103660 0x7fe3c419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 -- 172.21.15.103:0/1797656976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3c4103660 msgr2=0x7fe3c419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3c4103660 0x7fe3c419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 -- 172.21.15.103:0/1797656976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c4102d40 msgr2=0x7fe3c419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c4102d40 0x7fe3c419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 -- 172.21.15.103:0/1797656976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c419f680 con 0x7fe3c410c170 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3ca681640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c410c170 0x7fe3c419efa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00eaa0 tx=0x7fe3bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3baffd640 1 -- 172.21.15.103:0/1797656976 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc00cdb0 con 0x7fe3c410c170 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3baffd640 1 -- 172.21.15.103:0/1797656976 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3bc005020 con 0x7fe3c410c170 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3baffd640 1 -- 172.21.15.103:0/1797656976 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0116b0 con 0x7fe3c410c170 2024-09-19T07:48:45.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.278+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c419f970 con 0x7fe3c410c170 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.279+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a7250 con 0x7fe3c410c170 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.279+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c4103120 con 0x7fe3c410c170 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.281+0000 7fe3baffd640 1 -- 172.21.15.103:0/1797656976 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe3bc005560 con 0x7fe3c410c170 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.282+0000 7fe3baffd640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3b00a33b0 0x7fe3b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.282+0000 7fe3c9e80640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3b00a33b0 0x7fe3b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.282+0000 7fe3baffd640 1 -- 172.21.15.103:0/1797656976 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fe3bc0d0380 con 0x7fe3c410c170 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.282+0000 7fe3c9e80640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3b00a33b0 0x7fe3b00a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009b60 tx=0x7fe3c0005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:45.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.285+0000 7fe3baffd640 1 -- 172.21.15.103:0/1797656976 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe3bc016030 con 0x7fe3c410c170 2024-09-19T07:48:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 610.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:45.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.487+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3c419ba30 con 0x7fe3c410c170 2024-09-19T07:48:45.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.488+0000 7fe3baffd640 1 -- 172.21.15.103:0/1797656976 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7fe3c419ba30 con 0x7fe3c410c170 2024-09-19T07:48:45.488 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:48:45.488 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":46,"btime":"2024-09-19T07:48:00:225048+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:47:59.861225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:48:45.488 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 46 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3b00a33b0 msgr2=0x7fe3b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3b00a33b0 0x7fe3b00a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009b60 tx=0x7fe3c0005f90 comp rx=0 tx=0).stop 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c410c170 msgr2=0x7fe3c419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c410c170 0x7fe3c419efa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00eaa0 tx=0x7fe3bc00ef60 comp rx=0 tx=0).stop 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 shutdown_connections 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3b00a33b0 0x7fe3b00a5870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c410c170 0x7fe3c419efa0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3c4103660 0x7fe3c419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 --2- 172.21.15.103:0/1797656976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c4102d40 0x7fe3c419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 >> 172.21.15.103:0/1797656976 conn(0x7fe3c40fc7a0 msgr2=0x7fe3c410d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 shutdown_connections 2024-09-19T07:48:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:45.490+0000 7fe3cc0e4640 1 -- 172.21.15.103:0/1797656976 wait complete. 2024-09-19T07:48:46.412 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:46 smithi103 ceph-mon[31201]: pgmap v334: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-19T07:48:46.412 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:46 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1797656976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:46.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:48:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:46 smithi134 ceph-mon[35414]: pgmap v334: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-19T07:48:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:46 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1797656976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:46.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:46 smithi174 ceph-mon[35370]: pgmap v334: 97 pgs: 97 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-19T07:48:46.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:46 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1797656976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:46.698 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 -- 172.21.15.103:0/1095516153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0106f10 msgr2=0x7f4bf010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/1095516153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0106f10 0x7f4bf010ea90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009ca0 tx=0x7f4be802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 -- 172.21.15.103:0/1095516153 shutdown_connections 2024-09-19T07:48:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/1095516153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0106f10 0x7f4bf010ea90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/1095516153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf0102790 0x7f4bf01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/1095516153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0101de0 0x7f4bf01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 -- 172.21.15.103:0/1095516153 >> 172.21.15.103:0/1095516153 conn(0x7f4bf00fd8c0 msgr2=0x7f4bf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 -- 172.21.15.103:0/1095516153 shutdown_connections 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 -- 172.21.15.103:0/1095516153 wait complete. 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.416+0000 7f4bf6da1640 1 Processor -- start 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf6da1640 1 -- start start 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf6da1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf0101de0 0x7f4bf01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf6da1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0102790 0x7f4bf01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf6da1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0106f10 0x7f4bf019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf6da1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0111f70 con 0x7f4bf0102790 2024-09-19T07:48:47.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf6da1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0111df0 con 0x7f4bf0106f10 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf6da1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01120f0 con 0x7f4bf0101de0 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0102790 0x7f4bf01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0102790 0x7f4bf01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40794/0 (socket says 172.21.15.103:40794) 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 -- 172.21.15.103:0/957283299 learned_addr learned my addr 172.21.15.103:0/957283299 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf0101de0 0x7f4bf01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4bf533e640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0106f10 0x7f4bf019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 -- 172.21.15.103:0/957283299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf0101de0 msgr2=0x7f4bf01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf0101de0 0x7f4bf01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 -- 172.21.15.103:0/957283299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0106f10 msgr2=0x7f4bf019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0106f10 0x7f4bf019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 -- 172.21.15.103:0/957283299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf019d360 con 0x7f4bf0102790 2024-09-19T07:48:47.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.417+0000 7f4beffff640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0102790 0x7f4bf01988f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00e9f0 tx=0x7f4bdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.418+0000 7f4bedffb640 1 -- 172.21.15.103:0/957283299 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00cd80 con 0x7f4bf0102790 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.418+0000 7f4bedffb640 1 -- 172.21.15.103:0/957283299 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bdc005500 con 0x7f4bf0102790 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.418+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf019d650 con 0x7f4bf0102790 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.418+0000 7f4bedffb640 1 -- 172.21.15.103:0/957283299 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc010690 con 0x7f4bf0102790 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.418+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf01a4f30 con 0x7f4bf0102790 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.420+0000 7f4bedffb640 1 -- 172.21.15.103:0/957283299 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4bdc0046c0 con 0x7f4bf0102790 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.420+0000 7f4bf917a740 1 Processor -- start 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.420+0000 7f4bf917a740 1 -- start start 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaa0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bedffb640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4be00a33b0 0x7f4be00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaa0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecab1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf917a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6eca062f0 con 0x55e6ecaa1b40 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf917a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e6ec9adcf0 con 0x55e6ecaaf780 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf917a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6eca022b0 con 0x55e6ecaa05d0 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaa0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4be00a33b0 0x7f4be00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaa0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40800/0 (socket says 172.21.15.103:40800) 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 -- 172.21.15.103:0/2887512854 learned_addr learned my addr 172.21.15.103:0/2887512854 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bedffb640 1 -- 172.21.15.103:0/957283299 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f4bdc0d00d0 con 0x7f4bf0102790 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4beffff640 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaa0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4be00a33b0 0x7f4be00a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009bd0 tx=0x7f4be4005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 -- 172.21.15.103:0/2887512854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa05d0 msgr2=0x55e6ecaa0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaa0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 -- 172.21.15.103:0/2887512854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 msgr2=0x55e6ecab1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecab1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 -- 172.21.15.103:0/2887512854 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ec9923a0 con 0x55e6ecaa1b40 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.421+0000 7f4bf533e640 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaa0090 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4be8004af0 tx=0x7f4be8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6130791de5fec6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bdaffd640 1 -- 172.21.15.103:0/2887512854 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8031db0 con 0x55e6ecaa1b40 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bdaffd640 1 -- 172.21.15.103:0/2887512854 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be8036040 con 0x55e6ecaa1b40 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bdaffd640 1 -- 172.21.15.103:0/2887512854 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8004330 con 0x55e6ecaa1b40 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 -- 172.21.15.103:0/2887512854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa1b40 msgr2=0x55e6ecaa0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaa0090 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4be8004af0 tx=0x7f4be8009850 comp rx=0 tx=0).stop 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 -- 172.21.15.103:0/2887512854 shutdown_connections 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecab1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaa0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 --2- 172.21.15.103:0/2887512854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaa0090 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 -- 172.21.15.103:0/2887512854 >> 172.21.15.103:0/2887512854 conn(0x55e6eca69310 msgr2=0x55e6eca9cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 -- 172.21.15.103:0/2887512854 shutdown_connections 2024-09-19T07:48:47.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.422+0000 7f4bf917a740 1 -- 172.21.15.103:0/2887512854 wait complete. 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 -- start start 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 Processor -- start 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaaf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecaac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6ecab2420 con 0x55e6ecaaf780 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e6ec9c1060 con 0x55e6ecaa05d0 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6ecab25a0 con 0x55e6ecaa1b40 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf917a740 10 client.0 _ll_register_callbacks cb 0x55e6ec7148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf533e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecaac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf533e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecaac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40814/0 (socket says 172.21.15.103:40814) 2024-09-19T07:48:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.423+0000 7f4bf533e640 1 -- 172.21.15.103:0/787157238 learned_addr learned my addr 172.21.15.103:0/787157238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaaf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4beffff640 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf533e640 1 -- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 msgr2=0x55e6ecaac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf533e640 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf533e640 1 -- 172.21.15.103:0/787157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 msgr2=0x55e6ecaaf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf533e640 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaaf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf533e640 1 -- 172.21.15.103:0/787157238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ec7773c0 con 0x55e6ecaaf780 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf533e640 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecaac9e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4be80095c0 tx=0x7f4be8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10bafde0862228e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bcffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bcffff640 1 -- 172.21.15.103:0/787157238 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be80317e0 con 0x55e6ecaaf780 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bcffff640 1 -- 172.21.15.103:0/787157238 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f4be8003b70 con 0x55e6ecaaf780 2024-09-19T07:48:47.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be8036040 con 0x55e6ecaaf780 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bcffff640 1 -- 172.21.15.103:0/787157238 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4be8003d90 con 0x55e6ecaaf780 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf917a740 10 client.17592 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e6ecab2420 con 0x55e6ecaaf780 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.424+0000 7f4bcffff640 1 -- 172.21.15.103:0/787157238 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be8036040 con 0x55e6ecaaf780 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bcffff640 1 -- 172.21.15.103:0/787157238 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f4be802f9b0 con 0x55e6ecaaf780 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf917a740 10 client.17592 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf917a740 10 client.17592 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf917a740 20 client.17592 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf917a740 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecabf000 0x55e6ecac13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf917a740 4 client.17592 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:47.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6eca13860 con 0x55e6ecabf000 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecabf000 0x55e6ecac13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bcffff640 10 client.17592 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.425+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecabf000 0x55e6ecac13f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2751b1b02b8b1881 server_cookie=58e4115de6d180f3 in_seq=0 out_seq=0 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bcffff640 1 -- 172.21.15.103:0/787157238 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6eca13860 con 0x55e6ecabf000 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bcffff640 10 client.17592 handle_command_reply: tid=0 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 client.17592 shutdown 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 msgr2=0x55e6ecaac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecaac9e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4be80095c0 tx=0x7f4be8005ef0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 shutdown_connections 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecabf000 0x55e6ecac13f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecaac9e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bcffff640 0 client.17592 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:47.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 --2- 172.21.15.103:0/787157238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecaaf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 >> 172.21.15.103:0/787157238 conn(0x55e6eca69310 msgr2=0x55e6ecaa2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 shutdown_connections 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 1 -- 172.21.15.103:0/787157238 wait complete. 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.426+0000 7f4bf917a740 20 client.17592 trim_cache size 0 max 16384 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 Processor -- start 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 -- start start 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa05d0 0x55e6ecabb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaaacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecac6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01ab590 con 0x55e6ecaa05d0 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01ad100 con 0x55e6ecaaf780 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf917a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01abc90 con 0x55e6ecaa1b40 2024-09-19T07:48:47.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf4b3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa05d0 0x55e6ecabb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf4b3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa05d0 0x55e6ecabb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40818/0 (socket says 172.21.15.103:40818) 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.440+0000 7f4bf4b3d640 1 -- 172.21.15.103:0/317623998 learned_addr learned my addr 172.21.15.103:0/317623998 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4beffff640 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaaacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf533e640 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecac6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf4b3d640 1 -- 172.21.15.103:0/317623998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 msgr2=0x55e6ecaaacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaaacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf4b3d640 1 -- 172.21.15.103:0/317623998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 msgr2=0x55e6ecac6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecac6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf4b3d640 1 -- 172.21.15.103:0/317623998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ecaaf4b0 con 0x55e6ecaa05d0 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4beffff640 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaaacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf533e640 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecac6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa05d0 0x55e6ecabb6c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x55e6ecac1a20 tx=0x7f4be4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a653c535722419c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bda7fc640 1 -- 172.21.15.103:0/317623998 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be401f0d0 con 0x55e6ecaa05d0 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bda7fc640 1 -- 172.21.15.103:0/317623998 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be401f270 con 0x55e6ecaa05d0 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.441+0000 7f4bda7fc640 1 -- 172.21.15.103:0/317623998 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4028410 con 0x55e6ecaa05d0 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- 172.21.15.103:0/317623998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa05d0 msgr2=0x55e6ecabb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa05d0 0x55e6ecabb6c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x55e6ecac1a20 tx=0x7f4be4014000 comp rx=0 tx=0).stop 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- 172.21.15.103:0/317623998 shutdown_connections 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaaf780 0x55e6ecac6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecaaacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 --2- 172.21.15.103:0/317623998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaa05d0 0x55e6ecabb6c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- 172.21.15.103:0/317623998 >> 172.21.15.103:0/317623998 conn(0x55e6eca69310 msgr2=0x55e6ecaae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- 172.21.15.103:0/317623998 shutdown_connections 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- 172.21.15.103:0/317623998 wait complete. 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- start start 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 Processor -- start 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecab8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecab3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecab3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01aca30 con 0x55e6ecaaf780 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01ac360 con 0x55e6ecaa05d0 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.442+0000 7f4bf917a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01a9590 con 0x55e6ecaa1b40 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf917a740 10 client.0 _ll_register_callbacks cb 0x7f4bdc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:47.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecab3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecab3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40824/0 (socket says 172.21.15.103:40824) 2024-09-19T07:48:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 -- 172.21.15.103:0/1726905456 learned_addr learned my addr 172.21.15.103:0/1726905456 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecab8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 -- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 msgr2=0x55e6ecab3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4beffff640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecab3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecab3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 -- 172.21.15.103:0/1726905456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 msgr2=0x55e6ecab8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecab8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 -- 172.21.15.103:0/1726905456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ecb28a80 con 0x55e6ecaaf780 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecab8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf533e640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecab3c10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4be802fe00 tx=0x7f4be8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14579deab74c86e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bcffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bcffff640 1 -- 172.21.15.103:0/1726905456 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be805e020 con 0x55e6ecaaf780 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bcffff640 1 -- 172.21.15.103:0/1726905456 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f4be802f610 con 0x55e6ecaaf780 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e6eca0dd70 con 0x55e6ecaaf780 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bcffff640 1 -- 172.21.15.103:0/1726905456 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4be8069050 con 0x55e6ecaaf780 2024-09-19T07:48:47.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf917a740 10 client.17598 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.443+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e6ecb29b80 con 0x55e6ecaaf780 2024-09-19T07:48:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bcffff640 1 -- 172.21.15.103:0/1726905456 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e6eca0dd70 con 0x55e6ecaaf780 2024-09-19T07:48:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bcffff640 1 -- 172.21.15.103:0/1726905456 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f4be8003f30 con 0x55e6ecaaf780 2024-09-19T07:48:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf917a740 10 client.17598 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf917a740 10 client.17598 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:48:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf917a740 20 client.17598 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:47.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf917a740 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecb2a2c0 0x55e6ecb2c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf917a740 4 client.17598 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6ec98a170 con 0x55e6ecb2a2c0 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecb2a2c0 0x55e6ecb2c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bf4b3d640 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecb2a2c0 0x55e6ecb2c6e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f4cd003baa84a0b server_cookie=c3c9d57c60998329 in_seq=0 out_seq=0 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.444+0000 7f4bcffff640 10 client.17598 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bcffff640 1 -- 172.21.15.103:0/1726905456 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6ec98a170 con 0x55e6ecb2a2c0 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bcffff640 10 client.17598 handle_command_reply: tid=0 2024-09-19T07:48:47.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 client.17598 shutdown 2024-09-19T07:48:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 msgr2=0x55e6ecab3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecab3c10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4be802fe00 tx=0x7f4be8005fd0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 shutdown_connections 2024-09-19T07:48:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e6ecb2a2c0 0x55e6ecb2c6e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6ecaaf780 0x55e6ecab3c10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6ecaa1b40 0x55e6ecab3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 --2- 172.21.15.103:0/1726905456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6ecaa05d0 0x55e6ecab8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 >> 172.21.15.103:0/1726905456 conn(0x55e6eca69310 msgr2=0x55e6ecaa2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bcffff640 0 client.17598 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:48:47.474 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 shutdown_connections 2024-09-19T07:48:47.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 1 -- 172.21.15.103:0/1726905456 wait complete. 2024-09-19T07:48:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.445+0000 7f4bf917a740 20 client.17598 trim_cache size 0 max 16384 2024-09-19T07:48:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4be00a33b0 msgr2=0x7f4be00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4be00a33b0 0x7f4be00a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009bd0 tx=0x7f4be4005f90 comp rx=0 tx=0).stop 2024-09-19T07:48:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0102790 msgr2=0x7f4bf01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:47.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0102790 0x7f4bf01988f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00e9f0 tx=0x7f4bdc00edf0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 shutdown_connections 2024-09-19T07:48:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f4be00a33b0 0x7f4be00a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4bf0106f10 0x7f4bf019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bf0102790 0x7f4bf01988f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 --2- 172.21.15.103:0/957283299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf0101de0 0x7f4bf01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:47.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 >> 172.21.15.103:0/957283299 conn(0x7f4bf00fd8c0 msgr2=0x7f4bf00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:47.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 shutdown_connections 2024-09-19T07:48:47.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:47.446+0000 7f4bf6da1640 1 -- 172.21.15.103:0/957283299 wait complete. 2024-09-19T07:48:48.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:48:48.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:48 smithi134 ceph-mon[35414]: pgmap v335: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 623 op/s 2024-09-19T07:48:48.599 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:48 smithi174 ceph-mon[35370]: pgmap v335: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 623 op/s 2024-09-19T07:48:48.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:48 smithi103 ceph-mon[31201]: pgmap v335: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 623 op/s 2024-09-19T07:48:49.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.051+0000 7f26f9317640 1 -- 172.21.15.103:0/775667599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 msgr2=0x7f26f4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.051+0000 7f26f9317640 1 --2- 172.21.15.103:0/775667599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4100ee0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009930 tx=0x7f26e802a420 comp rx=0 tx=0).stop 2024-09-19T07:48:49.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 -- 172.21.15.103:0/775667599 shutdown_connections 2024-09-19T07:48:49.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 --2- 172.21.15.103:0/775667599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f4101420 0x7f26f410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 --2- 172.21.15.103:0/775667599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4100ee0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 --2- 172.21.15.103:0/775667599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f4106a80 0x7f26f4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 -- 172.21.15.103:0/775667599 >> 172.21.15.103:0/775667599 conn(0x7f26f40fc7a0 msgr2=0x7f26f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 -- 172.21.15.103:0/775667599 shutdown_connections 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 -- 172.21.15.103:0/775667599 wait complete. 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 Processor -- start 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.052+0000 7f26f9317640 1 -- start start 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f4101420 0x7f26f4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f4106a80 0x7f26f419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4110e40 con 0x7f26f4100a80 2024-09-19T07:48:49.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4110cc0 con 0x7f26f4106a80 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4110fc0 con 0x7f26f4101420 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56874/0 (socket says 172.21.15.103:56874) 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1127452327 learned_addr learned my addr 172.21.15.103:0/1127452327 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1127452327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f4101420 msgr2=0x7f26f4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f4101420 0x7f26f4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1127452327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f4106a80 msgr2=0x7f26f419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f4106a80 0x7f26f419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1127452327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f419d3c0 con 0x7f26f4100a80 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4198410 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f26e400eab0 tx=0x7f26e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26dbfff640 1 -- 172.21.15.103:0/1127452327 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e400ce40 con 0x7f26f4100a80 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f419d6b0 con 0x7f26f4100a80 2024-09-19T07:48:49.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26dbfff640 1 -- 172.21.15.103:0/1127452327 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e4011950 con 0x7f26f4100a80 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26dbfff640 1 -- 172.21.15.103:0/1127452327 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4010ae0 con 0x7f26f4100a80 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.053+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f41a4f90 con 0x7f26f4100a80 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.056+0000 7f26dbfff640 1 -- 172.21.15.103:0/1127452327 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f26e4011b00 con 0x7f26f4100a80 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.056+0000 7f26fb6f0740 1 Processor -- start 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.056+0000 7f26fb6f0740 1 -- start start 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc56370 0x56330cc58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56330cbb12f0 con 0x56330cc56370 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56330cb58cf0 con 0x56330cc4cb40 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56330cbad2b0 con 0x56330cc4b5d0 2024-09-19T07:48:49.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc56370 0x56330cc58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc56370 0x56330cc58760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56878/0 (socket says 172.21.15.103:56878) 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 -- 172.21.15.103:0/176725148 learned_addr learned my addr 172.21.15.103:0/176725148 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26dbfff640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26dc0a33b0 0x7f26dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f37fe640 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 -- 172.21.15.103:0/176725148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 msgr2=0x56330cc4ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc4ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f37fe640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26dc0a33b0 0x7f26dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 -- 172.21.15.103:0/176725148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 msgr2=0x56330cc4b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc4b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 -- 172.21.15.103:0/176725148 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330cb3d3a0 con 0x56330cc56370 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26dbfff640 1 -- 172.21.15.103:0/1127452327 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f26e40d0420 con 0x7f26f4100a80 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.057+0000 7f26f27fc640 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc56370 0x56330cc58760 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f26e8031530 tx=0x7f26e8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5080f87745c52f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26f37fe640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26dc0a33b0 0x7f26dc0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0058c0 tx=0x7f26ec005850 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26d97fa640 1 -- 172.21.15.103:0/176725148 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8040070 con 0x56330cc56370 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26d97fa640 1 -- 172.21.15.103:0/176725148 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e8004040 con 0x56330cc56370 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26d97fa640 1 -- 172.21.15.103:0/176725148 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8035040 con 0x56330cc56370 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 -- 172.21.15.103:0/176725148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc56370 msgr2=0x56330cc58760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc56370 0x56330cc58760 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f26e8031530 tx=0x7f26e8005d50 comp rx=0 tx=0).stop 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 -- 172.21.15.103:0/176725148 shutdown_connections 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc56370 0x56330cc58760 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc4ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/176725148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 -- 172.21.15.103:0/176725148 >> 172.21.15.103:0/176725148 conn(0x56330cc14310 msgr2=0x56330cc47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 -- 172.21.15.103:0/176725148 shutdown_connections 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.058+0000 7f26fb6f0740 1 -- 172.21.15.103:0/176725148 wait complete. 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 -- start start 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 Processor -- start 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc55c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc4cb40 0x56330cc52f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc56370 0x56330cc53480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56330cc59010 con 0x56330cc4cb40 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56330cb6c060 con 0x56330cc56370 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56330cc59190 con 0x56330cc4b5d0 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26fb6f0740 10 client.0 _ll_register_callbacks cb 0x56330c8bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc4cb40 0x56330cc52f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc4cb40 0x56330cc52f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56882/0 (socket says 172.21.15.103:56882) 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 -- 172.21.15.103:0/2895773263 learned_addr learned my addr 172.21.15.103:0/2895773263 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc55c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f37fe640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc56370 0x56330cc53480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 -- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 msgr2=0x56330cc55c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc55c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 -- 172.21.15.103:0/2895773263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc56370 msgr2=0x56330cc53480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc56370 0x56330cc53480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f27fc640 1 -- 172.21.15.103:0/2895773263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330c9223c0 con 0x56330cc4cb40 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.059+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc55c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26f27fc640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc4cb40 0x56330cc52f40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f26e8005c40 tx=0x7f26e8032ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e116b964b173bd92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26d6ff5640 1 -- 172.21.15.103:0/2895773263 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8035070 con 0x56330cc4cb40 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e8004040 con 0x56330cc4cb40 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26d6ff5640 1 -- 172.21.15.103:0/2895773263 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f26e80054b0 con 0x56330cc4cb40 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26d6ff5640 1 -- 172.21.15.103:0/2895773263 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f26e8040050 con 0x56330cc4cb40 2024-09-19T07:48:49.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 10 client.17610 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56330cc64a00 con 0x56330cc4cb40 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26d6ff5640 1 -- 172.21.15.103:0/2895773263 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e8004040 con 0x56330cc4cb40 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26d6ff5640 1 -- 172.21.15.103:0/2895773263 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f26e80035f0 con 0x56330cc4cb40 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 10 client.17610 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 10 client.17610 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 20 client.17610 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc65d50 0x56330cc68140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 4 client.17610 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.060+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56330cbbe860 con 0x56330cc65d50 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.061+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc65d50 0x56330cc68140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.061+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc65d50 0x56330cc68140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3dc8574960c9afb server_cookie=ff0ca22730a4edac in_seq=0 out_seq=0 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.061+0000 7f26d6ff5640 10 client.17610 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26d6ff5640 1 -- 172.21.15.103:0/2895773263 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56330cbbe860 con 0x56330cc65d50 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26d6ff5640 10 client.17610 handle_command_reply: tid=0 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 client.17610 shutdown 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc4cb40 msgr2=0x56330cc52f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc4cb40 0x56330cc52f40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f26e8005c40 tx=0x7f26e8032ee0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 shutdown_connections 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc65d50 0x56330cc68140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc56370 0x56330cc53480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc4cb40 0x56330cc52f40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26d6ff5640 0 client.17610 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/2895773263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc55c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 >> 172.21.15.103:0/2895773263 conn(0x56330cc14310 msgr2=0x56330cc4d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 shutdown_connections 2024-09-19T07:48:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 1 -- 172.21.15.103:0/2895773263 wait complete. 2024-09-19T07:48:49.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.062+0000 7f26fb6f0740 20 client.17610 trim_cache size 0 max 16384 2024-09-19T07:48:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 Processor -- start 2024-09-19T07:48:49.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 -- start start 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc51f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc6d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41ad830 con 0x56330cc14310 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41af3a0 con 0x56330cc4cb40 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41adf30 con 0x56330cc4b5d0 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.075+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56888/0 (socket says 172.21.15.103:56888) 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1893365579 learned_addr learned my addr 172.21.15.103:0/1893365579 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1893365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 msgr2=0x56330cc51f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/1893365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc51f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1893365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 msgr2=0x56330cc6d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/1893365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc6d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26f2ffd640 1 -- 172.21.15.103:0/1893365579 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330cc48e20 con 0x56330cc14310 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/1893365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc622e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f26e400c700 tx=0x7f26e40d2930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0076c894ecf88b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26d8ff9640 1 -- 172.21.15.103:0/1893365579 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e409a9f0 con 0x56330cc14310 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26d8ff9640 1 -- 172.21.15.103:0/1893365579 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e40cdde0 con 0x56330cc14310 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.076+0000 7f26d8ff9640 1 -- 172.21.15.103:0/1893365579 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e40202b0 con 0x56330cc14310 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- 172.21.15.103:0/1893365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 msgr2=0x56330cc622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/1893365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc622e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f26e400c700 tx=0x7f26e40d2930 comp rx=0 tx=0).stop 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- 172.21.15.103:0/1893365579 shutdown_connections 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/1893365579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc6d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/1893365579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc51f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/1893365579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc622e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- 172.21.15.103:0/1893365579 >> 172.21.15.103:0/1893365579 conn(0x56330cc535e0 msgr2=0x56330cc51810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- 172.21.15.103:0/1893365579 shutdown_connections 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- 172.21.15.103:0/1893365579 wait complete. 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- start start 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 Processor -- start 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc5a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc5a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc5acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41aecd0 con 0x56330cc14310 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41ae600 con 0x56330cc4cb40 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41ab830 con 0x56330cc4b5d0 2024-09-19T07:48:49.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.077+0000 7f26fb6f0740 10 client.0 _ll_register_callbacks cb 0x7f26e8031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc5a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc5a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56892/0 (socket says 172.21.15.103:56892) 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 -- 172.21.15.103:0/4173878599 learned_addr learned my addr 172.21.15.103:0/4173878599 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f27fc640 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc5a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f37fe640 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc5acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 -- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 msgr2=0x56330cc5a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc5a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 -- 172.21.15.103:0/4173878599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 msgr2=0x56330cc5acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc5acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 -- 172.21.15.103:0/4173878599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330cc685a0 con 0x56330cc14310 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26f2ffd640 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc5a270 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f26e409cfa0 tx=0x7f26e4010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab71d7ebd068a99f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:49.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26d6ff5640 1 -- 172.21.15.103:0/4173878599 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e40b0c10 con 0x56330cc14310 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26d6ff5640 1 -- 172.21.15.103:0/4173878599 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f26e40202b0 con 0x56330cc14310 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e40cdde0 con 0x56330cc14310 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26d6ff5640 1 -- 172.21.15.103:0/4173878599 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f26e40204b0 con 0x56330cc14310 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26fb6f0740 10 client.17616 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56330ccd0980 con 0x56330cc14310 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.078+0000 7f26d6ff5640 1 -- 172.21.15.103:0/4173878599 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e40cdde0 con 0x56330cc14310 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26d6ff5640 1 -- 172.21.15.103:0/4173878599 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f26e400f040 con 0x56330cc14310 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26fb6f0740 10 client.17616 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26fb6f0740 10 client.17616 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26fb6f0740 20 client.17616 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc56370 0x56330ccd2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:49.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26fb6f0740 4 client.17616 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56330cbb8d70 con 0x56330cc56370 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26f27fc640 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc56370 0x56330ccd2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26f27fc640 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc56370 0x56330ccd2fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b06c5b6c726c70 server_cookie=12d74c7f9e61b055 in_seq=0 out_seq=0 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.079+0000 7f26d6ff5640 10 client.17616 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26d6ff5640 1 -- 172.21.15.103:0/4173878599 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56330cbb8d70 con 0x56330cc56370 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26d6ff5640 10 client.17616 handle_command_reply: tid=0 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 client.17616 shutdown 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 msgr2=0x56330cc5a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc5a270 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f26e409cfa0 tx=0x7f26e4010540 comp rx=0 tx=0).stop 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 shutdown_connections 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56330cc56370 0x56330ccd2fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56330cc4cb40 0x56330cc5acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56330cc4b5d0 0x56330cc5a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26d6ff5640 0 client.17616 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 --2- 172.21.15.103:0/4173878599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56330cc14310 0x56330cc5a270 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 >> 172.21.15.103:0/4173878599 conn(0x56330cc535e0 msgr2=0x56330cc52030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 shutdown_connections 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 1 -- 172.21.15.103:0/4173878599 wait complete. 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26fb6f0740 20 client.17616 trim_cache size 0 max 16384 2024-09-19T07:48:49.093 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26dc0a33b0 msgr2=0x7f26dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26f9317640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26dc0a33b0 0x7f26dc0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0058c0 tx=0x7f26ec005850 comp rx=0 tx=0).stop 2024-09-19T07:48:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 msgr2=0x7f26f4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.080+0000 7f26f9317640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4198410 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f26e400eab0 tx=0x7f26e400ef70 comp rx=0 tx=0).stop 2024-09-19T07:48:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 shutdown_connections 2024-09-19T07:48:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26dc0a33b0 0x7f26dc0a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26f4106a80 0x7f26f419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f4101420 0x7f26f4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 --2- 172.21.15.103:0/1127452327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f4100a80 0x7f26f4198410 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:49.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 >> 172.21.15.103:0/1127452327 conn(0x7f26f40fc7a0 msgr2=0x7f26f4104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:49.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 shutdown_connections 2024-09-19T07:48:49.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:49.081+0000 7f26f9317640 1 -- 172.21.15.103:0/1127452327 wait complete. 2024-09-19T07:48:49.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:48:50.144 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:50.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 615.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:50 smithi134 ceph-mon[35414]: pgmap v336: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 620 op/s 2024-09-19T07:48:50.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:50 smithi103 ceph-mon[31201]: pgmap v336: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 620 op/s 2024-09-19T07:48:50.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 -- 172.21.15.103:0/2653240558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 msgr2=0x7f73b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 --2- 172.21.15.103:0/2653240558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4100ee0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009930 tx=0x7f73a402a420 comp rx=0 tx=0).stop 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 -- 172.21.15.103:0/2653240558 shutdown_connections 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 --2- 172.21.15.103:0/2653240558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b4101420 0x7f73b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 --2- 172.21.15.103:0/2653240558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4100ee0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 --2- 172.21.15.103:0/2653240558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73b4106a80 0x7f73b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 -- 172.21.15.103:0/2653240558 >> 172.21.15.103:0/2653240558 conn(0x7f73b40fc7a0 msgr2=0x7f73b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 -- 172.21.15.103:0/2653240558 shutdown_connections 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 -- 172.21.15.103:0/2653240558 wait complete. 2024-09-19T07:48:50.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.661+0000 7f73b9000640 1 Processor -- start 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b9000640 1 -- start start 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b4101420 0x7f73b41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b9000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73b4106a80 0x7f73b419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b9000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4110e80 con 0x7f73b4100a80 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b9000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4110d00 con 0x7f73b4106a80 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b9000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4111000 con 0x7f73b4101420 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56904/0 (socket says 172.21.15.103:56904) 2024-09-19T07:48:50.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 -- 172.21.15.103:0/1430435554 learned_addr learned my addr 172.21.15.103:0/1430435554 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 -- 172.21.15.103:0/1430435554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b4101420 msgr2=0x7f73b41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b4101420 0x7f73b41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73b4106a80 0x7f73b419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b4101420 0x7f73b41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 -- 172.21.15.103:0/1430435554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73b4106a80 msgr2=0x7f73b419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73b4106a80 0x7f73b419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 -- 172.21.15.103:0/1430435554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b419d410 con 0x7f73b4100a80 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b4101420 0x7f73b41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73b4106a80 0x7f73b419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73b259c640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4198460 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c890 tx=0x7f739c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73ab7fe640 1 -- 172.21.15.103:0/1430435554 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c004220 con 0x7f73b4100a80 2024-09-19T07:48:50.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73ab7fe640 1 -- 172.21.15.103:0/1430435554 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f739c0043c0 con 0x7f73b4100a80 2024-09-19T07:48:50.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.662+0000 7f73ab7fe640 1 -- 172.21.15.103:0/1430435554 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c0049e0 con 0x7f73b4100a80 2024-09-19T07:48:50.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.663+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b419d700 con 0x7f73b4100a80 2024-09-19T07:48:50.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.663+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b41a4fe0 con 0x7f73b4100a80 2024-09-19T07:48:50.669 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:50 smithi174 ceph-mon[35370]: pgmap v336: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 620 op/s 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.665+0000 7f73ab7fe640 1 -- 172.21.15.103:0/1430435554 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f739c0079b0 con 0x7f73b4100a80 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 Processor -- start 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 -- start start 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e68765d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e68765da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e68766a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6875c32f0 con 0x55e68765eb40 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e68756acf0 con 0x55e687668370 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6875bf2b0 con 0x55e68765d5d0 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e68765d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73b259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e68765da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e68765d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56910/0 (socket says 172.21.15.103:56910) 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73b2d9d640 1 -- 172.21.15.103:0/2107173262 learned_addr learned my addr 172.21.15.103:0/2107173262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e68766a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.666+0000 7f73ab7fe640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f73a00a3430 0x7f73a00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f73a00a3430 0x7f73a00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73ab7fe640 1 -- 172.21.15.103:0/1430435554 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f739c0cfd40 con 0x7f73b4100a80 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73b2d9d640 1 -- 172.21.15.103:0/2107173262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 msgr2=0x55e68765da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e68765da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73b2d9d640 1 -- 172.21.15.103:0/2107173262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 msgr2=0x55e68766a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e68766a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73b2d9d640 1 -- 172.21.15.103:0/2107173262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68754f3a0 con 0x55e68765eb40 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e68765d090 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f73ac013950 tx=0x7f73ac013e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a052c26ab1921ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73a8ff9640 1 -- 172.21.15.103:0/2107173262 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac013420 con 0x55e68765eb40 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73a8ff9640 1 -- 172.21.15.103:0/2107173262 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73ac011c40 con 0x55e68765eb40 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.667+0000 7f73a8ff9640 1 -- 172.21.15.103:0/2107173262 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac005280 con 0x55e68765eb40 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 -- 172.21.15.103:0/2107173262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 msgr2=0x55e68765d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f73a00a3430 0x7f73a00a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73ac020000 tx=0x7f73ac01f000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e68765d090 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f73ac013950 tx=0x7f73ac013e10 comp rx=0 tx=0).stop 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 -- 172.21.15.103:0/2107173262 shutdown_connections 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e68766a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e68765da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/2107173262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e68765d090 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 -- 172.21.15.103:0/2107173262 >> 172.21.15.103:0/2107173262 conn(0x55e687626310 msgr2=0x55e687659c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 -- 172.21.15.103:0/2107173262 shutdown_connections 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 -- 172.21.15.103:0/2107173262 wait complete. 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 -- start start 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.668+0000 7f73bb3d9740 1 Processor -- start 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e687667d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765eb40 0x55e687665010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687665600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e68766b070 con 0x55e687668370 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e68757e060 con 0x55e68765eb40 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e68766b1f0 con 0x55e68765d5d0 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73bb3d9740 10 client.0 _ll_register_callbacks cb 0x55e6872d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687665600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687665600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56918/0 (socket says 172.21.15.103:56918) 2024-09-19T07:48:50.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 -- 172.21.15.103:0/1028633038 learned_addr learned my addr 172.21.15.103:0/1028633038 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765eb40 0x55e687665010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 -- 172.21.15.103:0/1028633038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 msgr2=0x55e687667d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b259c640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e687667d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e687667d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 -- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765eb40 msgr2=0x55e687665010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765eb40 0x55e687665010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 -- 172.21.15.103:0/1028633038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6873343c0 con 0x55e687668370 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765eb40 0x55e687665010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b259c640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e687667d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687665600 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0096c0 tx=0x7f73ac009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ef1da2494acaba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.669+0000 7f7399ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f7399ffb640 1 -- 172.21.15.103:0/1028633038 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011df0 con 0x55e687668370 2024-09-19T07:48:50.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f7399ffb640 1 -- 172.21.15.103:0/1028633038 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f73ac005000 con 0x55e687668370 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73ac011c40 con 0x55e687668370 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f7399ffb640 1 -- 172.21.15.103:0/1028633038 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f73ac03c050 con 0x55e687668370 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 10 client.17628 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e687676b10 con 0x55e687668370 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f7399ffb640 1 -- 172.21.15.103:0/1028633038 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73ac011c40 con 0x55e687668370 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f7399ffb640 1 -- 172.21.15.103:0/1028633038 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f73ac037030 con 0x55e687668370 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 10 client.17628 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 10 client.17628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 20 client.17628 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e68767a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 4 client.17628 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.670+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6875d0860 con 0x55e687677ec0 2024-09-19T07:48:50.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.671+0000 7f73b259c640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e68767a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.671+0000 7f73b259c640 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e68767a2b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8eda0b18def9ca server_cookie=50d5796b9ad87516 in_seq=0 out_seq=0 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.671+0000 7f7399ffb640 10 client.17628 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f7399ffb640 1 -- 172.21.15.103:0/1028633038 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6875d0860 con 0x55e687677ec0 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f7399ffb640 10 client.17628 handle_command_reply: tid=0 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 client.17628 shutdown 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 msgr2=0x55e687665600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687665600 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0096c0 tx=0x7f73ac009890 comp rx=0 tx=0).stop 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 shutdown_connections 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e68767a2b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f7399ffb640 0 client.17628 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687665600 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765eb40 0x55e687665010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1028633038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e687667d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 >> 172.21.15.103:0/1028633038 conn(0x55e687626310 msgr2=0x55e68765f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 shutdown_connections 2024-09-19T07:48:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1028633038 wait complete. 2024-09-19T07:48:50.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.672+0000 7f73bb3d9740 20 client.17628 trim_cache size 0 max 16384 2024-09-19T07:48:50.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 Processor -- start 2024-09-19T07:48:50.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 -- start start 2024-09-19T07:48:50.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765d5d0 0x55e687674310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765eb40 0x55e687660120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687660660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73b41ada70 con 0x55e687668370 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73b41af5e0 con 0x55e68765d5d0 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73b41ae170 con 0x55e68765eb40 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687660660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687660660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56922/0 (socket says 172.21.15.103:56922) 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 -- 172.21.15.103:0/3312262964 learned_addr learned my addr 172.21.15.103:0/3312262964 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b259c640 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765d5d0 0x55e687674310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 -- 172.21.15.103:0/3312262964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765eb40 msgr2=0x55e687660120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765eb40 0x55e687660120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 -- 172.21.15.103:0/3312262964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765d5d0 msgr2=0x55e687674310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765d5d0 0x55e687674310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 -- 172.21.15.103:0/3312262964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6876680d0 con 0x55e687668370 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b259c640 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765d5d0 0x55e687674310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:50.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.685+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687660660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f73ac013950 tx=0x7f73ac009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630a8531833482a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f739bfff640 1 -- 172.21.15.103:0/3312262964 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011750 con 0x55e687668370 2024-09-19T07:48:50.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f739bfff640 1 -- 172.21.15.103:0/3312262964 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73ac0059c0 con 0x55e687668370 2024-09-19T07:48:50.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f739bfff640 1 -- 172.21.15.103:0/3312262964 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011c40 con 0x55e687668370 2024-09-19T07:48:50.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 -- 172.21.15.103:0/3312262964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 msgr2=0x55e687660660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687660660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f73ac013950 tx=0x7f73ac009f10 comp rx=0 tx=0).stop 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 -- 172.21.15.103:0/3312262964 shutdown_connections 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e687668370 0x55e687660660 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765eb40 0x55e687660120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/3312262964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e68765d5d0 0x55e687674310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 -- 172.21.15.103:0/3312262964 >> 172.21.15.103:0/3312262964 conn(0x55e687626310 msgr2=0x55e687664160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 -- 172.21.15.103:0/3312262964 shutdown_connections 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 -- 172.21.15.103:0/3312262964 wait complete. 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.686+0000 7f73bb3d9740 1 -- start start 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 1 Processor -- start 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e6876708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e687661a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e687661f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73b41aef10 con 0x55e68765eb40 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73b41ae840 con 0x55e687668370 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73b41aba70 con 0x55e68765d5d0 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73bb3d9740 10 client.0 _ll_register_callbacks cb 0x7f73a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e687661a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e687661a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56932/0 (socket says 172.21.15.103:56932) 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 -- 172.21.15.103:0/1721642865 learned_addr learned my addr 172.21.15.103:0/1721642865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b259c640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e6876708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 -- 172.21.15.103:0/1721642865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 msgr2=0x55e6876708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e687661f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e6876708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 -- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 msgr2=0x55e687661f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e687661f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b1d9b640 1 -- 172.21.15.103:0/1721642865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e687662490 con 0x55e68765eb40 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b259c640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e6876708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.687+0000 7f73b2d9d640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e687661f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73b1d9b640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e687661a10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009900 tx=0x7f73a40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33898076427c7deb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f7399ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f7399ffb640 1 -- 172.21.15.103:0/1721642865 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4035040 con 0x55e68765eb40 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f7399ffb640 1 -- 172.21.15.103:0/1721642865 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f73a4004040 con 0x55e68765eb40 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73ac0059c0 con 0x55e68765eb40 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f7399ffb640 1 -- 172.21.15.103:0/1721642865 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f73a4004240 con 0x55e68765eb40 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 10 client.17634 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e6876e2510 con 0x55e68765eb40 2024-09-19T07:48:50.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f7399ffb640 1 -- 172.21.15.103:0/1721642865 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73ac0059c0 con 0x55e68765eb40 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f7399ffb640 1 -- 172.21.15.103:0/1721642865 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f73a40048c0 con 0x55e68765eb40 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 10 client.17634 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 10 client.17634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 20 client.17634 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e6876e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 4 client.17634 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.688+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6875cad70 con 0x55e687677ec0 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73b259c640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e6876e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73b259c640 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e6876e4d50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2490848505970640 server_cookie=8ce4193af2dad81e in_seq=0 out_seq=0 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f7399ffb640 10 client.17634 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f7399ffb640 1 -- 172.21.15.103:0/1721642865 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6875cad70 con 0x55e687677ec0 2024-09-19T07:48:50.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f7399ffb640 10 client.17634 handle_command_reply: tid=0 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 client.17634 shutdown 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 msgr2=0x55e687661a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e687661a10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009900 tx=0x7f73a40317e0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 shutdown_connections 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55e687677ec0 0x55e6876e4d50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e687668370 0x55e687661f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f7399ffb640 0 client.17634 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e68765eb40 0x55e687661a10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 --2- 172.21.15.103:0/1721642865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e68765d5d0 0x55e6876708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.689+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 >> 172.21.15.103:0/1721642865 conn(0x55e687626310 msgr2=0x55e687666870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 shutdown_connections 2024-09-19T07:48:50.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73bb3d9740 1 -- 172.21.15.103:0/1721642865 wait complete. 2024-09-19T07:48:50.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73bb3d9740 20 client.17634 trim_cache size 0 max 16384 2024-09-19T07:48:50.697 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:50.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f73a00a3430 msgr2=0x7f73a00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f73a00a3430 0x7f73a00a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73ac020000 tx=0x7f73ac01f000 comp rx=0 tx=0).stop 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 msgr2=0x7f73b4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4198460 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f739c00c890 tx=0x7f739c00cd50 comp rx=0 tx=0).stop 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 shutdown_connections 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f73a00a3430 0x7f73a00a58f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73b4106a80 0x7f73b419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b4101420 0x7f73b41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 --2- 172.21.15.103:0/1430435554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73b4100a80 0x7f73b4198460 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 >> 172.21.15.103:0/1430435554 conn(0x7f73b40fc7a0 msgr2=0x7f73b4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 shutdown_connections 2024-09-19T07:48:50.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:50.690+0000 7f73b9000640 1 -- 172.21.15.103:0/1430435554 wait complete. 2024-09-19T07:48:51.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:48:51.660 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:52.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 -- 172.21.15.103:0/3891288725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 msgr2=0x7f3b4c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3891288725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c103190 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009930 tx=0x7f3b4002a420 comp rx=0 tx=0).stop 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 -- 172.21.15.103:0/3891288725 shutdown_connections 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3891288725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b4c1082e0 0x7f3b4c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3891288725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c103760 0x7f3b4c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3891288725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c103190 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 -- 172.21.15.103:0/3891288725 >> 172.21.15.103:0/3891288725 conn(0x7f3b4c0fc7d0 msgr2=0x7f3b4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 -- 172.21.15.103:0/3891288725 shutdown_connections 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.150+0000 7f3b52e19640 1 -- 172.21.15.103:0/3891288725 wait complete. 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 Processor -- start 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 -- start start 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b4c103760 0x7f3b4c100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c1082e0 0x7f3b4c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1120b0 con 0x7f3b4c102db0 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c111f30 con 0x7f3b4c103760 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b52e19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c112230 con 0x7f3b4c1082e0 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b50bb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b50bb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56956/0 (socket says 172.21.15.103:56956) 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.151+0000 7f3b50bb5640 1 -- 172.21.15.103:0/3164199516 learned_addr learned my addr 172.21.15.103:0/3164199516 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b4c103760 0x7f3b4c100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b513b6640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c1082e0 0x7f3b4c101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b50bb5640 1 -- 172.21.15.103:0/3164199516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c1082e0 msgr2=0x7f3b4c101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c1082e0 0x7f3b4c101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b50bb5640 1 -- 172.21.15.103:0/3164199516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b4c103760 msgr2=0x7f3b4c100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b4c103760 0x7f3b4c100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b50bb5640 1 -- 172.21.15.103:0/3164199516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c1a6b40 con 0x7f3b4c102db0 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b4c103760 0x7f3b4c100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c100a10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3b40004770 tx=0x7f3b40003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b49ffb640 1 -- 172.21.15.103:0/3164199516 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4003f070 con 0x7f3b4c102db0 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1a6dd0 con 0x7f3b4c102db0 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b49ffb640 1 -- 172.21.15.103:0/3164199516 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b40031c20 con 0x7f3b4c102db0 2024-09-19T07:48:52.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b49ffb640 1 -- 172.21.15.103:0/3164199516 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40035030 con 0x7f3b4c102db0 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.152+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a7250 con 0x7f3b4c102db0 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b49ffb640 1 -- 172.21.15.103:0/3164199516 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3b40042070 con 0x7f3b4c102db0 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 Processor -- start 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 -- start start 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fb1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564067f172f0 con 0x564067fb2b40 2024-09-19T07:48:52.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564067ebecf0 con 0x564067fbc370 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b551f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564067f132b0 con 0x564067fb15d0 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56970/0 (socket says 172.21.15.103:56970) 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 -- 172.21.15.103:0/1213672380 learned_addr learned my addr 172.21.15.103:0/1213672380 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fb1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b49ffb640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fbe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 -- 172.21.15.103:0/1213672380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 msgr2=0x564067fb1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fb1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 -- 172.21.15.103:0/1213672380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 msgr2=0x564067fbe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fbe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b513b6640 1 -- 172.21.15.103:0/1213672380 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564067ea33a0 con 0x564067fb2b40 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.155+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fbe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b49ffb640 1 -- 172.21.15.103:0/3164199516 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f3b400f0610 con 0x7f3b4c102db0 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b513b6640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b513b6640 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb1090 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3b44008e00 tx=0x7f3b4400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b654c39f149db12d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b36ffd640 1 -- 172.21.15.103:0/1213672380 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4401f070 con 0x564067fb2b40 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b36ffd640 1 -- 172.21.15.103:0/1213672380 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b44019430 con 0x564067fb2b40 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b36ffd640 1 -- 172.21.15.103:0/1213672380 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44010650 con 0x564067fb2b40 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b551f2740 1 -- 172.21.15.103:0/1213672380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 msgr2=0x564067fb1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b513b6640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3b4401c000 tx=0x7f3b44010d80 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.156+0000 7f3b551f2740 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb1090 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3b44008e00 tx=0x7f3b4400d710 comp rx=0 tx=0).stop 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- 172.21.15.103:0/1213672380 shutdown_connections 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fbe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fb1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 --2- 172.21.15.103:0/1213672380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb1090 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- 172.21.15.103:0/1213672380 >> 172.21.15.103:0/1213672380 conn(0x564067f7a310 msgr2=0x564067fadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- 172.21.15.103:0/1213672380 shutdown_connections 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- 172.21.15.103:0/1213672380 wait complete. 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- start start 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 Processor -- start 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fbbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fb9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564067fbf070 con 0x564067fb2b40 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564067ed2060 con 0x564067fbc370 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564067fbf1f0 con 0x564067fb15d0 2024-09-19T07:48:52.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.157+0000 7f3b551f2740 10 client.0 _ll_register_callbacks cb 0x564067c258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56984/0 (socket says 172.21.15.103:56984) 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 -- 172.21.15.103:0/3140258542 learned_addr learned my addr 172.21.15.103:0/3140258542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b513b6640 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fb9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fbbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 -- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 msgr2=0x564067fbbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fbbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 -- 172.21.15.103:0/3140258542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 msgr2=0x564067fb9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fb9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 -- 172.21.15.103:0/3140258542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564067c883c0 con 0x564067fb2b40 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb9010 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800de60 tx=0x7f3b3800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353fd08a0c299fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:52.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3140258542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38011850 con 0x564067fb2b40 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3140258542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f3b380119f0 con 0x564067fb2b40 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b44019430 con 0x564067fb2b40 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3140258542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3b38011bf0 con 0x564067fb2b40 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b551f2740 10 client.17646 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564067fcaae0 con 0x564067fb2b40 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.158+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3140258542 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b44019430 con 0x564067fb2b40 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3140258542 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f3b380103d0 con 0x564067fb2b40 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b551f2740 10 client.17646 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b551f2740 10 client.17646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b551f2740 20 client.17646 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564067fcbff0 0x564067fce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b551f2740 4 client.17646 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:52.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564067f24860 con 0x564067fcbff0 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.159+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564067fcbff0 0x564067fce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b2bfff640 10 client.17646 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564067fcbff0 0x564067fce430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2145d45259d5fb21 server_cookie=d1031ca502921d94 in_seq=0 out_seq=0 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3140258542 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564067f24860 con 0x564067fcbff0 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b2bfff640 10 client.17646 handle_command_reply: tid=0 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 client.17646 shutdown 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 msgr2=0x564067fb9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb9010 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800de60 tx=0x7f3b3800f730 comp rx=0 tx=0).stop 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 shutdown_connections 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564067fcbff0 0x564067fce430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fbc370 0x564067fb9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb2b40 0x564067fb9010 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3140258542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb15d0 0x564067fbbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 >> 172.21.15.103:0/3140258542 conn(0x564067f7a310 msgr2=0x564067fb3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.160+0000 7f3b2bfff640 0 client.17646 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:52.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.161+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 shutdown_connections 2024-09-19T07:48:52.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.161+0000 7f3b551f2740 1 -- 172.21.15.103:0/3140258542 wait complete. 2024-09-19T07:48:52.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.161+0000 7f3b551f2740 20 client.17646 trim_cache size 0 max 16384 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 Processor -- start 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 -- start start 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb15d0 0x564067fb8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb2b40 0x564067fc8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fbc370 0x564067fd3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1afb70 con 0x564067fb15d0 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1b16e0 con 0x564067fb2b40 2024-09-19T07:48:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b551f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1b0270 con 0x564067fbc370 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb15d0 0x564067fb8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb15d0 0x564067fb8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56994/0 (socket says 172.21.15.103:56994) 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 -- 172.21.15.103:0/2441257942 learned_addr learned my addr 172.21.15.103:0/2441257942 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb2b40 0x564067fc8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b513b6640 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fbc370 0x564067fd3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 -- 172.21.15.103:0/2441257942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fbc370 msgr2=0x564067fd3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fbc370 0x564067fd3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 -- 172.21.15.103:0/2441257942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb2b40 msgr2=0x564067fc8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb2b40 0x564067fc8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b50bb5640 1 -- 172.21.15.103:0/2441257942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564067faded0 con 0x564067fb15d0 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.174+0000 7f3b513b6640 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fbc370 0x564067fd3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb15d0 0x564067fb8fe0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3b40000c00 tx=0x7f3b400bc6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a979d6ca56359f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b367fc640 1 -- 172.21.15.103:0/2441257942 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40049070 con 0x564067fb15d0 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b367fc640 1 -- 172.21.15.103:0/2441257942 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b400c0c90 con 0x564067fb15d0 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b367fc640 1 -- 172.21.15.103:0/2441257942 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400b8f50 con 0x564067fb15d0 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b551f2740 1 -- 172.21.15.103:0/2441257942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb15d0 msgr2=0x564067fb8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b551f2740 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb15d0 0x564067fb8fe0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3b40000c00 tx=0x7f3b400bc6b0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b551f2740 1 -- 172.21.15.103:0/2441257942 shutdown_connections 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b551f2740 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fbc370 0x564067fd3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b551f2740 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb2b40 0x564067fc8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b551f2740 1 --2- 172.21.15.103:0/2441257942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fb15d0 0x564067fb8fe0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.175+0000 7f3b551f2740 1 -- 172.21.15.103:0/2441257942 >> 172.21.15.103:0/2441257942 conn(0x564067f7a310 msgr2=0x564067fb88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 -- 172.21.15.103:0/2441257942 shutdown_connections 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 -- 172.21.15.103:0/2441257942 wait complete. 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 -- start start 2024-09-19T07:48:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 Processor -- start 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb15d0 0x564067fc4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb2b40 0x564067fc4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fbc370 0x564067fb5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1b1010 con 0x564067fbc370 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1b0940 con 0x564067fb15d0 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1adb70 con 0x564067fb2b40 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b551f2740 10 client.0 _ll_register_callbacks cb 0x7f3b3800d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b513b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fbc370 0x564067fb5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b513b6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fbc370 0x564067fb5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57008/0 (socket says 172.21.15.103:57008) 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b513b6640 1 -- 172.21.15.103:0/3910186301 learned_addr learned my addr 172.21.15.103:0/3910186301 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.176+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb15d0 0x564067fc4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b4bfff640 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb2b40 0x564067fc4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b513b6640 1 -- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb2b40 msgr2=0x564067fc4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b513b6640 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb2b40 0x564067fc4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b513b6640 1 -- 172.21.15.103:0/3910186301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb15d0 msgr2=0x564067fc4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b513b6640 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb15d0 0x564067fc4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b513b6640 1 -- 172.21.15.103:0/3910186301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564067fb4120 con 0x564067fbc370 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b513b6640 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fbc370 0x564067fb5bf0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d750 tx=0x7f3b44014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728ff3bec758eff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3910186301 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44033060 con 0x564067fbc370 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3910186301 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f3b44010450 con 0x564067fbc370 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b400c0c90 con 0x564067fbc370 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3910186301 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3b4401f040 con 0x564067fbc370 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b551f2740 10 client.17652 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.177+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564068036740 con 0x564067fbc370 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3910186301 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b400c0c90 con 0x564067fbc370 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3910186301 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f3b4402e470 con 0x564067fbc370 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b551f2740 10 client.17652 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b551f2740 10 client.17652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b551f2740 20 client.17652 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564068036b20 0x564068038f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b551f2740 4 client.17652 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564067f1ed70 con 0x564068036b20 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.178+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564068036b20 0x564068038f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b2bfff640 10 client.17652 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b50bb5640 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564068036b20 0x564068038f40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f96f40570818f024 server_cookie=de59c645366c5528 in_seq=0 out_seq=0 2024-09-19T07:48:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b2bfff640 1 -- 172.21.15.103:0/3910186301 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564067f1ed70 con 0x564068036b20 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b2bfff640 10 client.17652 handle_command_reply: tid=0 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 client.17652 shutdown 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fbc370 msgr2=0x564067fb5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fbc370 0x564067fb5bf0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d750 tx=0x7f3b44014000 comp rx=0 tx=0).stop 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 shutdown_connections 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x564068036b20 0x564068038f40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564067fbc370 0x564067fb5bf0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564067fb2b40 0x564067fc4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 --2- 172.21.15.103:0/3910186301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564067fb15d0 0x564067fc4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 >> 172.21.15.103:0/3910186301 conn(0x564067f7a310 msgr2=0x564067fb3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b2bfff640 0 client.17652 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 shutdown_connections 2024-09-19T07:48:52.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 1 -- 172.21.15.103:0/3910186301 wait complete. 2024-09-19T07:48:52.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.179+0000 7f3b551f2740 20 client.17652 trim_cache size 0 max 16384 2024-09-19T07:48:52.191 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b3c0a33b0 msgr2=0x7f3b3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3b4401c000 tx=0x7f3b44010d80 comp rx=0 tx=0).stop 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 msgr2=0x7f3b4c100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c100a10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3b40004770 tx=0x7f3b40003b90 comp rx=0 tx=0).stop 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 shutdown_connections 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3b3c0a33b0 0x7f3b3c0a5870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4c1082e0 0x7f3b4c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b4c103760 0x7f3b4c100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 --2- 172.21.15.103:0/3164199516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b4c102db0 0x7f3b4c100a10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:52.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 >> 172.21.15.103:0/3164199516 conn(0x7f3b4c0fc7d0 msgr2=0x7f3b4c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:52.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 shutdown_connections 2024-09-19T07:48:52.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:52.180+0000 7f3b52e19640 1 -- 172.21.15.103:0/3164199516 wait complete. 2024-09-19T07:48:52.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:52 smithi103 ceph-mon[31201]: pgmap v337: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 626 op/s 2024-09-19T07:48:52.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:52 smithi134 ceph-mon[35414]: pgmap v337: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 626 op/s 2024-09-19T07:48:52.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:52 smithi174 ceph-mon[35370]: pgmap v337: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 626 op/s 2024-09-19T07:48:52.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:48:53.237 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 -- 172.21.15.103:0/1379408190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc10c180 msgr2=0x7f37bc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 --2- 172.21.15.103:0/1379408190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc10c180 0x7f37bc10e600 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009ca0 tx=0x7f37b402a950 comp rx=0 tx=0).stop 2024-09-19T07:48:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 -- 172.21.15.103:0/1379408190 shutdown_connections 2024-09-19T07:48:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 --2- 172.21.15.103:0/1379408190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc10c180 0x7f37bc10e600 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 --2- 172.21.15.103:0/1379408190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc1036c0 0x7f37bc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 --2- 172.21.15.103:0/1379408190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc102da0 0x7f37bc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 -- 172.21.15.103:0/1379408190 >> 172.21.15.103:0/1379408190 conn(0x7f37bc0fc7a0 msgr2=0x7f37bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 -- 172.21.15.103:0/1379408190 shutdown_connections 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.832+0000 7f37c3810640 1 -- 172.21.15.103:0/1379408190 wait complete. 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 Processor -- start 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 -- start start 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc102da0 0x7f37bc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc1036c0 0x7f37bc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc10c180 0x7f37bc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111fa0 con 0x7f37bc1036c0 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111e20 con 0x7f37bc102da0 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c3810640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc112120 con 0x7f37bc10c180 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc1036c0 0x7f37bc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc1036c0 0x7f37bc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57024/0 (socket says 172.21.15.103:57024) 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 -- 172.21.15.103:0/2764593987 learned_addr learned my addr 172.21.15.103:0/2764593987 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc10c180 0x7f37bc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c15ac640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc102da0 0x7f37bc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 -- 172.21.15.103:0/2764593987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc10c180 msgr2=0x7f37bc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc10c180 0x7f37bc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 -- 172.21.15.103:0/2764593987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc102da0 msgr2=0x7f37bc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc102da0 0x7f37bc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 -- 172.21.15.103:0/2764593987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc19d350 con 0x7f37bc1036c0 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c15ac640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc102da0 0x7f37bc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:53.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.833+0000 7f37c0dab640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc1036c0 0x7f37bc1988e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c920 tx=0x7f37b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.834+0000 7f37ae7fc640 1 -- 172.21.15.103:0/2764593987 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8007c70 con 0x7f37bc1036c0 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.834+0000 7f37ae7fc640 1 -- 172.21.15.103:0/2764593987 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b8007e10 con 0x7f37bc1036c0 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.834+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc19d640 con 0x7f37bc1036c0 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.834+0000 7f37ae7fc640 1 -- 172.21.15.103:0/2764593987 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8018890 con 0x7f37bc1036c0 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.834+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a4f20 con 0x7f37bc1036c0 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.836+0000 7f37ae7fc640 1 -- 172.21.15.103:0/2764593987 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f37b8018a70 con 0x7f37bc1036c0 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.836+0000 7f37c5be9740 1 Processor -- start 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.836+0000 7f37c5be9740 1 -- start start 2024-09-19T07:48:53.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63127b40 0x560b63126090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b631265d0 0x560b63126a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63131370 0x560b63133760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c5be9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b6308c2f0 con 0x560b631265d0 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c5be9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b63033cf0 con 0x560b63131370 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c5be9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b630882b0 con 0x560b63127b40 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37ae7fc640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37a80a33b0 0x7f37a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b631265d0 0x560b63126a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b631265d0 0x560b63126a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57026/0 (socket says 172.21.15.103:57026) 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 -- 172.21.15.103:0/3179019284 learned_addr learned my addr 172.21.15.103:0/3179019284 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c1dad640 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63127b40 0x560b63126090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37ae7fc640 1 -- 172.21.15.103:0/2764593987 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f37b80cff30 con 0x7f37bc1036c0 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c0dab640 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63131370 0x560b63133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37a80a33b0 0x7f37a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 -- 172.21.15.103:0/3179019284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63127b40 msgr2=0x560b63126090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63127b40 0x560b63126090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 -- 172.21.15.103:0/3179019284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63131370 msgr2=0x560b63133760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63131370 0x560b63133760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 -- 172.21.15.103:0/3179019284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b630183a0 con 0x560b631265d0 2024-09-19T07:48:53.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c0dab640 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63131370 0x560b63133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c15ac640 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b631265d0 0x560b63126a10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f37b000aa10 tx=0x7f37b000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd059c1882837e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37a80a33b0 0x7f37a80a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009a60 tx=0x7f37b40095c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.837+0000 7f37a77fe640 1 -- 172.21.15.103:0/3179019284 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0015070 con 0x560b631265d0 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37a77fe640 1 -- 172.21.15.103:0/3179019284 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b000cc60 con 0x560b631265d0 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37a77fe640 1 -- 172.21.15.103:0/3179019284 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0018700 con 0x560b631265d0 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 -- 172.21.15.103:0/3179019284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b631265d0 msgr2=0x560b63126a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b631265d0 0x560b63126a10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f37b000aa10 tx=0x7f37b000aed0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 -- 172.21.15.103:0/3179019284 shutdown_connections 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63131370 0x560b63133760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b631265d0 0x560b63126a10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 --2- 172.21.15.103:0/3179019284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63127b40 0x560b63126090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 -- 172.21.15.103:0/3179019284 >> 172.21.15.103:0/3179019284 conn(0x560b630ef310 msgr2=0x560b63122c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 -- 172.21.15.103:0/3179019284 shutdown_connections 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.838+0000 7f37c5be9740 1 -- 172.21.15.103:0/3179019284 wait complete. 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 -- start start 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 Processor -- start 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b63130d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b6312dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b6312e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b63134070 con 0x560b63131370 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b63047060 con 0x560b63127b40 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b631341f0 con 0x560b631265d0 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c5be9740 10 client.0 _ll_register_callbacks cb 0x560b62d9a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:53.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c1dad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b6312e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c1dad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b6312e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57028/0 (socket says 172.21.15.103:57028) 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c1dad640 1 -- 172.21.15.103:0/2708747159 learned_addr learned my addr 172.21.15.103:0/2708747159 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c15ac640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b63130d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c0dab640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b6312dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c1dad640 1 -- 172.21.15.103:0/2708747159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 msgr2=0x560b63130d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.839+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b63130d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c1dad640 1 -- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 msgr2=0x560b6312dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b6312dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c1dad640 1 -- 172.21.15.103:0/2708747159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b62dfd3c0 con 0x560b63131370 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c0dab640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b6312dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b6312e530 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f37b402fcb0 tx=0x7f37b402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f1f2d12533dde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2708747159 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b404f030 con 0x560b63131370 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2708747159 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f37b4004040 con 0x560b63131370 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b000cc60 con 0x560b63131370 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2708747159 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f37b4034040 con 0x560b63131370 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 10 client.17664 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560b6313fa10 con 0x560b63131370 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2708747159 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b000cc60 con 0x560b63131370 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2708747159 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f37b404b4f0 con 0x560b63131370 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 10 client.17664 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 20 client.17664 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b63140e20 0x560b631432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 4 client.17664 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.840+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b63099860 con 0x560b63140e20 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.841+0000 7f37c15ac640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b63140e20 0x560b631432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.841+0000 7f37a4ff9640 10 client.17664 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.841+0000 7f37c15ac640 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b63140e20 0x560b631432c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a8786e4e55cfe2 server_cookie=dff6b5387c0fc20d in_seq=0 out_seq=0 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2708747159 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b63099860 con 0x560b63140e20 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37a4ff9640 10 client.17664 handle_command_reply: tid=0 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 client.17664 shutdown 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 msgr2=0x560b6312e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b6312e530 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f37b402fcb0 tx=0x7f37b402a950 comp rx=0 tx=0).stop 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 shutdown_connections 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b63140e20 0x560b631432c0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b6312e530 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b6312dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37a4ff9640 0 client.17664 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:53.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2708747159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b63130d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 >> 172.21.15.103:0/2708747159 conn(0x560b630ef310 msgr2=0x560b63128960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 shutdown_connections 2024-09-19T07:48:53.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 1 -- 172.21.15.103:0/2708747159 wait complete. 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.842+0000 7f37c5be9740 20 client.17664 trim_cache size 0 max 16384 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 Processor -- start 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 -- start start 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b631265d0 0x560b6312ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63127b40 0x560b63129120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63131370 0x560b63129660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1ad840 con 0x560b63127b40 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1af3b0 con 0x560b631265d0 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.855+0000 7f37c5be9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1adf40 con 0x560b63131370 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63127b40 0x560b63129120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63127b40 0x560b63129120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57032/0 (socket says 172.21.15.103:57032) 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 -- 172.21.15.103:0/520982513 learned_addr learned my addr 172.21.15.103:0/520982513 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:53.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c15ac640 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b631265d0 0x560b6312ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 -- 172.21.15.103:0/520982513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63131370 msgr2=0x560b63129660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63131370 0x560b63129660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 -- 172.21.15.103:0/520982513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b631265d0 msgr2=0x560b6312ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b631265d0 0x560b6312ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 -- 172.21.15.103:0/520982513 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b63129ba0 con 0x560b63127b40 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c15ac640 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b631265d0 0x560b6312ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37c0dab640 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63127b40 0x560b63129120 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f37b8007590 tx=0x7f37b80bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83697aeef69d3db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37a6ffd640 1 -- 172.21.15.103:0/520982513 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8067050 con 0x560b63127b40 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37a6ffd640 1 -- 172.21.15.103:0/520982513 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b809c200 con 0x560b63127b40 2024-09-19T07:48:53.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.856+0000 7f37a6ffd640 1 -- 172.21.15.103:0/520982513 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b800fc50 con 0x560b63127b40 2024-09-19T07:48:53.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 -- 172.21.15.103:0/520982513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63127b40 msgr2=0x560b63129120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63127b40 0x560b63129120 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f37b8007590 tx=0x7f37b80bc040 comp rx=0 tx=0).stop 2024-09-19T07:48:53.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 -- 172.21.15.103:0/520982513 shutdown_connections 2024-09-19T07:48:53.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b63131370 0x560b63129660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63127b40 0x560b63129120 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 --2- 172.21.15.103:0/520982513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b631265d0 0x560b6312ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 -- 172.21.15.103:0/520982513 >> 172.21.15.103:0/520982513 conn(0x560b630ef310 msgr2=0x560b6312f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 -- 172.21.15.103:0/520982513 shutdown_connections 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 -- 172.21.15.103:0/520982513 wait complete. 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 -- start start 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.857+0000 7f37c5be9740 1 Processor -- start 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b631358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b63135e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c5be9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b631aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c5be9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1aece0 con 0x560b63131370 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c5be9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1ae610 con 0x560b63127b40 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c5be9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1ab840 con 0x560b631265d0 2024-09-19T07:48:53.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b631aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c5be9740 10 client.0 _ll_register_callbacks cb 0x7f37b40098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c15ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b631358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b631aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57042/0 (socket says 172.21.15.103:57042) 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c15ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b631358c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:57038/0 (socket says 172.21.15.103:57038) 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 -- 172.21.15.103:0/2859372155 learned_addr learned my addr 172.21.15.103:0/2859372155 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c0dab640 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b63135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 -- 172.21.15.103:0/2859372155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 msgr2=0x560b631358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b631358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 -- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 msgr2=0x560b63135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b63135e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 -- 172.21.15.103:0/2859372155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b631ab0f0 con 0x560b63131370 2024-09-19T07:48:53.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37c1dad640 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b631aabb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f37b4030040 tx=0x7f37b404b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cb119aeab927b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:53.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.858+0000 7f37a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:53.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2859372155 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4060030 con 0x560b63131370 2024-09-19T07:48:53.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2859372155 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f37b4058070 con 0x560b63131370 2024-09-19T07:48:53.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b809c200 con 0x560b63131370 2024-09-19T07:48:53.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2859372155 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f37b406a050 con 0x560b63131370 2024-09-19T07:48:53.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 10 client.17670 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560b631ac390 con 0x560b63131370 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2859372155 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b809c200 con 0x560b63131370 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2859372155 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f37b40046a0 con 0x560b63131370 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 10 client.17670 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 20 client.17670 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b631acab0 0x560b631aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 4 client.17670 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.859+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b63093d70 con 0x560b631acab0 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c15ac640 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b631acab0 0x560b631aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:53.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c15ac640 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b631acab0 0x560b631aeed0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c525564517f399a8 server_cookie=e215dc300a01efeb in_seq=0 out_seq=0 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37a4ff9640 10 client.17670 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37a4ff9640 1 -- 172.21.15.103:0/2859372155 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b63093d70 con 0x560b631acab0 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37a4ff9640 10 client.17670 handle_command_reply: tid=0 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 client.17670 shutdown 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 msgr2=0x560b631aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b631aabb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f37b4030040 tx=0x7f37b404b710 comp rx=0 tx=0).stop 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 shutdown_connections 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x560b631acab0 0x560b631aeed0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b63131370 0x560b631aabb0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b63127b40 0x560b63135e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 --2- 172.21.15.103:0/2859372155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b631265d0 0x560b631358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 >> 172.21.15.103:0/2859372155 conn(0x560b630ef310 msgr2=0x560b63130030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.860+0000 7f37a4ff9640 0 client.17670 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:53.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 shutdown_connections 2024-09-19T07:48:53.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c5be9740 1 -- 172.21.15.103:0/2859372155 wait complete. 2024-09-19T07:48:53.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c5be9740 20 client.17670 trim_cache size 0 max 16384 2024-09-19T07:48:53.880 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:48:53.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37a80a33b0 msgr2=0x7f37a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37a80a33b0 0x7f37a80a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009a60 tx=0x7f37b40095c0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc1036c0 msgr2=0x7f37bc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:53.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc1036c0 0x7f37bc1988e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c920 tx=0x7f37b800cde0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 shutdown_connections 2024-09-19T07:48:53.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37a80a33b0 0x7f37a80a5870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37bc10c180 0x7f37bc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37bc1036c0 0x7f37bc1988e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 --2- 172.21.15.103:0/2764593987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37bc102da0 0x7f37bc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:53.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 >> 172.21.15.103:0/2764593987 conn(0x7f37bc0fc7a0 msgr2=0x7f37bc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:53.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 shutdown_connections 2024-09-19T07:48:53.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:53.861+0000 7f37c3810640 1 -- 172.21.15.103:0/2764593987 wait complete. 2024-09-19T07:48:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:54 smithi134 ceph-mon[35414]: pgmap v338: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 621 op/s 2024-09-19T07:48:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:54 smithi174 ceph-mon[35370]: pgmap v338: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 621 op/s 2024-09-19T07:48:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:54 smithi103 ceph-mon[31201]: pgmap v338: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 621 op/s 2024-09-19T07:48:55.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 620.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:48:55.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:48:55.809 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:48:55.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:48:55.959 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:56.350 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:56 smithi103 ceph-mon[31201]: pgmap v339: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 624 op/s 2024-09-19T07:48:56.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 -- 172.21.15.103:0/394725897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 msgr2=0x7fb834100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:56.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 --2- 172.21.15.103:0/394725897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb834100ee0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb828009930 tx=0x7fb82802a420 comp rx=0 tx=0).stop 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 -- 172.21.15.103:0/394725897 shutdown_connections 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 --2- 172.21.15.103:0/394725897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb834101420 0x7fb83410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 --2- 172.21.15.103:0/394725897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb834100ee0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 --2- 172.21.15.103:0/394725897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb834106a80 0x7fb834106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 -- 172.21.15.103:0/394725897 >> 172.21.15.103:0/394725897 conn(0x7fb8340fc7a0 msgr2=0x7fb8340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 -- 172.21.15.103:0/394725897 shutdown_connections 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 -- 172.21.15.103:0/394725897 wait complete. 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 Processor -- start 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.429+0000 7fb83a832640 1 -- start start 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb8340fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb834101420 0x7fb8340fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:56.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb834106a80 0x7fb8340fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb834110ec0 con 0x7fb834100a80 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb834110d40 con 0x7fb834101420 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb834111040 con 0x7fb834106a80 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb8340fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb8340fcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57064/0 (socket says 172.21.15.103:57064) 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 -- 172.21.15.103:0/150748759 learned_addr learned my addr 172.21.15.103:0/150748759 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb838dcf640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb834106a80 0x7fb8340fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:56.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb8337fe640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb834101420 0x7fb8340fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 -- 172.21.15.103:0/150748759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb834106a80 msgr2=0x7fb8340fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb834106a80 0x7fb8340fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 -- 172.21.15.103:0/150748759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb834101420 msgr2=0x7fb8340fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb834101420 0x7fb8340fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 -- 172.21.15.103:0/150748759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8341a69b0 con 0x7fb834100a80 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb833fff640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb8340fcd30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d770 tx=0x7fb82400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb8317fa640 1 -- 172.21.15.103:0/150748759 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824017070 con 0x7fb834100a80 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8341a6ca0 con 0x7fb834100a80 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb8317fa640 1 -- 172.21.15.103:0/150748759 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824005500 con 0x7fb834100a80 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb8317fa640 1 -- 172.21.15.103:0/150748759 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824004e00 con 0x7fb834100a80 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.430+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8341a7190 con 0x7fb834100a80 2024-09-19T07:48:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.431+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb834100ee0 con 0x7fb834100a80 2024-09-19T07:48:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.432+0000 7fb8317fa640 1 -- 172.21.15.103:0/150748759 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb824004fa0 con 0x7fb834100a80 2024-09-19T07:48:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.433+0000 7fb8317fa640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb81c0a33b0 0x7fb81c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.433+0000 7fb8337fe640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb81c0a33b0 0x7fb81c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.433+0000 7fb8317fa640 1 -- 172.21.15.103:0/150748759 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fb8240cf8c0 con 0x7fb834100a80 2024-09-19T07:48:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.434+0000 7fb8337fe640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb81c0a33b0 0x7fb81c0a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8280096c0 tx=0x7fb828000bc0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.437+0000 7fb8317fa640 1 -- 172.21.15.103:0/150748759 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb82409c230 con 0x7fb834100a80 2024-09-19T07:48:56.510 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:56 smithi134 ceph-mon[35414]: pgmap v339: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 624 op/s 2024-09-19T07:48:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:56 smithi174 ceph-mon[35370]: pgmap v339: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 624 op/s 2024-09-19T07:48:56.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.667+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb834109a90 con 0x7fb834100a80 2024-09-19T07:48:56.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.668+0000 7fb8317fa640 1 -- 172.21.15.103:0/150748759 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7fb82409bb30 con 0x7fb834100a80 2024-09-19T07:48:56.669 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:48:56.669 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":46,"btime":"2024-09-19T07:48:00:225048+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:47:59.861225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:48:56.669 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 46 2024-09-19T07:48:56.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb81c0a33b0 msgr2=0x7fb81c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb81c0a33b0 0x7fb81c0a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8280096c0 tx=0x7fb828000bc0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 msgr2=0x7fb8340fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb8340fcd30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d770 tx=0x7fb82400dc30 comp rx=0 tx=0).stop 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 shutdown_connections 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb81c0a33b0 0x7fb81c0a5870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb834106a80 0x7fb8340fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb834101420 0x7fb8340fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 --2- 172.21.15.103:0/150748759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb834100a80 0x7fb8340fcd30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 >> 172.21.15.103:0/150748759 conn(0x7fb8340fc7a0 msgr2=0x7fb83406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 shutdown_connections 2024-09-19T07:48:56.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:56.670+0000 7fb83a832640 1 -- 172.21.15.103:0/150748759 wait complete. 2024-09-19T07:48:57.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 -- 172.21.15.103:0/3832167529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 msgr2=0x7f80e410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/3832167529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e410d950 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009cd0 tx=0x7f80dc02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 -- 172.21.15.103:0/3832167529 shutdown_connections 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/3832167529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e410d950 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/3832167529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80e4100a80 0x7f80e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/3832167529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80e4106a80 0x7f80e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 -- 172.21.15.103:0/3832167529 >> 172.21.15.103:0/3832167529 conn(0x7f80e40fc7a0 msgr2=0x7f80e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 -- 172.21.15.103:0/3832167529 shutdown_connections 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 -- 172.21.15.103:0/3832167529 wait complete. 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 Processor -- start 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.174+0000 7f80eb5ba640 1 -- start start 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80eb5ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80e4100a80 0x7f80e41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80eb5ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80eb5ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80e4106a80 0x7f80e419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80eb5ba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4110e80 con 0x7f80e4101420 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80eb5ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4110d00 con 0x7f80e4106a80 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80eb5ba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4111000 con 0x7f80e4100a80 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57084/0 (socket says 172.21.15.103:57084) 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 -- 172.21.15.103:0/881642460 learned_addr learned my addr 172.21.15.103:0/881642460 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e9b57640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80e4106a80 0x7f80e419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e9356640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80e4100a80 0x7f80e41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 -- 172.21.15.103:0/881642460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80e4100a80 msgr2=0x7f80e41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80e4100a80 0x7f80e41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 -- 172.21.15.103:0/881642460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80e4106a80 msgr2=0x7f80e419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80e4106a80 0x7f80e419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 -- 172.21.15.103:0/881642460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e419d460 con 0x7f80e4101420 2024-09-19T07:48:57.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80e8b55640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e41989f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d950 tx=0x7f80d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.175+0000 7f80da7fc640 1 -- 172.21.15.103:0/881642460 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4017030 con 0x7f80e4101420 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.176+0000 7f80da7fc640 1 -- 172.21.15.103:0/881642460 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d4004020 con 0x7f80e4101420 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.176+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e419d750 con 0x7f80e4101420 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.176+0000 7f80da7fc640 1 -- 172.21.15.103:0/881642460 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40047a0 con 0x7f80e4101420 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.176+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e41a5030 con 0x7f80e4101420 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.178+0000 7f80da7fc640 1 -- 172.21.15.103:0/881642460 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f80d40041d0 con 0x7f80e4101420 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.178+0000 7f80ed993740 1 Processor -- start 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.178+0000 7f80ed993740 1 -- start start 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80ed993740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098ce8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80ed993740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce9480 0x565098cf3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80ed993740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098cf3d40 0x565098cf6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80ed993740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565098c4e280 con 0x565098cf3d40 2024-09-19T07:48:57.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80ed993740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565098bf5b10 con 0x565098ce8b60 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80ed993740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565098c4a240 con 0x565098ce9480 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80da7fc640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f80d00a33b0 0x7f80d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098cf3d40 0x565098cf6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9b57640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f80d00a33b0 0x7f80d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098cf3d40 0x565098cf6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57094/0 (socket says 172.21.15.103:57094) 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9356640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce9480 0x565098cf3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e8b55640 1 -- 172.21.15.103:0/3920027029 learned_addr learned my addr 172.21.15.103:0/3920027029 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9b57640 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098ce8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80da7fc640 1 -- 172.21.15.103:0/881642460 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f80d40cf660 con 0x7f80e4101420 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9356640 1 -- 172.21.15.103:0/3920027029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 msgr2=0x565098ce8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9356640 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098ce8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9356640 1 -- 172.21.15.103:0/3920027029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098cf3d40 msgr2=0x565098cf6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9356640 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098cf3d40 0x565098cf6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9356640 1 -- 172.21.15.103:0/3920027029 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565098c021f0 con 0x565098ce9480 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e8b55640 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098cf3d40 0x565098cf6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9b57640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f80d00a33b0 0x7f80d00a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009430 tx=0x7f80dc009360 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.179+0000 7f80e9356640 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce9480 0x565098cf3800 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80e00045b0 tx=0x7f80e000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d15a21fda3c6bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80cf7fe640 1 -- 172.21.15.103:0/3920027029 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0003bd0 con 0x565098ce9480 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80cf7fe640 1 -- 172.21.15.103:0/3920027029 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0015070 con 0x565098ce9480 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80cf7fe640 1 -- 172.21.15.103:0/3920027029 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004d70 con 0x565098ce9480 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 -- 172.21.15.103:0/3920027029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce9480 msgr2=0x565098cf3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce9480 0x565098cf3800 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80e00045b0 tx=0x7f80e000bb20 comp rx=0 tx=0).stop 2024-09-19T07:48:57.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 -- 172.21.15.103:0/3920027029 shutdown_connections 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098cf3d40 0x565098cf6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce9480 0x565098cf3800 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 --2- 172.21.15.103:0/3920027029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098ce8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 -- 172.21.15.103:0/3920027029 >> 172.21.15.103:0/3920027029 conn(0x565098ce50f0 msgr2=0x565098ce54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 -- 172.21.15.103:0/3920027029 shutdown_connections 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.180+0000 7f80ed993740 1 -- 172.21.15.103:0/3920027029 wait complete. 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 -- start start 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 Processor -- start 2024-09-19T07:48:57.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565098cf6910 con 0x565098ce9480 2024-09-19T07:48:57.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565098c08f10 con 0x565098ce8b60 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565098cf6a90 con 0x565098cf3d40 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80ed993740 10 client.0 _ll_register_callbacks cb 0x565098bd1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57096/0 (socket says 172.21.15.103:57096) 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.181+0000 7f80e8b55640 1 -- 172.21.15.103:0/3660881175 learned_addr learned my addr 172.21.15.103:0/3660881175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80e9356640 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80e8b55640 1 -- 172.21.15.103:0/3660881175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 msgr2=0x565098cf1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80e8b55640 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80e8b55640 1 -- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 msgr2=0x565098cf0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80e8b55640 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80e8b55640 1 -- 172.21.15.103:0/3660881175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650989bf6a0 con 0x565098ce9480 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80e8b55640 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf10f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009fd0 tx=0x7f80d409b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41db7bc5d9c921bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:57.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ccff9640 1 -- 172.21.15.103:0/3660881175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4014070 con 0x565098ce9480 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ccff9640 1 -- 172.21.15.103:0/3660881175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f80d40c1560 con 0x565098ce9480 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e0015070 con 0x565098ce9480 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ccff9640 1 -- 172.21.15.103:0/3660881175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f80d409c030 con 0x565098ce9480 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ed993740 10 client.17688 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565098d02500 con 0x565098ce9480 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.182+0000 7f80ccff9640 1 -- 172.21.15.103:0/3660881175 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0015070 con 0x565098ce9480 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ccff9640 1 -- 172.21.15.103:0/3660881175 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f80d40a4110 con 0x565098ce9480 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ed993740 10 client.17688 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ed993740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ed993740 20 client.17688 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ed993740 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d03910 0x565098d05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ed993740 4 client.17688 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:57.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565098c5b7f0 con 0x565098d03910 2024-09-19T07:48:57.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80e9356640 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d03910 0x565098d05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80ccff9640 10 client.17688 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.183+0000 7f80e9356640 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d03910 0x565098d05da0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15633adc75ce30d4 server_cookie=798eed0cc69ff1b1 in_seq=0 out_seq=0 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ccff9640 1 -- 172.21.15.103:0/3660881175 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565098c5b7f0 con 0x565098d03910 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ccff9640 10 client.17688 handle_command_reply: tid=0 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 client.17688 shutdown 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 msgr2=0x565098cf10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf10f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009fd0 tx=0x7f80d409b470 comp rx=0 tx=0).stop 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 shutdown_connections 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d03910 0x565098d05da0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ccff9640 0 client.17688 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf10f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 --2- 172.21.15.103:0/3660881175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 >> 172.21.15.103:0/3660881175 conn(0x565098ce50f0 msgr2=0x565098ce4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 shutdown_connections 2024-09-19T07:48:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 1 -- 172.21.15.103:0/3660881175 wait complete. 2024-09-19T07:48:57.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.184+0000 7f80ed993740 20 client.17688 trim_cache size 0 max 16384 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 Processor -- start 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 -- start start 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce8b60 0x565098cefa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098d0b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098cf3d40 0x565098d0bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41ad910 con 0x565098ce9480 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41af480 con 0x565098cf3d40 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80ed993740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41ae010 con 0x565098ce8b60 2024-09-19T07:48:57.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098d0b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098d0b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57100/0 (socket says 172.21.15.103:57100) 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 -- 172.21.15.103:0/997226056 learned_addr learned my addr 172.21.15.103:0/997226056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e9356640 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce8b60 0x565098cefa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 -- 172.21.15.103:0/997226056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce8b60 msgr2=0x565098cefa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce8b60 0x565098cefa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 -- 172.21.15.103:0/997226056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098cf3d40 msgr2=0x565098d0bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098cf3d40 0x565098d0bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e8b55640 1 -- 172.21.15.103:0/997226056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565098ce4b90 con 0x565098ce9480 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.202+0000 7f80e9356640 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce8b60 0x565098cefa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.203+0000 7f80e8b55640 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098d0b670 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f80d4097d40 tx=0x7f80d40cfa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f920c34dcafb4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.203+0000 7f80ceffd640 1 -- 172.21.15.103:0/997226056 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4014070 con 0x565098ce9480 2024-09-19T07:48:57.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.203+0000 7f80ceffd640 1 -- 172.21.15.103:0/997226056 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d40afe50 con 0x565098ce9480 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.203+0000 7f80ceffd640 1 -- 172.21.15.103:0/997226056 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d40a4110 con 0x565098ce9480 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 -- 172.21.15.103:0/997226056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 msgr2=0x565098d0b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098d0b670 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f80d4097d40 tx=0x7f80d40cfa10 comp rx=0 tx=0).stop 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 -- 172.21.15.103:0/997226056 shutdown_connections 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098cf3d40 0x565098d0bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098d0b670 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 --2- 172.21.15.103:0/997226056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098ce8b60 0x565098cefa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 -- 172.21.15.103:0/997226056 >> 172.21.15.103:0/997226056 conn(0x565098ce50f0 msgr2=0x565098cef350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 -- 172.21.15.103:0/997226056 shutdown_connections 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 -- 172.21.15.103:0/997226056 wait complete. 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 -- start start 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.204+0000 7f80ed993740 1 Processor -- start 2024-09-19T07:48:57.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41aedb0 con 0x565098ce9480 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41ae6e0 con 0x565098ce8b60 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41ab910 con 0x565098cf3d40 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 10 client.0 _ll_register_callbacks cb 0x7f80d40cf960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57102/0 (socket says 172.21.15.103:57102) 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 -- 172.21.15.103:0/3842082387 learned_addr learned my addr 172.21.15.103:0/3842082387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e9b57640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 -- 172.21.15.103:0/3842082387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 msgr2=0x565098cf85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 -- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 msgr2=0x565098cf7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e9356640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 -- 172.21.15.103:0/3842082387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565098d6d7a0 con 0x565098ce9480 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e9b57640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e9356640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80e8b55640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf80a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005fd0 tx=0x7f80d4098250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e4a94fb8bfa5d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ccff9640 1 -- 172.21.15.103:0/3842082387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4014070 con 0x565098ce9480 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ccff9640 1 -- 172.21.15.103:0/3842082387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f80d409ad80 con 0x565098ce9480 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.205+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80d40afe50 con 0x565098ce9480 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ccff9640 1 -- 172.21.15.103:0/3842082387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f80d409fce0 con 0x565098ce9480 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 10 client.17694 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565098d6d7a0 con 0x565098ce9480 2024-09-19T07:48:57.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ccff9640 1 -- 172.21.15.103:0/3842082387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d40afe50 con 0x565098ce9480 2024-09-19T07:48:57.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ccff9640 1 -- 172.21.15.103:0/3842082387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f80d40cf110 con 0x565098ce9480 2024-09-19T07:48:57.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 10 client.17694 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:57.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:57.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 20 client.17694 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:57.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d6ecf0 0x565098d710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:57.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 4 client.17694 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:48:57.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565098c55d00 con 0x565098d6ecf0 2024-09-19T07:48:57.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.206+0000 7f80e9356640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d6ecf0 0x565098d710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.207+0000 7f80ccff9640 10 client.17694 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.207+0000 7f80e9356640 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d6ecf0 0x565098d710e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90cd7e3602fe7daf server_cookie=335b7c6ba91db18d in_seq=0 out_seq=0 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ccff9640 1 -- 172.21.15.103:0/3842082387 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565098c55d00 con 0x565098d6ecf0 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ccff9640 10 client.17694 handle_command_reply: tid=0 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 client.17694 shutdown 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 msgr2=0x565098cf80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf80a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005fd0 tx=0x7f80d4098250 comp rx=0 tx=0).stop 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 shutdown_connections 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x565098d6ecf0 0x565098d710e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565098cf3d40 0x565098cf85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565098ce9480 0x565098cf80a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ccff9640 0 client.17694 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 --2- 172.21.15.103:0/3842082387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565098ce8b60 0x565098cf7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.208+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 >> 172.21.15.103:0/3842082387 conn(0x565098ce50f0 msgr2=0x565098ce4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 shutdown_connections 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80ed993740 1 -- 172.21.15.103:0/3842082387 wait complete. 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80ed993740 20 client.17694 trim_cache size 0 max 16384 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:48:57.215 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "e38e59f5-bf18-4850-968f-63d6e0af9ff8", 2024-09-19T07:48:57.216 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:48:57.216 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:48:57.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f80d00a33b0 msgr2=0x7f80d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f80d00a33b0 0x7f80d00a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009430 tx=0x7f80dc009360 comp rx=0 tx=0).stop 2024-09-19T07:48:57.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 msgr2=0x7f80e41989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e41989f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d950 tx=0x7f80d400de10 comp rx=0 tx=0).stop 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 shutdown_connections 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f80d00a33b0 0x7f80d00a5870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80e4106a80 0x7f80e419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80e4101420 0x7f80e41989f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 --2- 172.21.15.103:0/881642460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80e4100a80 0x7f80e41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 >> 172.21.15.103:0/881642460 conn(0x7f80e40fc7a0 msgr2=0x7f80e406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 shutdown_connections 2024-09-19T07:48:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:57.209+0000 7f80eb5ba640 1 -- 172.21.15.103:0/881642460 wait complete. 2024-09-19T07:48:57.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:57 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/150748759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:57.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:57 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/150748759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:57.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:57 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/150748759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:48:57.970 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:48:58.362 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 8ac5c642 --release --await' 2024-09-19T07:48:58.363 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 8ac5c642 --release --await 2024-09-19T07:48:58.454 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:58.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: pgmap v340: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:48:58.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:48:58 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: pgmap v340: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:48:58.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.583 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.583 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.583 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:48:58 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: pgmap v340: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.3 MiB/s wr, 443 op/s 2024-09-19T07:48:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:48:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:58.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:48:58 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:48:59.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.963+0000 7fec1758a640 1 -- 172.21.15.103:0/2295367065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10102d30 msgr2=0x7fec10103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.963+0000 7fec1758a640 1 --2- 172.21.15.103:0/2295367065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10102d30 0x7fec10103110 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009900 tx=0x7fec0c02a480 comp rx=0 tx=0).stop 2024-09-19T07:48:59.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 -- 172.21.15.103:0/2295367065 shutdown_connections 2024-09-19T07:48:59.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 --2- 172.21.15.103:0/2295367065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1010c160 0x7fec1010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 --2- 172.21.15.103:0/2295367065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec10103650 0x7fec1010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 --2- 172.21.15.103:0/2295367065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10102d30 0x7fec10103110 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 -- 172.21.15.103:0/2295367065 >> 172.21.15.103:0/2295367065 conn(0x7fec100fc790 msgr2=0x7fec100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 -- 172.21.15.103:0/2295367065 shutdown_connections 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 -- 172.21.15.103:0/2295367065 wait complete. 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 Processor -- start 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.964+0000 7fec1758a640 1 -- start start 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec1758a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec10102d30 0x7fec10198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec1758a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10103650 0x7fec10198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec1758a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1010c160 0x7fec1019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec1758a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec10111f60 con 0x7fec10103650 2024-09-19T07:48:59.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec1758a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec10111de0 con 0x7fec10102d30 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec1758a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec101120e0 con 0x7fec1010c160 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10103650 0x7fec10198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10103650 0x7fec10198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39776/0 (socket says 172.21.15.103:39776) 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 -- 172.21.15.103:0/3153853223 learned_addr learned my addr 172.21.15.103:0/3153853223 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec15326640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec10102d30 0x7fec10198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 -- 172.21.15.103:0/3153853223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1010c160 msgr2=0x7fec1019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1010c160 0x7fec1019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 -- 172.21.15.103:0/3153853223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec10102d30 msgr2=0x7fec10198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec10102d30 0x7fec10198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 -- 172.21.15.103:0/3153853223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1019d3b0 con 0x7fec10103650 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec15326640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec10102d30 0x7fec10198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec14b25640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10103650 0x7fec10198940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fec0000c970 tx=0x7fec0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec067fc640 1 -- 172.21.15.103:0/3153853223 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00007bb0 con 0x7fec10103650 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec067fc640 1 -- 172.21.15.103:0/3153853223 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec00007d50 con 0x7fec10103650 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.965+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1019d6a0 con 0x7fec10103650 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.966+0000 7fec067fc640 1 -- 172.21.15.103:0/3153853223 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00018820 con 0x7fec10103650 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.966+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec101a4f80 con 0x7fec10103650 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.968+0000 7fec067fc640 1 -- 172.21.15.103:0/3153853223 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fec000189c0 con 0x7fec10103650 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.968+0000 7fec19963740 1 Processor -- start 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.968+0000 7fec19963740 1 -- start start 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.968+0000 7fec067fc640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7febfc0a33b0 0x7febfc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.968+0000 7fec19963740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e099f0 0x5607f0e08360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec19963740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e088a0 0x5607f0e08d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec19963740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec19963740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607f0d6e2e0 con 0x5607f0e1bd20 2024-09-19T07:48:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec19963740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607f0d15ce0 con 0x5607f0e088a0 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec19963740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607f0d6a2a0 con 0x5607f0e099f0 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7febfc0a33b0 0x7febfc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec067fc640 1 -- 172.21.15.103:0/3153853223 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fec000cfea0 con 0x7fec10103650 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e1e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39784/0 (socket says 172.21.15.103:39784) 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 -- 172.21.15.103:0/2607132998 learned_addr learned my addr 172.21.15.103:0/2607132998 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec14b25640 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e088a0 0x5607f0e08d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15326640 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e099f0 0x5607f0e08360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7febfc0a33b0 0x7febfc0a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec08013510 tx=0x7fec08007f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 -- 172.21.15.103:0/2607132998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e099f0 msgr2=0x5607f0e08360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e099f0 0x5607f0e08360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 -- 172.21.15.103:0/2607132998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e088a0 msgr2=0x5607f0e08d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e088a0 0x5607f0e08d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 -- 172.21.15.103:0/2607132998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f0cfa440 con 0x5607f0e1bd20 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7fec15b27640 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e1e110 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fec08007a20 tx=0x7fec0801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93fdf1e5ba6793d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7febfb7fe640 1 -- 172.21.15.103:0/2607132998 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0801d920 con 0x5607f0e1bd20 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7febfb7fe640 1 -- 172.21.15.103:0/2607132998 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0801c470 con 0x5607f0e1bd20 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.969+0000 7febfb7fe640 1 -- 172.21.15.103:0/2607132998 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec080115e0 con 0x5607f0e1bd20 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 -- 172.21.15.103:0/2607132998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 msgr2=0x5607f0e1e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e1e110 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fec08007a20 tx=0x7fec0801a000 comp rx=0 tx=0).stop 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 -- 172.21.15.103:0/2607132998 shutdown_connections 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e1e110 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e088a0 0x5607f0e08d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 --2- 172.21.15.103:0/2607132998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e099f0 0x5607f0e08360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 -- 172.21.15.103:0/2607132998 >> 172.21.15.103:0/2607132998 conn(0x5607f0e04a30 msgr2=0x5607f0e04e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 -- 172.21.15.103:0/2607132998 shutdown_connections 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.970+0000 7fec19963740 1 -- 172.21.15.103:0/2607132998 wait complete. 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 -- start start 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 Processor -- start 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e0c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607f0e1e9f0 con 0x5607f0e1bd20 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5607f0d29050 con 0x5607f0e099f0 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607f0e1eb70 con 0x5607f0e088a0 2024-09-19T07:48:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec19963740 10 client.0 _ll_register_callbacks cb 0x5607f0d69160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e0c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e0c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39792/0 (socket says 172.21.15.103:39792) 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 -- 172.21.15.103:0/3489689030 learned_addr learned my addr 172.21.15.103:0/3489689030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15326640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec14b25640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 -- 172.21.15.103:0/3489689030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 msgr2=0x5607f0e0ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e0ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 -- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 msgr2=0x5607f0e0bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec15b27640 1 -- 172.21.15.103:0/3489689030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f0adf3c0 con 0x5607f0e1bd20 2024-09-19T07:48:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.971+0000 7fec14b25640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec15b27640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e0c330 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fec08016040 tx=0x7fec08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d04e7ed0652af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7febf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7febf8ff9640 1 -- 172.21.15.103:0/3489689030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08032060 con 0x5607f0e1bd20 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7febf8ff9640 1 -- 172.21.15.103:0/3489689030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fec0801dd60 con 0x5607f0e1bd20 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec0801c470 con 0x5607f0e1bd20 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7febf8ff9640 1 -- 172.21.15.103:0/3489689030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fec0801c9b0 con 0x5607f0e1bd20 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 10 client.17706 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5607f0e2a400 con 0x5607f0e1bd20 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7febf8ff9640 1 -- 172.21.15.103:0/3489689030 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0801c470 con 0x5607f0e1bd20 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7febf8ff9640 1 -- 172.21.15.103:0/3489689030 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fec0800f040 con 0x5607f0e1bd20 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 10 client.17706 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:59.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 20 client.17706 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e2b750 0x5607f0e2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 4 client.17706 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.972+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607f0d7b850 con 0x5607f0e2b750 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.973+0000 7fec15326640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e2b750 0x5607f0e2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.973+0000 7febf8ff9640 10 client.17706 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.973+0000 7fec15326640 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e2b750 0x5607f0e2db40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9937e4bfe4b83c server_cookie=88832d6b962b7b25 in_seq=0 out_seq=0 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7febf8ff9640 1 -- 172.21.15.103:0/3489689030 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607f0d7b850 con 0x5607f0e2b750 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7febf8ff9640 10 client.17706 handle_command_reply: tid=0 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 client.17706 shutdown 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 msgr2=0x5607f0e0c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e0c330 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fec08016040 tx=0x7fec08004a40 comp rx=0 tx=0).stop 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 shutdown_connections 2024-09-19T07:48:59.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e2b750 0x5607f0e2db40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e1bd20 0x5607f0e0c330 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7febf8ff9640 0 client.17706 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 --2- 172.21.15.103:0/3489689030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e0ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 >> 172.21.15.103:0/3489689030 conn(0x5607f0e04a30 msgr2=0x5607f0e0a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 shutdown_connections 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 1 -- 172.21.15.103:0/3489689030 wait complete. 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.974+0000 7fec19963740 20 client.17706 trim_cache size 0 max 16384 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 Processor -- start 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 -- start start 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e088a0 0x5607f0e27c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e1bd20 0x5607f0e33230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec101ab710 con 0x5607f0e088a0 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec101ad280 con 0x5607f0e099f0 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec19963740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec101abe10 con 0x5607f0e1bd20 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e088a0 0x5607f0e27c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e088a0 0x5607f0e27c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39804/0 (socket says 172.21.15.103:39804) 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 -- 172.21.15.103:0/3450591785 learned_addr learned my addr 172.21.15.103:0/3450591785 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 -- 172.21.15.103:0/3450591785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e1bd20 msgr2=0x5607f0e33230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 --2- 172.21.15.103:0/3450591785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e1bd20 0x5607f0e33230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 -- 172.21.15.103:0/3450591785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 msgr2=0x5607f0e0c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 --2- 172.21.15.103:0/3450591785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 -- 172.21.15.103:0/3450591785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f0e1af60 con 0x5607f0e088a0 2024-09-19T07:48:59.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7fec15326640 1 --2- 172.21.15.103:0/3450591785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e088a0 0x5607f0e27c90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009230 tx=0x7fec0c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11614200cae3eac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.991+0000 7febfaffd640 1 -- 172.21.15.103:0/3450591785 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c048020 con 0x5607f0e088a0 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7febfaffd640 1 -- 172.21.15.103:0/3450591785 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0c032870 con 0x5607f0e088a0 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7febfaffd640 1 -- 172.21.15.103:0/3450591785 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c035040 con 0x5607f0e088a0 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 -- 172.21.15.103:0/3450591785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e088a0 msgr2=0x5607f0e27c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 --2- 172.21.15.103:0/3450591785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e088a0 0x5607f0e27c90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009230 tx=0x7fec0c005a40 comp rx=0 tx=0).stop 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 -- 172.21.15.103:0/3450591785 shutdown_connections 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 --2- 172.21.15.103:0/3450591785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e1bd20 0x5607f0e33230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 --2- 172.21.15.103:0/3450591785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e099f0 0x5607f0e0c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 --2- 172.21.15.103:0/3450591785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e088a0 0x5607f0e27c90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 -- 172.21.15.103:0/3450591785 >> 172.21.15.103:0/3450591785 conn(0x5607f0e04a30 msgr2=0x5607f0e03a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 -- 172.21.15.103:0/3450591785 shutdown_connections 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 -- 172.21.15.103:0/3450591785 wait complete. 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.992+0000 7fec19963740 1 -- start start 2024-09-19T07:48:59.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 1 Processor -- start 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e1fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e099f0 0x5607f0e20040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e1bd20 0x5607f0e20580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec101acbb0 con 0x5607f0e099f0 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec101ac4e0 con 0x5607f0e1bd20 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec101a9710 con 0x5607f0e088a0 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec19963740 10 client.0 _ll_register_callbacks cb 0x7fec0000a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e099f0 0x5607f0e20040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e099f0 0x5607f0e20040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39818/0 (socket says 172.21.15.103:39818) 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 -- 172.21.15.103:0/4058712556 learned_addr learned my addr 172.21.15.103:0/4058712556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec15326640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e1fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec15b27640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e1bd20 0x5607f0e20580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 -- 172.21.15.103:0/4058712556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 msgr2=0x5607f0e1fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e1fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 -- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e1bd20 msgr2=0x5607f0e20580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e1bd20 0x5607f0e20580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec14b25640 1 -- 172.21.15.103:0/4058712556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f0e95200 con 0x5607f0e099f0 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.993+0000 7fec15b27640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e1bd20 0x5607f0e20580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec14b25640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e099f0 0x5607f0e20040 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fec0009be70 tx=0x7fec0009a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f17a9c278ea3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7febf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7febf8ff9640 1 -- 172.21.15.103:0/4058712556 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0009c170 con 0x5607f0e099f0 2024-09-19T07:48:59.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7febf8ff9640 1 -- 172.21.15.103:0/4058712556 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fec000a0150 con 0x5607f0e099f0 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec0c032870 con 0x5607f0e099f0 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7febf8ff9640 1 -- 172.21.15.103:0/4058712556 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fec000a0520 con 0x5607f0e099f0 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 10 client.17712 fetch_fsmap learned FSMap version 46 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5607f0e964b0 con 0x5607f0e099f0 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7febf8ff9640 1 -- 172.21.15.103:0/4058712556 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0c032870 con 0x5607f0e099f0 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7febf8ff9640 1 -- 172.21.15.103:0/4058712556 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fec00096ce0 con 0x5607f0e099f0 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 10 client.17712 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 20 client.17712 populate_metadata read hostname 'smithi103' 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e96c30 0x5607f0e99050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 4 client.17712 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607f0d75d60 con 0x5607f0e96c30 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.994+0000 7fec15326640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e96c30 0x5607f0e99050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:48:59.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7febf8ff9640 10 client.17712 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec15326640 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e96c30 0x5607f0e99050 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0080cd960ea79a9 server_cookie=4f11216c9f0f8690 in_seq=0 out_seq=0 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7febf8ff9640 1 -- 172.21.15.103:0/4058712556 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5607f0d75d60 con 0x5607f0e96c30 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7febf8ff9640 10 client.17712 handle_command_reply: tid=0 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 client.17712 shutdown 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e099f0 msgr2=0x5607f0e20040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e099f0 0x5607f0e20040 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fec0009be70 tx=0x7fec0009a680 comp rx=0 tx=0).stop 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 shutdown_connections 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5607f0e96c30 0x5607f0e99050 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5607f0e1bd20 0x5607f0e20580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607f0e099f0 0x5607f0e20040 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7febf8ff9640 0 client.17712 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 --2- 172.21.15.103:0/4058712556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607f0e088a0 0x5607f0e1fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 >> 172.21.15.103:0/4058712556 conn(0x5607f0e04a30 msgr2=0x5607f0e0bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 shutdown_connections 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 1 -- 172.21.15.103:0/4058712556 wait complete. 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout: "e38e59f5-bf18-4850-968f-63d6e0af9ff8": { 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "e38e59f5-bf18-4850-968f-63d6e0af9ff8", 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:48:59.017 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.995+0000 7fec19963740 20 client.17712 trim_cache size 0 max 16384 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7febfc0a33b0 msgr2=0x7febfc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7febfc0a33b0 0x7febfc0a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec08013510 tx=0x7fec08007f90 comp rx=0 tx=0).stop 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10103650 msgr2=0x7fec10198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10103650 0x7fec10198940 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fec0000c970 tx=0x7fec0000ce30 comp rx=0 tx=0).stop 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 shutdown_connections 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7febfc0a33b0 0x7febfc0a5870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1010c160 0x7fec1019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec10103650 0x7fec10198940 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 --2- 172.21.15.103:0/3153853223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec10102d30 0x7fec10198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:48:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 >> 172.21.15.103:0/3153853223 conn(0x7fec100fc790 msgr2=0x7fec100fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:48:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 shutdown_connections 2024-09-19T07:48:59.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:48:58.996+0000 7fec1758a640 1 -- 172.21.15.103:0/3153853223 wait complete. 2024-09-19T07:48:59.255 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:48:59.782 INFO:tasks.cephfs.filesystem:scrub status for tag:e38e59f5-bf18-4850-968f-63d6e0af9ff8 - {'path': '/', 'tag': 'e38e59f5-bf18-4850-968f-63d6e0af9ff8', 'options': 'recursive,force'} 2024-09-19T07:49:00.338 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:00 smithi103 ceph-mon[31201]: pgmap v341: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/2091959446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94101de0 msgr2=0x7f8e941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/2091959446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94101de0 0x7f8e941021c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009900 tx=0x7f8e9002a410 comp rx=0 tx=0).stop 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/2091959446 shutdown_connections 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/2091959446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e94106f10 0x7f8e9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/2091959446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e94102790 0x7f8e941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/2091959446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94101de0 0x7f8e941021c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/2091959446 >> 172.21.15.103:0/2091959446 conn(0x7f8e940fd8c0 msgr2=0x7f8e940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/2091959446 shutdown_connections 2024-09-19T07:49:00.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.337+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/2091959446 wait complete. 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 Processor -- start 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 -- start start 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e94101de0 0x7f8e940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94102790 0x7f8e94074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e94106f10 0x7f8e94074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e94111f70 con 0x7f8e94102790 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e94111df0 con 0x7f8e94101de0 2024-09-19T07:49:00.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9bfbe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e941120f0 con 0x7f8e94106f10 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94102790 0x7f8e94074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94102790 0x7f8e94074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39836/0 (socket says 172.21.15.103:39836) 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e99559640 1 -- 172.21.15.103:0/1364293591 learned_addr learned my addr 172.21.15.103:0/1364293591 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e94101de0 0x7f8e940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.338+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e94106f10 0x7f8e94074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e99559640 1 -- 172.21.15.103:0/1364293591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e94106f10 msgr2=0x7f8e94074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e99559640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e94106f10 0x7f8e94074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e99559640 1 -- 172.21.15.103:0/1364293591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e94101de0 msgr2=0x7f8e940744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e99559640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e94101de0 0x7f8e940744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e99559640 1 -- 172.21.15.103:0/1364293591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e941a68b0 con 0x7f8e94102790 2024-09-19T07:49:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e99559640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94102790 0x7f8e94074a30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c890 tx=0x7f8e8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e8affd640 1 -- 172.21.15.103:0/1364293591 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84004220 con 0x7f8e94102790 2024-09-19T07:49:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e941a6ba0 con 0x7f8e94102790 2024-09-19T07:49:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e8affd640 1 -- 172.21.15.103:0/1364293591 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e840043c0 con 0x7f8e94102790 2024-09-19T07:49:00.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e8affd640 1 -- 172.21.15.103:0/1364293591 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84004a00 con 0x7f8e94102790 2024-09-19T07:49:00.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.339+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e941a7130 con 0x7f8e94102790 2024-09-19T07:49:00.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.341+0000 7f8e8affd640 1 -- 172.21.15.103:0/1364293591 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8e840079b0 con 0x7f8e94102790 2024-09-19T07:49:00.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.341+0000 7f8e9e397740 1 Processor -- start 2024-09-19T07:49:00.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9e397740 1 -- start start 2024-09-19T07:49:00.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f63440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f6dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f70630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9e397740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d837ec8620 con 0x55d837f6e240 2024-09-19T07:49:00.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9e397740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d837e6fe50 con 0x55d837f63060 2024-09-19T07:49:00.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9e397740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d837ec45e0 con 0x55d837f63980 2024-09-19T07:49:00.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f70630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f70630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39846/0 (socket says 172.21.15.103:39846) 2024-09-19T07:49:00.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e99559640 1 -- 172.21.15.103:0/1164640261 learned_addr learned my addr 172.21.15.103:0/1164640261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:00.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e8affd640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e800a33b0 0x7f8e800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f63440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f6dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e800a33b0 0x7f8e800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e8affd640 1 -- 172.21.15.103:0/1364293591 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8e840cfae0 con 0x7f8e94102790 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9a55b640 1 -- 172.21.15.103:0/1164640261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 msgr2=0x55d837f6dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.342+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f6dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e9a55b640 1 -- 172.21.15.103:0/1164640261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 msgr2=0x55d837f70630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f70630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e9a55b640 1 -- 172.21.15.103:0/1164640261 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d837e60d40 con 0x55d837f63060 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e99559640 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f70630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f63440 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c014880 tx=0x7f8e8c014d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8aac58b447cf25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e7ffff640 1 -- 172.21.15.103:0/1164640261 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c014250 con 0x55d837f63060 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e800a33b0 0x7f8e800a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c021000 tx=0x7f8e8c01f000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e7ffff640 1 -- 172.21.15.103:0/1164640261 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8c009e20 con 0x55d837f63060 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.343+0000 7f8e7ffff640 1 -- 172.21.15.103:0/1164640261 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c013b40 con 0x55d837f63060 2024-09-19T07:49:00.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 -- 172.21.15.103:0/1164640261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 msgr2=0x55d837f63440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f63440 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c014880 tx=0x7f8e8c014d40 comp rx=0 tx=0).stop 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 -- 172.21.15.103:0/1164640261 shutdown_connections 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f70630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f6dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 --2- 172.21.15.103:0/1164640261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f63440 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 -- 172.21.15.103:0/1164640261 >> 172.21.15.103:0/1164640261 conn(0x55d837f5f5f0 msgr2=0x55d837f5f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 -- 172.21.15.103:0/1164640261 shutdown_connections 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 -- 172.21.15.103:0/1164640261 wait complete. 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 -- start start 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.344+0000 7f8e9e397740 1 Processor -- start 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63060 0x55d837f6b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f6b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f6e240 0x55d837f6bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9e397740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d837f70e10 con 0x55d837f63980 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9e397740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d837e50f10 con 0x55d837f6e240 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9e397740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d837f70f90 con 0x55d837f63060 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9e397740 10 client.0 _ll_register_callbacks cb 0x55d837e74db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f6b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f6b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39852/0 (socket says 172.21.15.103:39852) 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 -- 172.21.15.103:0/258944999 learned_addr learned my addr 172.21.15.103:0/258944999 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63060 0x55d837f6b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f6e240 0x55d837f6bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 -- 172.21.15.103:0/258944999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63060 msgr2=0x55d837f6b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63060 0x55d837f6b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 -- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f6e240 msgr2=0x55d837f6bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f6e240 0x55d837f6bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e99559640 1 -- 172.21.15.103:0/258944999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d837bd7580 con 0x55d837f63980 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.345+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f6e240 0x55d837f6bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e99559640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f6b6d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8e84000c00 tx=0x7f8e840d3140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62de9f93800448cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/258944999 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8401d040 con 0x55d837f63980 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/258944999 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8e84007ca0 con 0x55d837f63980 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d837ed5b90 con 0x55d837f63980 2024-09-19T07:49:00.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/258944999 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8e8409b710 con 0x55d837f63980 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e9e397740 10 client.17724 fetch_fsmap learned FSMap version 46 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d837ca1960 con 0x55d837f63980 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/258944999 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d837ed5b90 con 0x55d837f63980 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/258944999 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f8e8409b200 con 0x55d837f63980 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e9e397740 10 client.17724 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e9e397740 10 client.17724 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e9e397740 20 client.17724 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.346+0000 7f8e9e397740 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837f7dec0 0x55d837f802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.347+0000 7f8e9e397740 4 client.17724 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.347+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d837ed00a0 con 0x55d837f7dec0 2024-09-19T07:49:00.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.347+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837f7dec0 0x55d837f802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.347+0000 7f8e7d7fa640 10 client.17724 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:00.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.347+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837f7dec0 0x55d837f802f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba23a6e38cb77462 server_cookie=4fdf4746d805480d in_seq=0 out_seq=0 2024-09-19T07:49:00.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/258944999 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d837ed00a0 con 0x55d837f7dec0 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e7d7fa640 10 client.17724 handle_command_reply: tid=0 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 client.17724 shutdown 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 msgr2=0x55d837f6b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f6b6d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8e84000c00 tx=0x7f8e840d3140 comp rx=0 tx=0).stop 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 shutdown_connections 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837f7dec0 0x55d837f802f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f6e240 0x55d837f6bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f6b6d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e7d7fa640 0 client.17724 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:00.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 --2- 172.21.15.103:0/258944999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63060 0x55d837f6b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 >> 172.21.15.103:0/258944999 conn(0x55d837f5f5f0 msgr2=0x55d837f652d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:00.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 shutdown_connections 2024-09-19T07:49:00.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 1 -- 172.21.15.103:0/258944999 wait complete. 2024-09-19T07:49:00.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.348+0000 7f8e9e397740 20 client.17724 trim_cache size 0 max 16384 2024-09-19T07:49:00.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 Processor -- start 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 -- start start 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f85bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f86100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f6e240 0x55d837f68450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e941afbc0 con 0x55d837f63980 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e941b1730 con 0x55d837f63060 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9e397740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e941b02c0 con 0x55d837f6e240 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e99559640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f86100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39860/0 (socket says 172.21.15.103:39860) 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e99559640 1 -- 172.21.15.103:0/3435414756 learned_addr learned my addr 172.21.15.103:0/3435414756 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f6e240 0x55d837f68450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.365+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f85bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e99559640 1 -- 172.21.15.103:0/3435414756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f6e240 msgr2=0x55d837f68450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e99559640 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f6e240 0x55d837f68450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e99559640 1 -- 172.21.15.103:0/3435414756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 msgr2=0x55d837f85bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e99559640 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f85bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e99559640 1 -- 172.21.15.103:0/3435414756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d837f64200 con 0x55d837f63980 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f6e240 0x55d837f68450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e99559640 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f86100 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8e8409b5c0 tx=0x7f8e840d31b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfc677f69ef8c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e7f7fe640 1 -- 172.21.15.103:0/3435414756 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8401d070 con 0x55d837f63980 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e7f7fe640 1 -- 172.21.15.103:0/3435414756 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8409f020 con 0x55d837f63980 2024-09-19T07:49:00.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.366+0000 7f8e7f7fe640 1 -- 172.21.15.103:0/3435414756 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e840662d0 con 0x55d837f63980 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- 172.21.15.103:0/3435414756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 msgr2=0x55d837f86100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f86100 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8e8409b5c0 tx=0x7f8e840d31b0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- 172.21.15.103:0/3435414756 shutdown_connections 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f6e240 0x55d837f68450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f63980 0x55d837f86100 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 --2- 172.21.15.103:0/3435414756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f85bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- 172.21.15.103:0/3435414756 >> 172.21.15.103:0/3435414756 conn(0x55d837f5f5f0 msgr2=0x55d837f69aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:00.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- 172.21.15.103:0/3435414756 shutdown_connections 2024-09-19T07:49:00.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- 172.21.15.103:0/3435414756 wait complete. 2024-09-19T07:49:00.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- start start 2024-09-19T07:49:00.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 Processor -- start 2024-09-19T07:49:00.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f68380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f688c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f68e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e941b1060 con 0x55d837f6e240 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e941b0990 con 0x55d837f63060 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e941adbc0 con 0x55d837f63980 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.367+0000 7f8e9e397740 10 client.0 _ll_register_callbacks cb 0x7f8e84004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f68e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f68e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39876/0 (socket says 172.21.15.103:39876) 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 -- 172.21.15.103:0/2822084493 learned_addr learned my addr 172.21.15.103:0/2822084493 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e99d5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f68380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e99559640 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 -- 172.21.15.103:0/2822084493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 msgr2=0x55d837f688c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f688c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 -- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 msgr2=0x55d837f68380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f68380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 -- 172.21.15.103:0/2822084493 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d837fe7d40 con 0x55d837f6e240 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f68380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9a55b640 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f68e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c014fd0 tx=0x7f8e8c012890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999cbe5d1cbb48a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/2822084493 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c004be0 con 0x55d837f6e240 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/2822084493 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8e8c004d80 con 0x55d837f6e240 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e8409f020 con 0x55d837f6e240 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/2822084493 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8e8c01e880 con 0x55d837f6e240 2024-09-19T07:49:00.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9e397740 10 client.17730 fetch_fsmap learned FSMap version 46 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.368+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d837fe9040 con 0x55d837f6e240 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/2822084493 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8409f020 con 0x55d837f6e240 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/2822084493 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f8e8c031020 con 0x55d837f6e240 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e9e397740 10 client.17730 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e9e397740 10 client.17730 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e9e397740 20 client.17730 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e9e397740 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837fe94d0 0x55d837feb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e9e397740 4 client.17730 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8ac5c642", "release": true, "await": true}] 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8ac5c642", "release": true, "await": true}) -- 0x55d837e4c5a0 con 0x55d837fe94d0 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837fe94d0 0x55d837feb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e7d7fa640 10 client.17730 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:00.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.369+0000 7f8e99d5a640 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837fe94d0 0x55d837feb8c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c2a4c2ee937c9c8 server_cookie=dede05e58935db95 in_seq=0 out_seq=0 2024-09-19T07:49:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.420+0000 7f8e7d7fa640 1 -- 172.21.15.103:0/2822084493 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d837e4c5a0 con 0x55d837fe94d0 2024-09-19T07:49:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.420+0000 7f8e7d7fa640 10 client.17730 handle_command_reply: tid=0 2024-09-19T07:49:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.420+0000 7f8e9e397740 1 client.17730 shutdown 2024-09-19T07:49:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 msgr2=0x55d837f68e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f68e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c014fd0 tx=0x7f8e8c012890 comp rx=0 tx=0).stop 2024-09-19T07:49:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 shutdown_connections 2024-09-19T07:49:00.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d837fe94d0 0x55d837feb8c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d837f6e240 0x55d837f68e00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d837f63980 0x55d837f688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 --2- 172.21.15.103:0/2822084493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d837f63060 0x55d837f68380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 >> 172.21.15.103:0/2822084493 conn(0x55d837f5f5f0 msgr2=0x55d837f6b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e7d7fa640 0 client.17730 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 shutdown_connections 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 1 -- 172.21.15.103:0/2822084493 wait complete. 2024-09-19T07:49:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.421+0000 7f8e9e397740 20 client.17730 trim_cache size 0 max 16384 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 46, 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 40, 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "8ac5c642": { 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 40, 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:49:00.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:49:00.425 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:49:00.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:49:00.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:49:00.426 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:49:00.426 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e800a33b0 msgr2=0x7f8e800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e800a33b0 0x7f8e800a5870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c021000 tx=0x7f8e8c01f000 comp rx=0 tx=0).stop 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94102790 msgr2=0x7f8e94074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94102790 0x7f8e94074a30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c890 tx=0x7f8e8400cd50 comp rx=0 tx=0).stop 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 shutdown_connections 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e800a33b0 0x7f8e800a5870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e94106f10 0x7f8e94074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e94102790 0x7f8e94074a30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 --2- 172.21.15.103:0/1364293591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e94101de0 0x7f8e940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 >> 172.21.15.103:0/1364293591 conn(0x7f8e940fd8c0 msgr2=0x7f8e940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:00.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 shutdown_connections 2024-09-19T07:49:00.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:00.422+0000 7f8e9bfbe640 1 -- 172.21.15.103:0/1364293591 wait complete. 2024-09-19T07:49:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 625.0 s, 23.6 tps, lat 19726.666 ms stddev 14925.500 2024-09-19T07:49:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:00 smithi134 ceph-mon[35414]: pgmap v341: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-19T07:49:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:00 smithi174 ceph-mon[35370]: pgmap v341: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-19T07:49:01.203 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8ac5c642', total seconds elapsed: 35.5 2024-09-19T07:49:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:02 smithi134 ceph-mon[35414]: pgmap v342: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 33 KiB/s wr, 17 op/s 2024-09-19T07:49:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:02 smithi174 ceph-mon[35370]: pgmap v342: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 33 KiB/s wr, 17 op/s 2024-09-19T07:49:02.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:02 smithi103 ceph-mon[31201]: pgmap v342: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 33 KiB/s wr, 17 op/s 2024-09-19T07:49:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:04 smithi134 ceph-mon[35414]: pgmap v343: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-19T07:49:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:04 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:04 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:04 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:04 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:04 smithi174 ceph-mon[35370]: pgmap v343: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-19T07:49:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:04 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:04 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:04 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:04 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:04.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:04 smithi103 ceph-mon[31201]: pgmap v343: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-09-19T07:49:04.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:04 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:04.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:04 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:04.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:04 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:04.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:04 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 630.0 s, 2352.6 tps, lat 108.643 ms stddev 1435.642 2024-09-19T07:49:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:05 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:05 smithi134 ceph-mon[35414]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:49:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:05 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:05 smithi174 ceph-mon[35370]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:49:05.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:05 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:05.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:05 smithi103 ceph-mon[31201]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T07:49:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:06 smithi134 ceph-mon[35414]: pgmap v345: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-19T07:49:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:06 smithi174 ceph-mon[35370]: pgmap v345: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-19T07:49:06.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:06 smithi103 ceph-mon[31201]: pgmap v345: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-19T07:49:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:08 smithi134 ceph-mon[35414]: pgmap v346: 97 pgs: 37 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 43 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-09-19T07:49:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:08 smithi174 ceph-mon[35370]: pgmap v346: 97 pgs: 37 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 43 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-09-19T07:49:08.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:08 smithi103 ceph-mon[31201]: pgmap v346: 97 pgs: 37 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 43 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-09-19T07:49:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 635.0 s, 2960.4 tps, lat 33.828 ms stddev 39.060 2024-09-19T07:49:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:10 smithi134 ceph-mon[35414]: pgmap v347: 97 pgs: 37 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 43 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-09-19T07:49:10.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:10 smithi174 ceph-mon[35370]: pgmap v347: 97 pgs: 37 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 43 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-09-19T07:49:10.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:10 smithi103 ceph-mon[31201]: pgmap v347: 97 pgs: 37 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 43 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-09-19T07:49:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:12 smithi134 ceph-mon[35414]: pgmap v348: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.7 MiB/s wr, 666 op/s 2024-09-19T07:49:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:12 smithi174 ceph-mon[35370]: pgmap v348: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.7 MiB/s wr, 666 op/s 2024-09-19T07:49:12.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:12 smithi103 ceph-mon[31201]: pgmap v348: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.7 MiB/s wr, 666 op/s 2024-09-19T07:49:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:14 smithi134 ceph-mon[35414]: pgmap v349: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.7 MiB/s wr, 666 op/s 2024-09-19T07:49:14.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:14 smithi174 ceph-mon[35370]: pgmap v349: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.7 MiB/s wr, 666 op/s 2024-09-19T07:49:14.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:14 smithi103 ceph-mon[31201]: pgmap v349: 97 pgs: 97 active+clean; 3.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 8.7 MiB/s wr, 666 op/s 2024-09-19T07:49:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 640.0 s, 2916.8 tps, lat 34.448 ms stddev 39.492 2024-09-19T07:49:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:16 smithi134 ceph-mon[35414]: pgmap v350: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 10 MiB/s wr, 773 op/s 2024-09-19T07:49:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:16 smithi134 ceph-mon[35414]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:49:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:16 smithi174 ceph-mon[35370]: pgmap v350: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 10 MiB/s wr, 773 op/s 2024-09-19T07:49:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:16 smithi174 ceph-mon[35370]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:49:16.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:16 smithi103 ceph-mon[31201]: pgmap v350: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 10 MiB/s wr, 773 op/s 2024-09-19T07:49:16.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:16 smithi103 ceph-mon[31201]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T07:49:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:18 smithi134 ceph-mon[35414]: pgmap v352: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-19T07:49:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:49:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:18 smithi174 ceph-mon[35370]: pgmap v352: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-19T07:49:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:49:18.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:18 smithi103 ceph-mon[31201]: pgmap v352: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-19T07:49:18.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:49:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:49:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:49:19.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:19 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:49:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:49:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:49:19.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:19 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:49:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:49:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:49:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:19 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:49:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 645.0 s, 2884.8 tps, lat 34.280 ms stddev 37.685 2024-09-19T07:49:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:20 smithi134 ceph-mon[35414]: pgmap v353: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-19T07:49:20.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:20 smithi174 ceph-mon[35370]: pgmap v353: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-19T07:49:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:20 smithi103 ceph-mon[31201]: pgmap v353: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-19T07:49:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:22 smithi134 ceph-mon[35414]: pgmap v354: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-09-19T07:49:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:22 smithi174 ceph-mon[35370]: pgmap v354: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-09-19T07:49:22.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:22 smithi103 ceph-mon[31201]: pgmap v354: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-09-19T07:49:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:24 smithi134 ceph-mon[35414]: pgmap v355: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-09-19T07:49:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:24 smithi174 ceph-mon[35370]: pgmap v355: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-09-19T07:49:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:24 smithi103 ceph-mon[31201]: pgmap v355: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-09-19T07:49:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 650.0 s, 2187.4 tps, lat 46.027 ms stddev 51.791 2024-09-19T07:49:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:26 smithi134 ceph-mon[35414]: pgmap v356: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-09-19T07:49:26.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:26 smithi174 ceph-mon[35370]: pgmap v356: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-09-19T07:49:26.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:26 smithi103 ceph-mon[31201]: pgmap v356: 97 pgs: 97 active+clean; 3.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-09-19T07:49:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:28 smithi134 ceph-mon[35414]: pgmap v357: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 650 op/s 2024-09-19T07:49:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:28 smithi174 ceph-mon[35370]: pgmap v357: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 650 op/s 2024-09-19T07:49:28.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:28 smithi103 ceph-mon[31201]: pgmap v357: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 650 op/s 2024-09-19T07:49:29.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:49:30.136 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 655.0 s, 2352.8 tps, lat 42.293 ms stddev 45.842 2024-09-19T07:49:30.548 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:30 smithi103 ceph-mon[31201]: pgmap v358: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.0 MiB/s wr, 570 op/s 2024-09-19T07:49:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:30 smithi134 ceph-mon[35414]: pgmap v358: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.0 MiB/s wr, 570 op/s 2024-09-19T07:49:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:30 smithi174 ceph-mon[35370]: pgmap v358: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.0 MiB/s wr, 570 op/s 2024-09-19T07:49:30.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.679+0000 7fc74ffff640 1 -- 172.21.15.103:0/4039842503 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480319b0 con 0x7fc758100510 2024-09-19T07:49:30.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.680+0000 7fc75f453640 1 -- 172.21.15.103:0/4039842503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 msgr2=0x7fc758112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.680+0000 7fc75f453640 1 --2- 172.21.15.103:0/4039842503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 0x7fc758112340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc748009900 tx=0x7fc74802a480 comp rx=0 tx=0).stop 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.681+0000 7fc75f453640 1 -- 172.21.15.103:0/4039842503 shutdown_connections 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.681+0000 7fc75f453640 1 --2- 172.21.15.103:0/4039842503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758112880 0x7fc758114c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.681+0000 7fc75f453640 1 --2- 172.21.15.103:0/4039842503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 0x7fc758112340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.681+0000 7fc75f453640 1 --2- 172.21.15.103:0/4039842503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7580ffbf0 0x7fc7580fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.681+0000 7fc75f453640 1 -- 172.21.15.103:0/4039842503 >> 172.21.15.103:0/4039842503 conn(0x7fc7580fba90 msgr2=0x7fc7580fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.681+0000 7fc75f453640 1 -- 172.21.15.103:0/4039842503 shutdown_connections 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.681+0000 7fc75f453640 1 -- 172.21.15.103:0/4039842503 wait complete. 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 Processor -- start 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 -- start start 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7580ffbf0 0x7fc758110980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 0x7fc758109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc758112880 0x7fc758109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc758118560 con 0x7fc758112880 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581183e0 con 0x7fc758100510 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75f453640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581186e0 con 0x7fc7580ffbf0 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75d1ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7580ffbf0 0x7fc758110980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75d1ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7580ffbf0 0x7fc758110980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43784/0 (socket says 172.21.15.103:43784) 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75d9f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc758112880 0x7fc758109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75d1ef640 1 -- 172.21.15.103:0/1781742540 learned_addr learned my addr 172.21.15.103:0/1781742540 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.682+0000 7fc75c9ee640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 0x7fc758109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75d1ef640 1 -- 172.21.15.103:0/1781742540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 msgr2=0x7fc758109a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 0x7fc758109a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75d1ef640 1 -- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc758112880 msgr2=0x7fc758109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc758112880 0x7fc758109f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75d1ef640 1 -- 172.21.15.103:0/1781742540 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75810a590 con 0x7fc7580ffbf0 2024-09-19T07:49:30.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc758112880 0x7fc758109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:30.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7580ffbf0 0x7fc758110980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc75400e990 tx=0x7fc75400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc74e7fc640 1 -- 172.21.15.103:0/1781742540 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75400cbe0 con 0x7fc7580ffbf0 2024-09-19T07:49:30.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75810e4b0 con 0x7fc7580ffbf0 2024-09-19T07:49:30.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc74e7fc640 1 -- 172.21.15.103:0/1781742540 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75400cd80 con 0x7fc7580ffbf0 2024-09-19T07:49:30.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc74e7fc640 1 -- 172.21.15.103:0/1781742540 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754011470 con 0x7fc7580ffbf0 2024-09-19T07:49:30.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.683+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75810e9c0 con 0x7fc7580ffbf0 2024-09-19T07:49:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.689+0000 7fc76182c740 1 Processor -- start 2024-09-19T07:49:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.689+0000 7fc76182c740 1 -- start start 2024-09-19T07:49:30.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc74e7fc640 1 -- 172.21.15.103:0/1781742540 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc754005000 con 0x7fc7580ffbf0 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc76182c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd33880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc76182c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd3e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc75d9f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd33880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc75d9f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd33880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35652/0 (socket says 172.21.15.103:35652) 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc75d9f0640 1 -- 172.21.15.103:0/3066575782 learned_addr learned my addr 172.21.15.103:0/3066575782 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc76182c740 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd40a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f70bc992e0 con 0x55f70bd334a0 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f70bc40ce0 con 0x55f70bd33dc0 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f70bc952a0 con 0x55f70bd3e680 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.690+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd3e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc75c9ee640 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd40a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc74e7fc640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc7440a35b0 0x7fc7440a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc75d9f0640 1 -- 172.21.15.103:0/3066575782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 msgr2=0x55f70bd40a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd40a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc75d9f0640 1 -- 172.21.15.103:0/3066575782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 msgr2=0x55f70bd3e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd3e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc75d9f0640 1 -- 172.21.15.103:0/3066575782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f70bc25440 con 0x55f70bd334a0 2024-09-19T07:49:30.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.691+0000 7fc74e7fc640 1 -- 172.21.15.103:0/1781742540 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fc754004e00 con 0x7fc7580ffbf0 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.692+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc7440a35b0 0x7fc7440a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.692+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd33880 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc7500107a0 tx=0x7fc750010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f449ce4a61ff184 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.692+0000 7fc7437fe640 1 -- 172.21.15.103:0/3066575782 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750022070 con 0x55f70bd334a0 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.692+0000 7fc7437fe640 1 -- 172.21.15.103:0/3066575782 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75000eab0 con 0x55f70bd334a0 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.692+0000 7fc7437fe640 1 -- 172.21.15.103:0/3066575782 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75001b6e0 con 0x55f70bd334a0 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.692+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc7440a35b0 0x7fc7440a5a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc75001f000 tx=0x7fc750020000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 msgr2=0x55f70bd33880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd33880 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc7500107a0 tx=0x7fc750010c60 comp rx=0 tx=0).stop 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 shutdown_connections 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd40a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd3e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 --2- 172.21.15.103:0/3066575782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd33880 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 >> 172.21.15.103:0/3066575782 conn(0x55f70bd2fa30 msgr2=0x55f70bd2fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 shutdown_connections 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.693+0000 7fc76182c740 1 -- 172.21.15.103:0/3066575782 wait complete. 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc76182c740 1 -- start start 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc76182c740 1 Processor -- start 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc76182c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd334a0 0x55f70bd3e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc76182c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd3b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc76182c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd3e680 0x55f70bd3b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc75c9ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd3b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc75d1ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd334a0 0x55f70bd3e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc75c9ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd3b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43796/0 (socket says 172.21.15.103:43796) 2024-09-19T07:49:30.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc75d1ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd334a0 0x55f70bd3e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56374/0 (socket says 172.21.15.103:56374) 2024-09-19T07:49:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc75c9ee640 1 -- 172.21.15.103:0/3230585766 learned_addr learned my addr 172.21.15.103:0/3230585766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.694+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f70bd41250 con 0x55f70bd3e680 2024-09-19T07:49:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f70bc54050 con 0x55f70bd334a0 2024-09-19T07:49:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f70bd413d0 con 0x55f70bd33dc0 2024-09-19T07:49:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd3e680 0x55f70bd3b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc76182c740 10 client.0 _ll_register_callbacks cb 0x55f70bc94160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc75c9ee640 1 -- 172.21.15.103:0/3230585766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd334a0 msgr2=0x55f70bd3e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc75c9ee640 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd334a0 0x55f70bd3e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc75c9ee640 1 -- 172.21.15.103:0/3230585766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd3e680 msgr2=0x55f70bd3b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc75c9ee640 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd3e680 0x55f70bd3b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc75c9ee640 1 -- 172.21.15.103:0/3230585766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70ba0a3c0 con 0x55f70bd33dc0 2024-09-19T07:49:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc75c9ee640 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd3b2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc748009900 tx=0x7fc748004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2bd0f6b698e42b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.695+0000 7fc740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:49:30.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.696+0000 7fc740ff9640 1 -- 172.21.15.103:0/3230585766 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748035040 con 0x55f70bd33dc0 2024-09-19T07:49:30.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.696+0000 7fc740ff9640 1 -- 172.21.15.103:0/3230585766 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fc748005450 con 0x55f70bd33dc0 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.696+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75000eab0 con 0x55f70bd33dc0 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.696+0000 7fc740ff9640 1 -- 172.21.15.103:0/3230585766 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc748005650 con 0x55f70bd33dc0 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.696+0000 7fc76182c740 10 client.27043 fetch_fsmap learned FSMap version 46 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.696+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f70bd4cc20 con 0x55f70bd33dc0 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.696+0000 7fc740ff9640 1 -- 172.21.15.103:0/3230585766 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75000eab0 con 0x55f70bd33dc0 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc740ff9640 1 -- 172.21.15.103:0/3230585766 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fc748003ba0 con 0x55f70bd33dc0 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc76182c740 10 client.27043 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc76182c740 10 client.27043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc76182c740 20 client.27043 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:30.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc76182c740 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bd4e0e0 0x55f70bd50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc76182c740 4 client.27043 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f70bca6850 con 0x55f70bd4e0e0 2024-09-19T07:49:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.697+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bd4e0e0 0x55f70bd50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.701+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bd4e0e0 0x55f70bd50560 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af170d16685f0ccb server_cookie=99903b0031e4b8b5 in_seq=0 out_seq=0 2024-09-19T07:49:30.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.701+0000 7fc740ff9640 10 client.27043 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.701+0000 7fc740ff9640 1 -- 172.21.15.103:0/3230585766 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f70bca6850 con 0x55f70bd4e0e0 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.701+0000 7fc740ff9640 10 client.27043 handle_command_reply: tid=0 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.701+0000 7fc76182c740 1 client.27043 shutdown 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 msgr2=0x55f70bd3b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd3b2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc748009900 tx=0x7fc748004440 comp rx=0 tx=0).stop 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 shutdown_connections 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bd4e0e0 0x55f70bd50560 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd3e680 0x55f70bd3b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd3b2c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 --2- 172.21.15.103:0/3230585766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd334a0 0x55f70bd3e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 >> 172.21.15.103:0/3230585766 conn(0x55f70bd2fa30 msgr2=0x55f70bd2eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc740ff9640 0 client.27043 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:30.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 shutdown_connections 2024-09-19T07:49:30.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 1 -- 172.21.15.103:0/3230585766 wait complete. 2024-09-19T07:49:30.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.702+0000 7fc76182c740 20 client.27043 trim_cache size 0 max 16384 2024-09-19T07:49:30.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 Processor -- start 2024-09-19T07:49:30.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 -- start start 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd55e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd3e680 0x55f70bd56370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581bc880 con 0x55f70bd334a0 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581be3f0 con 0x55f70bd3e680 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc76182c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581bcf80 con 0x55f70bd33dc0 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.724+0000 7fc75d1ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75c9ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd55e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75c9ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd55e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43802/0 (socket says 172.21.15.103:43802) 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75c9ee640 1 -- 172.21.15.103:0/3237187857 learned_addr learned my addr 172.21.15.103:0/3237187857 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd3e680 0x55f70bd56370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75d9f0640 1 -- 172.21.15.103:0/3237187857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 msgr2=0x55f70bd55e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd55e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75d9f0640 1 -- 172.21.15.103:0/3237187857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 msgr2=0x55f70bd4a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd4a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75d9f0640 1 -- 172.21.15.103:0/3237187857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f70bd30af0 con 0x55f70bd3e680 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75c9ee640 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd55e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:30.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd3e680 0x55f70bd56370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc750005fd0 tx=0x7fc750025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91bf924b293f0bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.725+0000 7fc742ffd640 1 -- 172.21.15.103:0/3237187857 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750036070 con 0x55f70bd3e680 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc742ffd640 1 -- 172.21.15.103:0/3237187857 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75000eab0 con 0x55f70bd3e680 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc742ffd640 1 -- 172.21.15.103:0/3237187857 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750022040 con 0x55f70bd3e680 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 -- 172.21.15.103:0/3237187857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd3e680 msgr2=0x55f70bd56370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd3e680 0x55f70bd56370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc750005fd0 tx=0x7fc750025000 comp rx=0 tx=0).stop 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 -- 172.21.15.103:0/3237187857 shutdown_connections 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd3e680 0x55f70bd56370 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd33dc0 0x55f70bd55e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 --2- 172.21.15.103:0/3237187857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd4a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 -- 172.21.15.103:0/3237187857 >> 172.21.15.103:0/3237187857 conn(0x55f70bd2fa30 msgr2=0x55f70bd3a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.726+0000 7fc76182c740 1 -- 172.21.15.103:0/3237187857 shutdown_connections 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.727+0000 7fc76182c740 1 -- 172.21.15.103:0/3237187857 wait complete. 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.727+0000 7fc76182c740 1 -- start start 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.727+0000 7fc76182c740 1 Processor -- start 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc76182c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc76182c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd46f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc76182c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd43f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc76182c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581bdd20 con 0x55f70bd334a0 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc76182c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581bd650 con 0x55f70bd33dc0 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc76182c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581ba880 con 0x55f70bd3e680 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc76182c740 10 client.0 _ll_register_callbacks cb 0x7fc748009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d1ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d1ef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35678/0 (socket says 172.21.15.103:35678) 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d1ef640 1 -- 172.21.15.103:0/393254745 learned_addr learned my addr 172.21.15.103:0/393254745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd43f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75c9ee640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd46f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d9f0640 1 -- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 msgr2=0x55f70bd46f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd46f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d9f0640 1 -- 172.21.15.103:0/393254745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 msgr2=0x55f70bd469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d9f0640 1 -- 172.21.15.103:0/393254745 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f70bd44780 con 0x55f70bd3e680 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.728+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc75d9f0640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd43f80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc750000f80 tx=0x7fc750005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a401dc695c5626c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:49:30.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc740ff9640 1 -- 172.21.15.103:0/393254745 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750022050 con 0x55f70bd3e680 2024-09-19T07:49:30.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc740ff9640 1 -- 172.21.15.103:0/393254745 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fc75000ed50 con 0x55f70bd3e680 2024-09-19T07:49:30.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75000eab0 con 0x55f70bd3e680 2024-09-19T07:49:30.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc740ff9640 1 -- 172.21.15.103:0/393254745 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc750036040 con 0x55f70bd3e680 2024-09-19T07:49:30.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc76182c740 10 client.27049 fetch_fsmap learned FSMap version 46 2024-09-19T07:49:30.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f70bdb8aa0 con 0x55f70bd3e680 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.729+0000 7fc740ff9640 1 -- 172.21.15.103:0/393254745 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75000eab0 con 0x55f70bd3e680 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc740ff9640 1 -- 172.21.15.103:0/393254745 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fc750039070 con 0x55f70bd3e680 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc76182c740 10 client.27049 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc76182c740 10 client.27049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc76182c740 20 client.27049 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc76182c740 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bdb8e70 0x55f70bdbb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc76182c740 4 client.27049 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f70bca0d60 con 0x55f70bdb8e70 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bdb8e70 0x55f70bdbb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:30.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc740ff9640 10 client.27049 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.730+0000 7fc75d1ef640 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bdb8e70 0x55f70bdbb260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c87eea24e6a8e94 server_cookie=f6cd3e32de2be309 in_seq=0 out_seq=0 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc740ff9640 1 -- 172.21.15.103:0/393254745 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f70bca0d60 con 0x55f70bdb8e70 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc740ff9640 10 client.27049 handle_command_reply: tid=0 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 client.27049 shutdown 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 msgr2=0x55f70bd43f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd43f80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc750000f80 tx=0x7fc750005e30 comp rx=0 tx=0).stop 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 shutdown_connections 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55f70bdb8e70 0x55f70bdbb260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f70bd3e680 0x55f70bd43f80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc740ff9640 0 client.27049 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f70bd33dc0 0x55f70bd46f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 --2- 172.21.15.103:0/393254745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f70bd334a0 0x55f70bd469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 >> 172.21.15.103:0/393254745 conn(0x55f70bd2fa30 msgr2=0x55f70bd2e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 shutdown_connections 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.731+0000 7fc76182c740 1 -- 172.21.15.103:0/393254745 wait complete. 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.732+0000 7fc76182c740 20 client.27049 trim_cache size 0 max 16384 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:49:30.736 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:49:30.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc7440a35b0 msgr2=0x7fc7440a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc7440a35b0 0x7fc7440a5a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc75001f000 tx=0x7fc750020000 comp rx=0 tx=0).stop 2024-09-19T07:49:30.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7580ffbf0 msgr2=0x7fc758110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:30.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7580ffbf0 0x7fc758110980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc75400e990 tx=0x7fc75400ee50 comp rx=0 tx=0).stop 2024-09-19T07:49:30.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 shutdown_connections 2024-09-19T07:49:30.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc7440a35b0 0x7fc7440a5a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc758112880 0x7fc758109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100510 0x7fc758109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 --2- 172.21.15.103:0/1781742540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7580ffbf0 0x7fc758110980 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:30.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 >> 172.21.15.103:0/1781742540 conn(0x7fc7580fba90 msgr2=0x7fc758111830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:30.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 shutdown_connections 2024-09-19T07:49:30.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:30.735+0000 7fc75f453640 1 -- 172.21.15.103:0/1781742540 wait complete. 2024-09-19T07:49:31.844 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:49:31.844 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:49:32.242 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:32.644 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:32 smithi103 ceph-mon[31201]: pgmap v359: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.8 MiB/s wr, 849 op/s 2024-09-19T07:49:32.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:32 smithi174 ceph-mon[35370]: pgmap v359: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.8 MiB/s wr, 849 op/s 2024-09-19T07:49:32.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 -- 172.21.15.103:0/2261569877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c8100c60 msgr2=0x7f45c8104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/2261569877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c8100c60 0x7f45c8104cb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009900 tx=0x7f45bc02a410 comp rx=0 tx=0).stop 2024-09-19T07:49:32.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 -- 172.21.15.103:0/2261569877 shutdown_connections 2024-09-19T07:49:32.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/2261569877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45c81053e0 0x7f45c810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/2261569877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c8100c60 0x7f45c8104cb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/2261569877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c81002b0 0x7f45c8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 -- 172.21.15.103:0/2261569877 >> 172.21.15.103:0/2261569877 conn(0x7f45c8077070 msgr2=0x7f45c80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:32.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 -- 172.21.15.103:0/2261569877 shutdown_connections 2024-09-19T07:49:32.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.757+0000 7f45ccd9f640 1 -- 172.21.15.103:0/2261569877 wait complete. 2024-09-19T07:49:32.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.759+0000 7f45ccd9f640 1 Processor -- start 2024-09-19T07:49:32.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.759+0000 7f45ccd9f640 1 -- start start 2024-09-19T07:49:32.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.759+0000 7f45ccd9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c81002b0 0x7f45c819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.760+0000 7f45ccd9f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8100c60 0x7f45c819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.760+0000 7f45c77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c81002b0 0x7f45c819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.760+0000 7f45c77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c81002b0 0x7f45c819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56412/0 (socket says 172.21.15.103:56412) 2024-09-19T07:49:32.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.760+0000 7f45c77fe640 1 -- 172.21.15.103:0/93490140 learned_addr learned my addr 172.21.15.103:0/93490140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:32.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.760+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45c81053e0 0x7f45c8196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.760+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45c8110450 con 0x7f45c8100c60 2024-09-19T07:49:32.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.760+0000 7f45c7fff640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45c81053e0 0x7f45c8196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.761+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81102d0 con 0x7f45c81002b0 2024-09-19T07:49:32.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.761+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45c81105d0 con 0x7f45c81053e0 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.761+0000 7f45c77fe640 1 -- 172.21.15.103:0/93490140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45c81053e0 msgr2=0x7f45c8196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.761+0000 7f45c77fe640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45c81053e0 0x7f45c8196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.761+0000 7f45c77fe640 1 -- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8100c60 msgr2=0x7f45c819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.761+0000 7f45c77fe640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8100c60 0x7f45c819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.761+0000 7f45c77fe640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c8196f90 con 0x7f45c81002b0 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.762+0000 7f45c77fe640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c81002b0 0x7f45c819c650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c970 tx=0x7f45b800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.762+0000 7f45c4ff9640 1 -- 172.21.15.103:0/93490140 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800bcb0 con 0x7f45c81002b0 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.762+0000 7f45c4ff9640 1 -- 172.21.15.103:0/93490140 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b8013070 con 0x7f45c81002b0 2024-09-19T07:49:32.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.762+0000 7f45c4ff9640 1 -- 172.21.15.103:0/93490140 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b800e6f0 con 0x7f45c81002b0 2024-09-19T07:49:32.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.762+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c8197220 con 0x7f45c81002b0 2024-09-19T07:49:32.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.762+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c80698b0 con 0x7f45c81002b0 2024-09-19T07:49:32.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.763+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c81975e0 con 0x7f45c81002b0 2024-09-19T07:49:32.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.765+0000 7f45c4ff9640 1 -- 172.21.15.103:0/93490140 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f45b8010440 con 0x7f45c81002b0 2024-09-19T07:49:32.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.766+0000 7f45c4ff9640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f45b00a78c0 0x7f45b00a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:32.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.767+0000 7f45c4ff9640 1 -- 172.21.15.103:0/93490140 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f45b80d0690 con 0x7f45c81002b0 2024-09-19T07:49:32.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.767+0000 7f45c6ffd640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f45b00a78c0 0x7f45b00a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:32.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.767+0000 7f45c6ffd640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f45b00a78c0 0x7f45b00a9d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0096c0 tx=0x7f45bc000bc0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:32.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:32.769+0000 7f45c4ff9640 1 -- 172.21.15.103:0/93490140 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45b809d110 con 0x7f45c81002b0 2024-09-19T07:49:32.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:32 smithi134 ceph-mon[35414]: pgmap v359: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.8 MiB/s wr, 849 op/s 2024-09-19T07:49:33.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.094+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45c8101790 con 0x7f45c81002b0 2024-09-19T07:49:33.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.095+0000 7f45c4ff9640 1 -- 172.21.15.103:0/93490140 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7f45b809ca10 con 0x7f45c81002b0 2024-09-19T07:49:33.095 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:49:33.095 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":46,"btime":"2024-09-19T07:48:00:225048+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:47:59.861225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:49:33.095 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 46 2024-09-19T07:49:33.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f45b00a78c0 msgr2=0x7f45b00a9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:33.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f45b00a78c0 0x7f45b00a9d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0096c0 tx=0x7f45bc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c81002b0 msgr2=0x7f45c819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:33.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c81002b0 0x7f45c819c650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c970 tx=0x7f45b800ce30 comp rx=0 tx=0).stop 2024-09-19T07:49:33.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 shutdown_connections 2024-09-19T07:49:33.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f45b00a78c0 0x7f45b00a9d80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45c81053e0 0x7f45c8196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45c8100c60 0x7f45c819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 --2- 172.21.15.103:0/93490140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45c81002b0 0x7f45c819c650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:33.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.097+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 >> 172.21.15.103:0/93490140 conn(0x7f45c8077070 msgr2=0x7f45c8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:33.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.098+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 shutdown_connections 2024-09-19T07:49:33.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:33.098+0000 7f45ccd9f640 1 -- 172.21.15.103:0/93490140 wait complete. 2024-09-19T07:49:33.424 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:33 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/93490140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:49:33.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:33 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/93490140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:49:33.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:33 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/93490140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:49:34.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:34 smithi103 ceph-mon[31201]: pgmap v360: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-09-19T07:49:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:34 smithi174 ceph-mon[35370]: pgmap v360: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-09-19T07:49:34.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:49:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:34 smithi134 ceph-mon[35414]: pgmap v360: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-09-19T07:49:35.033 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 660.0 s, 1816.0 tps, lat 55.382 ms stddev 59.639 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 -- 172.21.15.103:0/4146303207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 msgr2=0x7f6ba81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 --2- 172.21.15.103:0/4146303207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba81021d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009900 tx=0x7f6b9802a410 comp rx=0 tx=0).stop 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 -- 172.21.15.103:0/4146303207 shutdown_connections 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 --2- 172.21.15.103:0/4146303207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8106f20 0x7f6ba810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 --2- 172.21.15.103:0/4146303207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba81027a0 0x7f6ba81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 --2- 172.21.15.103:0/4146303207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba81021d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 -- 172.21.15.103:0/4146303207 >> 172.21.15.103:0/4146303207 conn(0x7f6ba80fd8f0 msgr2=0x7f6ba80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 -- 172.21.15.103:0/4146303207 shutdown_connections 2024-09-19T07:49:35.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.646+0000 7f6bad410640 1 -- 172.21.15.103:0/4146303207 wait complete. 2024-09-19T07:49:35.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 Processor -- start 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 -- start start 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba81027a0 0x7f6ba8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8106f20 0x7f6ba8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35688/0 (socket says 172.21.15.103:35688) 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8111f80 con 0x7f6ba8101df0 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/2860181957 learned_addr learned my addr 172.21.15.103:0/2860181957 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8111e00 con 0x7f6ba81027a0 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.647+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8112100 con 0x7f6ba8106f20 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba81027a0 0x7f6ba8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8106f20 0x7f6ba8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/2860181957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8106f20 msgr2=0x7f6ba8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8106f20 0x7f6ba8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/2860181957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba81027a0 msgr2=0x7f6ba8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba81027a0 0x7f6ba8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/2860181957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba81a6880 con 0x7f6ba8101df0 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba81027a0 0x7f6ba8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8106f20 0x7f6ba8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba80744f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006010 tx=0x7f6b98004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6b8ffff640 1 -- 172.21.15.103:0/2860181957 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9803f030 con 0x7f6ba8101df0 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba81a6b10 con 0x7f6ba8101df0 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6b8ffff640 1 -- 172.21.15.103:0/2860181957 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98004440 con 0x7f6ba8101df0 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6b8ffff640 1 -- 172.21.15.103:0/2860181957 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98035070 con 0x7f6ba8101df0 2024-09-19T07:49:35.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.648+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81a7040 con 0x7f6ba8101df0 2024-09-19T07:49:35.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.652+0000 7f6baf7e9740 1 Processor -- start 2024-09-19T07:49:35.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6baf7e9740 1 -- start start 2024-09-19T07:49:35.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6b8ffff640 1 -- 172.21.15.103:0/2860181957 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6b98042030 con 0x7f6ba8101df0 2024-09-19T07:49:35.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490dca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490dcaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490ddff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e490d302f0 con 0x55e490dca5d0 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e490cd7cf0 con 0x55e490dcbb40 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.653+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e490d2c2b0 con 0x55e490dddb80 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490ddff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490ddff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:43868/0 (socket says 172.21.15.103:43868) 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 -- 172.21.15.103:0/310990107 learned_addr learned my addr 172.21.15.103:0/310990107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 -- 172.21.15.103:0/310990107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 msgr2=0x55e490dca090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:49:35.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490dcaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6b8ffff640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b900a3430 0x7f6b900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6b8ffff640 1 -- 172.21.15.103:0/2860181957 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f6b980f0080 con 0x7f6ba8101df0 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490dca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490dca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 -- 172.21.15.103:0/310990107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 msgr2=0x55e490dcaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490dcaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.654+0000 7f6ba67fc640 1 -- 172.21.15.103:0/310990107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e490cbc3a0 con 0x55e490dddb80 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.655+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490dcaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.655+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490dca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.655+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b900a3430 0x7f6b900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.655+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490ddff70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0114f0 tx=0x7f6b9c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93de276a6c654781 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.655+0000 7f6b8d7fa640 1 -- 172.21.15.103:0/310990107 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c01e070 con 0x55e490dddb80 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.655+0000 7f6b8d7fa640 1 -- 172.21.15.103:0/310990107 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9c0035f0 con 0x55e490dddb80 2024-09-19T07:49:35.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.655+0000 7f6b8d7fa640 1 -- 172.21.15.103:0/310990107 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00f820 con 0x55e490dddb80 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6baf7e9740 1 -- 172.21.15.103:0/310990107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 msgr2=0x55e490ddff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490ddff70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0114f0 tx=0x7f6b9c0119b0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b900a3430 0x7f6b900a58f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00094d0 tx=0x7f6ba0004a40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6baf7e9740 1 -- 172.21.15.103:0/310990107 shutdown_connections 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490ddff70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490dcaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/310990107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490dca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.656+0000 7f6baf7e9740 1 -- 172.21.15.103:0/310990107 >> 172.21.15.103:0/310990107 conn(0x55e490d93310 msgr2=0x55e490dc6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.657+0000 7f6baf7e9740 1 -- 172.21.15.103:0/310990107 shutdown_connections 2024-09-19T07:49:35.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.657+0000 7f6baf7e9740 1 -- 172.21.15.103:0/310990107 wait complete. 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.657+0000 7f6baf7e9740 1 -- start start 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.657+0000 7f6baf7e9740 1 Processor -- start 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490ddd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dcbb40 0x55e490dda730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dddb80 0x55e490ddac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e490de0880 con 0x55e490dca5d0 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e490ceb060 con 0x55e490dddb80 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e490de0a00 con 0x55e490dcbb40 2024-09-19T07:49:35.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6baf7e9740 10 client.0 _ll_register_callbacks cb 0x55e490a3e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490ddd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490ddd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35710/0 (socket says 172.21.15.103:35710) 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1047955312 learned_addr learned my addr 172.21.15.103:0/1047955312 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dddb80 0x55e490ddac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dcbb40 0x55e490dda730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dcbb40 msgr2=0x55e490dda730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dcbb40 0x55e490dda730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1047955312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dddb80 msgr2=0x55e490ddac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dddb80 0x55e490ddac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1047955312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e490aa13c0 con 0x55e490dca5d0 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dddb80 0x55e490ddac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490ddd430 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b8d20 tx=0x7f6b980bc120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f76b065d4949801 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.658+0000 7f6b8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1047955312 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98049050 con 0x55e490dca5d0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1047955312 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f6b980b89c0 con 0x55e490dca5d0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9c0035f0 con 0x55e490dca5d0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1047955312 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6b980bca50 con 0x55e490dca5d0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 10 client.17760 fetch_fsmap learned FSMap version 46 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e490dec1a0 con 0x55e490dca5d0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1047955312 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9c0035f0 con 0x55e490dca5d0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1047955312 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f6b980d5120 con 0x55e490dca5d0 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 10 client.17760 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 10 client.17760 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 20 client.17760 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490ded590 0x55e490def9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 4 client.17760 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:35.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.659+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e490d3d860 con 0x55e490ded590 2024-09-19T07:49:35.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.660+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490ded590 0x55e490def9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.660+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490ded590 0x55e490def9c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f92a2113951e5bda server_cookie=e4aff96627810632 in_seq=0 out_seq=0 2024-09-19T07:49:35.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.660+0000 7f6b8aff5640 10 client.17760 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1047955312 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e490d3d860 con 0x55e490ded590 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6b8aff5640 10 client.17760 handle_command_reply: tid=0 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 client.17760 shutdown 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 msgr2=0x55e490ddd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490ddd430 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b8d20 tx=0x7f6b980bc120 comp rx=0 tx=0).stop 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 shutdown_connections 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490ded590 0x55e490def9c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dddb80 0x55e490ddac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dcbb40 0x55e490dda730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1047955312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490ddd430 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6b8aff5640 0 client.17760 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:49:35.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 >> 172.21.15.103:0/1047955312 conn(0x55e490d93310 msgr2=0x55e490dcc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.662+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 shutdown_connections 2024-09-19T07:49:35.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.663+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1047955312 wait complete. 2024-09-19T07:49:35.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.663+0000 7f6baf7e9740 20 client.17760 trim_cache size 0 max 16384 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.678+0000 7f6baf7e9740 1 Processor -- start 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.678+0000 7f6baf7e9740 1 -- start start 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dca5d0 0x55e490de9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490ddc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dddb80 0x55e490df52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ab6b0 con 0x55e490dddb80 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ad220 con 0x55e490dcbb40 2024-09-19T07:49:35.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6baf7e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81abdb0 con 0x55e490dca5d0 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dddb80 0x55e490df52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490ddc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490ddc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56458/0 (socket says 172.21.15.103:56458) 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 -- 172.21.15.103:0/744934981 learned_addr learned my addr 172.21.15.103:0/744934981 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dca5d0 0x55e490de9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 -- 172.21.15.103:0/744934981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dca5d0 msgr2=0x55e490de9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dca5d0 0x55e490de9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 -- 172.21.15.103:0/744934981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dddb80 msgr2=0x55e490df52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dddb80 0x55e490df52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 -- 172.21.15.103:0/744934981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e490dc5310 con 0x55e490dcbb40 2024-09-19T07:49:35.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.679+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490ddc6a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c000c00 tx=0x7f6b9c004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d79ded9e2466617 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.680+0000 7f6b8cff9640 1 -- 172.21.15.103:0/744934981 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c01e040 con 0x55e490dcbb40 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.680+0000 7f6b8cff9640 1 -- 172.21.15.103:0/744934981 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9c0035f0 con 0x55e490dcbb40 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.680+0000 7f6b8cff9640 1 -- 172.21.15.103:0/744934981 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00f5c0 con 0x55e490dcbb40 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.680+0000 7f6baf7e9740 1 -- 172.21.15.103:0/744934981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 msgr2=0x55e490ddc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.680+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490ddc6a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c000c00 tx=0x7f6b9c004f20 comp rx=0 tx=0).stop 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.681+0000 7f6baf7e9740 1 -- 172.21.15.103:0/744934981 shutdown_connections 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.681+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dddb80 0x55e490df52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.681+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490ddc6a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.681+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/744934981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dca5d0 0x55e490de9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.681+0000 7f6baf7e9740 1 -- 172.21.15.103:0/744934981 >> 172.21.15.103:0/744934981 conn(0x55e490d93310 msgr2=0x55e490ddbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.681+0000 7f6baf7e9740 1 -- 172.21.15.103:0/744934981 shutdown_connections 2024-09-19T07:49:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.681+0000 7f6baf7e9740 1 -- 172.21.15.103:0/744934981 wait complete. 2024-09-19T07:49:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.682+0000 7f6baf7e9740 1 -- start start 2024-09-19T07:49:35.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.682+0000 7f6baf7e9740 1 Processor -- start 2024-09-19T07:49:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.682+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490de5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490de61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6baf7e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490de2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490de5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490de5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35734/0 (socket says 172.21.15.103:35734) 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1239636308 learned_addr learned my addr 172.21.15.103:0/1239636308 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490de61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba77fe640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490de2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81acb50 con 0x55e490dca5d0 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81ac480 con 0x55e490dcbb40 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81a96d0 con 0x55e490dddb80 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6baf7e9740 10 client.0 _ll_register_callbacks cb 0x7f6b98005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 msgr2=0x55e490de2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490de2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1239636308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 msgr2=0x55e490de61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490de61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba6ffd640 1 -- 172.21.15.103:0/1239636308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e490dc6510 con 0x55e490dca5d0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.683+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490de61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6ba6ffd640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490de5c90 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b87b0 tx=0x7f6b980b8c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa04604b628d33e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6b8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1239636308 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98049070 con 0x55e490dca5d0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1239636308 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f6b980c4d30 con 0x55e490dca5d0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9c0035f0 con 0x55e490dca5d0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1239636308 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6b980bc840 con 0x55e490dca5d0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6baf7e9740 10 client.17766 fetch_fsmap learned FSMap version 46 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e490e58610 con 0x55e490dca5d0 2024-09-19T07:49:35.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.684+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1239636308 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b980d5330 con 0x55e490dca5d0 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.685+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1239636308 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f6b9803bab0 con 0x55e490dca5d0 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.685+0000 7f6baf7e9740 10 client.17766 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.685+0000 7f6baf7e9740 10 client.17766 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.685+0000 7f6baf7e9740 20 client.17766 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.686+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490e58bd0 0x55e490e5aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.686+0000 7f6baf7e9740 4 client.17766 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.686+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6b9c0035f0 con 0x55e490e58bd0 2024-09-19T07:49:35.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.686+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490e58bd0 0x55e490e5aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.702+0000 7f6b8aff5640 10 client.17766 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:49:35.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.702+0000 7f6ba67fc640 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490e58bd0 0x55e490e5aff0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4aad3a583b29b08 server_cookie=b21250ccd8a7061b in_seq=0 out_seq=0 2024-09-19T07:49:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.705+0000 7f6b8aff5640 1 -- 172.21.15.103:0/1239636308 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6b9c0035f0 con 0x55e490e58bd0 2024-09-19T07:49:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.705+0000 7f6b8aff5640 10 client.17766 handle_command_reply: tid=0 2024-09-19T07:49:35.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.705+0000 7f6baf7e9740 1 client.17766 shutdown 2024-09-19T07:49:35.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.709+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 msgr2=0x55e490de5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.709+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490de5c90 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b87b0 tx=0x7f6b980b8c70 comp rx=0 tx=0).stop 2024-09-19T07:49:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.710+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 shutdown_connections 2024-09-19T07:49:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.710+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e490e58bd0 0x55e490e5aff0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.710+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e490dddb80 0x55e490de2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.710+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e490dcbb40 0x55e490de61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.710+0000 7f6b8aff5640 0 client.17766 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:49:35.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.710+0000 7f6baf7e9740 1 --2- 172.21.15.103:0/1239636308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e490dca5d0 0x55e490de5c90 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.710+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 >> 172.21.15.103:0/1239636308 conn(0x55e490d93310 msgr2=0x55e490ddc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.712+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 shutdown_connections 2024-09-19T07:49:35.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.712+0000 7f6baf7e9740 1 -- 172.21.15.103:0/1239636308 wait complete. 2024-09-19T07:49:35.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.712+0000 7f6baf7e9740 20 client.17766 trim_cache size 0 max 16384 2024-09-19T07:49:35.712 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:49:35.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b900a3430 msgr2=0x7f6b900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b900a3430 0x7f6b900a58f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00094d0 tx=0x7f6ba0004a40 comp rx=0 tx=0).stop 2024-09-19T07:49:35.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 msgr2=0x7f6ba80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:35.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba80744f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006010 tx=0x7f6b98004040 comp rx=0 tx=0).stop 2024-09-19T07:49:35.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 shutdown_connections 2024-09-19T07:49:35.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b900a3430 0x7f6b900a58f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba8106f20 0x7f6ba8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba81027a0 0x7f6ba8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 --2- 172.21.15.103:0/2860181957 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba8101df0 0x7f6ba80744f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:35.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 >> 172.21.15.103:0/2860181957 conn(0x7f6ba80fd8f0 msgr2=0x7f6ba80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:35.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.713+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 shutdown_connections 2024-09-19T07:49:35.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:35.714+0000 7f6bad410640 1 -- 172.21.15.103:0/2860181957 wait complete. 2024-09-19T07:49:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:36 smithi103 ceph-mon[31201]: pgmap v361: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-09-19T07:49:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:36 smithi174 ceph-mon[35370]: pgmap v361: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-09-19T07:49:36.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:36 smithi134 ceph-mon[35414]: pgmap v361: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.6 MiB/s wr, 650 op/s 2024-09-19T07:49:36.916 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:49:37.287 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:38.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.308+0000 7f6ba5717640 1 -- 172.21.15.103:0/2824362256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 msgr2=0x7f6ba0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.308+0000 7f6ba5717640 1 --2- 172.21.15.103:0/2824362256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba0100ca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009930 tx=0x7f6b9002a420 comp rx=0 tx=0).stop 2024-09-19T07:49:38.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.309+0000 7f6ba5717640 1 -- 172.21.15.103:0/2824362256 shutdown_connections 2024-09-19T07:49:38.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.309+0000 7f6ba5717640 1 --2- 172.21.15.103:0/2824362256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba0105ec0 0x7f6ba0106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.309+0000 7f6ba5717640 1 --2- 172.21.15.103:0/2824362256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba01011e0 0x7f6ba0105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.309+0000 7f6ba5717640 1 --2- 172.21.15.103:0/2824362256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba0100ca0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.309+0000 7f6ba5717640 1 -- 172.21.15.103:0/2824362256 >> 172.21.15.103:0/2824362256 conn(0x7f6ba00fc7d0 msgr2=0x7f6ba00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.309+0000 7f6ba5717640 1 -- 172.21.15.103:0/2824362256 shutdown_connections 2024-09-19T07:49:38.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.309+0000 7f6ba5717640 1 -- 172.21.15.103:0/2824362256 wait complete. 2024-09-19T07:49:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.310+0000 7f6ba5717640 1 Processor -- start 2024-09-19T07:49:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6ba5717640 1 -- start start 2024-09-19T07:49:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6ba5717640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba01a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6ba5717640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba01011e0 0x7f6ba01a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6ba5717640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba0105ec0 0x7f6ba019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6b9effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba01a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6b9effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba01a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38610/0 (socket says 172.21.15.103:38610) 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6b9effd640 1 -- 172.21.15.103:0/4142840151 learned_addr learned my addr 172.21.15.103:0/4142840151 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6ba5717640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0118660 con 0x7f6ba0105ec0 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01184e0 con 0x7f6ba01011e0 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01187e0 con 0x7f6ba01008c0 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba0105ec0 0x7f6ba019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.311+0000 7f6b97fff640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba01011e0 0x7f6ba01a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9effd640 1 -- 172.21.15.103:0/4142840151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba01011e0 msgr2=0x7f6ba01a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9effd640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba01011e0 0x7f6ba01a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9effd640 1 -- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba0105ec0 msgr2=0x7f6ba019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9effd640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba0105ec0 0x7f6ba019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9effd640 1 -- 172.21.15.103:0/4142840151 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba019f100 con 0x7f6ba01008c0 2024-09-19T07:49:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b97fff640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba01011e0 0x7f6ba01a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9effd640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba01a4810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b9002a940 tx=0x7f6b90003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9cff9640 1 -- 172.21.15.103:0/4142840151 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9003f070 con 0x7f6ba01008c0 2024-09-19T07:49:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba019f390 con 0x7f6ba01008c0 2024-09-19T07:49:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.312+0000 7f6b9cff9640 1 -- 172.21.15.103:0/4142840151 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b90031c90 con 0x7f6ba01008c0 2024-09-19T07:49:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.313+0000 7f6b9cff9640 1 -- 172.21.15.103:0/4142840151 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90035030 con 0x7f6ba01008c0 2024-09-19T07:49:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.313+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba01ab6e0 con 0x7f6ba01008c0 2024-09-19T07:49:38.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.315+0000 7f6b9cff9640 1 -- 172.21.15.103:0/4142840151 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f6b90042070 con 0x7f6ba01008c0 2024-09-19T07:49:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.316+0000 7f6b9cff9640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b800a3390 0x7f6b800a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.316+0000 7f6ba7af0740 1 Processor -- start 2024-09-19T07:49:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.316+0000 7f6ba7af0740 1 -- start start 2024-09-19T07:49:38.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.316+0000 7f6b9cff9640 1 -- 172.21.15.103:0/4142840151 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f6b900f00c0 con 0x7f6ba01008c0 2024-09-19T07:49:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.317+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 0x556f8c55c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.317+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c55c5d0 0x556f8c55ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.317+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c567370 0x556f8c569760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.317+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556f8c4c22f0 con 0x556f8c55c5d0 2024-09-19T07:49:38.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.317+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f8c469cf0 con 0x556f8c55db40 2024-09-19T07:49:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.317+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f8c4be2b0 con 0x556f8c567370 2024-09-19T07:49:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.317+0000 7f6b97fff640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b800a3390 0x7f6b800a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.318+0000 7f6b9f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c567370 0x556f8c569760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.318+0000 7f6b9f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c567370 0x556f8c569760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38622/0 (socket says 172.21.15.103:38622) 2024-09-19T07:49:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.318+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/1477587449 learned_addr learned my addr 172.21.15.103:0/1477587449 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:38.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.318+0000 7f6b97fff640 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c55c5d0 0x556f8c55ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/1477587449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 msgr2=0x556f8c55c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 0x556f8c55c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/1477587449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c55c5d0 msgr2=0x556f8c55ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c55c5d0 0x556f8c55ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/1477587449 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8c44e3a0 con 0x556f8c567370 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c567370 0x556f8c569760 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009f70 tx=0x7f6b98010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cfd63b84b46feb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b957fa640 1 -- 172.21.15.103:0/1477587449 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98016070 con 0x556f8c567370 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.319+0000 7f6b957fa640 1 -- 172.21.15.103:0/1477587449 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98004d20 con 0x556f8c567370 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6b957fa640 1 -- 172.21.15.103:0/1477587449 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98012950 con 0x556f8c567370 2024-09-19T07:49:38.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1477587449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c567370 msgr2=0x556f8c569760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c567370 0x556f8c569760 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009f70 tx=0x7f6b98010800 comp rx=0 tx=0).stop 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6b97fff640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b800a3390 0x7f6b800a5850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b88005d00 tx=0x7f6b88005c70 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1477587449 shutdown_connections 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c567370 0x556f8c569760 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c55c5d0 0x556f8c55ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1477587449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 0x556f8c55c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1477587449 >> 172.21.15.103:0/1477587449 conn(0x556f8c525310 msgr2=0x556f8c558c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1477587449 shutdown_connections 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.320+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1477587449 wait complete. 2024-09-19T07:49:38.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 -- start start 2024-09-19T07:49:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 Processor -- start 2024-09-19T07:49:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55c5d0 0x556f8c566d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 0x556f8c564020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c564610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556f8c56a110 con 0x556f8c567370 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f8c47d060 con 0x556f8c55db40 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f8c56a290 con 0x556f8c55c5d0 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6ba7af0740 10 client.0 _ll_register_callbacks cb 0x556f8c1d08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6b9effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55c5d0 0x556f8c566d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6b9effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55c5d0 0x556f8c566d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38628/0 (socket says 172.21.15.103:38628) 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6b9effd640 1 -- 172.21.15.103:0/384998500 learned_addr learned my addr 172.21.15.103:0/384998500 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c564610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.321+0000 7f6b97fff640 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 0x556f8c564020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b9effd640 1 -- 172.21.15.103:0/384998500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 msgr2=0x556f8c564020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b9effd640 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 0x556f8c564020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b9effd640 1 -- 172.21.15.103:0/384998500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 msgr2=0x556f8c564610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b9effd640 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c564610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b9effd640 1 -- 172.21.15.103:0/384998500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8c2333c0 con 0x556f8c55c5d0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b9effd640 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55c5d0 0x556f8c566d70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b8e10 tx=0x7f6b900bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a265a586f5186025 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/384998500 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b900d5060 con 0x556f8c55c5d0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/384998500 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f6b900d5200 con 0x556f8c55c5d0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.322+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b98004d20 con 0x556f8c55c5d0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.323+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/384998500 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6b90049040 con 0x556f8c55c5d0 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.323+0000 7f6ba7af0740 10 client.27082 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:38.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.323+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f8c56a110 con 0x556f8c55c5d0 2024-09-19T07:49:38.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/384998500 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98004d20 con 0x556f8c55c5d0 2024-09-19T07:49:38.329 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/384998500 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f6b900c4c20 con 0x556f8c55c5d0 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6ba7af0740 10 client.27082 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6ba7af0740 10 client.27082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6ba7af0740 20 client.27082 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c577190 0x556f8c579580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6ba7af0740 4 client.27082 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f8c4cf860 con 0x556f8c577190 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.324+0000 7f6b97fff640 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c577190 0x556f8c579580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6b8e7fc640 10 client.27082 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6b97fff640 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c577190 0x556f8c579580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6083c6bb54cace4c server_cookie=678e0b0f1eb991cd in_seq=0 out_seq=0 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/384998500 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f8c4cf860 con 0x556f8c577190 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6b8e7fc640 10 client.27082 handle_command_reply: tid=0 2024-09-19T07:49:38.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 client.27082 shutdown 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55c5d0 msgr2=0x556f8c566d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55c5d0 0x556f8c566d70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b8e10 tx=0x7f6b900bc040 comp rx=0 tx=0).stop 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 shutdown_connections 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c577190 0x556f8c579580 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c564610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55db40 0x556f8c564020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/384998500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55c5d0 0x556f8c566d70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 >> 172.21.15.103:0/384998500 conn(0x556f8c525310 msgr2=0x556f8c55e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6b8e7fc640 0 client.27082 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.325+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 shutdown_connections 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.326+0000 7f6ba7af0740 1 -- 172.21.15.103:0/384998500 wait complete. 2024-09-19T07:49:38.331 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.326+0000 7f6ba7af0740 20 client.27082 trim_cache size 0 max 16384 2024-09-19T07:49:38.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.345+0000 7f6ba7af0740 1 Processor -- start 2024-09-19T07:49:38.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.345+0000 7f6ba7af0740 1 -- start start 2024-09-19T07:49:38.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.345+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c558d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.345+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c573410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.345+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c55e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01b3d50 con 0x556f8c567370 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01b5890 con 0x556f8c55c5d0 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01b4420 con 0x556f8c55db40 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c55e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c55e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55474/0 (socket says 172.21.15.103:55474) 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c558d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9f7fe640 1 -- 172.21.15.103:0/222205654 learned_addr learned my addr 172.21.15.103:0/222205654 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b97fff640 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c573410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9effd640 1 -- 172.21.15.103:0/222205654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 msgr2=0x556f8c573410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9effd640 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c573410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9effd640 1 -- 172.21.15.103:0/222205654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 msgr2=0x556f8c55e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9effd640 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c55e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9effd640 1 -- 172.21.15.103:0/222205654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8c5584f0 con 0x556f8c55c5d0 2024-09-19T07:49:38.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c55e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:38.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.346+0000 7f6b9effd640 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c558d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b900c4b80 tx=0x7f6b900bc0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d92f33ba6f20d2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.349+0000 7f6b94ff9640 1 -- 172.21.15.103:0/222205654 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b900d5040 con 0x556f8c55c5d0 2024-09-19T07:49:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.349+0000 7f6b94ff9640 1 -- 172.21.15.103:0/222205654 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b900bcb40 con 0x556f8c55c5d0 2024-09-19T07:49:38.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.349+0000 7f6b94ff9640 1 -- 172.21.15.103:0/222205654 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9009f020 con 0x556f8c55c5d0 2024-09-19T07:49:38.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 -- 172.21.15.103:0/222205654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 msgr2=0x556f8c558d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c558d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b900c4b80 tx=0x7f6b900bc0b0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 -- 172.21.15.103:0/222205654 shutdown_connections 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c55e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c573410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/222205654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c558d70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 -- 172.21.15.103:0/222205654 >> 172.21.15.103:0/222205654 conn(0x556f8c525310 msgr2=0x556f8c55e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 -- 172.21.15.103:0/222205654 shutdown_connections 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.350+0000 7f6ba7af0740 1 -- 172.21.15.103:0/222205654 wait complete. 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 -- start start 2024-09-19T07:49:38.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 Processor -- start 2024-09-19T07:49:38.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c56fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c560bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c561110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01b51c0 con 0x556f8c567370 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01b4af0 con 0x556f8c55c5d0 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01b1d50 con 0x556f8c55db40 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6b9effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c56fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.351+0000 7f6ba7af0740 10 client.0 _ll_register_callbacks cb 0x7f6b900dc040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c56fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40764/0 (socket says 172.21.15.103:40764) 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 -- 172.21.15.103:0/1340113963 learned_addr learned my addr 172.21.15.103:0/1340113963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b97fff640 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c560bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9f7fe640 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c561110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 -- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 msgr2=0x556f8c560bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c560bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 -- 172.21.15.103:0/1340113963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 msgr2=0x556f8c561110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c561110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 -- 172.21.15.103:0/1340113963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8c561780 con 0x556f8c55c5d0 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b9effd640 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c56fb40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b8e10 tx=0x7f6b900bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5e33571183bf4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.352+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/1340113963 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9009f050 con 0x556f8c55c5d0 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/1340113963 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f6b900d53e0 con 0x556f8c55c5d0 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b900bcb40 con 0x556f8c55c5d0 2024-09-19T07:49:38.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/1340113963 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6b900d5040 con 0x556f8c55c5d0 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6ba7af0740 10 client.27128 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f8c561780 con 0x556f8c55c5d0 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/1340113963 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9003b740 con 0x556f8c55c5d0 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/1340113963 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f6b900bb890 con 0x556f8c55c5d0 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6ba7af0740 10 client.27128 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6ba7af0740 10 client.27128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.353+0000 7f6ba7af0740 20 client.27128 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.354+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c5e1eb0 0x556f8c5e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.354+0000 7f6ba7af0740 4 client.27128 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.354+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6b900bcb40 con 0x556f8c5e1eb0 2024-09-19T07:49:38.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.354+0000 7f6b97fff640 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c5e1eb0 0x556f8c5e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.354+0000 7f6b8e7fc640 10 client.27128 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:49:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.354+0000 7f6b97fff640 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c5e1eb0 0x556f8c5e42d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b0b004bd2560be5 server_cookie=7e98c509c4fedcfe in_seq=0 out_seq=0 2024-09-19T07:49:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6b8e7fc640 1 -- 172.21.15.103:0/1340113963 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6b900bcb40 con 0x556f8c5e1eb0 2024-09-19T07:49:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6b8e7fc640 10 client.27128 handle_command_reply: tid=0 2024-09-19T07:49:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 client.27128 shutdown 2024-09-19T07:49:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 msgr2=0x556f8c56fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c56fb40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b8e10 tx=0x7f6b900bc040 comp rx=0 tx=0).stop 2024-09-19T07:49:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 shutdown_connections 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x556f8c5e1eb0 0x556f8c5e42d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556f8c567370 0x556f8c561110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f8c55db40 0x556f8c560bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 --2- 172.21.15.103:0/1340113963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f8c55c5d0 0x556f8c56fb40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 >> 172.21.15.103:0/1340113963 conn(0x556f8c525310 msgr2=0x556f8c557d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6b8e7fc640 0 client.27128 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 shutdown_connections 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 1 -- 172.21.15.103:0/1340113963 wait complete. 2024-09-19T07:49:38.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.355+0000 7f6ba7af0740 20 client.27128 trim_cache size 0 max 16384 2024-09-19T07:49:38.360 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:49:38.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b800a3390 msgr2=0x7f6b800a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b800a3390 0x7f6b800a5850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b88005d00 tx=0x7f6b88005c70 comp rx=0 tx=0).stop 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 msgr2=0x7f6ba01a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba01a4810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6b9002a940 tx=0x7f6b90003b90 comp rx=0 tx=0).stop 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 shutdown_connections 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f6b800a3390 0x7f6b800a5850 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ba0105ec0 0x7f6ba019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ba01011e0 0x7f6ba01a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 --2- 172.21.15.103:0/4142840151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba01008c0 0x7f6ba01a4810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 >> 172.21.15.103:0/4142840151 conn(0x7f6ba00fc7d0 msgr2=0x7f6ba010f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:38.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 shutdown_connections 2024-09-19T07:49:38.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:38.356+0000 7f6ba5717640 1 -- 172.21.15.103:0/4142840151 wait complete. 2024-09-19T07:49:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:38 smithi103 ceph-mon[31201]: pgmap v362: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 776 op/s 2024-09-19T07:49:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:38 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:49:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:38 smithi174 ceph-mon[35370]: pgmap v362: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 776 op/s 2024-09-19T07:49:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:38 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:49:38.833 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:38 smithi134 ceph-mon[35414]: pgmap v362: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.3 MiB/s wr, 776 op/s 2024-09-19T07:49:38.833 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:38 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:49:39.327 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:49:39.685 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:40.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.260+0000 7fa8631c3640 1 -- 172.21.15.103:0/3039488924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c102570 msgr2=0x7fa85c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.260+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3039488924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c102570 0x7fa85c1029b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009900 tx=0x7fa84c02a480 comp rx=0 tx=0).stop 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.260+0000 7fa8631c3640 1 -- 172.21.15.103:0/3039488924 shutdown_connections 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.260+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3039488924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa85c102ef0 0x7fa85c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.260+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3039488924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c102570 0x7fa85c1029b0 secure :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009900 tx=0x7fa84c02a480 comp rx=0 tx=0).stop 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.260+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3039488924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c100e20 0x7fa85c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.260+0000 7fa8631c3640 1 -- 172.21.15.103:0/3039488924 >> 172.21.15.103:0/3039488924 conn(0x7fa85c0fc7d0 msgr2=0x7fa85c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.261+0000 7fa8631c3640 1 -- 172.21.15.103:0/3039488924 shutdown_connections 2024-09-19T07:49:40.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.261+0000 7fa8631c3640 1 -- 172.21.15.103:0/3039488924 wait complete. 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.261+0000 7fa8631c3640 1 Processor -- start 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.261+0000 7fa8631c3640 1 -- start start 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa8631c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa85c100e20 0x7fa85c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa8631c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c102ef0 0x7fa85c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa8631c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c1a02b0 0x7fa85c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa8631c3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c10ff30 con 0x7fa85c1a02b0 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa8631c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c06ae40 con 0x7fa85c100e20 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa8631c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1100b0 con 0x7fa85c102ef0 2024-09-19T07:49:40.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c1a02b0 0x7fa85c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa860f5f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa85c100e20 0x7fa85c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa853fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c102ef0 0x7fa85c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c1a02b0 0x7fa85c1a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55502/0 (socket says 172.21.15.103:55502) 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 -- 172.21.15.103:0/3406048779 learned_addr learned my addr 172.21.15.103:0/3406048779 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 -- 172.21.15.103:0/3406048779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c102ef0 msgr2=0x7fa85c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c102ef0 0x7fa85c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 -- 172.21.15.103:0/3406048779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa85c100e20 msgr2=0x7fa85c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa85c100e20 0x7fa85c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 -- 172.21.15.103:0/3406048779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c1a0d70 con 0x7fa85c1a02b0 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.262+0000 7fa861760640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c1a02b0 0x7fa85c1a0760 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa85400efd0 tx=0x7fa85400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.263+0000 7fa851ffb640 1 -- 172.21.15.103:0/3406048779 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854003b70 con 0x7fa85c1a02b0 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.263+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c1a1060 con 0x7fa85c1a02b0 2024-09-19T07:49:40.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.263+0000 7fa851ffb640 1 -- 172.21.15.103:0/3406048779 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa854003d10 con 0x7fa85c1a02b0 2024-09-19T07:49:40.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.263+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c1a76a0 con 0x7fa85c1a02b0 2024-09-19T07:49:40.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.263+0000 7fa851ffb640 1 -- 172.21.15.103:0/3406048779 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8540108d0 con 0x7fa85c1a02b0 2024-09-19T07:49:40.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.268+0000 7fa851ffb640 1 -- 172.21.15.103:0/3406048779 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa854004020 con 0x7fa85c1a02b0 2024-09-19T07:49:40.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.268+0000 7fa86559c740 1 Processor -- start 2024-09-19T07:49:40.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.268+0000 7fa86559c740 1 -- start start 2024-09-19T07:49:40.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.268+0000 7fa86559c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb928268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa86559c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92826de0 0x55cb92827220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa86559c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92831b80 0x55cb92833f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa86559c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb927842f0 con 0x55cb92828350 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa86559c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb9272bcf0 con 0x55cb92831b80 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa86559c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb927802b0 con 0x55cb92826de0 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa853fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb928268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa853fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb928268a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55514/0 (socket says 172.21.15.103:55514) 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa853fff640 1 -- 172.21.15.103:0/2539197443 learned_addr learned my addr 172.21.15.103:0/2539197443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92826de0 0x55cb92827220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa853fff640 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92831b80 0x55cb92833f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa851ffb640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8440a3430 0x7fa8440a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa860f5f640 1 -- 172.21.15.103:0/2539197443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92831b80 msgr2=0x55cb92833f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92831b80 0x55cb92833f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa860f5f640 1 -- 172.21.15.103:0/2539197443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 msgr2=0x55cb928268a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb928268a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa860f5f640 1 -- 172.21.15.103:0/2539197443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb927103a0 con 0x55cb92826de0 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.269+0000 7fa851ffb640 1 -- 172.21.15.103:0/3406048779 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fa8540d00c0 con 0x7fa85c1a02b0 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.270+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92826de0 0x55cb92827220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa85800abf0 tx=0x7fa85800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c9131f26ea23ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.270+0000 7fa861760640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8440a3430 0x7fa8440a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.270+0000 7fa842ffd640 1 -- 172.21.15.103:0/2539197443 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858005220 con 0x55cb92826de0 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.270+0000 7fa842ffd640 1 -- 172.21.15.103:0/2539197443 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa858004c80 con 0x55cb92826de0 2024-09-19T07:49:40.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.270+0000 7fa842ffd640 1 -- 172.21.15.103:0/2539197443 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858018900 con 0x55cb92826de0 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 -- 172.21.15.103:0/2539197443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92826de0 msgr2=0x55cb92827220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa861760640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8440a3430 0x7fa8440a58f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8540987f0 tx=0x7fa85400cd50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92826de0 0x55cb92827220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa85800abf0 tx=0x7fa85800f790 comp rx=0 tx=0).stop 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 -- 172.21.15.103:0/2539197443 shutdown_connections 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92831b80 0x55cb92833f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92826de0 0x55cb92827220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 --2- 172.21.15.103:0/2539197443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb928268a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 -- 172.21.15.103:0/2539197443 >> 172.21.15.103:0/2539197443 conn(0x55cb927e7310 msgr2=0x55cb92823470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 -- 172.21.15.103:0/2539197443 shutdown_connections 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 -- 172.21.15.103:0/2539197443 wait complete. 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 -- start start 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.271+0000 7fa86559c740 1 Processor -- start 2024-09-19T07:49:40.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa86559c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb92831530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa86559c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92828350 0x55cb9282e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa86559c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92831b80 0x55cb9282ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa86559c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb92834880 con 0x55cb92831b80 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa86559c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb9273f060 con 0x55cb92826de0 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa86559c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb92834a00 con 0x55cb92828350 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa86559c740 10 client.0 _ll_register_callbacks cb 0x55cb924928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa853fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb92831530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa853fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb92831530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40796/0 (socket says 172.21.15.103:40796) 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa853fff640 1 -- 172.21.15.103:0/1124441495 learned_addr learned my addr 172.21.15.103:0/1124441495 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa860f5f640 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92828350 0x55cb9282e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa853fff640 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92831b80 0x55cb9282ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.272+0000 7fa853fff640 1 -- 172.21.15.103:0/1124441495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92828350 msgr2=0x55cb9282e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.273+0000 7fa853fff640 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92828350 0x55cb9282e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.273+0000 7fa853fff640 1 -- 172.21.15.103:0/1124441495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92831b80 msgr2=0x55cb9282ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.273+0000 7fa853fff640 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92831b80 0x55cb9282ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.273+0000 7fa853fff640 1 -- 172.21.15.103:0/1124441495 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb924f53c0 con 0x55cb92826de0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.274+0000 7fa853fff640 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb92831530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa85c101be0 tx=0x7fa84c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611a73a61ad50bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.274+0000 7fa83bfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.274+0000 7fa83bfff640 1 -- 172.21.15.103:0/1124441495 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c035040 con 0x55cb92826de0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.274+0000 7fa83bfff640 1 -- 172.21.15.103:0/1124441495 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fa84c004030 con 0x55cb92826de0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.274+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa858004c80 con 0x55cb92826de0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa83bfff640 1 -- 172.21.15.103:0/1124441495 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa84c004230 con 0x55cb92826de0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 10 client.27137 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cb928402d0 con 0x55cb92826de0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa83bfff640 1 -- 172.21.15.103:0/1124441495 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa858004c80 con 0x55cb92826de0 2024-09-19T07:49:40.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa83bfff640 1 -- 172.21.15.103:0/1124441495 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fa84c004ab0 con 0x55cb92826de0 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 10 client.27137 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 10 client.27137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 20 client.27137 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb92841790 0x55cb92843bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 4 client.27137 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb92791860 con 0x55cb92841790 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.275+0000 7fa860f5f640 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb92841790 0x55cb92843bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.276+0000 7fa860f5f640 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb92841790 0x55cb92843bc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac7c9ce7a0528f89 server_cookie=94722342d10bde2e in_seq=0 out_seq=0 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.276+0000 7fa83bfff640 10 client.27137 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa83bfff640 1 -- 172.21.15.103:0/1124441495 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb92791860 con 0x55cb92841790 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa83bfff640 10 client.27137 handle_command_reply: tid=0 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 client.27137 shutdown 2024-09-19T07:49:40.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 msgr2=0x55cb92831530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb92831530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa85c101be0 tx=0x7fa84c0058d0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 shutdown_connections 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb92841790 0x55cb92843bc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92831b80 0x55cb9282ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92828350 0x55cb9282e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 --2- 172.21.15.103:0/1124441495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb92831530 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 >> 172.21.15.103:0/1124441495 conn(0x55cb927e7310 msgr2=0x55cb92829170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa83bfff640 0 client.27137 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 shutdown_connections 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 1 -- 172.21.15.103:0/1124441495 wait complete. 2024-09-19T07:49:40.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.277+0000 7fa86559c740 20 client.27137 trim_cache size 0 max 16384 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.296+0000 7fa86559c740 1 Processor -- start 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.296+0000 7fa86559c740 1 -- start start 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa86559c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92826de0 0x55cb92830750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa86559c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92828350 0x55cb92829930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa86559c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb92829e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa86559c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1abb80 con 0x55cb92826de0 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa86559c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1ad6c0 con 0x55cb92828350 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa86559c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1ac250 con 0x55cb92831b80 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92826de0 0x55cb92830750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92826de0 0x55cb92830750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55526/0 (socket says 172.21.15.103:55526) 2024-09-19T07:49:40.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 -- 172.21.15.103:0/3452572179 learned_addr learned my addr 172.21.15.103:0/3452572179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:40.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb92829e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 -- 172.21.15.103:0/3452572179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92828350 msgr2=0x55cb92829930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92828350 0x55cb92829930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 -- 172.21.15.103:0/3452572179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92826de0 msgr2=0x55cb92830750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92826de0 0x55cb92830750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.297+0000 7fa853fff640 1 -- 172.21.15.103:0/3452572179 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb92827f90 con 0x55cb92831b80 2024-09-19T07:49:40.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.298+0000 7fa853fff640 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb92829e70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0317d0 tx=0x7fa84c005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b0ff65e1db1073c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.298+0000 7fa8427fc640 1 -- 172.21.15.103:0/3452572179 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c035070 con 0x55cb92831b80 2024-09-19T07:49:40.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.298+0000 7fa8427fc640 1 -- 172.21.15.103:0/3452572179 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84c004c30 con 0x55cb92831b80 2024-09-19T07:49:40.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.298+0000 7fa8427fc640 1 -- 172.21.15.103:0/3452572179 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c005230 con 0x55cb92831b80 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 -- 172.21.15.103:0/3452572179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 msgr2=0x55cb92829e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb92829e70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0317d0 tx=0x7fa84c005e60 comp rx=0 tx=0).stop 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 -- 172.21.15.103:0/3452572179 shutdown_connections 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb92829e70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92828350 0x55cb92829930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 --2- 172.21.15.103:0/3452572179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92826de0 0x55cb92830750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 -- 172.21.15.103:0/3452572179 >> 172.21.15.103:0/3452572179 conn(0x55cb927e7310 msgr2=0x55cb92830040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 -- 172.21.15.103:0/3452572179 shutdown_connections 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 -- 172.21.15.103:0/3452572179 wait complete. 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.299+0000 7fa86559c740 1 -- start start 2024-09-19T07:49:40.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.300+0000 7fa86559c740 1 Processor -- start 2024-09-19T07:49:40.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa86559c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb9282b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa86559c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb9282bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa853fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb9282b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa853fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb9282b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40812/0 (socket says 172.21.15.103:40812) 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa853fff640 1 -- 172.21.15.103:0/2965295037 learned_addr learned my addr 172.21.15.103:0/2965295037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa86559c740 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb9282c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1acff0 con 0x55cb92828350 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1ac920 con 0x55cb92826de0 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c1a9b80 con 0x55cb92831b80 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb9282bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.301+0000 7fa86559c740 10 client.0 _ll_register_callbacks cb 0x7fa84c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa860f5f640 1 -- 172.21.15.103:0/2965295037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 msgr2=0x55cb9282c1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:49:40.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb9282c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa860f5f640 1 -- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 msgr2=0x55cb9282b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb9282b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa860f5f640 1 -- 172.21.15.103:0/2965295037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb928ab4b0 con 0x55cb92828350 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa853fff640 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb9282b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa860f5f640 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb9282bc80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa85800fc90 tx=0x7fa85800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b444a1f33ab930fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.302+0000 7fa83bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.303+0000 7fa83bfff640 1 -- 172.21.15.103:0/2965295037 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858033060 con 0x55cb92828350 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.303+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa84c004c30 con 0x55cb92828350 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.303+0000 7fa83bfff640 1 -- 172.21.15.103:0/2965295037 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fa85801e040 con 0x55cb92828350 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.303+0000 7fa83bfff640 1 -- 172.21.15.103:0/2965295037 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa85800a7e0 con 0x55cb92828350 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.303+0000 7fa86559c740 10 client.17802 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.303+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cb928ac690 con 0x55cb92828350 2024-09-19T07:49:40.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.303+0000 7fa83bfff640 1 -- 172.21.15.103:0/2965295037 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa84c004c30 con 0x55cb92828350 2024-09-19T07:49:40.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.304+0000 7fa83bfff640 1 -- 172.21.15.103:0/2965295037 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fa858018620 con 0x55cb92828350 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.304+0000 7fa86559c740 10 client.17802 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.304+0000 7fa86559c740 10 client.17802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.304+0000 7fa86559c740 20 client.17802 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.304+0000 7fa86559c740 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb928acbe0 0x55cb928af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.304+0000 7fa86559c740 4 client.17802 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.304+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb9278bd70 con 0x55cb928acbe0 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.305+0000 7fa853fff640 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb928acbe0 0x55cb928af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.305+0000 7fa853fff640 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb928acbe0 0x55cb928af060 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9f1c7f3b9391894 server_cookie=5c0362d8d49102b5 in_seq=0 out_seq=0 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa83bfff640 10 client.17802 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa83bfff640 1 -- 172.21.15.103:0/2965295037 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb9278bd70 con 0x55cb928acbe0 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa83bfff640 10 client.17802 handle_command_reply: tid=0 2024-09-19T07:49:40.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa86559c740 1 client.17802 shutdown 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 msgr2=0x55cb9282bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa86559c740 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb9282bc80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa85800fc90 tx=0x7fa85800a6a0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 shutdown_connections 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa86559c740 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55cb928acbe0 0x55cb928af060 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa86559c740 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb92831b80 0x55cb9282c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.306+0000 7fa83bfff640 0 client.17802 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.307+0000 7fa86559c740 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb92828350 0x55cb9282bc80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.307+0000 7fa86559c740 1 --2- 172.21.15.103:0/2965295037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb92826de0 0x55cb9282b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.307+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 >> 172.21.15.103:0/2965295037 conn(0x55cb927e7310 msgr2=0x55cb92830860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.307+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 shutdown_connections 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.307+0000 7fa86559c740 1 -- 172.21.15.103:0/2965295037 wait complete. 2024-09-19T07:49:40.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.307+0000 7fa86559c740 20 client.17802 trim_cache size 0 max 16384 2024-09-19T07:49:40.309 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8440a3430 msgr2=0x7fa8440a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8440a3430 0x7fa8440a58f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8540987f0 tx=0x7fa85400cd50 comp rx=0 tx=0).stop 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c1a02b0 msgr2=0x7fa85c1a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c1a02b0 0x7fa85c1a0760 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa85400efd0 tx=0x7fa85400c6a0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 shutdown_connections 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa8440a3430 0x7fa8440a58f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85c1a02b0 0x7fa85c1a0760 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c102ef0 0x7fa85c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 --2- 172.21.15.103:0/3406048779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa85c100e20 0x7fa85c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 >> 172.21.15.103:0/3406048779 conn(0x7fa85c0fc7d0 msgr2=0x7fa85c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.308+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 shutdown_connections 2024-09-19T07:49:40.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:40.309+0000 7fa8631c3640 1 -- 172.21.15.103:0/3406048779 wait complete. 2024-09-19T07:49:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 665.0 s, 2529.2 tps, lat 39.563 ms stddev 44.775 2024-09-19T07:49:40.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:40 smithi103 ceph-mon[31201]: pgmap v363: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 590 op/s 2024-09-19T07:49:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:40 smithi174 ceph-mon[35370]: pgmap v363: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 590 op/s 2024-09-19T07:49:40.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:40 smithi134 ceph-mon[35414]: pgmap v363: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 590 op/s 2024-09-19T07:49:41.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:49:41.530 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 -- 172.21.15.103:0/3372095018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768106a80 msgr2=0x7f8768106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 --2- 172.21.15.103:0/3372095018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768106a80 0x7f8768106e60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8764009930 tx=0x7f876402a420 comp rx=0 tx=0).stop 2024-09-19T07:49:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 -- 172.21.15.103:0/3372095018 shutdown_connections 2024-09-19T07:49:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 --2- 172.21.15.103:0/3372095018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8768101420 0x7f876810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 --2- 172.21.15.103:0/3372095018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768100a80 0x7f8768100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 --2- 172.21.15.103:0/3372095018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768106a80 0x7f8768106e60 secure :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8764009930 tx=0x7f876402a420 comp rx=0 tx=0).stop 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 -- 172.21.15.103:0/3372095018 >> 172.21.15.103:0/3372095018 conn(0x7f87680fc7a0 msgr2=0x7f87680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 -- 172.21.15.103:0/3372095018 shutdown_connections 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.042+0000 7f876fcc3640 1 -- 172.21.15.103:0/3372095018 wait complete. 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 Processor -- start 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 -- start start 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768100a80 0x7f8768109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101420 0x7f876810a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f876810a6e0 0x7f87681aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f876810fdc0 con 0x7f8768100a80 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f876810fc40 con 0x7f8768101420 2024-09-19T07:49:42.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876fcc3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f876810ff40 con 0x7f876810a6e0 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876da5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768100a80 0x7f8768109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876d25e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101420 0x7f876810a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876da5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768100a80 0x7f8768109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55550/0 (socket says 172.21.15.103:55550) 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876da5f640 1 -- 172.21.15.103:0/459725873 learned_addr learned my addr 172.21.15.103:0/459725873 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.043+0000 7f876e260640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f876810a6e0 0x7f87681aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876da5f640 1 -- 172.21.15.103:0/459725873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f876810a6e0 msgr2=0x7f87681aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876da5f640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f876810a6e0 0x7f87681aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876da5f640 1 -- 172.21.15.103:0/459725873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101420 msgr2=0x7f876810a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876da5f640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101420 0x7f876810a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876da5f640 1 -- 172.21.15.103:0/459725873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87681ab290 con 0x7f8768100a80 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876d25e640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101420 0x7f876810a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:42.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876da5f640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768100a80 0x7f8768109c60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f8764003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f875effd640 1 -- 172.21.15.103:0/459725873 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876403f020 con 0x7f8768100a80 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87681ab520 con 0x7f8768100a80 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f875effd640 1 -- 172.21.15.103:0/459725873 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8764031c90 con 0x7f8768100a80 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f875effd640 1 -- 172.21.15.103:0/459725873 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764035070 con 0x7f8768100a80 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.044+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87681ab9b0 con 0x7f8768100a80 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.047+0000 7f877209c740 1 Processor -- start 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.047+0000 7f877209c740 1 -- start start 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f877209c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f877209c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbefa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f877209c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f877209c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5625abb552f0 con 0x5625abbfa370 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f877209c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625abafccf0 con 0x5625abbf0b40 2024-09-19T07:49:42.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f877209c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625abb512b0 con 0x5625abbef5d0 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876d25e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876d25e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55556/0 (socket says 172.21.15.103:55556) 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876e260640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876d25e640 1 -- 172.21.15.103:0/2508918171 learned_addr learned my addr 172.21.15.103:0/2508918171 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f875effd640 1 -- 172.21.15.103:0/459725873 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8764042020 con 0x7f8768100a80 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876e260640 1 -- 172.21.15.103:0/2508918171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 msgr2=0x5625abbefa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876e260640 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbefa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876e260640 1 -- 172.21.15.103:0/2508918171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 msgr2=0x5625abbfc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876e260640 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876e260640 1 -- 172.21.15.103:0/2508918171 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625abae13a0 con 0x5625abbf0b40 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.048+0000 7f876d25e640 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:42.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.049+0000 7f876e260640 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbef090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8760009eb0 tx=0x7f87600107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bea048b91f9b53a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.049+0000 7f8753fff640 1 -- 172.21.15.103:0/2508918171 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760003b70 con 0x5625abbf0b40 2024-09-19T07:49:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.049+0000 7f8753fff640 1 -- 172.21.15.103:0/2508918171 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8760015070 con 0x5625abbf0b40 2024-09-19T07:49:42.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.049+0000 7f8753fff640 1 -- 172.21.15.103:0/2508918171 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876001a940 con 0x5625abbf0b40 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.049+0000 7f875effd640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f87540a33b0 0x7f87540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.049+0000 7f876da5f640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f87540a33b0 0x7f87540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.049+0000 7f875effd640 1 -- 172.21.15.103:0/459725873 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f87640ef900 con 0x7f8768100a80 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.050+0000 7f877209c740 1 -- 172.21.15.103:0/2508918171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 msgr2=0x5625abbef090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.050+0000 7f877209c740 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbef090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8760009eb0 tx=0x7f87600107a0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.051+0000 7f876da5f640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f87540a33b0 0x7f87540a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5625abbecac0 tx=0x7f87640b7920 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.051+0000 7f877209c740 1 -- 172.21.15.103:0/2508918171 shutdown_connections 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.051+0000 7f877209c740 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.051+0000 7f877209c740 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbefa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.051+0000 7f877209c740 1 --2- 172.21.15.103:0/2508918171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbef090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.051+0000 7f877209c740 1 -- 172.21.15.103:0/2508918171 >> 172.21.15.103:0/2508918171 conn(0x5625abbb8310 msgr2=0x5625abbebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.052+0000 7f877209c740 1 -- 172.21.15.103:0/2508918171 shutdown_connections 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.052+0000 7f877209c740 1 -- 172.21.15.103:0/2508918171 wait complete. 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.052+0000 7f877209c740 1 -- start start 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.052+0000 7f877209c740 1 Processor -- start 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.053+0000 7f877209c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbf9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.053+0000 7f877209c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbf7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.053+0000 7f877209c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.054+0000 7f876e260640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.054+0000 7f876e260640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55564/0 (socket says 172.21.15.103:55564) 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.055+0000 7f876e260640 1 -- 172.21.15.103:0/4287285238 learned_addr learned my addr 172.21.15.103:0/4287285238 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.055+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5625abbfd070 con 0x5625abbfa370 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.055+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625abb10060 con 0x5625abbf0b40 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.055+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625abbfd1f0 con 0x5625abbef5d0 2024-09-19T07:49:42.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.055+0000 7f876e260640 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbf9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.056+0000 7f877209c740 10 client.0 _ll_register_callbacks cb 0x5625ab8638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.057+0000 7f876e260640 1 -- 172.21.15.103:0/4287285238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 msgr2=0x5625abbf7070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.057+0000 7f876e260640 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbf7070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.057+0000 7f876e260640 1 -- 172.21.15.103:0/4287285238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 msgr2=0x5625abbf7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.057+0000 7f876e260640 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.057+0000 7f876e260640 1 -- 172.21.15.103:0/4287285238 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ab8c63c0 con 0x5625abbef5d0 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.064+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8760015070 con 0x5625abbef5d0 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.064+0000 7f876e260640 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbf9dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f876001a5e0 tx=0x7f8760005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9f23583f174397 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.064+0000 7f87517fa640 10 client.27109 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:49:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.065+0000 7f87517fa640 1 -- 172.21.15.103:0/4287285238 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760023040 con 0x5625abbef5d0 2024-09-19T07:49:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.065+0000 7f87517fa640 1 -- 172.21.15.103:0/4287285238 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f8760004200 con 0x5625abbef5d0 2024-09-19T07:49:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.065+0000 7f87517fa640 1 -- 172.21.15.103:0/4287285238 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8760004460 con 0x5625abbef5d0 2024-09-19T07:49:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.066+0000 7f877209c740 10 client.27109 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.066+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=47}) -- 0x5625abc08a30 con 0x5625abbef5d0 2024-09-19T07:49:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.070+0000 7f87517fa640 1 -- 172.21.15.103:0/4287285238 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f876001a680 con 0x5625abbef5d0 2024-09-19T07:49:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.070+0000 7f877209c740 10 client.27109 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.070+0000 7f877209c740 10 client.27109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:49:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.070+0000 7f877209c740 20 client.27109 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.070+0000 7f877209c740 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc09d80 0x5625abc0c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.070+0000 7f877209c740 4 client.27109 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.070+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8760015070 con 0x5625abc09d80 2024-09-19T07:49:42.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.071+0000 7f876d25e640 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc09d80 0x5625abc0c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.073+0000 7f87517fa640 10 client.27109 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:49:42.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.074+0000 7f876d25e640 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc09d80 0x5625abc0c170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be14f3faaaeac7b9 server_cookie=e2970873c8ff4672 in_seq=0 out_seq=0 2024-09-19T07:49:42.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.075+0000 7f87517fa640 1 -- 172.21.15.103:0/4287285238 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8760015070 con 0x5625abc09d80 2024-09-19T07:49:42.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.075+0000 7f87517fa640 10 client.27109 handle_command_reply: tid=0 2024-09-19T07:49:42.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.075+0000 7f877209c740 1 client.27109 shutdown 2024-09-19T07:49:42.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 msgr2=0x5625abbf9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbf9dc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f876001a5e0 tx=0x7f8760005fd0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 shutdown_connections 2024-09-19T07:49:42.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc09d80 0x5625abc0c170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbf7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 --2- 172.21.15.103:0/4287285238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abbf9dc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.076+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 >> 172.21.15.103:0/4287285238 conn(0x5625abbb8310 msgr2=0x5625abbf1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:42.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.077+0000 7f87517fa640 0 client.27109 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:49:42.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.077+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 shutdown_connections 2024-09-19T07:49:42.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.077+0000 7f877209c740 1 -- 172.21.15.103:0/4287285238 wait complete. 2024-09-19T07:49:42.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.077+0000 7f877209c740 20 client.27109 trim_cache size 0 max 16384 2024-09-19T07:49:42.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 Processor -- start 2024-09-19T07:49:42.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 -- start start 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbef5d0 0x5625abbf7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbf0b40 0x5625abbf2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87681b4510 con 0x5625abbfa370 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87681b6050 con 0x5625abbef5d0 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f877209c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87681b4be0 con 0x5625abbf0b40 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55570/0 (socket says 172.21.15.103:55570) 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 -- 172.21.15.103:0/2865137790 learned_addr learned my addr 172.21.15.103:0/2865137790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbef5d0 0x5625abbf7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876d25e640 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbf0b40 0x5625abbf2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 -- 172.21.15.103:0/2865137790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbf0b40 msgr2=0x5625abbf2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbf0b40 0x5625abbf2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 -- 172.21.15.103:0/2865137790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbef5d0 msgr2=0x5625abbf7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbef5d0 0x5625abbf7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876e260640 1 -- 172.21.15.103:0/2865137790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625abbeb4f0 con 0x5625abbfa370 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.097+0000 7f876d25e640 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbf0b40 0x5625abbf2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.098+0000 7f876e260640 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf2660 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8760009a80 tx=0x7f8760013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3bb7c2424d6e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.098+0000 7f87537fe640 1 -- 172.21.15.103:0/2865137790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876003c040 con 0x5625abbfa370 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.098+0000 7f87537fe640 1 -- 172.21.15.103:0/2865137790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f876003d040 con 0x5625abbfa370 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.098+0000 7f87537fe640 1 -- 172.21.15.103:0/2865137790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760004000 con 0x5625abbfa370 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.098+0000 7f877209c740 1 -- 172.21.15.103:0/2865137790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 msgr2=0x5625abbf2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.098+0000 7f877209c740 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf2660 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8760009a80 tx=0x7f8760013070 comp rx=0 tx=0).stop 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 -- 172.21.15.103:0/2865137790 shutdown_connections 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbf2660 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbf0b40 0x5625abbf2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 --2- 172.21.15.103:0/2865137790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbef5d0 0x5625abbf7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 -- 172.21.15.103:0/2865137790 >> 172.21.15.103:0/2865137790 conn(0x5625abbb8310 msgr2=0x5625abbf6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 -- 172.21.15.103:0/2865137790 shutdown_connections 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 -- 172.21.15.103:0/2865137790 wait complete. 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 -- start start 2024-09-19T07:49:42.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.099+0000 7f877209c740 1 Processor -- start 2024-09-19T07:49:42.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.100+0000 7f877209c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abc02890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.100+0000 7f877209c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbfdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.100+0000 7f877209c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.100+0000 7f877209c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87681b5980 con 0x5625abbfa370 2024-09-19T07:49:42.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.100+0000 7f877209c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87681b52b0 con 0x5625abbf0b40 2024-09-19T07:49:42.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.100+0000 7f877209c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87681b2510 con 0x5625abbef5d0 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.100+0000 7f877209c740 10 client.0 _ll_register_callbacks cb 0x7f875800a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876d25e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbfdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876e260640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abc02890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876e260640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abc02890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38770/0 (socket says 172.21.15.103:38770) 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876e260640 1 -- 172.21.15.103:0/2559798969 learned_addr learned my addr 172.21.15.103:0/2559798969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876d25e640 1 -- 172.21.15.103:0/2559798969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 msgr2=0x5625abc02890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876d25e640 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abc02890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876d25e640 1 -- 172.21.15.103:0/2559798969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 msgr2=0x5625abbfe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876d25e640 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.101+0000 7f876d25e640 1 -- 172.21.15.103:0/2559798969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625abbfe690 con 0x5625abbf0b40 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.102+0000 7f87517fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:49:42.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.102+0000 7f876d25e640 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbfdc10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5625abc0c7d0 tx=0x7f8758010af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dbe887518e8f939 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.102+0000 7f87517fa640 1 -- 172.21.15.103:0/2559798969 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758021030 con 0x5625abbf0b40 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.102+0000 7f87517fa640 1 -- 172.21.15.103:0/2559798969 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f875800f910 con 0x5625abbf0b40 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.102+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f876003d040 con 0x5625abbf0b40 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.102+0000 7f87517fa640 1 -- 172.21.15.103:0/2559798969 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f875800fb10 con 0x5625abbf0b40 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.102+0000 7f877209c740 10 client.27149 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5625abc744a0 con 0x5625abbf0b40 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f87517fa640 1 -- 172.21.15.103:0/2559798969 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875800fcb0 con 0x5625abbf0b40 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f87517fa640 1 -- 172.21.15.103:0/2559798969 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f8758026670 con 0x5625abbf0b40 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f877209c740 10 client.27149 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f877209c740 10 client.27149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f877209c740 20 client.27149 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:42.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f877209c740 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc74c60 0x5625abc77050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f877209c740 4 client.27149 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f876003d040 con 0x5625abc74c60 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.103+0000 7f876e260640 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc74c60 0x5625abc77050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.104+0000 7f87517fa640 10 client.27149 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.104+0000 7f876e260640 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc74c60 0x5625abc77050 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=272f18f3ed62be06 server_cookie=256ce6d50b9a6242 in_seq=0 out_seq=0 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.104+0000 7f87517fa640 1 -- 172.21.15.103:0/2559798969 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f876003d040 con 0x5625abc74c60 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.104+0000 7f87517fa640 10 client.27149 handle_command_reply: tid=0 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 client.27149 shutdown 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 msgr2=0x5625abbfdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbfdc10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5625abc0c7d0 tx=0x7f8758010af0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 shutdown_connections 2024-09-19T07:49:42.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5625abc74c60 0x5625abc77050 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f87517fa640 0 client.27149 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:49:42.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625abbfa370 0x5625abbfe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625abbf0b40 0x5625abbfdc10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 --2- 172.21.15.103:0/2559798969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625abbef5d0 0x5625abc02890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 >> 172.21.15.103:0/2559798969 conn(0x5625abbb8310 msgr2=0x5625abbea4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:42.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 shutdown_connections 2024-09-19T07:49:42.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 1 -- 172.21.15.103:0/2559798969 wait complete. 2024-09-19T07:49:42.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.105+0000 7f877209c740 20 client.27149 trim_cache size 0 max 16384 2024-09-19T07:49:42.115 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f87540a33b0 msgr2=0x7f87540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f87540a33b0 0x7f87540a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5625abbecac0 tx=0x7f87640b7920 comp rx=0 tx=0).stop 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768100a80 msgr2=0x7f8768109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768100a80 0x7f8768109c60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f8764003c10 comp rx=0 tx=0).stop 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 shutdown_connections 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f87540a33b0 0x7f87540a5870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f876810a6e0 0x7f87681aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8768101420 0x7f876810a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 --2- 172.21.15.103:0/459725873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8768100a80 0x7f8768109c60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:42.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 >> 172.21.15.103:0/459725873 conn(0x7f87680fc7a0 msgr2=0x7f87680fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:42.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 shutdown_connections 2024-09-19T07:49:42.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:42.106+0000 7f876fcc3640 1 -- 172.21.15.103:0/459725873 wait complete. 2024-09-19T07:49:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:42 smithi174 ceph-mon[35370]: pgmap v364: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 870 op/s 2024-09-19T07:49:42.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:42 smithi103 ceph-mon[31201]: pgmap v364: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 870 op/s 2024-09-19T07:49:42.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:42 smithi134 ceph-mon[35414]: pgmap v364: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 870 op/s 2024-09-19T07:49:42.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:49:43.476 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:44.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.009+0000 7f3061fc1640 1 -- 172.21.15.103:0/2533229920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c100980 msgr2=0x7f305c100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.009+0000 7f3061fc1640 1 --2- 172.21.15.103:0/2533229920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c100980 0x7f305c100d60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3058009860 tx=0x7f305802a400 comp rx=0 tx=0).stop 2024-09-19T07:49:44.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.013+0000 7f3061fc1640 1 -- 172.21.15.103:0/2533229920 shutdown_connections 2024-09-19T07:49:44.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.013+0000 7f3061fc1640 1 --2- 172.21.15.103:0/2533229920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c105d90 0x7f305c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.013+0000 7f3061fc1640 1 --2- 172.21.15.103:0/2533229920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1012a0 0x7f305c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.013+0000 7f3061fc1640 1 --2- 172.21.15.103:0/2533229920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c100980 0x7f305c100d60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.013+0000 7f3061fc1640 1 -- 172.21.15.103:0/2533229920 >> 172.21.15.103:0/2533229920 conn(0x7f305c0fc610 msgr2=0x7f305c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.015+0000 7f3061fc1640 1 -- 172.21.15.103:0/2533229920 shutdown_connections 2024-09-19T07:49:44.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.015+0000 7f3061fc1640 1 -- 172.21.15.103:0/2533229920 wait complete. 2024-09-19T07:49:44.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.015+0000 7f3061fc1640 1 Processor -- start 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.019+0000 7f3061fc1640 1 -- start start 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.019+0000 7f3061fc1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c100980 0x7f305c114c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.019+0000 7f3061fc1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c1012a0 0x7f305c10dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.019+0000 7f3061fc1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c105d90 0x7f305c10e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.019+0000 7f3061fc1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1183a0 con 0x7f305c1012a0 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.019+0000 7f3061fc1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f305c118220 con 0x7f305c100980 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.019+0000 7f3061fc1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f305c118520 con 0x7f305c105d90 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.020+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c100980 0x7f305c114c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.020+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c100980 0x7f305c114c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40894/0 (socket says 172.21.15.103:40894) 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.020+0000 7f3060fbf640 1 -- 172.21.15.103:0/1028386509 learned_addr learned my addr 172.21.15.103:0/1028386509 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:44.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.020+0000 7f3053fff640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c1012a0 0x7f305c10dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.021+0000 7f3053fff640 1 -- 172.21.15.103:0/1028386509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c105d90 msgr2=0x7f305c10e1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:49:44.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.021+0000 7f3053fff640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c105d90 0x7f305c10e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.021+0000 7f3053fff640 1 -- 172.21.15.103:0/1028386509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c100980 msgr2=0x7f305c114c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.021+0000 7f3053fff640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c100980 0x7f305c114c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.021+0000 7f3053fff640 1 -- 172.21.15.103:0/1028386509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c10e860 con 0x7f305c1012a0 2024-09-19T07:49:44.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.021+0000 7f3053fff640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c1012a0 0x7f305c10dcb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f304c00e6c0 tx=0x7f304c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.022+0000 7f3051ffb640 1 -- 172.21.15.103:0/1028386509 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c00cab0 con 0x7f305c1012a0 2024-09-19T07:49:44.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.022+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c112750 con 0x7f305c1012a0 2024-09-19T07:49:44.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.022+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c112c90 con 0x7f305c1012a0 2024-09-19T07:49:44.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.022+0000 7f3051ffb640 1 -- 172.21.15.103:0/1028386509 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304c00cc50 con 0x7f305c1012a0 2024-09-19T07:49:44.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.022+0000 7f3051ffb640 1 -- 172.21.15.103:0/1028386509 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c018840 con 0x7f305c1012a0 2024-09-19T07:49:44.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.026+0000 7f3051ffb640 1 -- 172.21.15.103:0/1028386509 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f304c0189e0 con 0x7f305c1012a0 2024-09-19T07:49:44.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.027+0000 7f30655fc740 1 Processor -- start 2024-09-19T07:49:44.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f30655fc740 1 -- start start 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f3051ffb640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f30440a35c0 0x7f30440a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f30655fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f30655fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f30655fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f30655fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cd0a2432f0 con 0x55cd0a2decf0 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f30655fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cd0a1eacf0 con 0x55cd0a2e8520 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f30655fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cd0a23f2b0 con 0x55cd0a2dd780 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.028+0000 7f3051ffb640 1 -- 172.21.15.103:0/1028386509 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f304c0d0e30 con 0x7f305c1012a0 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.029+0000 7f30617c0640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f30440a35c0 0x7f30440a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.029+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.029+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2dd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55596/0 (socket says 172.21.15.103:55596) 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.029+0000 7f3060fbf640 1 -- 172.21.15.103:0/1670863296 learned_addr learned my addr 172.21.15.103:0/1670863296 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:44.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.029+0000 7f30617c0640 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.029+0000 7f3053fff640 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.030+0000 7f3060fbf640 1 -- 172.21.15.103:0/1670863296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 msgr2=0x55cd0a2ddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.030+0000 7f3060fbf640 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2ddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.030+0000 7f3060fbf640 1 -- 172.21.15.103:0/1670863296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 msgr2=0x55cd0a2ea910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.030+0000 7f3060fbf640 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ea910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.030+0000 7f3060fbf640 1 -- 172.21.15.103:0/1670863296 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0a1cf3a0 con 0x55cd0a2decf0 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.030+0000 7f3060fbf640 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2dd240 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3058000c00 tx=0x7f3058030ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bb2c2be5b7bd3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.031+0000 7f30617c0640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f30440a35c0 0x7f30440a5a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55cd0a2de220 tx=0x7f30540095e0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.031+0000 7f3042ffd640 1 -- 172.21.15.103:0/1670863296 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058004020 con 0x55cd0a2decf0 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.031+0000 7f3042ffd640 1 -- 172.21.15.103:0/1670863296 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3058030880 con 0x55cd0a2decf0 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.031+0000 7f3042ffd640 1 -- 172.21.15.103:0/1670863296 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580046c0 con 0x55cd0a2decf0 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.032+0000 7f30655fc740 1 -- 172.21.15.103:0/1670863296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 msgr2=0x55cd0a2dd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.032+0000 7f30655fc740 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2dd240 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3058000c00 tx=0x7f3058030ba0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.032+0000 7f30655fc740 1 -- 172.21.15.103:0/1670863296 shutdown_connections 2024-09-19T07:49:44.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.032+0000 7f30655fc740 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ea910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.032+0000 7f30655fc740 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2ddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.032+0000 7f30655fc740 1 --2- 172.21.15.103:0/1670863296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2dd240 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.032+0000 7f30655fc740 1 -- 172.21.15.103:0/1670863296 >> 172.21.15.103:0/1670863296 conn(0x55cd0a2a6310 msgr2=0x55cd0a2d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.033+0000 7f30655fc740 1 -- 172.21.15.103:0/1670863296 shutdown_connections 2024-09-19T07:49:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.033+0000 7f30655fc740 1 -- 172.21.15.103:0/1670863296 wait complete. 2024-09-19T07:49:44.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.034+0000 7f30655fc740 1 -- start start 2024-09-19T07:49:44.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.034+0000 7f30655fc740 1 Processor -- start 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.034+0000 7f30655fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.034+0000 7f30655fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55608/0 (socket says 172.21.15.103:55608) 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f3060fbf640 1 -- 172.21.15.103:0/78909664 learned_addr learned my addr 172.21.15.103:0/78909664 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30655fc740 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cd0a2eb1c0 con 0x55cd0a2dd780 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cd0a1fe060 con 0x55cd0a2decf0 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cd0a2eb340 con 0x55cd0a2e8520 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30617c0640 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30655fc740 10 client.0 _ll_register_callbacks cb 0x55cd09f518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:44.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30617c0640 1 -- 172.21.15.103:0/78909664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 msgr2=0x55cd0a2e50f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30617c0640 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30617c0640 1 -- 172.21.15.103:0/78909664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 msgr2=0x55cd0a2e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30617c0640 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.035+0000 7f30617c0640 1 -- 172.21.15.103:0/78909664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd09fb43c0 con 0x55cd0a2e8520 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f30617c0640 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e5630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f30540092f0 tx=0x7f3054009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f8b7dddffc7ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f303bfff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f303bfff640 1 -- 172.21.15.103:0/78909664 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054025060 con 0x55cd0a2e8520 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f303bfff640 1 -- 172.21.15.103:0/78909664 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f305400f970 con 0x55cd0a2e8520 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3058030880 con 0x55cd0a2e8520 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f303bfff640 1 -- 172.21.15.103:0/78909664 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f305400fb70 con 0x55cd0a2e8520 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f30655fc740 10 client.27121 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:44.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.036+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cd0a2f6c20 con 0x55cd0a2e8520 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f303bfff640 1 -- 172.21.15.103:0/78909664 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3058030880 con 0x55cd0a2e8520 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f303bfff640 1 -- 172.21.15.103:0/78909664 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f3054010050 con 0x55cd0a2e8520 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f30655fc740 10 client.27121 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f30655fc740 10 client.27121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f30655fc740 20 client.27121 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f30655fc740 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a2f8130 0x55cd0a2fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f30655fc740 4 client.27121 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd0a250860 con 0x55cd0a2f8130 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.037+0000 7f3060fbf640 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a2f8130 0x55cd0a2fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.038+0000 7f303bfff640 10 client.27121 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:44.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.038+0000 7f3060fbf640 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a2f8130 0x55cd0a2fa5f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7195880759d6e56c server_cookie=3ea74943b6d9bd39 in_seq=0 out_seq=0 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.038+0000 7f303bfff640 1 -- 172.21.15.103:0/78909664 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd0a250860 con 0x55cd0a2f8130 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.038+0000 7f303bfff640 10 client.27121 handle_command_reply: tid=0 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 client.27121 shutdown 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 msgr2=0x55cd0a2e5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e5630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f30540092f0 tx=0x7f3054009200 comp rx=0 tx=0).stop 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 shutdown_connections 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a2f8130 0x55cd0a2fa5f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e5630 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 --2- 172.21.15.103:0/78909664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 >> 172.21.15.103:0/78909664 conn(0x55cd0a2a6310 msgr2=0x55cd0a2dfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f303bfff640 0 client.27121 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:44.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 shutdown_connections 2024-09-19T07:49:44.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 1 -- 172.21.15.103:0/78909664 wait complete. 2024-09-19T07:49:44.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.039+0000 7f30655fc740 20 client.27121 trim_cache size 0 max 16384 2024-09-19T07:49:44.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.056+0000 7f30655fc740 1 Processor -- start 2024-09-19T07:49:44.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f30655fc740 1 -- start start 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f30655fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f30655fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f30655fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f30655fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f304800e7e0 con 0x55cd0a2dd780 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f30655fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3048010320 con 0x55cd0a2decf0 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f30655fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f304800eeb0 con 0x55cd0a2e8520 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55610/0 (socket says 172.21.15.103:55610) 2024-09-19T07:49:44.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f3060fbf640 1 -- 172.21.15.103:0/506078542 learned_addr learned my addr 172.21.15.103:0/506078542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.057+0000 7f3053fff640 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f30617c0640 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f3060fbf640 1 -- 172.21.15.103:0/506078542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 msgr2=0x55cd0a2e0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f3060fbf640 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f3060fbf640 1 -- 172.21.15.103:0/506078542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 msgr2=0x55cd0a2e02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f3060fbf640 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f3060fbf640 1 -- 172.21.15.103:0/506078542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0a2dab10 con 0x55cd0a2dd780 2024-09-19T07:49:44.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f3053fff640 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:44.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.058+0000 7f3060fbf640 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7060 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3058009230 tx=0x7f3058047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3309b038b940df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.059+0000 7f30427fc640 1 -- 172.21.15.103:0/506078542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058030e40 con 0x55cd0a2dd780 2024-09-19T07:49:44.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.059+0000 7f30427fc640 1 -- 172.21.15.103:0/506078542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3058030880 con 0x55cd0a2dd780 2024-09-19T07:49:44.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.059+0000 7f30427fc640 1 -- 172.21.15.103:0/506078542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058004e40 con 0x55cd0a2dd780 2024-09-19T07:49:44.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.059+0000 7f30655fc740 1 -- 172.21.15.103:0/506078542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 msgr2=0x55cd0a2e7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.059+0000 7f30655fc740 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7060 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3058009230 tx=0x7f3058047040 comp rx=0 tx=0).stop 2024-09-19T07:49:44.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 -- 172.21.15.103:0/506078542 shutdown_connections 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2e8520 0x55cd0a2e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2decf0 0x55cd0a2e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 --2- 172.21.15.103:0/506078542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2dd780 0x55cd0a2e7060 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 -- 172.21.15.103:0/506078542 >> 172.21.15.103:0/506078542 conn(0x55cd0a2a6310 msgr2=0x55cd0a2e6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 -- 172.21.15.103:0/506078542 shutdown_connections 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 -- 172.21.15.103:0/506078542 wait complete. 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 -- start start 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.060+0000 7f30655fc740 1 Processor -- start 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f30655fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2f0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f30655fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f30655fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f3053fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f3053fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2f0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55626/0 (socket says 172.21.15.103:55626) 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f3060fbf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f3053fff640 1 -- 172.21.15.103:0/441153587 learned_addr learned my addr 172.21.15.103:0/441153587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f304800fc50 con 0x55cd0a2decf0 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f30617c0640 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f304800f580 con 0x55cd0a2e8520 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.061+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f304800c7e0 con 0x55cd0a2dd780 2024-09-19T07:49:44.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.062+0000 7f30655fc740 10 client.0 _ll_register_callbacks cb 0x7f3054009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:44.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.062+0000 7f3053fff640 1 -- 172.21.15.103:0/441153587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 msgr2=0x55cd0a2f0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.062+0000 7f3053fff640 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2f0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.062+0000 7f3053fff640 1 -- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 msgr2=0x55cd0a2ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:44.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.062+0000 7f3053fff640 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.062+0000 7f3053fff640 1 -- 172.21.15.103:0/441153587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0a2ec4c0 con 0x55cd0a2decf0 2024-09-19T07:49:44.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.062+0000 7f3060fbf640 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2f0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:44.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.063+0000 7f3053fff640 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2f0e40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f304c099a70 tx=0x7f304c00e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4647dfbf9846fa8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:44.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.063+0000 7f303bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:44.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.064+0000 7f303bfff640 1 -- 172.21.15.103:0/441153587 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c0b6060 con 0x55cd0a2decf0 2024-09-19T07:49:44.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.064+0000 7f303bfff640 1 -- 172.21.15.103:0/441153587 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f304c0ce790 con 0x55cd0a2decf0 2024-09-19T07:49:44.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.064+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3058030880 con 0x55cd0a2decf0 2024-09-19T07:49:44.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.065+0000 7f303bfff640 1 -- 172.21.15.103:0/441153587 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f304c0ce990 con 0x55cd0a2decf0 2024-09-19T07:49:44.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.065+0000 7f30655fc740 10 client.17835 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:44.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.065+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55cd0a3629e0 con 0x55cd0a2decf0 2024-09-19T07:49:44.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f303bfff640 1 -- 172.21.15.103:0/441153587 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304c0a5d10 con 0x55cd0a2decf0 2024-09-19T07:49:44.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f303bfff640 1 -- 172.21.15.103:0/441153587 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f304c0b8030 con 0x55cd0a2decf0 2024-09-19T07:49:44.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f30655fc740 10 client.17835 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:44.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f30655fc740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:44.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f30655fc740 20 client.17835 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:44.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f30655fc740 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a363190 0x55cd0a3655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:44.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f30655fc740 4 client.17835 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:49:44.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3058030880 con 0x55cd0a363190 2024-09-19T07:49:44.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.066+0000 7f3060fbf640 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a363190 0x55cd0a3655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:44.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f303bfff640 10 client.17835 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:44.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f3060fbf640 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a363190 0x55cd0a3655b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed306ac356a4ac72 server_cookie=e1e9bccce930f27a in_seq=0 out_seq=0 2024-09-19T07:49:44.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f303bfff640 1 -- 172.21.15.103:0/441153587 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3058030880 con 0x55cd0a363190 2024-09-19T07:49:44.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f303bfff640 10 client.17835 handle_command_reply: tid=0 2024-09-19T07:49:44.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 client.17835 shutdown 2024-09-19T07:49:44.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 msgr2=0x55cd0a2f0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2f0e40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f304c099a70 tx=0x7f304c00e640 comp rx=0 tx=0).stop 2024-09-19T07:49:44.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 shutdown_connections 2024-09-19T07:49:44.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cd0a363190 0x55cd0a3655b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f303bfff640 0 client.17835 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:44.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cd0a2e8520 0x55cd0a2ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd0a2decf0 0x55cd0a2f0e40 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 --2- 172.21.15.103:0/441153587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cd0a2dd780 0x55cd0a2f0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.067+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 >> 172.21.15.103:0/441153587 conn(0x55cd0a2a6310 msgr2=0x55cd0a2e6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 shutdown_connections 2024-09-19T07:49:44.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f30655fc740 1 -- 172.21.15.103:0/441153587 wait complete. 2024-09-19T07:49:44.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f30655fc740 20 client.17835 trim_cache size 0 max 16384 2024-09-19T07:49:44.075 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:49:44.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f30440a35c0 msgr2=0x7f30440a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f30440a35c0 0x7f30440a5a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55cd0a2de220 tx=0x7f30540095e0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c1012a0 msgr2=0x7f305c10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:44.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c1012a0 0x7f305c10dcb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f304c00e6c0 tx=0x7f304c00eb80 comp rx=0 tx=0).stop 2024-09-19T07:49:44.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 shutdown_connections 2024-09-19T07:49:44.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f30440a35c0 0x7f30440a5a80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c105d90 0x7f305c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f305c1012a0 0x7f305c10dcb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.068+0000 7f3061fc1640 1 --2- 172.21.15.103:0/1028386509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c100980 0x7f305c114c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:44.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.069+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 >> 172.21.15.103:0/1028386509 conn(0x7f305c0fc610 msgr2=0x7f305c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:44.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.069+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 shutdown_connections 2024-09-19T07:49:44.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:44.069+0000 7f3061fc1640 1 -- 172.21.15.103:0/1028386509 wait complete. 2024-09-19T07:49:44.621 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:44 smithi103 ceph-mon[31201]: pgmap v365: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 591 op/s 2024-09-19T07:49:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:44 smithi174 ceph-mon[35370]: pgmap v365: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 591 op/s 2024-09-19T07:49:44.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:44 smithi134 ceph-mon[35414]: pgmap v365: 97 pgs: 97 active+clean; 4.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 591 op/s 2024-09-19T07:49:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 670.0 s, 1698.6 tps, lat 58.625 ms stddev 102.538 2024-09-19T07:49:46.234 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:49:46.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:49:46.602 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:46.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:46 smithi174 ceph-mon[35370]: pgmap v366: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 928 op/s 2024-09-19T07:49:46.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:46 smithi103 ceph-mon[31201]: pgmap v366: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 928 op/s 2024-09-19T07:49:46.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:46 smithi134 ceph-mon[35414]: pgmap v366: 97 pgs: 97 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 928 op/s 2024-09-19T07:49:47.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 -- 172.21.15.103:0/3111043007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c100a90 msgr2=0x7f462c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 --2- 172.21.15.103:0/3111043007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c100a90 0x7f462c100ef0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f461c009930 tx=0x7f461c02a420 comp rx=0 tx=0).stop 2024-09-19T07:49:47.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 -- 172.21.15.103:0/3111043007 shutdown_connections 2024-09-19T07:49:47.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 --2- 172.21.15.103:0/3111043007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f462c101430 0x7f462c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 --2- 172.21.15.103:0/3111043007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c100a90 0x7f462c100ef0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 --2- 172.21.15.103:0/3111043007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f462c106a90 0x7f462c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 -- 172.21.15.103:0/3111043007 >> 172.21.15.103:0/3111043007 conn(0x7f462c0fc7d0 msgr2=0x7f462c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:47.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 -- 172.21.15.103:0/3111043007 shutdown_connections 2024-09-19T07:49:47.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.135+0000 7f46335fe640 1 -- 172.21.15.103:0/3111043007 wait complete. 2024-09-19T07:49:47.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 Processor -- start 2024-09-19T07:49:47.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 -- start start 2024-09-19T07:49:47.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f462c100a90 0x7f462c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f462c101430 0x7f462c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c106a90 0x7f462c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f462c110e50 con 0x7f462c106a90 2024-09-19T07:49:47.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f462c110cd0 con 0x7f462c100a90 2024-09-19T07:49:47.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f46335fe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f462c110fd0 con 0x7f462c101430 2024-09-19T07:49:47.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4630b99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f462c101430 0x7f462c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c106a90 0x7f462c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f463139a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f462c100a90 0x7f462c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c106a90 0x7f462c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55648/0 (socket says 172.21.15.103:55648) 2024-09-19T07:49:47.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4630b99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f462c101430 0x7f462c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38840/0 (socket says 172.21.15.103:38840) 2024-09-19T07:49:47.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 -- 172.21.15.103:0/810895745 learned_addr learned my addr 172.21.15.103:0/810895745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:47.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 -- 172.21.15.103:0/810895745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f462c101430 msgr2=0x7f462c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f462c101430 0x7f462c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 -- 172.21.15.103:0/810895745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f462c100a90 msgr2=0x7f462c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f462c100a90 0x7f462c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.136+0000 7f4631b9b640 1 -- 172.21.15.103:0/810895745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c19d400 con 0x7f462c106a90 2024-09-19T07:49:47.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.137+0000 7f4631b9b640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c106a90 0x7f462c19cd20 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f462400eb10 tx=0x7f462400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:47.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.138+0000 7f46227fc640 1 -- 172.21.15.103:0/810895745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624004b20 con 0x7f462c106a90 2024-09-19T07:49:47.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.138+0000 7f46227fc640 1 -- 172.21.15.103:0/810895745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4624004cc0 con 0x7f462c106a90 2024-09-19T07:49:47.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.138+0000 7f46227fc640 1 -- 172.21.15.103:0/810895745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624010760 con 0x7f462c106a90 2024-09-19T07:49:47.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.138+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c19d6f0 con 0x7f462c106a90 2024-09-19T07:49:47.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.138+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c0fccf0 con 0x7f462c106a90 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.141+0000 7f46359d7740 1 Processor -- start 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46227fc640 1 -- 172.21.15.103:0/810895745 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f4624005020 con 0x7f462c106a90 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46359d7740 1 -- start start 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46359d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46359d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b13501800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46359d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b13504130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46359d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564b1345c280 con 0x564b13501d40 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46359d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564b13403b10 con 0x564b134f7480 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f46359d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564b13458240 con 0x564b134f6b60 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f4630b99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f463139a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b13501800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.142+0000 7f4630b99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38852/0 (socket says 172.21.15.103:38852) 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.143+0000 7f4630b99640 1 -- 172.21.15.103:0/4001226639 learned_addr learned my addr 172.21.15.103:0/4001226639 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.143+0000 7f4630b99640 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b13504130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.143+0000 7f46227fc640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46180a3500 0x7f46180a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.143+0000 7f463139a640 1 -- 172.21.15.103:0/4001226639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 msgr2=0x564b134f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.143+0000 7f463139a640 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.143+0000 7f463139a640 1 -- 172.21.15.103:0/4001226639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 msgr2=0x564b13504130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.143+0000 7f46227fc640 1 -- 172.21.15.103:0/810895745 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f46240d02b0 con 0x7f462c106a90 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.144+0000 7f4631b9b640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46180a3500 0x7f46180a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.144+0000 7f463139a640 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b13504130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.144+0000 7f463139a640 1 -- 172.21.15.103:0/4001226639 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b134101f0 con 0x564b134f7480 2024-09-19T07:49:47.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.145+0000 7f4631b9b640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46180a3500 0x7f46180a59c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46240cdff0 tx=0x7f4624005b40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.145+0000 7f463139a640 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b13501800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4628005fd0 tx=0x7f46280107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8961baf338ecbbb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.145+0000 7f4630b99640 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b13504130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.145+0000 7f46177fe640 1 -- 172.21.15.103:0/4001226639 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462801c480 con 0x564b134f7480 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.145+0000 7f46177fe640 1 -- 172.21.15.103:0/4001226639 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f462801cb00 con 0x564b134f7480 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.145+0000 7f46177fe640 1 -- 172.21.15.103:0/4001226639 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628011bd0 con 0x564b134f7480 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.146+0000 7f46359d7740 1 -- 172.21.15.103:0/4001226639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 msgr2=0x564b13501800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.146+0000 7f46359d7740 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b13501800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4628005fd0 tx=0x7f46280107a0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 -- 172.21.15.103:0/4001226639 shutdown_connections 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b13504130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b13501800 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 --2- 172.21.15.103:0/4001226639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 -- 172.21.15.103:0/4001226639 >> 172.21.15.103:0/4001226639 conn(0x564b134f30f0 msgr2=0x564b134f34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 -- 172.21.15.103:0/4001226639 shutdown_connections 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 -- 172.21.15.103:0/4001226639 wait complete. 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 -- start start 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.147+0000 7f46359d7740 1 Processor -- start 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f46359d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f46359d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f46359d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b134ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f46359d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564b13504910 con 0x564b13501d40 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f46359d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564b13416f10 con 0x564b134f7480 2024-09-19T07:49:47.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f46359d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564b13504a90 con 0x564b134f6b60 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f46359d7740 10 client.0 _ll_register_callbacks cb 0x564b133dfa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f4630b99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40956/0 (socket says 172.21.15.103:40956) 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f4630b99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134fec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38854/0 (socket says 172.21.15.103:38854) 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 -- 172.21.15.103:0/3585612723 learned_addr learned my addr 172.21.15.103:0/3585612723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f4630b99640 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b134ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 -- 172.21.15.103:0/3585612723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 msgr2=0x564b134fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134fec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 -- 172.21.15.103:0/3585612723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 msgr2=0x564b134ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b134ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 -- 172.21.15.103:0/3585612723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b131cd6a0 con 0x564b134f7480 2024-09-19T07:49:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f4630b99640 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f463139a640 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff140 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4628011010 tx=0x7f462800bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694e283fef3216c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:47.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.148+0000 7f4614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:49:47.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.150+0000 7f4614ff9640 1 -- 172.21.15.103:0/3585612723 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462802b060 con 0x564b134f7480 2024-09-19T07:49:47.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.150+0000 7f4614ff9640 1 -- 172.21.15.103:0/3585612723 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f462801c690 con 0x564b134f7480 2024-09-19T07:49:47.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.150+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462801cb00 con 0x564b134f7480 2024-09-19T07:49:47.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.150+0000 7f4614ff9640 1 -- 172.21.15.103:0/3585612723 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f462800b7e0 con 0x564b134f7480 2024-09-19T07:49:47.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.150+0000 7f46359d7740 10 client.27173 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:47.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.150+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564b13518c70 con 0x564b134f7480 2024-09-19T07:49:47.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f4614ff9640 1 -- 172.21.15.103:0/3585612723 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4628011df0 con 0x564b134f7480 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f4614ff9640 1 -- 172.21.15.103:0/3585612723 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f4628026af0 con 0x564b134f7480 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f46359d7740 10 client.27173 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f46359d7740 10 client.27173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f46359d7740 20 client.27173 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f46359d7740 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1351c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f46359d7740 4 client.27173 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f462801cb00 con 0x564b1351a130 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f4630b99640 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1351c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f4614ff9640 10 client.27173 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.151+0000 7f4630b99640 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1351c520 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eadcab38cf777a0 server_cookie=3339ec4ef2ab73ae in_seq=0 out_seq=0 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.152+0000 7f4614ff9640 1 -- 172.21.15.103:0/3585612723 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f462801cb00 con 0x564b1351a130 2024-09-19T07:49:47.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.152+0000 7f4614ff9640 10 client.27173 handle_command_reply: tid=0 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 client.27173 shutdown 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 msgr2=0x564b134ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff140 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4628011010 tx=0x7f462800bf50 comp rx=0 tx=0).stop 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 shutdown_connections 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1351c520 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b13501d40 0x564b134ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f4614ff9640 0 client.27173 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff140 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 --2- 172.21.15.103:0/3585612723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b134fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 >> 172.21.15.103:0/3585612723 conn(0x564b134f30f0 msgr2=0x564b134f2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 shutdown_connections 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 1 -- 172.21.15.103:0/3585612723 wait complete. 2024-09-19T07:49:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.153+0000 7f46359d7740 20 client.27173 trim_cache size 0 max 16384 2024-09-19T07:49:47.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.175+0000 7f46359d7740 1 Processor -- start 2024-09-19T07:49:47.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.175+0000 7f46359d7740 1 -- start start 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f46359d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f6b60 0x564b134f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f46359d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f46359d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b13501d40 0x564b1351f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f46359d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ab8b0 con 0x564b134f6b60 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f46359d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ad3f0 con 0x564b134f7480 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f46359d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1abf80 con 0x564b13501d40 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f463139a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f463139a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40962/0 (socket says 172.21.15.103:40962) 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.176+0000 7f463139a640 1 -- 172.21.15.103:0/1207335624 learned_addr learned my addr 172.21.15.103:0/1207335624 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:47.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f4630b99640 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b13501d40 0x564b1351f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f4630b99640 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f6b60 0x564b134f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f463139a640 1 -- 172.21.15.103:0/1207335624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b13501d40 msgr2=0x564b1351f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f463139a640 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b13501d40 0x564b1351f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f463139a640 1 -- 172.21.15.103:0/1207335624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f6b60 msgr2=0x564b134f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f463139a640 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f6b60 0x564b134f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f463139a640 1 -- 172.21.15.103:0/1207335624 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b134f4280 con 0x564b134f7480 2024-09-19T07:49:47.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.177+0000 7f463139a640 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4628029040 tx=0x7f462802f4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cefb9920942c021 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:47.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.178+0000 7f4616ffd640 1 -- 172.21.15.103:0/1207335624 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628035080 con 0x564b134f7480 2024-09-19T07:49:47.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.178+0000 7f4616ffd640 1 -- 172.21.15.103:0/1207335624 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4628005a70 con 0x564b134f7480 2024-09-19T07:49:47.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.178+0000 7f4616ffd640 1 -- 172.21.15.103:0/1207335624 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462801c910 con 0x564b134f7480 2024-09-19T07:49:47.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.178+0000 7f46359d7740 1 -- 172.21.15.103:0/1207335624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 msgr2=0x564b134ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.178+0000 7f46359d7740 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4628029040 tx=0x7f462802f4a0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.179+0000 7f46359d7740 1 -- 172.21.15.103:0/1207335624 shutdown_connections 2024-09-19T07:49:47.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.179+0000 7f46359d7740 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b13501d40 0x564b1351f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.179+0000 7f46359d7740 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b134f7480 0x564b134ff4d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.179+0000 7f46359d7740 1 --2- 172.21.15.103:0/1207335624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f6b60 0x564b134f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.179+0000 7f46359d7740 1 -- 172.21.15.103:0/1207335624 >> 172.21.15.103:0/1207335624 conn(0x564b134f30f0 msgr2=0x564b13515d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.179+0000 7f46359d7740 1 -- 172.21.15.103:0/1207335624 shutdown_connections 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.179+0000 7f46359d7740 1 -- 172.21.15.103:0/1207335624 wait complete. 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 -- start start 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 Processor -- start 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b1351fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f7480 0x564b1357d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b13501d40 0x564b1357da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1acd20 con 0x564b134f7480 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1ac650 con 0x564b13501d40 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1a98b0 con 0x564b134f6b60 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f46359d7740 10 client.0 _ll_register_callbacks cb 0x7f462800ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:47.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f463139a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f7480 0x564b1357d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f4630b99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b1351fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f463139a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f7480 0x564b1357d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55686/0 (socket says 172.21.15.103:55686) 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f463139a640 1 -- 172.21.15.103:0/2066401114 learned_addr learned my addr 172.21.15.103:0/2066401114 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.180+0000 7f4630b99640 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b13501d40 0x564b1357da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f463139a640 1 -- 172.21.15.103:0/2066401114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 msgr2=0x564b1351fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f463139a640 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b1351fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f463139a640 1 -- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b13501d40 msgr2=0x564b1357da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f463139a640 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b13501d40 0x564b1357da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f463139a640 1 -- 172.21.15.103:0/2066401114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1357e260 con 0x564b134f7480 2024-09-19T07:49:47.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f4630b99640 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b1351fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f463139a640 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f7480 0x564b1357d540 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4628026ef0 tx=0x7f462802f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70d1dd84013851a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f4614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f4614ff9640 1 -- 172.21.15.103:0/2066401114 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628035040 con 0x564b134f7480 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f4614ff9640 1 -- 172.21.15.103:0/2066401114 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f462801c690 con 0x564b134f7480 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4628005a70 con 0x564b134f7480 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f4614ff9640 1 -- 172.21.15.103:0/2066401114 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4628005250 con 0x564b134f7480 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f46359d7740 10 client.17850 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564b1350ad30 con 0x564b134f7480 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.181+0000 7f4614ff9640 1 -- 172.21.15.103:0/2066401114 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4628005a70 con 0x564b134f7480 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.183+0000 7f4614ff9640 1 -- 172.21.15.103:0/2066401114 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f462802f7c0 con 0x564b134f7480 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.183+0000 7f46359d7740 10 client.17850 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.183+0000 7f46359d7740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.183+0000 7f46359d7740 20 client.17850 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.183+0000 7f46359d7740 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1358fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.183+0000 7f46359d7740 4 client.17850 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:49:47.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.183+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564b134697f0 con 0x564b1351a130 2024-09-19T07:49:47.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.186+0000 7f4630b99640 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1358fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:47.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.187+0000 7f4630b99640 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1358fa40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=721621a0cd9cd8cf server_cookie=eac40301aa3665e1 in_seq=0 out_seq=0 2024-09-19T07:49:47.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.187+0000 7f4614ff9640 10 client.17850 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:47.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.192+0000 7f4614ff9640 1 -- 172.21.15.103:0/2066401114 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564b134697f0 con 0x564b1351a130 2024-09-19T07:49:47.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.192+0000 7f4614ff9640 10 client.17850 handle_command_reply: tid=0 2024-09-19T07:49:47.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.193+0000 7f46359d7740 1 client.17850 shutdown 2024-09-19T07:49:47.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.198+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f7480 msgr2=0x564b1357d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.198+0000 7f46359d7740 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f7480 0x564b1357d540 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4628026ef0 tx=0x7f462802f3c0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.204+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 shutdown_connections 2024-09-19T07:49:47.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.204+0000 7f46359d7740 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x564b1351a130 0x564b1358fa40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.204+0000 7f46359d7740 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b13501d40 0x564b1357da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.204+0000 7f46359d7740 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564b134f7480 0x564b1357d540 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.204+0000 7f46359d7740 1 --2- 172.21.15.103:0/2066401114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b134f6b60 0x564b1351fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.204+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 >> 172.21.15.103:0/2066401114 conn(0x564b134f30f0 msgr2=0x564b134f1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:47.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.204+0000 7f4614ff9640 0 client.17850 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:47.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.208+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 shutdown_connections 2024-09-19T07:49:47.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.209+0000 7f46359d7740 1 -- 172.21.15.103:0/2066401114 wait complete. 2024-09-19T07:49:47.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.209+0000 7f46359d7740 20 client.17850 trim_cache size 0 max 16384 2024-09-19T07:49:47.212 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:49:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:49:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "e3b44e7a-579b-4223-b5e7-065a96db4fe4", 2024-09-19T07:49:47.212 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:49:47.212 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:49:47.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46180a3500 msgr2=0x7f46180a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46180a3500 0x7f46180a59c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46240cdff0 tx=0x7f4624005b40 comp rx=0 tx=0).stop 2024-09-19T07:49:47.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c106a90 msgr2=0x7f462c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:47.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c106a90 0x7f462c19cd20 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f462400eb10 tx=0x7f462400c2d0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 shutdown_connections 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f46180a3500 0x7f46180a59c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f462c106a90 0x7f462c19cd20 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f462c101430 0x7f462c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 --2- 172.21.15.103:0/810895745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f462c100a90 0x7f462c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 >> 172.21.15.103:0/810895745 conn(0x7f462c0fc7d0 msgr2=0x7f462c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 shutdown_connections 2024-09-19T07:49:47.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:47.212+0000 7f46335fe640 1 -- 172.21.15.103:0/810895745 wait complete. 2024-09-19T07:49:48.251 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:49:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: pgmap v367: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2024-09-19T07:49:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:49:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:48 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.619 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:49:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: pgmap v367: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2024-09-19T07:49:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:49:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:48 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: pgmap v367: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2024-09-19T07:49:48.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:49:48.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:49:48.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:48.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:48 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:49:49.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 -- 172.21.15.103:0/3538310216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8100a80 msgr2=0x7ff9c8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/3538310216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8100a80 0x7ff9c8100ee0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009930 tx=0x7ff9b802a420 comp rx=0 tx=0).stop 2024-09-19T07:49:49.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 -- 172.21.15.103:0/3538310216 shutdown_connections 2024-09-19T07:49:49.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/3538310216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8101420 0x7ff9c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/3538310216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8100a80 0x7ff9c8100ee0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/3538310216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c8106a80 0x7ff9c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 -- 172.21.15.103:0/3538310216 >> 172.21.15.103:0/3538310216 conn(0x7ff9c80fc7a0 msgr2=0x7ff9c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:49.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 -- 172.21.15.103:0/3538310216 shutdown_connections 2024-09-19T07:49:49.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.228+0000 7ff9cffa7640 1 -- 172.21.15.103:0/3538310216 wait complete. 2024-09-19T07:49:49.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 Processor -- start 2024-09-19T07:49:49.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 -- start start 2024-09-19T07:49:49.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8100a80 0x7ff9c81a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8101420 0x7ff9c81a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c8106a80 0x7ff9c819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8110e80 con 0x7ff9c8101420 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8110d00 con 0x7ff9c8100a80 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cffa7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8111000 con 0x7ff9c8106a80 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cd542640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8101420 0x7ff9c81a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cd542640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8101420 0x7ff9c81a3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52358/0 (socket says 172.21.15.103:52358) 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cd542640 1 -- 172.21.15.103:0/4067478338 learned_addr learned my addr 172.21.15.103:0/4067478338 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8100a80 0x7ff9c81a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.230+0000 7ff9ce544640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c8106a80 0x7ff9c819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.231+0000 7ff9cd542640 1 -- 172.21.15.103:0/4067478338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c8106a80 msgr2=0x7ff9c819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.231+0000 7ff9cd542640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c8106a80 0x7ff9c819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.231+0000 7ff9cd542640 1 -- 172.21.15.103:0/4067478338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8100a80 msgr2=0x7ff9c81a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.231+0000 7ff9cd542640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8100a80 0x7ff9c81a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.231+0000 7ff9cd542640 1 -- 172.21.15.103:0/4067478338 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c819f230 con 0x7ff9c8101420 2024-09-19T07:49:49.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.231+0000 7ff9cd542640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8101420 0x7ff9c81a3b90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009230 tx=0x7ff9b8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.232+0000 7ff9beffd640 1 -- 172.21.15.103:0/4067478338 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b803f030 con 0x7ff9c8101420 2024-09-19T07:49:49.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.233+0000 7ff9beffd640 1 -- 172.21.15.103:0/4067478338 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b8004440 con 0x7ff9c8101420 2024-09-19T07:49:49.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.233+0000 7ff9beffd640 1 -- 172.21.15.103:0/4067478338 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8035070 con 0x7ff9c8101420 2024-09-19T07:49:49.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.233+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c819f4c0 con 0x7ff9c8101420 2024-09-19T07:49:49.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.233+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81ab5d0 con 0x7ff9c8101420 2024-09-19T07:49:49.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.235+0000 7ff9beffd640 1 -- 172.21.15.103:0/4067478338 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff9b8042030 con 0x7ff9c8101420 2024-09-19T07:49:49.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.235+0000 7ff9d2380740 1 Processor -- start 2024-09-19T07:49:49.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.236+0000 7ff9d2380740 1 -- start start 2024-09-19T07:49:49.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c509880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c509dc0 0x55ab3c514140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c516a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9d2380740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ab3c46f2e0 con 0x55ab3c509dc0 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9d2380740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ab3c416ce0 con 0x55ab3c5094a0 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9d2380740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab3c46b2a0 con 0x55ab3c514680 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9ce544640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c509880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9ce544640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c509880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:53492/0 (socket says 172.21.15.103:53492) 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9ce544640 1 -- 172.21.15.103:0/165672215 learned_addr learned my addr 172.21.15.103:0/165672215 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c509dc0 0x55ab3c514140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.237+0000 7ff9cd542640 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c516a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9beffd640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9b40a3430 0x7ff9b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9cdd43640 1 -- 172.21.15.103:0/165672215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 msgr2=0x55ab3c516a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c516a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9cdd43640 1 -- 172.21.15.103:0/165672215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 msgr2=0x55ab3c509880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c509880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9cdd43640 1 -- 172.21.15.103:0/165672215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3c3fb440 con 0x55ab3c509dc0 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9ce544640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9b40a3430 0x7ff9b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9beffd640 1 -- 172.21.15.103:0/4067478338 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff9b80efc90 con 0x7ff9c8101420 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c509dc0 0x55ab3c514140 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009bf0 tx=0x7ff9c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622dadaa4113de17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9cd542640 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c516a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9b3fff640 1 -- 172.21.15.103:0/165672215 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4004050 con 0x55ab3c509dc0 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9b3fff640 1 -- 172.21.15.103:0/165672215 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4005500 con 0x55ab3c509dc0 2024-09-19T07:49:49.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9ce544640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9b40a3430 0x7ff9b40a58f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fd0 tx=0x7ff9c0004a40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.238+0000 7ff9b3fff640 1 -- 172.21.15.103:0/165672215 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4004d90 con 0x55ab3c509dc0 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.239+0000 7ff9d2380740 1 -- 172.21.15.103:0/165672215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c509dc0 msgr2=0x55ab3c514140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.239+0000 7ff9d2380740 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c509dc0 0x55ab3c514140 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009bf0 tx=0x7ff9c4011810 comp rx=0 tx=0).stop 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.240+0000 7ff9d2380740 1 -- 172.21.15.103:0/165672215 shutdown_connections 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.240+0000 7ff9d2380740 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c516a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.240+0000 7ff9d2380740 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c509dc0 0x55ab3c514140 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.240+0000 7ff9d2380740 1 --2- 172.21.15.103:0/165672215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c509880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.240+0000 7ff9d2380740 1 -- 172.21.15.103:0/165672215 >> 172.21.15.103:0/165672215 conn(0x55ab3c505a30 msgr2=0x55ab3c505e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.240+0000 7ff9d2380740 1 -- 172.21.15.103:0/165672215 shutdown_connections 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.240+0000 7ff9d2380740 1 -- 172.21.15.103:0/165672215 wait complete. 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.241+0000 7ff9d2380740 1 -- start start 2024-09-19T07:49:49.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.241+0000 7ff9d2380740 1 Processor -- start 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.241+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c513f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.241+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c5111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.241+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c514680 0x55ab3c511700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9d2380740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ab3c517250 con 0x55ab3c514680 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c513f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9ce544640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c514680 0x55ab3c511700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c513f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:53500/0 (socket says 172.21.15.103:53500) 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 -- 172.21.15.103:0/3946616663 learned_addr learned my addr 172.21.15.103:0/3946616663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ab3c42a050 con 0x55ab3c5094a0 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab3c5173d0 con 0x55ab3c509dc0 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9d2380740 10 client.0 _ll_register_callbacks cb 0x55ab3c46a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cd542640 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c5111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 -- 172.21.15.103:0/3946616663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 msgr2=0x55ab3c5111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c5111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 -- 172.21.15.103:0/3946616663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c514680 msgr2=0x55ab3c511700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c514680 0x55ab3c511700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 -- 172.21.15.103:0/3946616663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3c1e03c0 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c513f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4011d10 tx=0x7ff9c4009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1ed3b4453ef140 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.242+0000 7ff9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.243+0000 7ff9b17fa640 1 -- 172.21.15.103:0/3946616663 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4029070 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.243+0000 7ff9b17fa640 1 -- 172.21.15.103:0/3946616663 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff9c4004f20 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.243+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c4005500 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.243+0000 7ff9b17fa640 1 -- 172.21.15.103:0/3946616663 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff9c4033050 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.243+0000 7ff9d2380740 10 client.27191 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.243+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ab3c522b60 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9b17fa640 1 -- 172.21.15.103:0/3946616663 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4005500 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9b17fa640 1 -- 172.21.15.103:0/3946616663 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7ff9c402e440 con 0x55ab3c5094a0 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9d2380740 10 client.27191 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:49.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9d2380740 10 client.27191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:49.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9d2380740 20 client.27191 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:49.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9d2380740 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c523fc0 0x55ab3c5263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9d2380740 4 client.27191 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:49:49.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab3c47c850 con 0x55ab3c523fc0 2024-09-19T07:49:49.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.244+0000 7ff9cd542640 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c523fc0 0x55ab3c5263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.246+0000 7ff9cd542640 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c523fc0 0x55ab3c5263b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7086e3c48d8d1ff8 server_cookie=2d942875fd88c629 in_seq=0 out_seq=0 2024-09-19T07:49:49.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.246+0000 7ff9b17fa640 10 client.27191 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:49.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.246+0000 7ff9b17fa640 1 -- 172.21.15.103:0/3946616663 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab3c47c850 con 0x55ab3c523fc0 2024-09-19T07:49:49.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.246+0000 7ff9b17fa640 10 client.27191 handle_command_reply: tid=0 2024-09-19T07:49:49.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 client.27191 shutdown 2024-09-19T07:49:49.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 msgr2=0x55ab3c513f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c513f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4011d10 tx=0x7ff9c4009fd0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 shutdown_connections 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c523fc0 0x55ab3c5263b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c514680 0x55ab3c511700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c5111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 --2- 172.21.15.103:0/3946616663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c5094a0 0x55ab3c513f10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9b17fa640 0 client.27191 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 >> 172.21.15.103:0/3946616663 conn(0x55ab3c505a30 msgr2=0x55ab3c50b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 shutdown_connections 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 1 -- 172.21.15.103:0/3946616663 wait complete. 2024-09-19T07:49:49.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.247+0000 7ff9d2380740 20 client.27191 trim_cache size 0 max 16384 2024-09-19T07:49:49.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 Processor -- start 2024-09-19T07:49:49.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 -- start start 2024-09-19T07:49:49.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c5204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c509dc0 0x55ab3c50fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c513130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b3fc0 con 0x55ab3c5094a0 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b5b00 con 0x55ab3c509dc0 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.271+0000 7ff9d2380740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b4690 con 0x55ab3c514680 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9cdd43640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c5204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9cdd43640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c5204b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52378/0 (socket says 172.21.15.103:52378) 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9ce544640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c513130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9cdd43640 1 -- 172.21.15.103:0/2585528620 learned_addr learned my addr 172.21.15.103:0/2585528620 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9cd542640 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c509dc0 0x55ab3c50fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9ce544640 1 -- 172.21.15.103:0/2585528620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c509dc0 msgr2=0x55ab3c50fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9ce544640 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c509dc0 0x55ab3c50fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9ce544640 1 -- 172.21.15.103:0/2585528620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 msgr2=0x55ab3c5204b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9ce544640 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c5204b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9ce544640 1 -- 172.21.15.103:0/2585528620 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3c514280 con 0x55ab3c514680 2024-09-19T07:49:49.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9ce544640 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c513130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00044d0 tx=0x7ff9c0013030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9a08b356fc2671 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9b37fe640 1 -- 172.21.15.103:0/2585528620 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0004d20 con 0x55ab3c514680 2024-09-19T07:49:49.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9b37fe640 1 -- 172.21.15.103:0/2585528620 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c001ea90 con 0x55ab3c514680 2024-09-19T07:49:49.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.272+0000 7ff9b37fe640 1 -- 172.21.15.103:0/2585528620 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00057e0 con 0x55ab3c514680 2024-09-19T07:49:49.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 -- 172.21.15.103:0/2585528620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 msgr2=0x55ab3c513130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c513130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00044d0 tx=0x7ff9c0013030 comp rx=0 tx=0).stop 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 -- 172.21.15.103:0/2585528620 shutdown_connections 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c514680 0x55ab3c513130 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c509dc0 0x55ab3c50fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 --2- 172.21.15.103:0/2585528620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c5204b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 -- 172.21.15.103:0/2585528620 >> 172.21.15.103:0/2585528620 conn(0x55ab3c505a30 msgr2=0x55ab3c512a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 -- 172.21.15.103:0/2585528620 shutdown_connections 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 -- 172.21.15.103:0/2585528620 wait complete. 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.273+0000 7ff9d2380740 1 -- start start 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 1 Processor -- start 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c51cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c517e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c514680 0x55ab3c518390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b5430 con 0x55ab3c5094a0 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b4d60 con 0x55ab3c514680 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b1fc0 con 0x55ab3c509dc0 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c51cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9d2380740 10 client.0 _ll_register_callbacks cb 0x7ff9c0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c51cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52390/0 (socket says 172.21.15.103:52390) 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 -- 172.21.15.103:0/866566366 learned_addr learned my addr 172.21.15.103:0/866566366 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9ce544640 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c514680 0x55ab3c518390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cd542640 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c517e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 -- 172.21.15.103:0/866566366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 msgr2=0x55ab3c517e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c517e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 -- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c514680 msgr2=0x55ab3c518390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c514680 0x55ab3c518390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.274+0000 7ff9cdd43640 1 -- 172.21.15.103:0/866566366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3c518900 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9cdd43640 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c51cad0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009bf0 tx=0x7ff9c402e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645ba9336a15b0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9b17fa640 1 -- 172.21.15.103:0/866566366 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4029040 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9b17fa640 1 -- 172.21.15.103:0/866566366 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff9c40035f0 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c001ea90 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9b17fa640 1 -- 172.21.15.103:0/866566366 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff9c4046050 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9d2380740 10 client.17868 fetch_fsmap learned FSMap version 47 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ab3c58e860 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.275+0000 7ff9b17fa640 1 -- 172.21.15.103:0/866566366 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c001ea90 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9b17fa640 1 -- 172.21.15.103:0/866566366 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7ff9c4004e40 con 0x55ab3c5094a0 2024-09-19T07:49:49.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9d2380740 10 client.17868 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9d2380740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9d2380740 20 client.17868 populate_metadata read hostname 'smithi103' 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9d2380740 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c58ef10 0x55ab3c591330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9d2380740 4 client.17868 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab3c476d60 con 0x55ab3c58ef10 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9cd542640 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c58ef10 0x55ab3c591330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.276+0000 7ff9cd542640 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c58ef10 0x55ab3c591330 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aa6f1f26211c70f server_cookie=1d47c695b15e890c in_seq=0 out_seq=0 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.277+0000 7ff9b17fa640 10 client.17868 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.277+0000 7ff9b17fa640 1 -- 172.21.15.103:0/866566366 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ab3c476d60 con 0x55ab3c58ef10 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.277+0000 7ff9b17fa640 10 client.17868 handle_command_reply: tid=0 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.277+0000 7ff9d2380740 1 client.17868 shutdown 2024-09-19T07:49:49.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 msgr2=0x55ab3c51cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c51cad0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009bf0 tx=0x7ff9c402e750 comp rx=0 tx=0).stop 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 shutdown_connections 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ab3c58ef10 0x55ab3c591330 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab3c514680 0x55ab3c518390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab3c509dc0 0x55ab3c517e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 --2- 172.21.15.103:0/866566366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab3c5094a0 0x55ab3c51cad0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9b17fa640 0 client.17868 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 >> 172.21.15.103:0/866566366 conn(0x55ab3c505a30 msgr2=0x55ab3c513240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 shutdown_connections 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 1 -- 172.21.15.103:0/866566366 wait complete. 2024-09-19T07:49:49.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.278+0000 7ff9d2380740 20 client.17868 trim_cache size 0 max 16384 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "e3b44e7a-579b-4223-b5e7-065a96db4fe4": { 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "e3b44e7a-579b-4223-b5e7-065a96db4fe4", 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:49:49.285 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:49:49.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9b40a3430 msgr2=0x7ff9b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9b40a3430 0x7ff9b40a58f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fd0 tx=0x7ff9c0004a40 comp rx=0 tx=0).stop 2024-09-19T07:49:49.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8101420 msgr2=0x7ff9c81a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:49:49.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8101420 0x7ff9c81a3b90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009230 tx=0x7ff9b8004040 comp rx=0 tx=0).stop 2024-09-19T07:49:49.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 shutdown_connections 2024-09-19T07:49:49.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff9b40a3430 0x7ff9b40a58f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c8106a80 0x7ff9c819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c8101420 0x7ff9c81a3b90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 --2- 172.21.15.103:0/4067478338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8100a80 0x7ff9c81a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:49:49.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 >> 172.21.15.103:0/4067478338 conn(0x7ff9c80fc7a0 msgr2=0x7ff9c806a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:49:49.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 shutdown_connections 2024-09-19T07:49:49.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:49:49.279+0000 7ff9cffa7640 1 -- 172.21.15.103:0/4067478338 wait complete. 2024-09-19T07:49:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 675.0 s, 2873.8 tps, lat 35.028 ms stddev 38.863 2024-09-19T07:49:50.662 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:50 smithi103 ceph-mon[31201]: pgmap v368: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 1.50k op/s 2024-09-19T07:49:50.664 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:50 smithi174 ceph-mon[35370]: pgmap v368: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 1.50k op/s 2024-09-19T07:49:50.697 INFO:tasks.cephfs.filesystem:scrub status for tag:e3b44e7a-579b-4223-b5e7-065a96db4fe4 - {'path': '/', 'tag': 'e3b44e7a-579b-4223-b5e7-065a96db4fe4', 'options': 'recursive,force'} 2024-09-19T07:49:50.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:50 smithi134 ceph-mon[35414]: pgmap v368: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 1.50k op/s 2024-09-19T07:49:52.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:52 smithi174 ceph-mon[35370]: pgmap v369: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 36 MiB/s wr, 1.79k op/s 2024-09-19T07:49:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:52 smithi103 ceph-mon[31201]: pgmap v369: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 36 MiB/s wr, 1.79k op/s 2024-09-19T07:49:52.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:52 smithi134 ceph-mon[35414]: pgmap v369: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 36 MiB/s wr, 1.79k op/s 2024-09-19T07:49:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:54 smithi174 ceph-mon[35370]: pgmap v370: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.51k op/s 2024-09-19T07:49:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:54 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:54 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:54 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:54 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:54 smithi103 ceph-mon[31201]: pgmap v370: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.51k op/s 2024-09-19T07:49:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:54 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:54 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:54 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:54.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:54 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:54.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:54 smithi134 ceph-mon[35414]: pgmap v370: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 33 MiB/s wr, 1.51k op/s 2024-09-19T07:49:54.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:54 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:54.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:54 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:54.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:54 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:54.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:54 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 680.0 s, 2118.8 tps, lat 46.977 ms stddev 54.739 2024-09-19T07:49:55.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:55 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:49:55.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:55 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:49:55.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:55 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:49:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:56 smithi174 ceph-mon[35370]: pgmap v371: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 35 MiB/s wr, 1.64k op/s 2024-09-19T07:49:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:56 smithi103 ceph-mon[31201]: pgmap v371: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 35 MiB/s wr, 1.64k op/s 2024-09-19T07:49:56.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:56 smithi134 ceph-mon[35414]: pgmap v371: 97 pgs: 97 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 35 MiB/s wr, 1.64k op/s 2024-09-19T07:49:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:49:58 smithi174 ceph-mon[35370]: pgmap v372: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 27 MiB/s wr, 1.46k op/s 2024-09-19T07:49:58.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:49:58 smithi103 ceph-mon[31201]: pgmap v372: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 27 MiB/s wr, 1.46k op/s 2024-09-19T07:49:58.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:49:58 smithi134 ceph-mon[35414]: pgmap v372: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 27 MiB/s wr, 1.46k op/s 2024-09-19T07:50:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 685.0 s, 1806.4 tps, lat 54.169 ms stddev 59.361 2024-09-19T07:50:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:00 smithi174 ceph-mon[35370]: pgmap v373: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 5.6 MiB/s wr, 576 op/s 2024-09-19T07:50:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:00 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:00 smithi174 ceph-mon[35370]: overall HEALTH_OK 2024-09-19T07:50:00.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:00 smithi103 ceph-mon[31201]: pgmap v373: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 5.6 MiB/s wr, 576 op/s 2024-09-19T07:50:00.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:00 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:00.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:00 smithi103 ceph-mon[31201]: overall HEALTH_OK 2024-09-19T07:50:00.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:00 smithi134 ceph-mon[35414]: pgmap v373: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 5.6 MiB/s wr, 576 op/s 2024-09-19T07:50:00.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:00 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:00.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:00 smithi134 ceph-mon[35414]: overall HEALTH_OK 2024-09-19T07:50:01.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:01 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:01.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:01 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:01.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:01 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:02 smithi174 ceph-mon[35370]: pgmap v374: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 8.0 MiB/s wr, 833 op/s 2024-09-19T07:50:02.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:02 smithi103 ceph-mon[31201]: pgmap v374: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 8.0 MiB/s wr, 833 op/s 2024-09-19T07:50:02.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:02 smithi134 ceph-mon[35414]: pgmap v374: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 8.0 MiB/s wr, 833 op/s 2024-09-19T07:50:04.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:04 smithi103 ceph-mon[31201]: pgmap v375: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-09-19T07:50:04.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:04 smithi134 ceph-mon[35414]: pgmap v375: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-09-19T07:50:04.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:04 smithi174 ceph-mon[35370]: pgmap v375: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2024-09-19T07:50:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 690.0 s, 463.4 tps, lat 216.653 ms stddev 189.942 2024-09-19T07:50:05.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:05 smithi103 ceph-mon[31201]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:50:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:05 smithi134 ceph-mon[35414]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:50:05.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:05 smithi174 ceph-mon[35370]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T07:50:06.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:06 smithi103 ceph-mon[31201]: pgmap v377: 97 pgs: 6 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 85 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-09-19T07:50:06.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:06 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:06.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:06 smithi134 ceph-mon[35414]: pgmap v377: 97 pgs: 6 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 85 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-09-19T07:50:06.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:06 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:06 smithi174 ceph-mon[35370]: pgmap v377: 97 pgs: 6 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 85 active+clean; 4.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.6 MiB/s wr, 694 op/s 2024-09-19T07:50:06.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:06 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:08.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:08 smithi103 ceph-mon[31201]: pgmap v378: 97 pgs: 39 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 40 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:08.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:08 smithi134 ceph-mon[35414]: pgmap v378: 97 pgs: 39 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 40 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:08.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:08 smithi174 ceph-mon[35370]: pgmap v378: 97 pgs: 39 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 40 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 695.0 s, 2103.4 tps, lat 47.912 ms stddev 59.801 2024-09-19T07:50:10.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:10 smithi103 ceph-mon[31201]: pgmap v379: 97 pgs: 39 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 40 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:10.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:10 smithi134 ceph-mon[35414]: pgmap v379: 97 pgs: 39 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 40 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:10.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:10 smithi174 ceph-mon[35370]: pgmap v379: 97 pgs: 39 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 40 active+clean; 4.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:11.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:11 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:11.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:11 smithi103 ceph-mon[31201]: pgmap v380: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:11 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:11.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:11 smithi134 ceph-mon[35414]: pgmap v380: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:11.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:11 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:50:11.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:11 smithi174 ceph-mon[35370]: pgmap v380: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:13.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:13 smithi134 ceph-mon[35414]: pgmap v381: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:13.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:13 smithi174 ceph-mon[35370]: pgmap v381: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:13.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:13 smithi103 ceph-mon[31201]: pgmap v381: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-19T07:50:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 700.0 s, 2376.2 tps, lat 42.602 ms stddev 46.493 2024-09-19T07:50:15.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:15 smithi134 ceph-mon[35414]: pgmap v382: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 22 MiB/s wr, 1.89k op/s 2024-09-19T07:50:15.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:15 smithi174 ceph-mon[35370]: pgmap v382: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 22 MiB/s wr, 1.89k op/s 2024-09-19T07:50:15.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:15 smithi103 ceph-mon[31201]: pgmap v382: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 22 MiB/s wr, 1.89k op/s 2024-09-19T07:50:17.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:16 smithi134 ceph-mon[35414]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:50:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:16 smithi174 ceph-mon[35370]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:50:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:16 smithi103 ceph-mon[31201]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T07:50:18.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:17 smithi134 ceph-mon[35414]: pgmap v384: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-09-19T07:50:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:17 smithi174 ceph-mon[35370]: pgmap v384: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-09-19T07:50:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:17 smithi103 ceph-mon[31201]: pgmap v384: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-09-19T07:50:19.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:18 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:50:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:18 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:50:19.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:18 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:50:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: pgmap v385: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-09-19T07:50:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:50:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:50:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:20 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: pgmap v385: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-09-19T07:50:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:50:20.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:50:20.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:20 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 705.0 s, 2140.4 tps, lat 46.518 ms stddev 50.864 2024-09-19T07:50:20.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: pgmap v385: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:50:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:20 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:50:20.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:50:21.062 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:21.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 -- 172.21.15.103:0/1371293604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 msgr2=0x7f8e8c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 --2- 172.21.15.103:0/1371293604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c1021d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009900 tx=0x7f8e8002a410 comp rx=0 tx=0).stop 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 -- 172.21.15.103:0/1371293604 shutdown_connections 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 --2- 172.21.15.103:0/1371293604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e8c106f20 0x7f8e8c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 --2- 172.21.15.103:0/1371293604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e8c1027a0 0x7f8e8c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 --2- 172.21.15.103:0/1371293604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c1021d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 -- 172.21.15.103:0/1371293604 >> 172.21.15.103:0/1371293604 conn(0x7f8e8c0fd8f0 msgr2=0x7f8e8c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 -- 172.21.15.103:0/1371293604 shutdown_connections 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.604+0000 7f8e90a55640 1 -- 172.21.15.103:0/1371293604 wait complete. 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 Processor -- start 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 -- start start 2024-09-19T07:50:21.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e8c1027a0 0x7f8e8c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e8c106f20 0x7f8e8c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c111f90 con 0x7f8e8c101df0 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c111e10 con 0x7f8e8c1027a0 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e90a55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c112110 con 0x7f8e8c106f20 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.605+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58522/0 (socket says 172.21.15.103:58522) 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8a59c640 1 -- 172.21.15.103:0/2593284807 learned_addr learned my addr 172.21.15.103:0/2593284807 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e8c1027a0 0x7f8e8c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e8c106f20 0x7f8e8c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8a59c640 1 -- 172.21.15.103:0/2593284807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e8c106f20 msgr2=0x7f8e8c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e8c106f20 0x7f8e8c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8a59c640 1 -- 172.21.15.103:0/2593284807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e8c1027a0 msgr2=0x7f8e8c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e8c1027a0 0x7f8e8c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8a59c640 1 -- 172.21.15.103:0/2593284807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c19d430 con 0x7f8e8c101df0 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e8c1027a0 0x7f8e8c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c198480 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004770 tx=0x7f8e80003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e7b7fe640 1 -- 172.21.15.103:0/2593284807 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8003f070 con 0x7f8e8c101df0 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c19d6c0 con 0x7f8e8c101df0 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e7b7fe640 1 -- 172.21.15.103:0/2593284807 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e80031ca0 con 0x7f8e8c101df0 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e7b7fe640 1 -- 172.21.15.103:0/2593284807 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80035030 con 0x7f8e8c101df0 2024-09-19T07:50:21.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.606+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a5000 con 0x7f8e8c101df0 2024-09-19T07:50:21.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.610+0000 7f8e7b7fe640 1 -- 172.21.15.103:0/2593284807 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8e80042070 con 0x7f8e8c101df0 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.610+0000 7f8e92e2e740 1 Processor -- start 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.610+0000 7f8e92e2e740 1 -- start start 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c42344a0 0x5587c4234880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c4234dc0 0x5587c423f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e7b7fe640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e740a3430 0x7f8e740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c423f680 0x5587c4241a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587c419a2e0 con 0x5587c4234dc0 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587c4141ce0 con 0x5587c42344a0 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587c41962a0 con 0x5587c423f680 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c423f680 0x5587c4241a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c423f680 0x5587c4241a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54060/0 (socket says 172.21.15.103:54060) 2024-09-19T07:50:21.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 -- 172.21.15.103:0/710373415 learned_addr learned my addr 172.21.15.103:0/710373415 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c4234dc0 0x5587c423f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e7b7fe640 1 -- 172.21.15.103:0/2593284807 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f8e800f0070 con 0x7f8e8c101df0 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 -- 172.21.15.103:0/710373415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c42344a0 msgr2=0x5587c4234880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c42344a0 0x5587c4234880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 -- 172.21.15.103:0/710373415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c4234dc0 msgr2=0x5587c423f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c4234dc0 0x5587c423f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 -- 172.21.15.103:0/710373415 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c4126440 con 0x5587c423f680 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e740a3430 0x7f8e740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.611+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c423f680 0x5587c4241a70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e84010820 tx=0x7f8e84010ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30a1159a7cd3955 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e78ff9640 1 -- 172.21.15.103:0/710373415 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84004d20 con 0x5587c423f680 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e78ff9640 1 -- 172.21.15.103:0/710373415 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e84015070 con 0x5587c423f680 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e78ff9640 1 -- 172.21.15.103:0/710373415 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e840058c0 con 0x5587c423f680 2024-09-19T07:50:21.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e92e2e740 1 -- 172.21.15.103:0/710373415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c423f680 msgr2=0x5587c4241a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e740a3430 0x7f8e740a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0099b0 tx=0x7f8e7c005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c423f680 0x5587c4241a70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e84010820 tx=0x7f8e84010ce0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e92e2e740 1 -- 172.21.15.103:0/710373415 shutdown_connections 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c423f680 0x5587c4241a70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c4234dc0 0x5587c423f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/710373415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c42344a0 0x5587c4234880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.612+0000 7f8e92e2e740 1 -- 172.21.15.103:0/710373415 >> 172.21.15.103:0/710373415 conn(0x5587c4230a30 msgr2=0x5587c4230e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 -- 172.21.15.103:0/710373415 shutdown_connections 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 -- 172.21.15.103:0/710373415 wait complete. 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 -- start start 2024-09-19T07:50:21.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 Processor -- start 2024-09-19T07:50:21.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 0x5587c423f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c423c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c423c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587c4242250 con 0x5587c423f680 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5587c4155050 con 0x5587c4234dc0 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587c42423d0 con 0x5587c42344a0 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.613+0000 7f8e92e2e740 10 client.0 _ll_register_callbacks cb 0x5587c4195160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8ad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c423c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c423c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c423c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40290/0 (socket says 172.21.15.103:40290) 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e89d9b640 1 -- 172.21.15.103:0/1664437136 learned_addr learned my addr 172.21.15.103:0/1664437136 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 0x5587c423f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/1664437136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 msgr2=0x5587c423f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:21.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 0x5587c423f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 msgr2=0x5587c423c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c423c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/1664437136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c3f0b3c0 con 0x5587c423f680 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c423c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c423c8f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c010740 tx=0x7f8e7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2892f7fd6f6314b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e71ffb640 1 -- 172.21.15.103:0/1664437136 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c023070 con 0x5587c423f680 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e71ffb640 1 -- 172.21.15.103:0/1664437136 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f8e7c01f650 con 0x5587c423f680 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.614+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e84015070 con 0x5587c423f680 2024-09-19T07:50:21.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.616+0000 7f8e71ffb640 1 -- 172.21.15.103:0/1664437136 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c02f050 con 0x5587c423f680 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.616+0000 7f8e92e2e740 10 client.17880 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.616+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5587c424dce0 con 0x5587c423f680 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.616+0000 7f8e71ffb640 1 -- 172.21.15.103:0/1664437136 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e84015070 con 0x5587c423f680 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e71ffb640 1 -- 172.21.15.103:0/1664437136 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f8e7c00f040 con 0x5587c423f680 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e92e2e740 10 client.17880 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e92e2e740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e92e2e740 20 client.17880 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c424f1b0 0x5587c4251630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e92e2e740 4 client.17880 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:21.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587c41a7850 con 0x5587c424f1b0 2024-09-19T07:50:21.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c424f1b0 0x5587c4251630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e71ffb640 10 client.17880 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:21.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.617+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c424f1b0 0x5587c4251630 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cceeffa189e9b0e server_cookie=6f9ccab57f0f33c in_seq=0 out_seq=0 2024-09-19T07:50:21.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.619+0000 7f8e71ffb640 1 -- 172.21.15.103:0/1664437136 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587c41a7850 con 0x5587c424f1b0 2024-09-19T07:50:21.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.619+0000 7f8e71ffb640 10 client.17880 handle_command_reply: tid=0 2024-09-19T07:50:21.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.619+0000 7f8e92e2e740 1 client.17880 shutdown 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.620+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 msgr2=0x5587c423c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.620+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c423c8f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c010740 tx=0x7f8e7c0046c0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.621+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 shutdown_connections 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.621+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c424f1b0 0x5587c4251630 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.621+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c423c8f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.621+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c423c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.621+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1664437136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 0x5587c423f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.621+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 >> 172.21.15.103:0/1664437136 conn(0x5587c4230a30 msgr2=0x5587c4236760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:21.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.622+0000 7f8e71ffb640 0 client.17880 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:21.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.627+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 shutdown_connections 2024-09-19T07:50:21.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.632+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1664437136 wait complete. 2024-09-19T07:50:21.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.633+0000 7f8e92e2e740 20 client.17880 trim_cache size 0 max 16384 2024-09-19T07:50:21.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.671+0000 7f8e92e2e740 1 Processor -- start 2024-09-19T07:50:21.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.671+0000 7f8e92e2e740 1 -- start start 2024-09-19T07:50:21.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c42344a0 0x5587c424b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c4234dc0 0x5587c423abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c423f680 0x5587c4237760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1afbf0 con 0x5587c42344a0 2024-09-19T07:50:21.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1b1760 con 0x5587c423f680 2024-09-19T07:50:21.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1b02c0 con 0x5587c4234dc0 2024-09-19T07:50:21.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c42344a0 0x5587c424b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e8ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c423f680 0x5587c4237760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c42344a0 0x5587c424b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58560/0 (socket says 172.21.15.103:58560) 2024-09-19T07:50:21.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.672+0000 7f8e8a59c640 1 -- 172.21.15.103:0/1280629477 learned_addr learned my addr 172.21.15.103:0/1280629477 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:21.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.673+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/1280629477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c4234dc0 msgr2=0x5587c423abd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:50:21.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.676+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c4234dc0 0x5587c423abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.676+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c4234dc0 0x5587c423abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.676+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/1280629477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c42344a0 msgr2=0x5587c424b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:21.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.676+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c42344a0 0x5587c424b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.676+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/1280629477 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c423b2b0 con 0x5587c423f680 2024-09-19T07:50:21.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.676+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c42344a0 0x5587c424b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:21.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.676+0000 7f8e89d9b640 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c4234dc0 0x5587c423abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:21.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.680+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c423f680 0x5587c4237760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c011bc0 tx=0x7f8e7c012d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dbdd9b9bf4c8485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:21.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.681+0000 7f8e73fff640 1 -- 172.21.15.103:0/1280629477 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c023040 con 0x5587c423f680 2024-09-19T07:50:21.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.681+0000 7f8e73fff640 1 -- 172.21.15.103:0/1280629477 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7c038070 con 0x5587c423f680 2024-09-19T07:50:21.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.681+0000 7f8e73fff640 1 -- 172.21.15.103:0/1280629477 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c027a60 con 0x5587c423f680 2024-09-19T07:50:21.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1280629477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c423f680 msgr2=0x5587c4237760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c423f680 0x5587c4237760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c011bc0 tx=0x7f8e7c012d50 comp rx=0 tx=0).stop 2024-09-19T07:50:21.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1280629477 shutdown_connections 2024-09-19T07:50:21.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c423f680 0x5587c4237760 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c4234dc0 0x5587c423abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/1280629477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c42344a0 0x5587c424b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1280629477 >> 172.21.15.103:0/1280629477 conn(0x5587c4230a30 msgr2=0x5587c4237050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1280629477 shutdown_connections 2024-09-19T07:50:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.682+0000 7f8e92e2e740 1 -- 172.21.15.103:0/1280629477 wait complete. 2024-09-19T07:50:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.683+0000 7f8e92e2e740 1 -- start start 2024-09-19T07:50:21.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.683+0000 7f8e92e2e740 1 Processor -- start 2024-09-19T07:50:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 0x5587c4247b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c4242ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e92e2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c4243400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1b1060 con 0x5587c423f680 2024-09-19T07:50:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1b0990 con 0x5587c4234dc0 2024-09-19T07:50:21.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e92e2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1adbf0 con 0x5587c42344a0 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e92e2e740 10 client.0 _ll_register_callbacks cb 0x7f8e7c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c4242ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c4242ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40310/0 (socket says 172.21.15.103:40310) 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e89d9b640 1 -- 172.21.15.103:0/2152997062 learned_addr learned my addr 172.21.15.103:0/2152997062 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.684+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c4243400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.685+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/2152997062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 msgr2=0x5587c4247b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.685+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 0x5587c4247b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.685+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/2152997062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 msgr2=0x5587c4242ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.685+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c4242ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.685+0000 7f8e8ad9d640 1 -- 172.21.15.103:0/2152997062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c423f3c0 con 0x5587c423f680 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.685+0000 7f8e8ad9d640 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c4243400 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c011a50 tx=0x7f8e7c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6aabf9c0d6aef3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.685+0000 7f8e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:50:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.686+0000 7f8e71ffb640 1 -- 172.21.15.103:0/2152997062 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c040070 con 0x5587c423f680 2024-09-19T07:50:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.686+0000 7f8e71ffb640 1 -- 172.21.15.103:0/2152997062 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f8e7c0279e0 con 0x5587c423f680 2024-09-19T07:50:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.686+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e7c038070 con 0x5587c423f680 2024-09-19T07:50:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.686+0000 7f8e71ffb640 1 -- 172.21.15.103:0/2152997062 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c023040 con 0x5587c423f680 2024-09-19T07:50:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.686+0000 7f8e92e2e740 10 client.17886 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.686+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5587c42b9a90 con 0x5587c423f680 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e71ffb640 1 -- 172.21.15.103:0/2152997062 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e7c038070 con 0x5587c423f680 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e71ffb640 1 -- 172.21.15.103:0/2152997062 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f8e7c043030 con 0x5587c423f680 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e92e2e740 10 client.17886 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e92e2e740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e92e2e740 20 client.17886 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c42b9ff0 0x5587c42bc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e92e2e740 4 client.17886 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.688+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587c41a1d60 con 0x5587c42b9ff0 2024-09-19T07:50:21.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.689+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c42b9ff0 0x5587c42bc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:21.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.699+0000 7f8e8a59c640 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c42b9ff0 0x5587c42bc3e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6885d79cb43dfd0 server_cookie=dab371ec0b2e230a in_seq=0 out_seq=0 2024-09-19T07:50:21.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.699+0000 7f8e71ffb640 10 client.17886 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:21.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.699+0000 7f8e71ffb640 1 -- 172.21.15.103:0/2152997062 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5587c41a1d60 con 0x5587c42b9ff0 2024-09-19T07:50:21.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.699+0000 7f8e71ffb640 10 client.17886 handle_command_reply: tid=0 2024-09-19T07:50:21.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 client.17886 shutdown 2024-09-19T07:50:21.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 msgr2=0x5587c4243400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c4243400 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c011a50 tx=0x7f8e7c0052f0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 shutdown_connections 2024-09-19T07:50:21.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5587c42b9ff0 0x5587c42bc3e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587c423f680 0x5587c4243400 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5587c4234dc0 0x5587c4242ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 --2- 172.21.15.103:0/2152997062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587c42344a0 0x5587c4247b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.700+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 >> 172.21.15.103:0/2152997062 conn(0x5587c4230a30 msgr2=0x5587c422f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:21.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.701+0000 7f8e71ffb640 0 client.17886 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:21.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.702+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 shutdown_connections 2024-09-19T07:50:21.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.703+0000 7f8e92e2e740 1 -- 172.21.15.103:0/2152997062 wait complete. 2024-09-19T07:50:21.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.703+0000 7f8e92e2e740 20 client.17886 trim_cache size 0 max 16384 2024-09-19T07:50:21.703 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:50:21.703 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:50:21.703 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:50:21.703 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:50:21.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e740a3430 msgr2=0x7f8e740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e740a3430 0x7f8e740a58f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0099b0 tx=0x7f8e7c005f90 comp rx=0 tx=0).stop 2024-09-19T07:50:21.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 msgr2=0x7f8e8c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:21.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c198480 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004770 tx=0x7f8e80003c10 comp rx=0 tx=0).stop 2024-09-19T07:50:21.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 shutdown_connections 2024-09-19T07:50:21.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8e740a3430 0x7f8e740a58f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e8c106f20 0x7f8e8c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e8c1027a0 0x7f8e8c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:21.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 --2- 172.21.15.103:0/2593284807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e8c101df0 0x7f8e8c198480 secure :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004770 tx=0x7f8e80003c10 comp rx=0 tx=0).stop 2024-09-19T07:50:21.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.704+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 >> 172.21.15.103:0/2593284807 conn(0x7f8e8c0fd8f0 msgr2=0x7f8e8c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:21.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.707+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 shutdown_connections 2024-09-19T07:50:21.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:21.707+0000 7f8e90a55640 1 -- 172.21.15.103:0/2593284807 wait complete. 2024-09-19T07:50:22.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:22 smithi134 ceph-mon[35414]: pgmap v386: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-09-19T07:50:22.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:22 smithi174 ceph-mon[35370]: pgmap v386: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-09-19T07:50:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:22 smithi103 ceph-mon[31201]: pgmap v386: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-09-19T07:50:22.545 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:50:22.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:50:22.901 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:23.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 -- 172.21.15.103:0/394593590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 msgr2=0x7f8500114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 --2- 172.21.15.103:0/394593590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 0x7f8500114570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009ca0 tx=0x7f84f002a950 comp rx=0 tx=0).stop 2024-09-19T07:50:23.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 -- 172.21.15.103:0/394593590 shutdown_connections 2024-09-19T07:50:23.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 --2- 172.21.15.103:0/394593590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8500114ab0 0x7f8500116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 --2- 172.21.15.103:0/394593590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 0x7f8500114570 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 --2- 172.21.15.103:0/394593590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85001091f0 0x7f85001095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 -- 172.21.15.103:0/394593590 >> 172.21.15.103:0/394593590 conn(0x7f850006e140 msgr2=0x7f850006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 -- 172.21.15.103:0/394593590 shutdown_connections 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.848+0000 7f8507234640 1 -- 172.21.15.103:0/394593590 wait complete. 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.849+0000 7f8507234640 1 Processor -- start 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.849+0000 7f8507234640 1 -- start start 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8507234640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85001091f0 0x7f85001a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8507234640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 0x7f85001a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8507234640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8500114ab0 0x7f85001a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8507234640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f850011a790 con 0x7f85001091f0 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8507234640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f850011a610 con 0x7f8500114ab0 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8507234640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f850011a910 con 0x7f8500109b10 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8504fd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85001091f0 0x7f85001a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f84fffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 0x7f85001a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8504fd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85001091f0 0x7f85001a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58596/0 (socket says 172.21.15.103:58596) 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f8504fd0640 1 -- 172.21.15.103:0/2473883178 learned_addr learned my addr 172.21.15.103:0/2473883178 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f85057d1640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8500114ab0 0x7f85001a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f84fffff640 1 -- 172.21.15.103:0/2473883178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8500114ab0 msgr2=0x7f85001a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f84fffff640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8500114ab0 0x7f85001a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f84fffff640 1 -- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85001091f0 msgr2=0x7f85001a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f84fffff640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85001091f0 0x7f85001a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:23.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f84fffff640 1 -- 172.21.15.103:0/2473883178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a11d0 con 0x7f8500109b10 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.850+0000 7f84fffff640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 0x7f85001a6e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006010 tx=0x7f84f002f5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.851+0000 7f84fdffb640 1 -- 172.21.15.103:0/2473883178 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0034040 con 0x7f8500109b10 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.851+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a1460 con 0x7f8500109b10 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.851+0000 7f84fdffb640 1 -- 172.21.15.103:0/2473883178 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f0030920 con 0x7f8500109b10 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.851+0000 7f84fdffb640 1 -- 172.21.15.103:0/2473883178 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004750 con 0x7f8500109b10 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.851+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850010a300 con 0x7f8500109b10 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.852+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500073a40 con 0x7f8500109b10 2024-09-19T07:50:23.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.854+0000 7f84fdffb640 1 -- 172.21.15.103:0/2473883178 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f84f0037020 con 0x7f8500109b10 2024-09-19T07:50:23.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.855+0000 7f84fdffb640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84ec0a33b0 0x7f84ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:23.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.855+0000 7f84fdffb640 1 -- 172.21.15.103:0/2473883178 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f84f00f0940 con 0x7f8500109b10 2024-09-19T07:50:23.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.855+0000 7f8504fd0640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84ec0a33b0 0x7f84ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:23.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.855+0000 7f8504fd0640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84ec0a33b0 0x7f84ec0a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f85001a1be0 tx=0x7f84f8005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:23.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:23.860+0000 7f84fdffb640 1 -- 172.21.15.103:0/2473883178 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84f00bd420 con 0x7f8500109b10 2024-09-19T07:50:24.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.154+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f850010a560 con 0x7f8500109b10 2024-09-19T07:50:24.155 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:24 smithi103 ceph-mon[31201]: pgmap v387: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-09-19T07:50:24.158 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:50:24.158 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":51,"btime":"2024-09-19T07:50:10:429478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:50:09.866710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:50:24.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.157+0000 7f84fdffb640 1 -- 172.21.15.103:0/2473883178 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11992 (secure 0 0 0) 0x7f84f00bcd20 con 0x7f8500109b10 2024-09-19T07:50:24.159 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 51 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84ec0a33b0 msgr2=0x7f84ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84ec0a33b0 0x7f84ec0a5870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f85001a1be0 tx=0x7f84f8005f90 comp rx=0 tx=0).stop 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 msgr2=0x7f85001a6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 0x7f85001a6e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006010 tx=0x7f84f002f5d0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 shutdown_connections 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84ec0a33b0 0x7f84ec0a5870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8500114ab0 0x7f85001a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8500109b10 0x7f85001a6e20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 --2- 172.21.15.103:0/2473883178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85001091f0 0x7f85001a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 >> 172.21.15.103:0/2473883178 conn(0x7f850006e140 msgr2=0x7f8500071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 shutdown_connections 2024-09-19T07:50:24.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:24.162+0000 7f8507234640 1 -- 172.21.15.103:0/2473883178 wait complete. 2024-09-19T07:50:24.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:24 smithi134 ceph-mon[35414]: pgmap v387: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-09-19T07:50:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:24 smithi174 ceph-mon[35370]: pgmap v387: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-09-19T07:50:24.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:50:25.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:25 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2473883178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:50:25.342 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:25.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:25 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2473883178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:50:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 710.0 s, 1770.1 tps, lat 56.158 ms stddev 57.912 2024-09-19T07:50:25.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:25 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2473883178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:50:25.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3806376312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 msgr2=0x7f5a9c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3806376312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 0x7f5a9c105920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009860 tx=0x7f5a8c02a3d0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3806376312 shutdown_connections 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3806376312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105f40 0x7f5a9c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3806376312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 0x7f5a9c105920 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3806376312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a9c100b30 0x7f5a9c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3806376312 >> 172.21.15.103:0/3806376312 conn(0x7f5a9c0fc7a0 msgr2=0x7f5a9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3806376312 shutdown_connections 2024-09-19T07:50:25.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.891+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3806376312 wait complete. 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 Processor -- start 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 -- start start 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a9c100b30 0x7f5a9c1a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 0x7f5a9c1a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105f40 0x7f5a9c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c118560 con 0x7f5a9c105f40 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1183e0 con 0x7f5a9c101450 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa39cb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1186e0 con 0x7f5a9c100b30 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa0f66640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 0x7f5a9c1a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a9c100b30 0x7f5a9c1a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa0f66640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 0x7f5a9c1a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40356/0 (socket says 172.21.15.103:40356) 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa0f66640 1 -- 172.21.15.103:0/3440013330 learned_addr learned my addr 172.21.15.103:0/3440013330 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.892+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105f40 0x7f5a9c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa1767640 1 -- 172.21.15.103:0/3440013330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 msgr2=0x7f5a9c1a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa1767640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 0x7f5a9c1a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa1767640 1 -- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105f40 msgr2=0x7f5a9c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa1767640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105f40 0x7f5a9c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa1767640 1 -- 172.21.15.103:0/3440013330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c19f0a0 con 0x7f5a9c100b30 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa1767640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a9c100b30 0x7f5a9c1a47b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d6d0 tx=0x7f5a9800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5a927fc640 1 -- 172.21.15.103:0/3440013330 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a98017070 con 0x7f5a9c100b30 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9c19f390 con 0x7f5a9c100b30 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5a927fc640 1 -- 172.21.15.103:0/3440013330 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a98010460 con 0x7f5a9c100b30 2024-09-19T07:50:25.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5a927fc640 1 -- 172.21.15.103:0/3440013330 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800f620 con 0x7f5a9c100b30 2024-09-19T07:50:25.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.893+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9c1ab680 con 0x7f5a9c100b30 2024-09-19T07:50:25.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.896+0000 7f5a927fc640 1 -- 172.21.15.103:0/3440013330 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5a9801a070 con 0x7f5a9c100b30 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.896+0000 7f5aa5da4740 1 Processor -- start 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.896+0000 7f5aa5da4740 1 -- start start 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4cb40 0x55e98de4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa5da4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e98ddb12f0 con 0x55e98de4b5d0 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa5da4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e98dd58cf0 con 0x55e98de4cb40 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa5da4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e98ddad2b0 con 0x55e98de56370 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4cb40 0x55e98de4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4cb40 0x55e98de4b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40366/0 (socket says 172.21.15.103:40366) 2024-09-19T07:50:25.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de4ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58634/0 (socket says 172.21.15.103:58634) 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 -- 172.21.15.103:0/1179752175 learned_addr learned my addr 172.21.15.103:0/1179752175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 -- 172.21.15.103:0/1179752175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 msgr2=0x55e98de58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5a927fc640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5a880a33b0 0x7f5a880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 -- 172.21.15.103:0/1179752175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 msgr2=0x55e98de4ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de4ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 -- 172.21.15.103:0/1179752175 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98dd3d3a0 con 0x55e98de4cb40 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5a927fc640 1 -- 172.21.15.103:0/3440013330 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5a980cf820 con 0x7f5a9c100b30 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5a880a33b0 0x7f5a880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.897+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4cb40 0x55e98de4b090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400a430 tx=0x7f5a94009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d6b8bc189a58f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.898+0000 7f5a877fe640 1 -- 172.21.15.103:0/1179752175 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94010c10 con 0x55e98de4cb40 2024-09-19T07:50:25.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.898+0000 7f5a877fe640 1 -- 172.21.15.103:0/1179752175 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a94014dd0 con 0x55e98de4cb40 2024-09-19T07:50:25.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.898+0000 7f5a877fe640 1 -- 172.21.15.103:0/1179752175 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9401f5a0 con 0x55e98de4cb40 2024-09-19T07:50:25.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.898+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1179752175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4cb40 msgr2=0x55e98de4b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.898+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5a880a33b0 0x7f5a880a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a9401e000 tx=0x7f5a9401f000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:25.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.898+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4cb40 0x55e98de4b090 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400a430 tx=0x7f5a94009650 comp rx=0 tx=0).stop 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1179752175 shutdown_connections 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de4ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1179752175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4cb40 0x55e98de4b090 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1179752175 >> 172.21.15.103:0/1179752175 conn(0x55e98de14310 msgr2=0x55e98de47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1179752175 shutdown_connections 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1179752175 wait complete. 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 -- start start 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.899+0000 7f5aa5da4740 1 Processor -- start 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.900+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de52fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de55d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40374/0 (socket says 172.21.15.103:40374) 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 -- 172.21.15.103:0/25617270 learned_addr learned my addr 172.21.15.103:0/25617270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de56370 0x55e98de535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e98de59070 con 0x55e98de56370 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e98dd6c060 con 0x55e98de4b5d0 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e98de591f0 con 0x55e98de4cb40 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de52fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 -- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 msgr2=0x55e98de52fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de52fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 -- 172.21.15.103:0/25617270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de56370 msgr2=0x55e98de535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de56370 0x55e98de535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 -- 172.21.15.103:0/25617270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98db223c0 con 0x55e98de4b5d0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.901+0000 7f5aa1767640 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de55d30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d6a0 tx=0x7f5a98097f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57c6eb482a1149d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5aa5da4740 10 client.0 _ll_register_callbacks cb 0x55e98dabf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5a84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5a84ff9640 1 -- 172.21.15.103:0/25617270 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a98021070 con 0x55e98de4b5d0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a94014dd0 con 0x55e98de4b5d0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5a84ff9640 1 -- 172.21.15.103:0/25617270 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5a980860e0 con 0x55e98de4b5d0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5a84ff9640 1 -- 172.21.15.103:0/25617270 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5a980cd3e0 con 0x55e98de4b5d0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5aa5da4740 10 client.27230 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=51}) -- 0x55e98de64910 con 0x55e98de4b5d0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5a84ff9640 1 -- 172.21.15.103:0/25617270 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f5a980a42d0 con 0x55e98de4b5d0 2024-09-19T07:50:25.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.902+0000 7f5aa5da4740 10 client.27230 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5aa5da4740 10 client.27230 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5aa5da4740 20 client.27230 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98de65cc0 0x55e98de680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5aa5da4740 4 client.27230 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a94014dd0 con 0x55e98de65cc0 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98de65cc0 0x55e98de680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98de65cc0 0x55e98de680f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f5a16c689b48777 server_cookie=9ff2565abe3ff6ea in_seq=0 out_seq=0 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.903+0000 7f5a84ff9640 10 client.27230 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5a84ff9640 1 -- 172.21.15.103:0/25617270 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5a94014dd0 con 0x55e98de65cc0 2024-09-19T07:50:25.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5a84ff9640 10 client.27230 handle_command_reply: tid=0 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 client.27230 shutdown 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 msgr2=0x55e98de55d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de55d30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d6a0 tx=0x7f5a98097f00 comp rx=0 tx=0).stop 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 shutdown_connections 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98de65cc0 0x55e98de680f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de56370 0x55e98de535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de52fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5a84ff9640 0 client.27230 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/25617270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de55d30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.904+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 >> 172.21.15.103:0/25617270 conn(0x55e98de14310 msgr2=0x55e98de4d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.905+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 shutdown_connections 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.905+0000 7f5aa5da4740 1 -- 172.21.15.103:0/25617270 wait complete. 2024-09-19T07:50:25.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.905+0000 7f5aa5da4740 20 client.27230 trim_cache size 0 max 16384 2024-09-19T07:50:25.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.920+0000 7f5aa5da4740 1 Processor -- start 2024-09-19T07:50:25.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.920+0000 7f5aa5da4740 1 -- start start 2024-09-19T07:50:25.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.920+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.920+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4cb40 0x55e98de52c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de62310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40390/0 (socket says 172.21.15.103:40390) 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de6da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa0f66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4cb40 0x55e98de52c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa0f66640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4cb40 0x55e98de52c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58644/0 (socket says 172.21.15.103:58644) 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 -- 172.21.15.103:0/1662325321 learned_addr learned my addr 172.21.15.103:0/1662325321 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1b3fb0 con 0x55e98de4cb40 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1b5b20 con 0x55e98de4b5d0 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1b4680 con 0x55e98de56370 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de6da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 -- 172.21.15.103:0/1662325321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 msgr2=0x55e98de6da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de6da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 -- 172.21.15.103:0/1662325321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4cb40 msgr2=0x55e98de52c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4cb40 0x55e98de52c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 -- 172.21.15.103:0/1662325321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98de48960 con 0x55e98de4b5d0 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4cb40 0x55e98de52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5aa1767640 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de62310 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a98098570 tx=0x7f5a98086000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0eb20b2d5bc69e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.921+0000 7f5a86ffd640 1 -- 172.21.15.103:0/1662325321 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9809b6e0 con 0x55e98de4b5d0 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5a86ffd640 1 -- 172.21.15.103:0/1662325321 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a980d5550 con 0x55e98de4b5d0 2024-09-19T07:50:25.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5a86ffd640 1 -- 172.21.15.103:0/1662325321 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9809aff0 con 0x55e98de4b5d0 2024-09-19T07:50:25.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 msgr2=0x55e98de62310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de62310 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a98098570 tx=0x7f5a98086000 comp rx=0 tx=0).stop 2024-09-19T07:50:25.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 shutdown_connections 2024-09-19T07:50:25.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de56370 0x55e98de6da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4cb40 0x55e98de52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/1662325321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de4b5d0 0x55e98de62310 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 >> 172.21.15.103:0/1662325321 conn(0x55e98de14310 msgr2=0x55e98de52520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 shutdown_connections 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.922+0000 7f5aa5da4740 1 -- 172.21.15.103:0/1662325321 wait complete. 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 -- start start 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 Processor -- start 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de5a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de5ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de56370 0x55e98dee09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1b5420 con 0x55e98de4b5d0 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1b4d50 con 0x55e98de56370 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c1b1fb0 con 0x55e98de4cb40 2024-09-19T07:50:25.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa5da4740 10 client.0 _ll_register_callbacks cb 0x7f5a980d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de5a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa0f66640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de5ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa1767640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de5a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58652/0 (socket says 172.21.15.103:58652) 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa1767640 1 -- 172.21.15.103:0/3221979388 learned_addr learned my addr 172.21.15.103:0/3221979388 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.923+0000 7f5aa1f68640 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de56370 0x55e98dee09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa0f66640 1 -- 172.21.15.103:0/3221979388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de56370 msgr2=0x55e98dee09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de56370 0x55e98dee09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa0f66640 1 -- 172.21.15.103:0/3221979388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 msgr2=0x55e98de5a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de5a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa0f66640 1 -- 172.21.15.103:0/3221979388 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98dee0f50 con 0x55e98de4cb40 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa0f66640 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de5ae70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005e00 tx=0x7f5a8c030e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de1ee22602b2fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:25.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5a84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5a84ff9640 1 -- 172.21.15.103:0/3221979388 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c030870 con 0x55e98de4cb40 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5a84ff9640 1 -- 172.21.15.103:0/3221979388 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5a8c004030 con 0x55e98de4cb40 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a980d5550 con 0x55e98de4cb40 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5a84ff9640 1 -- 172.21.15.103:0/3221979388 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5a8c058050 con 0x55e98de4cb40 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa5da4740 10 client.27193 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e98dee2290 con 0x55e98de4cb40 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.924+0000 7f5a84ff9640 1 -- 172.21.15.103:0/3221979388 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a980d5550 con 0x55e98de4cb40 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5a84ff9640 1 -- 172.21.15.103:0/3221979388 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f5a8c04f070 con 0x55e98de4cb40 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa5da4740 10 client.27193 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa5da4740 10 client.27193 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa5da4740 20 client.27193 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:25.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98dee29b0 0x55e98dee4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa5da4740 4 client.27193 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e98ddbe860 con 0x55e98dee29b0 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa1767640 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98dee29b0 0x55e98dee4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5a84ff9640 10 client.27193 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa1767640 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98dee29b0 0x55e98dee4dd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8203514147c5405 server_cookie=ccdb7950068863bf in_seq=0 out_seq=0 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5a84ff9640 1 -- 172.21.15.103:0/3221979388 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e98ddbe860 con 0x55e98dee29b0 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5a84ff9640 10 client.27193 handle_command_reply: tid=0 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.925+0000 7f5aa5da4740 1 client.27193 shutdown 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 msgr2=0x55e98de5ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de5ae70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c005e00 tx=0x7f5a8c030e80 comp rx=0 tx=0).stop 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 shutdown_connections 2024-09-19T07:50:25.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55e98dee29b0 0x55e98dee4dd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e98de56370 0x55e98dee09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e98de4cb40 0x55e98de5ae70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5a84ff9640 0 client.27193 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 --2- 172.21.15.103:0/3221979388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e98de4b5d0 0x55e98de5a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 >> 172.21.15.103:0/3221979388 conn(0x55e98de14310 msgr2=0x55e98de52d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 shutdown_connections 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 1 -- 172.21.15.103:0/3221979388 wait complete. 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.926+0000 7f5aa5da4740 20 client.27193 trim_cache size 0 max 16384 2024-09-19T07:50:25.934 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:50:25.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5a880a33b0 msgr2=0x7f5a880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5a880a33b0 0x7f5a880a5870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a9401e000 tx=0x7f5a9401f000 comp rx=0 tx=0).stop 2024-09-19T07:50:25.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a9c100b30 msgr2=0x7f5a9c1a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a9c100b30 0x7f5a9c1a47b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d6d0 tx=0x7f5a9800db90 comp rx=0 tx=0).stop 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 shutdown_connections 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5a880a33b0 0x7f5a880a5870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105f40 0x7f5a9c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c101450 0x7f5a9c1a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 --2- 172.21.15.103:0/3440013330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a9c100b30 0x7f5a9c1a47b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 >> 172.21.15.103:0/3440013330 conn(0x7f5a9c0fc7a0 msgr2=0x7f5a9c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 shutdown_connections 2024-09-19T07:50:25.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:25.927+0000 7f5aa39cb640 1 -- 172.21.15.103:0/3440013330 wait complete. 2024-09-19T07:50:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:26 smithi174 ceph-mon[35370]: pgmap v388: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.9 MiB/s wr, 636 op/s 2024-09-19T07:50:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:26 smithi103 ceph-mon[31201]: pgmap v388: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.9 MiB/s wr, 636 op/s 2024-09-19T07:50:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:26 smithi134 ceph-mon[35414]: pgmap v388: 97 pgs: 97 active+clean; 3.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.9 MiB/s wr, 636 op/s 2024-09-19T07:50:27.180 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:50:27.513 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.019+0000 7f06fe1e9640 1 -- 172.21.15.103:0/2063431090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f806ea30 msgr2=0x7f06f806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.019+0000 7f06fe1e9640 1 --2- 172.21.15.103:0/2063431090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f806ea30 0x7f06f806ee70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009900 tx=0x7f06f402a420 comp rx=0 tx=0).stop 2024-09-19T07:50:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.021+0000 7f06fe1e9640 1 -- 172.21.15.103:0/2063431090 shutdown_connections 2024-09-19T07:50:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.021+0000 7f06fe1e9640 1 --2- 172.21.15.103:0/2063431090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f811b140 0x7f06f811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.021+0000 7f06fe1e9640 1 --2- 172.21.15.103:0/2063431090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f806ea30 0x7f06f806ee70 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.021+0000 7f06fe1e9640 1 --2- 172.21.15.103:0/2063431090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f8110190 0x7f06f806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.021+0000 7f06fe1e9640 1 -- 172.21.15.103:0/2063431090 >> 172.21.15.103:0/2063431090 conn(0x7f06f8073180 msgr2=0x7f06f8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.025+0000 7f06fe1e9640 1 -- 172.21.15.103:0/2063431090 shutdown_connections 2024-09-19T07:50:28.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.025+0000 7f06fe1e9640 1 -- 172.21.15.103:0/2063431090 wait complete. 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 Processor -- start 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 -- start start 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f806ea30 0x7f06f8118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f8110190 0x7f06f81109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f811b140 0x7f06f8110f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06f8120c60 con 0x7f06f806ea30 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06f8120ae0 con 0x7f06f811b140 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.026+0000 7f06fe1e9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06f8120de0 con 0x7f06f8110190 2024-09-19T07:50:28.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.027+0000 7f06fd9e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f811b140 0x7f06f8110f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.027+0000 7f06fd9e8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f811b140 0x7f06f8110f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40408/0 (socket says 172.21.15.103:40408) 2024-09-19T07:50:28.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.027+0000 7f06fd9e8640 1 -- 172.21.15.103:0/2261443470 learned_addr learned my addr 172.21.15.103:0/2261443470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:28.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.029+0000 7f06fd9e8640 1 -- 172.21.15.103:0/2261443470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f8110190 msgr2=0x7f06f81109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.029+0000 7f06fd9e8640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f8110190 0x7f06f81109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.029+0000 7f06fd9e8640 1 -- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f806ea30 msgr2=0x7f06f8118a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T07:50:28.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.029+0000 7f06fd9e8640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f806ea30 0x7f06f8118a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.029+0000 7f06fd9e8640 1 -- 172.21.15.103:0/2261443470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f8111750 con 0x7f06f811b140 2024-09-19T07:50:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.031+0000 7f06fd9e8640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f811b140 0x7f06f8110f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f06f002fa20 tx=0x7f06f002fee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.033+0000 7f06ea7fc640 1 -- 172.21.15.103:0/2261443470 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0039070 con 0x7f06f811b140 2024-09-19T07:50:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.033+0000 7f06fe1e9640 1 -- 172.21.15.103:0/2261443470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f81a6fa0 con 0x7f06f811b140 2024-09-19T07:50:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.033+0000 7f06fe1e9640 1 -- 172.21.15.103:0/2261443470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f81a74e0 con 0x7f06f811b140 2024-09-19T07:50:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.034+0000 7f06ea7fc640 1 -- 172.21.15.103:0/2261443470 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0003e50 con 0x7f06f811b140 2024-09-19T07:50:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.034+0000 7f06ea7fc640 1 -- 172.21.15.103:0/2261443470 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0032470 con 0x7f06f811b140 2024-09-19T07:50:28.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 Processor -- start 2024-09-19T07:50:28.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 -- start start 2024-09-19T07:50:28.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139853d90 0x55a139852450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139852990 0x55a139852df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a1397b82f0 con 0x55a13985d730 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a13975fcf0 con 0x55a139852990 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.039+0000 7f0701824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a1397b42b0 con 0x55a139853d90 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fc9e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fc9e6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58666/0 (socket says 172.21.15.103:58666) 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fc9e6640 1 -- 172.21.15.103:0/3461722449 learned_addr learned my addr 172.21.15.103:0/3461722449 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139853d90 0x55a139852450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139852990 0x55a139852df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fd1e7640 1 -- 172.21.15.103:0/3461722449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139853d90 msgr2=0x55a139852450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139853d90 0x55a139852450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fd1e7640 1 -- 172.21.15.103:0/3461722449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 msgr2=0x55a13985fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.040+0000 7f06fd1e7640 1 -- 172.21.15.103:0/3461722449 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1397443a0 con 0x55a139852990 2024-09-19T07:50:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.041+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139852990 0x55a139852df0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009690 tx=0x7f06ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc441843e442a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.041+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139853d90 0x55a139852450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.042+0000 7f06ea7fc640 1 -- 172.21.15.103:0/2261443470 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f06f0032610 con 0x7f06f811b140 2024-09-19T07:50:28.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.042+0000 7f06fe1e9640 1 -- 172.21.15.103:0/3461722449 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec015020 con 0x55a139852990 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.043+0000 7f06fe1e9640 1 -- 172.21.15.103:0/3461722449 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec00bda0 con 0x55a139852990 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.043+0000 7f06fe1e9640 1 -- 172.21.15.103:0/3461722449 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec010380 con 0x55a139852990 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.043+0000 7f06ea7fc640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f06e40a35c0 0x7f06e40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.043+0000 7f0701824740 1 -- 172.21.15.103:0/3461722449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139852990 msgr2=0x55a139852df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.043+0000 7f0701824740 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139852990 0x55a139852df0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009690 tx=0x7f06ec009b50 comp rx=0 tx=0).stop 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f06ea7fc640 1 -- 172.21.15.103:0/2261443470 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f06f00f1fa0 con 0x7f06f811b140 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f0701824740 1 -- 172.21.15.103:0/3461722449 shutdown_connections 2024-09-19T07:50:28.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f0701824740 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f0701824740 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139852990 0x55a139852df0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f0701824740 1 --2- 172.21.15.103:0/3461722449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139853d90 0x55a139852450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f0701824740 1 -- 172.21.15.103:0/3461722449 >> 172.21.15.103:0/3461722449 conn(0x55a13981b310 msgr2=0x55a13984f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f06e40a35c0 0x7f06e40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.044+0000 7f0701824740 1 -- 172.21.15.103:0/3461722449 shutdown_connections 2024-09-19T07:50:28.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.045+0000 7f0701824740 1 -- 172.21.15.103:0/3461722449 wait complete. 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.045+0000 7f0701824740 1 -- start start 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.045+0000 7f0701824740 1 Processor -- start 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f0701824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a13985d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f0701824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a13985a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f0701824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f0701824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a139860240 con 0x55a13985d730 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f0701824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a139773060 con 0x55a139853d90 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f0701824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a1398603c0 con 0x55a139852990 2024-09-19T07:50:28.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f0701824740 10 client.0 _ll_register_callbacks cb 0x55a1394c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f06fd1e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a13985d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f06fd1e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a13985d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54190/0 (socket says 172.21.15.103:54190) 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.046+0000 7f06fd1e7640 1 -- 172.21.15.103:0/246457973 learned_addr learned my addr 172.21.15.103:0/246457973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.047+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a13985a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.047+0000 7f06fd1e7640 1 -- 172.21.15.103:0/246457973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 msgr2=0x55a13985a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.047+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a13985a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.047+0000 7f06fd1e7640 1 -- 172.21.15.103:0/246457973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 msgr2=0x55a13985a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.047+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.047+0000 7f06fd1e7640 1 -- 172.21.15.103:0/246457973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1395293c0 con 0x55a139852990 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.047+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a13985d000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011950 tx=0x7f06ec009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a455b72f6021f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.048+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f06e40a35c0 0x7f06e40a5a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06f402aa10 tx=0x7f06f40051d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.048+0000 7f06e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.048+0000 7f06e0ff9640 1 -- 172.21.15.103:0/246457973 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec015030 con 0x55a139852990 2024-09-19T07:50:28.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.048+0000 7f06e0ff9640 1 -- 172.21.15.103:0/246457973 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f06ec00b8c0 con 0x55a139852990 2024-09-19T07:50:28.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.048+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06ec00bda0 con 0x55a139852990 2024-09-19T07:50:28.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.049+0000 7f06e0ff9640 1 -- 172.21.15.103:0/246457973 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f06ec020070 con 0x55a139852990 2024-09-19T07:50:28.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.049+0000 7f0701824740 10 client.27199 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:28.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.050+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a13986bba0 con 0x55a139852990 2024-09-19T07:50:28.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.054+0000 7f06e0ff9640 1 -- 172.21.15.103:0/246457973 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec00bda0 con 0x55a139852990 2024-09-19T07:50:28.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.054+0000 7f06e0ff9640 1 -- 172.21.15.103:0/246457973 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f06ec0039f0 con 0x55a139852990 2024-09-19T07:50:28.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.055+0000 7f0701824740 10 client.27199 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:28.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.055+0000 7f0701824740 10 client.27199 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:50:28.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.055+0000 7f0701824740 20 client.27199 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:28.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.055+0000 7f0701824740 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a13986cf50 0x55a13986f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.055+0000 7f0701824740 4 client.27199 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:28.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.055+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1397c5860 con 0x55a13986cf50 2024-09-19T07:50:28.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.055+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a13986cf50 0x55a13986f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.056+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a13986cf50 0x55a13986f340 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6697e6f5631a150 server_cookie=9f7b2abb207abbd5 in_seq=0 out_seq=0 2024-09-19T07:50:28.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.056+0000 7f06e0ff9640 10 client.27199 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:50:28.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.057+0000 7f06e0ff9640 1 -- 172.21.15.103:0/246457973 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1397c5860 con 0x55a13986cf50 2024-09-19T07:50:28.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.057+0000 7f06e0ff9640 10 client.27199 handle_command_reply: tid=0 2024-09-19T07:50:28.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.057+0000 7f0701824740 1 client.27199 shutdown 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 msgr2=0x55a13985d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a13985d000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011950 tx=0x7f06ec009370 comp rx=0 tx=0).stop 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 shutdown_connections 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a13986cf50 0x55a13986f340 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a13985a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 --2- 172.21.15.103:0/246457973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a13985d000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 >> 172.21.15.103:0/246457973 conn(0x55a13981b310 msgr2=0x55a13984e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f06e0ff9640 0 client.27199 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:50:28.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 shutdown_connections 2024-09-19T07:50:28.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 1 -- 172.21.15.103:0/246457973 wait complete. 2024-09-19T07:50:28.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.058+0000 7f0701824740 20 client.27199 trim_cache size 0 max 16384 2024-09-19T07:50:28.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 Processor -- start 2024-09-19T07:50:28.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 -- start start 2024-09-19T07:50:28.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139869540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139859290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81afef0 con 0x55a13985d730 2024-09-19T07:50:28.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81b1a60 con 0x55a139853d90 2024-09-19T07:50:28.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f0701824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81b05c0 con 0x55a139852990 2024-09-19T07:50:28.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f06fd1e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139869540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f06fd9e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f06fd1e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139869540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:54198/0 (socket says 172.21.15.103:54198) 2024-09-19T07:50:28.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f06fd1e7640 1 -- 172.21.15.103:0/2830755369 learned_addr learned my addr 172.21.15.103:0/2830755369 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:28.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.077+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139859290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fd1e7640 1 -- 172.21.15.103:0/2830755369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 msgr2=0x55a139859290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139859290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fd1e7640 1 -- 172.21.15.103:0/2830755369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 msgr2=0x55a13985eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fd1e7640 1 -- 172.21.15.103:0/2830755369 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a139860180 con 0x55a139852990 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139859290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fd9e8640 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.078+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139869540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000c00 tx=0x7f06ec011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd2fbf521ebe412 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.079+0000 7f06e2ffd640 1 -- 172.21.15.103:0/2830755369 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec015070 con 0x55a139852990 2024-09-19T07:50:28.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.079+0000 7f06e2ffd640 1 -- 172.21.15.103:0/2830755369 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec01ad80 con 0x55a139852990 2024-09-19T07:50:28.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.079+0000 7f06e2ffd640 1 -- 172.21.15.103:0/2830755369 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec004210 con 0x55a139852990 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 -- 172.21.15.103:0/2830755369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 msgr2=0x55a139869540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139869540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000c00 tx=0x7f06ec011af0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 -- 172.21.15.103:0/2830755369 shutdown_connections 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a13985eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139859290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 --2- 172.21.15.103:0/2830755369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139869540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 -- 172.21.15.103:0/2830755369 >> 172.21.15.103:0/2830755369 conn(0x55a13981b310 msgr2=0x55a139858b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 -- 172.21.15.103:0/2830755369 shutdown_connections 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 -- 172.21.15.103:0/2830755369 wait complete. 2024-09-19T07:50:28.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.080+0000 7f0701824740 1 -- start start 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 1 Processor -- start 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139873c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139874190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a1398646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81b1360 con 0x55a13985d730 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81b0c90 con 0x55a139853d90 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81adef0 con 0x55a139852990 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f0701824740 10 client.0 _ll_register_callbacks cb 0x7f06ec009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f06fc9e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139874190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f06fc9e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139874190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40438/0 (socket says 172.21.15.103:40438) 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f06fc9e6640 1 -- 172.21.15.103:0/2442450262 learned_addr learned my addr 172.21.15.103:0/2442450262 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:28.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.081+0000 7f06fd9e8640 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a1398646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139873c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06fc9e6640 1 -- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 msgr2=0x55a139873c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139873c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06fc9e6640 1 -- 172.21.15.103:0/2442450262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 msgr2=0x55a1398646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a1398646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06fc9e6640 1 -- 172.21.15.103:0/2442450262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a139864bf0 con 0x55a139853d90 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06fc9e6640 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139874190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009900 tx=0x7f06f4040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93fca5a11004d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06e0ff9640 1 -- 172.21.15.103:0/2442450262 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4031d10 con 0x55a139853d90 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f06e0ff9640 1 -- 172.21.15.103:0/2442450262 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f06f40046c0 con 0x55a139853d90 2024-09-19T07:50:28.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.082+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06ec01ad80 con 0x55a139853d90 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f06e0ff9640 1 -- 172.21.15.103:0/2442450262 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f06f40048c0 con 0x55a139853d90 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 10 client.27251 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a139865f20 con 0x55a139853d90 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f06e0ff9640 1 -- 172.21.15.103:0/2442450262 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f4004f40 con 0x55a139853d90 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f06e0ff9640 1 -- 172.21.15.103:0/2442450262 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f06f403a040 con 0x55a139853d90 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 10 client.27251 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 10 client.27251 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:50:28.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 20 client.27251 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a1398d7af0 0x55a1398d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 4 client.27251 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f06ec01ad80 con 0x55a1398d7af0 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.090+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a1398d7af0 0x55a1398d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f06e0ff9640 10 client.27251 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f06fd1e7640 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a1398d7af0 0x55a1398d9f10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6f3ef0395d499f4 server_cookie=1729c96994258e68 in_seq=0 out_seq=0 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f06e0ff9640 1 -- 172.21.15.103:0/2442450262 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f06ec01ad80 con 0x55a1398d7af0 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f06e0ff9640 10 client.27251 handle_command_reply: tid=0 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 client.27251 shutdown 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 msgr2=0x55a139874190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139874190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009900 tx=0x7f06f4040040 comp rx=0 tx=0).stop 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 shutdown_connections 2024-09-19T07:50:28.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55a1398d7af0 0x55a1398d9f10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a13985d730 0x55a1398646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a139853d90 0x55a139874190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.091+0000 7f0701824740 1 --2- 172.21.15.103:0/2442450262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a139852990 0x55a139873c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 >> 172.21.15.103:0/2442450262 conn(0x55a13981b310 msgr2=0x55a1398593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f06e0ff9640 0 client.27251 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 shutdown_connections 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f0701824740 1 -- 172.21.15.103:0/2442450262 wait complete. 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f0701824740 20 client.27251 trim_cache size 0 max 16384 2024-09-19T07:50:28.094 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f06e3fff640 1 -- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f06e40a35c0 msgr2=0x7f06e40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f06e3fff640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f06e40a35c0 0x7f06e40a5a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06f402aa10 tx=0x7f06f40051d0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f06e3fff640 1 -- 172.21.15.103:0/2261443470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f811b140 msgr2=0x7f06f8110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.092+0000 7f06e3fff640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f811b140 0x7f06f8110f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f06f002fa20 tx=0x7f06f002fee0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 -- 172.21.15.103:0/2261443470 shutdown_connections 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f06e40a35c0 0x7f06e40a5a80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06f811b140 0x7f06f8110f30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f8110190 0x7f06f81109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 --2- 172.21.15.103:0/2261443470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06f806ea30 0x7f06f8118a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 -- 172.21.15.103:0/2261443470 >> 172.21.15.103:0/2261443470 conn(0x7f06f8073180 msgr2=0x7f06f8119370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:28.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 -- 172.21.15.103:0/2261443470 shutdown_connections 2024-09-19T07:50:28.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:28.093+0000 7f06e3fff640 1 -- 172.21.15.103:0/2261443470 wait complete. 2024-09-19T07:50:28.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:28 smithi103 ceph-mon[31201]: pgmap v389: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-09-19T07:50:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:28 smithi174 ceph-mon[35370]: pgmap v389: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-09-19T07:50:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:28 smithi134 ceph-mon[35414]: pgmap v389: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-09-19T07:50:29.323 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:50:29.667 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:30.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 -- 172.21.15.103:0/2788087088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 msgr2=0x7fe4c0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 --2- 172.21.15.103:0/2788087088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c0100f00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009900 tx=0x7fe4b002a480 comp rx=0 tx=0).stop 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 -- 172.21.15.103:0/2788087088 shutdown_connections 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 --2- 172.21.15.103:0/2788087088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0105f30 0x7fe4c01063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 --2- 172.21.15.103:0/2788087088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c0101440 0x7fe4c0105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 --2- 172.21.15.103:0/2788087088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c0100f00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 -- 172.21.15.103:0/2788087088 >> 172.21.15.103:0/2788087088 conn(0x7fe4c00fc7d0 msgr2=0x7fe4c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 -- 172.21.15.103:0/2788087088 shutdown_connections 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.251+0000 7fe4c6596640 1 -- 172.21.15.103:0/2788087088 wait complete. 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.252+0000 7fe4c6596640 1 Processor -- start 2024-09-19T07:50:30.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.252+0000 7fe4c6596640 1 -- start start 2024-09-19T07:50:30.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.252+0000 7fe4c6596640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c01a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4c6596640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0101440 0x7fe4c01a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4c6596640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c0105f30 0x7fe4c019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4c6596640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0118560 con 0x7fe4c0101440 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4c6596640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01183e0 con 0x7fe4c0100b20 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4c6596640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01186e0 con 0x7fe4c0105f30 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c01a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0101440 0x7fe4c01a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c01a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54084/0 (socket says 172.21.15.103:54084) 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0101440 0x7fe4c01a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49356/0 (socket says 172.21.15.103:49356) 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bffff640 1 -- 172.21.15.103:0/1841633556 learned_addr learned my addr 172.21.15.103:0/1841633556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c0105f30 0x7fe4c019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/1841633556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c0105f30 msgr2=0x7fe4c019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c0105f30 0x7fe4c019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/1841633556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 msgr2=0x7fe4c01a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c01a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/1841633556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c019f060 con 0x7fe4c0101440 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bffff640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c01a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0101440 0x7fe4c01a4cb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9f0 tx=0x7fe4b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4bd7fa640 1 -- 172.21.15.103:0/1841633556 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400ccc0 con 0x7fe4c0101440 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.253+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c019f350 con 0x7fe4c0101440 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.254+0000 7fe4bd7fa640 1 -- 172.21.15.103:0/1841633556 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b40035f0 con 0x7fe4c0101440 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.254+0000 7fe4bd7fa640 1 -- 172.21.15.103:0/1841633556 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400ce60 con 0x7fe4c0101440 2024-09-19T07:50:30.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.254+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c01ab640 con 0x7fe4c0101440 2024-09-19T07:50:30.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 Processor -- start 2024-09-19T07:50:30.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 -- start start 2024-09-19T07:50:30.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6022090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b6022a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b602d370 0x55d0b602f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0b5f882f0 con 0x55d0b60225d0 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0b5f2fcf0 con 0x55d0b602d370 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4c896f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0b5f842b0 con 0x55d0b6023b40 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4bd7fa640 1 -- 172.21.15.103:0/1841633556 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe4b40037a0 con 0x7fe4c0101440 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.257+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b6022a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b6022a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49370/0 (socket says 172.21.15.103:49370) 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4bffff640 1 -- 172.21.15.103:0/1729787787 learned_addr learned my addr 172.21.15.103:0/1729787787 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6022090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4c4b33640 1 -- 172.21.15.103:0/1729787787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b602d370 msgr2=0x55d0b602f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b602d370 0x55d0b602f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4c4b33640 1 -- 172.21.15.103:0/1729787787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 msgr2=0x55d0b6022a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b6022a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4c4b33640 1 -- 172.21.15.103:0/1729787787 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b5f143a0 con 0x55d0b6023b40 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6022090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009f70 tx=0x7fe4b8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720efca823a0bdcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4bd7fa640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe4a80a3430 0x7fe4a80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4a67fc640 1 -- 172.21.15.103:0/1729787787 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8013850 con 0x55d0b6023b40 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4a67fc640 1 -- 172.21.15.103:0/1729787787 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b8019070 con 0x55d0b6023b40 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4a67fc640 1 -- 172.21.15.103:0/1729787787 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8014e60 con 0x55d0b6023b40 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4bd7fa640 1 -- 172.21.15.103:0/1841633556 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fe4b40d0020 con 0x7fe4c0101440 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.258+0000 7fe4bffff640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe4a80a3430 0x7fe4a80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 -- 172.21.15.103:0/1729787787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 msgr2=0x55d0b6022090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6022090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009f70 tx=0x7fe4b8009650 comp rx=0 tx=0).stop 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4bffff640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe4a80a3430 0x7fe4a80a58f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55d0b601fac0 tx=0x7fe4b0005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 -- 172.21.15.103:0/1729787787 shutdown_connections 2024-09-19T07:50:30.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b602d370 0x55d0b602f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b6022a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 --2- 172.21.15.103:0/1729787787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6022090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 -- 172.21.15.103:0/1729787787 >> 172.21.15.103:0/1729787787 conn(0x55d0b5feb310 msgr2=0x55d0b601ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 -- 172.21.15.103:0/1729787787 shutdown_connections 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.259+0000 7fe4c896f740 1 -- 172.21.15.103:0/1729787787 wait complete. 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.260+0000 7fe4c896f740 1 -- start start 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.260+0000 7fe4c896f740 1 Processor -- start 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.260+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b60225d0 0x55d0b602cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6029f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b602d370 0x55d0b602a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c896f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0b6030070 con 0x55d0b602d370 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c896f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0b5f43060 con 0x55d0b60225d0 2024-09-19T07:50:30.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c896f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0b60301f0 con 0x55d0b6023b40 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b60225d0 0x55d0b602cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b60225d0 0x55d0b602cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54100/0 (socket says 172.21.15.103:54100) 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bffff640 1 -- 172.21.15.103:0/2844815859 learned_addr learned my addr 172.21.15.103:0/2844815859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6029f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b602d370 0x55d0b602a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c896f740 10 client.0 _ll_register_callbacks cb 0x55d0b5c968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/2844815859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b60225d0 msgr2=0x55d0b602cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b60225d0 0x55d0b602cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/2844815859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b602d370 msgr2=0x55d0b602a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b602d370 0x55d0b602a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/2844815859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b5cf93c0 con 0x55d0b6023b40 2024-09-19T07:50:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6029f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40cd700 tx=0x7fe4b409be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709ed97857354219 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe49f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe49f7fe640 1 -- 172.21.15.103:0/2844815859 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40b4020 con 0x55d0b6023b40 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b8019070 con 0x55d0b6023b40 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.261+0000 7fe49f7fe640 1 -- 172.21.15.103:0/2844815859 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fe4b40cd980 con 0x55d0b6023b40 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe49f7fe640 1 -- 172.21.15.103:0/2844815859 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe4b40b5250 con 0x55d0b6023b40 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 10 client.27217 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d0b603bb20 con 0x55d0b6023b40 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe49f7fe640 1 -- 172.21.15.103:0/2844815859 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b8019070 con 0x55d0b6023b40 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe49f7fe640 1 -- 172.21.15.103:0/2844815859 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fe4b40a0170 con 0x55d0b6023b40 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 10 client.27217 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 10 client.27217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 20 client.27217 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:30.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b603cf30 0x55d0b603f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 4 client.27217 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:30.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b5f95860 con 0x55d0b603cf30 2024-09-19T07:50:30.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.262+0000 7fe4bffff640 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b603cf30 0x55d0b603f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.263+0000 7fe4bffff640 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b603cf30 0x55d0b603f320 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b867805bc49cf8f0 server_cookie=155d82df07004907 in_seq=0 out_seq=0 2024-09-19T07:50:30.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.263+0000 7fe49f7fe640 10 client.27217 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:50:30.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.263+0000 7fe49f7fe640 1 -- 172.21.15.103:0/2844815859 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0b5f95860 con 0x55d0b603cf30 2024-09-19T07:50:30.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.263+0000 7fe49f7fe640 10 client.27217 handle_command_reply: tid=0 2024-09-19T07:50:30.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 client.27217 shutdown 2024-09-19T07:50:30.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 msgr2=0x55d0b6029f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6029f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40cd700 tx=0x7fe4b409be90 comp rx=0 tx=0).stop 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 shutdown_connections 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b603cf30 0x55d0b603f320 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b602d370 0x55d0b602a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b6023b40 0x55d0b6029f90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2844815859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b60225d0 0x55d0b602cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 >> 172.21.15.103:0/2844815859 conn(0x55d0b5feb310 msgr2=0x55d0b6024930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe49f7fe640 0 client.27217 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 shutdown_connections 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.266+0000 7fe4c896f740 1 -- 172.21.15.103:0/2844815859 wait complete. 2024-09-19T07:50:30.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.267+0000 7fe4c896f740 20 client.27217 trim_cache size 0 max 16384 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 Processor -- start 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 -- start start 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b60393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6025120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6025660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01b4040 con 0x55d0b60225d0 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01b5bb0 con 0x55d0b6023b40 2024-09-19T07:50:30.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.282+0000 7fe4c896f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01b4710 con 0x55d0b602d370 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b60393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b60393a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49380/0 (socket says 172.21.15.103:49380) 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 -- 172.21.15.103:0/2987483437 learned_addr learned my addr 172.21.15.103:0/2987483437 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6025120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6025660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 -- 172.21.15.103:0/2987483437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 msgr2=0x55d0b6025660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6025660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 -- 172.21.15.103:0/2987483437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 msgr2=0x55d0b6025120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6025120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 -- 172.21.15.103:0/2987483437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b6025ba0 con 0x55d0b60225d0 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6025120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:30.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4bffff640 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b60393a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0000c00 tx=0x7fe4b0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9effa104454c4770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4a5ffb640 1 -- 172.21.15.103:0/2987483437 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0035040 con 0x55d0b60225d0 2024-09-19T07:50:30.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4a5ffb640 1 -- 172.21.15.103:0/2987483437 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b0058030 con 0x55d0b60225d0 2024-09-19T07:50:30.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.283+0000 7fe4a5ffb640 1 -- 172.21.15.103:0/2987483437 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0003bc0 con 0x55d0b60225d0 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.285+0000 7fe4c896f740 1 -- 172.21.15.103:0/2987483437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 msgr2=0x55d0b60393a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.285+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b60393a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0000c00 tx=0x7fe4b0040000 comp rx=0 tx=0).stop 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.285+0000 7fe4c896f740 1 -- 172.21.15.103:0/2987483437 shutdown_connections 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.285+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6025660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.285+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6025120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.285+0000 7fe4c896f740 1 --2- 172.21.15.103:0/2987483437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b60225d0 0x55d0b60393a0 secure :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0000c00 tx=0x7fe4b0040000 comp rx=0 tx=0).stop 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.285+0000 7fe4c896f740 1 -- 172.21.15.103:0/2987483437 >> 172.21.15.103:0/2987483437 conn(0x55d0b5feb310 msgr2=0x55d0b6029080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.286+0000 7fe4c896f740 1 -- 172.21.15.103:0/2987483437 shutdown_connections 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.286+0000 7fe4c896f740 1 -- 172.21.15.103:0/2987483437 wait complete. 2024-09-19T07:50:30.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.286+0000 7fe4c896f740 1 -- start start 2024-09-19T07:50:30.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 1 Processor -- start 2024-09-19T07:50:30.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6035af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6026ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b6027000 0x55d0b6027480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01b54b0 con 0x55d0b6027000 2024-09-19T07:50:30.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01b4de0 con 0x55d0b6023b40 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c01b2040 con 0x55d0b602d370 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c896f740 10 client.0 _ll_register_callbacks cb 0x7fe4b409c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6026ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6026ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38068/0 (socket says 172.21.15.103:38068) 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/978177960 learned_addr learned my addr 172.21.15.103:0/978177960 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4bffff640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6035af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.287+0000 7fe4c4b33640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b6027000 0x55d0b6027480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 msgr2=0x55d0b6035af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6035af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/978177960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b6027000 msgr2=0x55d0b6027480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b6027000 0x55d0b6027480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4bf7fe640 1 -- 172.21.15.103:0/978177960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b60a6b50 con 0x55d0b602d370 2024-09-19T07:50:30.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4bffff640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6035af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4bf7fe640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6026ac0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40986f0 tx=0x7fe4b409c110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1634d9d876c11a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe49f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe49f7fe640 1 -- 172.21.15.103:0/978177960 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40b4070 con 0x55d0b602d370 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b0058030 con 0x55d0b602d370 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe49f7fe640 1 -- 172.21.15.103:0/978177960 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fe4b40a0170 con 0x55d0b602d370 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe49f7fe640 1 -- 172.21.15.103:0/978177960 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe4b4098530 con 0x55d0b602d370 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4c896f740 10 client.27223 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d0b60a7dc0 con 0x55d0b602d370 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.288+0000 7fe49f7fe640 1 -- 172.21.15.103:0/978177960 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b0058030 con 0x55d0b602d370 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe49f7fe640 1 -- 172.21.15.103:0/978177960 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fe4b40850f0 con 0x55d0b602d370 2024-09-19T07:50:30.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4c896f740 10 client.27223 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4c896f740 10 client.27223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4c896f740 20 client.27223 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4c896f740 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b60a83b0 0x55d0b60aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4c896f740 4 client.27223 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0b5f8fd70 con 0x55d0b60a83b0 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4bffff640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b60a83b0 0x55d0b60aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe4bffff640 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b60a83b0 0x55d0b60aa7d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bbcc0bb1b3006ce server_cookie=de1b641f382a83ec in_seq=0 out_seq=0 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe49f7fe640 10 client.27223 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe49f7fe640 1 -- 172.21.15.103:0/978177960 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0b5f8fd70 con 0x55d0b60a83b0 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.289+0000 7fe49f7fe640 10 client.27223 handle_command_reply: tid=0 2024-09-19T07:50:30.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 client.27223 shutdown 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 msgr2=0x55d0b6026ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6026ac0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40986f0 tx=0x7fe4b409c110 comp rx=0 tx=0).stop 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 shutdown_connections 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55d0b60a83b0 0x55d0b60aa7d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0b6027000 0x55d0b6027480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe49f7fe640 0 client.27223 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0b602d370 0x55d0b6026ac0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 --2- 172.21.15.103:0/978177960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b6023b40 0x55d0b6035af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 >> 172.21.15.103:0/978177960 conn(0x55d0b5feb310 msgr2=0x55d0b601d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 shutdown_connections 2024-09-19T07:50:30.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 1 -- 172.21.15.103:0/978177960 wait complete. 2024-09-19T07:50:30.296 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:50:30.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c896f740 20 client.27223 trim_cache size 0 max 16384 2024-09-19T07:50:30.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe4a80a3430 msgr2=0x7fe4a80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c6596640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe4a80a3430 0x7fe4a80a58f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55d0b601fac0 tx=0x7fe4b0005a20 comp rx=0 tx=0).stop 2024-09-19T07:50:30.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0101440 msgr2=0x7fe4c01a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:30.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.290+0000 7fe4c6596640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0101440 0x7fe4c01a4cb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e9f0 tx=0x7fe4b400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 shutdown_connections 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe4a80a3430 0x7fe4a80a58f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4c0105f30 0x7fe4c019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4c0101440 0x7fe4c01a4cb0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 --2- 172.21.15.103:0/1841633556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4c0100b20 0x7fe4c01a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 >> 172.21.15.103:0/1841633556 conn(0x7fe4c00fc7d0 msgr2=0x7fe4c010f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 shutdown_connections 2024-09-19T07:50:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:30.291+0000 7fe4c6596640 1 -- 172.21.15.103:0/1841633556 wait complete. 2024-09-19T07:50:30.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:30 smithi134 ceph-mon[35414]: pgmap v390: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-19T07:50:30.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:30 smithi174 ceph-mon[35370]: pgmap v390: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-19T07:50:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 715.0 s, 1877.0 tps, lat 53.592 ms stddev 61.724 2024-09-19T07:50:30.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:30 smithi103 ceph-mon[31201]: pgmap v390: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-09-19T07:50:31.217 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:50:31.567 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:32.392 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:32 smithi103 ceph-mon[31201]: pgmap v391: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-09-19T07:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.400+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/3439669379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c410c180 msgr2=0x7fb3c410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.400+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/3439669379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c410c180 0x7fb3c410e600 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009ca0 tx=0x7fb3bc02a950 comp rx=0 tx=0).stop 2024-09-19T07:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/3439669379 shutdown_connections 2024-09-19T07:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/3439669379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c410c180 0x7fb3c410e600 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/3439669379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3c41036c0 0x7fb3c410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/3439669379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c4102da0 0x7fb3c4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/3439669379 >> 172.21.15.103:0/3439669379 conn(0x7fb3c40fc7a0 msgr2=0x7fb3c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/3439669379 shutdown_connections 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/3439669379 wait complete. 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 Processor -- start 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.401+0000 7fb3cb5ae640 1 -- start start 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3cb5ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c4102da0 0x7fb3c41a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3cb5ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c41036c0 0x7fb3c41a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3cb5ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3c410c180 0x7fb3c419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3cb5ae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4100de0 con 0x7fb3c4102da0 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3cb5ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4100c60 con 0x7fb3c41036c0 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3cb5ae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4100f60 con 0x7fb3c410c180 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c4102da0 0x7fb3c41a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c8b49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c41036c0 0x7fb3c41a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c4102da0 0x7fb3c41a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49418/0 (socket says 172.21.15.103:49418) 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 -- 172.21.15.103:0/2307358662 learned_addr learned my addr 172.21.15.103:0/2307358662 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3c410c180 0x7fb3c419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 -- 172.21.15.103:0/2307358662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3c410c180 msgr2=0x7fb3c419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3c410c180 0x7fb3c419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 -- 172.21.15.103:0/2307358662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c41036c0 msgr2=0x7fb3c41a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c41036c0 0x7fb3c41a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 -- 172.21.15.103:0/2307358662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c419b3c0 con 0x7fb3c4102da0 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c41036c0 0x7fb3c41a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.402+0000 7fb3c934a640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c4102da0 0x7fb3c41a02e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000e9f0 tx=0x7fb3c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.403+0000 7fb3ba7fc640 1 -- 172.21.15.103:0/2307358662 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c000cd80 con 0x7fb3c4102da0 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.403+0000 7fb3ba7fc640 1 -- 172.21.15.103:0/2307358662 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c0010910 con 0x7fb3c4102da0 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.403+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c419b6b0 con 0x7fb3c4102da0 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.403+0000 7fb3ba7fc640 1 -- 172.21.15.103:0/2307358662 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0018990 con 0x7fb3c4102da0 2024-09-19T07:50:32.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.403+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c41a7140 con 0x7fb3c4102da0 2024-09-19T07:50:32.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.406+0000 7fb3cd987740 1 Processor -- start 2024-09-19T07:50:32.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3cd987740 1 -- start start 2024-09-19T07:50:32.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3ba7fc640 1 -- 172.21.15.103:0/2307358662 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb3c0010ac0 con 0x7fb3c4102da0 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc256f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc256fde0 0x558fc2570220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc257ab80 0x558fc257cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3cd987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fc24cd2f0 con 0x558fc256fde0 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3cd987740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558fc2474cf0 con 0x558fc2571350 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.407+0000 7fb3cd987740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fc24c92b0 con 0x558fc257ab80 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.408+0000 7fb3c9b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc256f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.408+0000 7fb3c934a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc256fde0 0x558fc2570220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.408+0000 7fb3c9b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc256f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54154/0 (socket says 172.21.15.103:54154) 2024-09-19T07:50:32.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.408+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/3527331497 learned_addr learned my addr 172.21.15.103:0/3527331497 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:32.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.408+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc257ab80 0x558fc257cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3c8b49640 1 -- 172.21.15.103:0/3527331497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 msgr2=0x558fc256f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc256f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3ba7fc640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb3b00a3430 0x7fb3b00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3c8b49640 1 -- 172.21.15.103:0/3527331497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc256fde0 msgr2=0x558fc2570220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc256fde0 0x558fc2570220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3c8b49640 1 -- 172.21.15.103:0/3527331497 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc24593a0 con 0x558fc257ab80 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb3b00a3430 0x7fb3b00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3ba7fc640 1 -- 172.21.15.103:0/2307358662 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fb3c00d0080 con 0x7fb3c4102da0 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc257ab80 0x558fc257cf70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb3c410b9a0 tx=0x7fb3b40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3b26a3ade1e5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.409+0000 7fb3af7fe640 1 -- 172.21.15.103:0/3527331497 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4011700 con 0x558fc257ab80 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.410+0000 7fb3af7fe640 1 -- 172.21.15.103:0/3527331497 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b4011d80 con 0x558fc257ab80 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.410+0000 7fb3af7fe640 1 -- 172.21.15.103:0/3527331497 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b401a380 con 0x558fc257ab80 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.410+0000 7fb3cd987740 1 -- 172.21.15.103:0/3527331497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc257ab80 msgr2=0x558fc257cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.410+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb3b00a3430 0x7fb3b00a58f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009a60 tx=0x7fb3bc005d80 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:32.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.410+0000 7fb3cd987740 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc257ab80 0x558fc257cf70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb3c410b9a0 tx=0x7fb3b40107a0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 -- 172.21.15.103:0/3527331497 shutdown_connections 2024-09-19T07:50:32.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc257ab80 0x558fc257cf70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc256fde0 0x558fc2570220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 --2- 172.21.15.103:0/3527331497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc256f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 -- 172.21.15.103:0/3527331497 >> 172.21.15.103:0/3527331497 conn(0x558fc2530310 msgr2=0x558fc256c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:32.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 -- 172.21.15.103:0/3527331497 shutdown_connections 2024-09-19T07:50:32.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 -- 172.21.15.103:0/3527331497 wait complete. 2024-09-19T07:50:32.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 -- start start 2024-09-19T07:50:32.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.411+0000 7fb3cd987740 1 Processor -- start 2024-09-19T07:50:32.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2577780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2577ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3cd987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fc257d920 con 0x558fc257ab80 2024-09-19T07:50:32.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3cd987740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558fc2488060 con 0x558fc2571350 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3cd987740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fc257daa0 con 0x558fc256fde0 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3cd987740 10 client.0 _ll_register_callbacks cb 0x558fc21db8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2577ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c8b49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2577780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2577ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49444/0 (socket says 172.21.15.103:49444) 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/455502789 learned_addr learned my addr 172.21.15.103:0/455502789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c934a640 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 msgr2=0x558fc257a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/455502789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 msgr2=0x558fc2577780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2577780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/455502789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc223e3c0 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2577ce0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc02af40 tx=0x7fb3bc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5388b78f1f3017 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3acff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.412+0000 7fb3acff9640 1 -- 172.21.15.103:0/455502789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc056060 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3acff9640 1 -- 172.21.15.103:0/455502789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fb3bc004250 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b4011d80 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3acff9640 1 -- 172.21.15.103:0/455502789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb3bc034040 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 10 client.17943 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558fc2589390 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3acff9640 1 -- 172.21.15.103:0/455502789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b4011d80 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3acff9640 1 -- 172.21.15.103:0/455502789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fb3bc030a90 con 0x558fc257ab80 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 10 client.17943 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 10 client.17943 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:50:32.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 20 client.17943 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:32.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc258a850 0x558fc258cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 4 client.17943 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:32.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fc24da860 con 0x558fc258a850 2024-09-19T07:50:32.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.413+0000 7fb3c934a640 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc258a850 0x558fc258cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:32 smithi174 ceph-mon[35370]: pgmap v391: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-09-19T07:50:32.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.416+0000 7fb3c934a640 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc258a850 0x558fc258cc60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9de23506648814df server_cookie=3a8d4c1f3ad0b7c in_seq=0 out_seq=0 2024-09-19T07:50:32.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.416+0000 7fb3acff9640 10 client.17943 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3acff9640 1 -- 172.21.15.103:0/455502789 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fc24da860 con 0x558fc258a850 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3acff9640 10 client.17943 handle_command_reply: tid=0 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 client.17943 shutdown 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 msgr2=0x558fc2577ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2577ce0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc02af40 tx=0x7fb3bc0095c0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 shutdown_connections 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc258a850 0x558fc258cc60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2577ce0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2577780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 --2- 172.21.15.103:0/455502789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 >> 172.21.15.103:0/455502789 conn(0x558fc2530310 msgr2=0x558fc2572140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:32.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.417+0000 7fb3acff9640 0 client.17943 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:50:32.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.418+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 shutdown_connections 2024-09-19T07:50:32.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.418+0000 7fb3cd987740 1 -- 172.21.15.103:0/455502789 wait complete. 2024-09-19T07:50:32.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.418+0000 7fb3cd987740 20 client.17943 trim_cache size 0 max 16384 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.434+0000 7fb3cd987740 1 Processor -- start 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.434+0000 7fb3cd987740 1 -- start start 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.434+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc25796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.434+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2572930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.434+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2572e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3cd987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41ab750 con 0x558fc257ab80 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3cd987740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41ad2c0 con 0x558fc2571350 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c8b49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2572930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c8b49640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2572930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54172/0 (socket says 172.21.15.103:54172) 2024-09-19T07:50:32.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c8b49640 1 -- 172.21.15.103:0/4005174314 learned_addr learned my addr 172.21.15.103:0/4005174314 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c934a640 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc25796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3cd987740 1 -- 172.21.15.103:0/4005174314 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41abe20 con 0x558fc256fde0 2024-09-19T07:50:32.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2572e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/4005174314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 msgr2=0x558fc25796f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc25796f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/4005174314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 msgr2=0x558fc2572930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2572930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c9b4b640 1 -- 172.21.15.103:0/4005174314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc257a0f0 con 0x558fc257ab80 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.435+0000 7fb3c934a640 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc25796f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.436+0000 7fb3c9b4b640 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2572e70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc02ae70 tx=0x7fb3bc02f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e0b368e2c3f817 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.436+0000 7fb3aeffd640 1 -- 172.21.15.103:0/4005174314 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc056040 con 0x558fc257ab80 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.436+0000 7fb3aeffd640 1 -- 172.21.15.103:0/4005174314 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3bc02f3a0 con 0x558fc257ab80 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.436+0000 7fb3aeffd640 1 -- 172.21.15.103:0/4005174314 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc004610 con 0x558fc257ab80 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 -- 172.21.15.103:0/4005174314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 msgr2=0x558fc2572e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2572e70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc02ae70 tx=0x7fb3bc02f8a0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 -- 172.21.15.103:0/4005174314 shutdown_connections 2024-09-19T07:50:32.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc257ab80 0x558fc2572e70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc2571350 0x558fc2572930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 --2- 172.21.15.103:0/4005174314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc25796f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 -- 172.21.15.103:0/4005174314 >> 172.21.15.103:0/4005174314 conn(0x558fc2530310 msgr2=0x558fc2578fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:32.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 -- 172.21.15.103:0/4005174314 shutdown_connections 2024-09-19T07:50:32.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.437+0000 7fb3cd987740 1 -- 172.21.15.103:0/4005174314 wait complete. 2024-09-19T07:50:32.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.438+0000 7fb3cd987740 1 -- start start 2024-09-19T07:50:32.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.438+0000 7fb3cd987740 1 Processor -- start 2024-09-19T07:50:32.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.439+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.439+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc2571350 0x558fc257ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.439+0000 7fb3cd987740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc257ab80 0x558fc257f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.439+0000 7fb3cd987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41acbc0 con 0x558fc2571350 2024-09-19T07:50:32.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.439+0000 7fb3cd987740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41ac4f0 con 0x558fc257ab80 2024-09-19T07:50:32.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.439+0000 7fb3cd987740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41a9790 con 0x558fc256fde0 2024-09-19T07:50:32.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.439+0000 7fb3cd987740 10 client.0 _ll_register_callbacks cb 0x7fb3bc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc2571350 0x558fc257ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc2571350 0x558fc257ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49472/0 (socket says 172.21.15.103:49472) 2024-09-19T07:50:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 -- 172.21.15.103:0/2050460104 learned_addr learned my addr 172.21.15.103:0/2050460104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:32.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c934a640 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 -- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 msgr2=0x558fc257ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 -- 172.21.15.103:0/2050460104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc257ab80 msgr2=0x558fc257f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc257ab80 0x558fc257f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.442+0000 7fb3c8b49640 1 -- 172.21.15.103:0/2050460104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc25f4500 con 0x558fc2571350 2024-09-19T07:50:32.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.445+0000 7fb3c8b49640 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc2571350 0x558fc257ef60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005890 tx=0x7fb3b401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d867b901a3fd5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:32.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.445+0000 7fb3acff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:50:32.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.454+0000 7fb3acff9640 1 -- 172.21.15.103:0/2050460104 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004020 con 0x558fc2571350 2024-09-19T07:50:32.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.454+0000 7fb3acff9640 1 -- 172.21.15.103:0/2050460104 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fb3b40041c0 con 0x558fc2571350 2024-09-19T07:50:32.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.454+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3bc02f3a0 con 0x558fc2571350 2024-09-19T07:50:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.461+0000 7fb3acff9640 1 -- 172.21.15.103:0/2050460104 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb3b40043c0 con 0x558fc2571350 2024-09-19T07:50:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.461+0000 7fb3cd987740 10 client.17949 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:32.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.461+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558fc25f5720 con 0x558fc2571350 2024-09-19T07:50:32.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.469+0000 7fb3acff9640 1 -- 172.21.15.103:0/2050460104 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b4005520 con 0x558fc2571350 2024-09-19T07:50:32.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.469+0000 7fb3acff9640 1 -- 172.21.15.103:0/2050460104 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fb3b40035f0 con 0x558fc2571350 2024-09-19T07:50:32.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.469+0000 7fb3cd987740 10 client.17949 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:32.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.469+0000 7fb3cd987740 10 client.17949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:50:32.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.469+0000 7fb3cd987740 20 client.17949 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:32.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.470+0000 7fb3cd987740 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc25f5ba0 0x558fc25f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:32.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.470+0000 7fb3cd987740 4 client.17949 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:50:32.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.470+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb3bc02f3a0 con 0x558fc25f5ba0 2024-09-19T07:50:32.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.471+0000 7fb3c934a640 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc25f5ba0 0x558fc25f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:32.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.475+0000 7fb3c934a640 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc25f5ba0 0x558fc25f7f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32b1e3dfaf07f467 server_cookie=3100b12b84dd2ea3 in_seq=0 out_seq=0 2024-09-19T07:50:32.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.475+0000 7fb3acff9640 10 client.17949 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:50:32.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.480+0000 7fb3acff9640 1 -- 172.21.15.103:0/2050460104 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb3bc02f3a0 con 0x558fc25f5ba0 2024-09-19T07:50:32.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.480+0000 7fb3acff9640 10 client.17949 handle_command_reply: tid=0 2024-09-19T07:50:32.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.480+0000 7fb3cd987740 1 client.17949 shutdown 2024-09-19T07:50:32.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.481+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc2571350 msgr2=0x558fc257ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.481+0000 7fb3cd987740 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc2571350 0x558fc257ef60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005890 tx=0x7fb3b401c9e0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.486+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 shutdown_connections 2024-09-19T07:50:32.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.486+0000 7fb3cd987740 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x558fc25f5ba0 0x558fc25f7f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.486+0000 7fb3cd987740 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558fc257ab80 0x558fc257f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.486+0000 7fb3cd987740 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fc2571350 0x558fc257ef60 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.486+0000 7fb3acff9640 0 client.17949 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:50:32.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.486+0000 7fb3cd987740 1 --2- 172.21.15.103:0/2050460104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fc256fde0 0x558fc257ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.486+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 >> 172.21.15.103:0/2050460104 conn(0x558fc2530310 msgr2=0x558fc2575c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:32.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.487+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 shutdown_connections 2024-09-19T07:50:32.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.487+0000 7fb3cd987740 1 -- 172.21.15.103:0/2050460104 wait complete. 2024-09-19T07:50:32.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.487+0000 7fb3cd987740 20 client.17949 trim_cache size 0 max 16384 2024-09-19T07:50:32.490 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb3b00a3430 msgr2=0x7fb3b00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb3b00a3430 0x7fb3b00a58f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009a60 tx=0x7fb3bc005d80 comp rx=0 tx=0).stop 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c4102da0 msgr2=0x7fb3c41a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c4102da0 0x7fb3c41a02e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000e9f0 tx=0x7fb3c000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 shutdown_connections 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb3b00a3430 0x7fb3b00a58f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3c410c180 0x7fb3c419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c41036c0 0x7fb3c41a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 --2- 172.21.15.103:0/2307358662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c4102da0 0x7fb3c41a02e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.490+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 >> 172.21.15.103:0/2307358662 conn(0x7fb3c40fc7a0 msgr2=0x7fb3c410c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.492+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 shutdown_connections 2024-09-19T07:50:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:32.492+0000 7fb3cb5ae640 1 -- 172.21.15.103:0/2307358662 wait complete. 2024-09-19T07:50:32.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:32 smithi134 ceph-mon[35414]: pgmap v391: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 799 op/s 2024-09-19T07:50:33.405 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:50:33.795 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:34.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:34 smithi103 ceph-mon[31201]: pgmap v392: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-09-19T07:50:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:34 smithi174 ceph-mon[35370]: pgmap v392: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-09-19T07:50:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:34 smithi134 ceph-mon[35414]: pgmap v392: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 -- 172.21.15.103:0/1130000035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c10a560 msgr2=0x7f5d0c10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/1130000035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c10a560 0x7f5d0c10a940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009900 tx=0x7f5d0402a400 comp rx=0 tx=0).stop 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 -- 172.21.15.103:0/1130000035 shutdown_connections 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/1130000035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c1139e0 0x7f5d0c115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/1130000035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d0c10ae80 0x7f5d0c113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/1130000035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c10a560 0x7f5d0c10a940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 -- 172.21.15.103:0/1130000035 >> 172.21.15.103:0/1130000035 conn(0x7f5d0c06e2d0 msgr2=0x7f5d0c06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 -- 172.21.15.103:0/1130000035 shutdown_connections 2024-09-19T07:50:35.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.106+0000 7f5d12cb7640 1 -- 172.21.15.103:0/1130000035 wait complete. 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 Processor -- start 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 -- start start 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c10a560 0x7f5d0c1a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d0c10ae80 0x7f5d0c1a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c1139e0 0x7f5d0c19f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1196b0 con 0x7f5d0c10a560 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c119530 con 0x7f5d0c10ae80 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d12cb7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c119830 con 0x7f5d0c1139e0 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d10a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c10a560 0x7f5d0c1a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d0c10ae80 0x7f5d0c1a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d11254640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c1139e0 0x7f5d0c19f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d0c10ae80 0x7f5d0c1a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54190/0 (socket says 172.21.15.103:54190) 2024-09-19T07:50:35.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d11254640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c1139e0 0x7f5d0c19f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38158/0 (socket says 172.21.15.103:38158) 2024-09-19T07:50:35.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d0bfff640 1 -- 172.21.15.103:0/413377704 learned_addr learned my addr 172.21.15.103:0/413377704 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:35.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.107+0000 7f5d10a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c10a560 0x7f5d0c1a57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49490/0 (socket says 172.21.15.103:49490) 2024-09-19T07:50:35.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.109+0000 7f5d10a53640 1 -- 172.21.15.103:0/413377704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c1139e0 msgr2=0x7f5d0c19f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.109+0000 7f5d10a53640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c1139e0 0x7f5d0c19f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.109+0000 7f5d10a53640 1 -- 172.21.15.103:0/413377704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d0c10ae80 msgr2=0x7f5d0c1a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.109+0000 7f5d10a53640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d0c10ae80 0x7f5d0c1a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.109+0000 7f5d10a53640 1 -- 172.21.15.103:0/413377704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c1a00a0 con 0x7f5d0c10a560 2024-09-19T07:50:35.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.110+0000 7f5d10a53640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c10a560 0x7f5d0c1a57b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5d04005940 tx=0x7f5d040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.111+0000 7f5d09ffb640 1 -- 172.21.15.103:0/413377704 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0403f030 con 0x7f5d0c10a560 2024-09-19T07:50:35.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.111+0000 7f5d09ffb640 1 -- 172.21.15.103:0/413377704 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d04003ce0 con 0x7f5d0c10a560 2024-09-19T07:50:35.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.111+0000 7f5d09ffb640 1 -- 172.21.15.103:0/413377704 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04035070 con 0x7f5d0c10a560 2024-09-19T07:50:35.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.111+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c1a0330 con 0x7f5d0c10a560 2024-09-19T07:50:35.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.111+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c1091f0 con 0x7f5d0c10a560 2024-09-19T07:50:35.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.116+0000 7f5d15090740 1 Processor -- start 2024-09-19T07:50:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.116+0000 7f5d15090740 1 -- start start 2024-09-19T07:50:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d15090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ee350 0x55c7cd5ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d15090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d15090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d15090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7cd54a2f0 con 0x55c7cd5ee350 2024-09-19T07:50:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d15090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c7cd4f1cf0 con 0x55c7cd5ecde0 2024-09-19T07:50:35.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d15090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7cd5462b0 con 0x55c7cd5f7b80 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d11254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ee350 0x55c7cd5ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38168/0 (socket says 172.21.15.103:38168) 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d0bfff640 1 -- 172.21.15.103:0/463783555 learned_addr learned my addr 172.21.15.103:0/463783555 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d09ffb640 1 -- 172.21.15.103:0/413377704 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5d04042030 con 0x7f5d0c10a560 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d10a53640 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.117+0000 7f5d0bfff640 1 -- 172.21.15.103:0/463783555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 msgr2=0x55c7cd5ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d0bfff640 1 -- 172.21.15.103:0/463783555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ee350 msgr2=0x55c7cd5ec8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ee350 0x55c7cd5ec8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d0bfff640 1 -- 172.21.15.103:0/463783555 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7cd4d63a0 con 0x55c7cd5f7b80 2024-09-19T07:50:35.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d11254640 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ee350 0x55c7cd5ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:35.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d10a53640 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:35.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f9f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00acd0 tx=0x7f5cfc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31ea6dbc63ebe47d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d02ffd640 1 -- 172.21.15.103:0/463783555 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc005220 con 0x55c7cd5f7b80 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.118+0000 7f5d09ffb640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5cf80a34b0 0x7f5cf80a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.119+0000 7f5d09ffb640 1 -- 172.21.15.103:0/413377704 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5d040efc90 con 0x7f5d0c10a560 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.119+0000 7f5d02ffd640 1 -- 172.21.15.103:0/463783555 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc004c80 con 0x55c7cd5f7b80 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.119+0000 7f5d02ffd640 1 -- 172.21.15.103:0/463783555 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc018a30 con 0x55c7cd5f7b80 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.119+0000 7f5d11254640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5cf80a34b0 0x7f5cf80a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 -- 172.21.15.103:0/463783555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 msgr2=0x55c7cd5f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d11254640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5cf80a34b0 0x7f5cf80a5970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4005fd0 tx=0x7f5cf4009600 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f9f70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00acd0 tx=0x7f5cfc00f790 comp rx=0 tx=0).stop 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 -- 172.21.15.103:0/463783555 shutdown_connections 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f9f70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 --2- 172.21.15.103:0/463783555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ee350 0x55c7cd5ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 -- 172.21.15.103:0/463783555 >> 172.21.15.103:0/463783555 conn(0x55c7cd5ad310 msgr2=0x55c7cd5e9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.120+0000 7f5d15090740 1 -- 172.21.15.103:0/463783555 shutdown_connections 2024-09-19T07:50:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.121+0000 7f5d15090740 1 -- 172.21.15.103:0/463783555 wait complete. 2024-09-19T07:50:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.121+0000 7f5d15090740 1 -- start start 2024-09-19T07:50:35.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.121+0000 7f5d15090740 1 Processor -- start 2024-09-19T07:50:35.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.121+0000 7f5d15090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.121+0000 7f5d15090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.121+0000 7f5d15090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.121+0000 7f5d15090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7cd5fa880 con 0x55c7cd5ecde0 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d15090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c7cd505060 con 0x55c7cd5ee350 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d15090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7cd5faa00 con 0x55c7cd5f7b80 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d15090740 10 client.0 _ll_register_callbacks cb 0x55c7cd2588c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49516/0 (socket says 172.21.15.103:49516) 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 -- 172.21.15.103:0/3771503416 learned_addr learned my addr 172.21.15.103:0/3771503416 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d11254640 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 -- 172.21.15.103:0/3771503416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 msgr2=0x55c7cd5f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 -- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 msgr2=0x55c7cd5f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 -- 172.21.15.103:0/3771503416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7cd2bb3c0 con 0x55c7cd5ecde0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d10a53640 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5f7510 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5d040bbcb0 tx=0x7f5d040bbe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0505a9ef59d058a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5cebfff640 1 -- 172.21.15.103:0/3771503416 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04049040 con 0x55c7cd5ecde0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5cebfff640 1 -- 172.21.15.103:0/3771503416 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5d040b8620 con 0x55c7cd5ecde0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.122+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cfc004c80 con 0x55c7cd5ecde0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5cebfff640 1 -- 172.21.15.103:0/3771503416 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5d040bbfb0 con 0x55c7cd5ecde0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 10 client.17961 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55c7cd606280 con 0x55c7cd5ecde0 2024-09-19T07:50:35.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5cebfff640 1 -- 172.21.15.103:0/3771503416 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc004c80 con 0x55c7cd5ecde0 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5cebfff640 1 -- 172.21.15.103:0/3771503416 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f5d0403bce0 con 0x55c7cd5ecde0 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 10 client.17961 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 10 client.17961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 20 client.17961 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd609bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 4 client.17961 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7cd557860 con 0x55c7cd607780 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.123+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd609bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.124+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd609bd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc7e00c376ef46e server_cookie=5d386a2450a0c245 in_seq=0 out_seq=0 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.124+0000 7f5cebfff640 10 client.17961 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.124+0000 7f5cebfff640 1 -- 172.21.15.103:0/3771503416 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7cd557860 con 0x55c7cd607780 2024-09-19T07:50:35.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.124+0000 7f5cebfff640 10 client.17961 handle_command_reply: tid=0 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.127+0000 7f5d15090740 1 client.17961 shutdown 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.127+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ecde0 msgr2=0x55c7cd5f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.127+0000 7f5d15090740 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5f7510 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5d040bbcb0 tx=0x7f5d040bbe40 comp rx=0 tx=0).stop 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 shutdown_connections 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd609bd0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5cebfff640 0 client.17961 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 --2- 172.21.15.103:0/3771503416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5f7510 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 >> 172.21.15.103:0/3771503416 conn(0x55c7cd5ad310 msgr2=0x55c7cd5ef140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 shutdown_connections 2024-09-19T07:50:35.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 1 -- 172.21.15.103:0/3771503416 wait complete. 2024-09-19T07:50:35.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.128+0000 7f5d15090740 20 client.17961 trim_cache size 0 max 16384 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.144+0000 7f5d15090740 1 Processor -- start 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.144+0000 7f5d15090740 1 -- start start 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d15090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5dfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d15090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d15090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d15090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1b3e70 con 0x55c7cd5f7b80 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d15090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1b59e0 con 0x55c7cd5ecde0 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d15090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1b4540 con 0x55c7cd5ee350 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38180/0 (socket says 172.21.15.103:38180) 2024-09-19T07:50:35.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.145+0000 7f5d0bfff640 1 -- 172.21.15.103:0/662720800 learned_addr learned my addr 172.21.15.103:0/662720800 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:35.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.146+0000 7f5d10a53640 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5dfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.146+0000 7f5d0bfff640 1 -- 172.21.15.103:0/662720800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 msgr2=0x55c7cd5dfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.146+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5dfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.146+0000 7f5d0bfff640 1 -- 172.21.15.103:0/662720800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 msgr2=0x55c7cd5f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.146+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.146+0000 7f5d0bfff640 1 -- 172.21.15.103:0/662720800 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7cd5f99f0 con 0x55c7cd5ee350 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.146+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f8f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc018520 tx=0x7f5cfc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c948627bb9550927 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.147+0000 7f5d027fc640 1 -- 172.21.15.103:0/662720800 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc030070 con 0x55c7cd5ee350 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.147+0000 7f5d027fc640 1 -- 172.21.15.103:0/662720800 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc018690 con 0x55c7cd5ee350 2024-09-19T07:50:35.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.147+0000 7f5d027fc640 1 -- 172.21.15.103:0/662720800 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc004520 con 0x55c7cd5ee350 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 -- 172.21.15.103:0/662720800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ee350 msgr2=0x55c7cd5f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f8f70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc018520 tx=0x7f5cfc000bc0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 -- 172.21.15.103:0/662720800 shutdown_connections 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd5f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ee350 0x55c7cd5f8f70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 --2- 172.21.15.103:0/662720800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5dfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 -- 172.21.15.103:0/662720800 >> 172.21.15.103:0/662720800 conn(0x55c7cd5ad310 msgr2=0x55c7cd5df4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 -- 172.21.15.103:0/662720800 shutdown_connections 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.148+0000 7f5d15090740 1 -- 172.21.15.103:0/662720800 wait complete. 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 -- start start 2024-09-19T07:50:35.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 Processor -- start 2024-09-19T07:50:35.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd600520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd60eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1b52e0 con 0x55c7cd5f7b80 2024-09-19T07:50:35.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1b4c10 con 0x55c7cd5ee350 2024-09-19T07:50:35.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1b1e70 con 0x55c7cd5ecde0 2024-09-19T07:50:35.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d15090740 10 client.0 _ll_register_callbacks cb 0x7f5d040095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:35.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d10a53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d11254640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd60eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d10a53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38192/0 (socket says 172.21.15.103:38192) 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d10a53640 1 -- 172.21.15.103:0/3884588066 learned_addr learned my addr 172.21.15.103:0/3884588066 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.149+0000 7f5d0bfff640 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd600520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5d11254640 1 -- 172.21.15.103:0/3884588066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ecde0 msgr2=0x55c7cd5fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5d11254640 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5d11254640 1 -- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 msgr2=0x55c7cd600520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5d11254640 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd600520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5d11254640 1 -- 172.21.15.103:0/3884588066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7cd60f3b0 con 0x55c7cd5f7b80 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5d11254640 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd60eb80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4013f70 tx=0x7f5cf4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920f4c108c5acfa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5cebfff640 1 -- 172.21.15.103:0/3884588066 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4020ab0 con 0x55c7cd5f7b80 2024-09-19T07:50:35.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5cebfff640 1 -- 172.21.15.103:0/3884588066 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5cf4020c50 con 0x55c7cd5f7b80 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.150+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cfc018690 con 0x55c7cd5f7b80 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5cebfff640 1 -- 172.21.15.103:0/3884588066 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5cf401d430 con 0x55c7cd5f7b80 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 10 client.17964 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55c7cd5ff9e0 con 0x55c7cd5f7b80 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5cebfff640 1 -- 172.21.15.103:0/3884588066 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc018690 con 0x55c7cd5f7b80 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5cebfff640 1 -- 172.21.15.103:0/3884588066 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f5cf400f040 con 0x55c7cd5f7b80 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 10 client.17964 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 20 client.17964 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd674980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 4 client.17964 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.151+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7cd551d70 con 0x55c7cd607780 2024-09-19T07:50:35.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.152+0000 7f5d10a53640 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd674980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.152+0000 7f5d10a53640 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd674980 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84ff610f20b4249 server_cookie=a7901aaaad4fadda in_seq=0 out_seq=0 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.152+0000 7f5cebfff640 10 client.17964 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.152+0000 7f5cebfff640 1 -- 172.21.15.103:0/3884588066 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7cd551d70 con 0x55c7cd607780 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.152+0000 7f5cebfff640 10 client.17964 handle_command_reply: tid=0 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.152+0000 7f5d15090740 1 client.17964 shutdown 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 msgr2=0x55c7cd60eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd60eb80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4013f70 tx=0x7f5cf4004ab0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 shutdown_connections 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c7cd607780 0x55c7cd674980 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7cd5f7b80 0x55c7cd60eb80 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5cebfff640 0 client.17964 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:35.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c7cd5ee350 0x55c7cd600520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 --2- 172.21.15.103:0/3884588066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7cd5ecde0 0x55c7cd5fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 >> 172.21.15.103:0/3884588066 conn(0x55c7cd5ad310 msgr2=0x55c7cd5f2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 shutdown_connections 2024-09-19T07:50:35.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 1 -- 172.21.15.103:0/3884588066 wait complete. 2024-09-19T07:50:35.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.153+0000 7f5d15090740 20 client.17964 trim_cache size 0 max 16384 2024-09-19T07:50:35.158 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5cf80a34b0 msgr2=0x7f5cf80a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5cf80a34b0 0x7f5cf80a5970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4005fd0 tx=0x7f5cf4009600 comp rx=0 tx=0).stop 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c10a560 msgr2=0x7f5d0c1a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c10a560 0x7f5d0c1a57b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5d04005940 tx=0x7f5d040042f0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 shutdown_connections 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5cf80a34b0 0x7f5cf80a5970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c1139e0 0x7f5d0c19f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d0c10ae80 0x7f5d0c1a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 --2- 172.21.15.103:0/413377704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d0c10a560 0x7f5d0c1a57b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:35.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.154+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 >> 172.21.15.103:0/413377704 conn(0x7f5d0c06e2d0 msgr2=0x7f5d0c070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:35.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.155+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 shutdown_connections 2024-09-19T07:50:35.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:35.156+0000 7f5d12cb7640 1 -- 172.21.15.103:0/413377704 wait complete. 2024-09-19T07:50:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 720.0 s, 2072.2 tps, lat 48.337 ms stddev 56.618 2024-09-19T07:50:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:36 smithi174 ceph-mon[35370]: pgmap v393: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 593 op/s 2024-09-19T07:50:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:36 smithi103 ceph-mon[31201]: pgmap v393: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 593 op/s 2024-09-19T07:50:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:36 smithi134 ceph-mon[35414]: pgmap v393: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 593 op/s 2024-09-19T07:50:37.178 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:50:37.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:50:37.505 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.084+0000 7f01da497640 1 -- 172.21.15.103:0/4211831187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d4102360 msgr2=0x7f01d410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.084+0000 7f01da497640 1 --2- 172.21.15.103:0/4211831187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d4102360 0x7f01d410e890 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009cd0 tx=0x7f01cc02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.084+0000 7f01da497640 1 -- 172.21.15.103:0/4211831187 shutdown_connections 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.084+0000 7f01da497640 1 --2- 172.21.15.103:0/4211831187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d4102360 0x7f01d410e890 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.084+0000 7f01da497640 1 --2- 172.21.15.103:0/4211831187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d41019c0 0x7f01d4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.084+0000 7f01da497640 1 --2- 172.21.15.103:0/4211831187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d41079c0 0x7f01d4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.085+0000 7f01da497640 1 -- 172.21.15.103:0/4211831187 >> 172.21.15.103:0/4211831187 conn(0x7f01d40fd740 msgr2=0x7f01d40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.085+0000 7f01da497640 1 -- 172.21.15.103:0/4211831187 shutdown_connections 2024-09-19T07:50:38.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.085+0000 7f01da497640 1 -- 172.21.15.103:0/4211831187 wait complete. 2024-09-19T07:50:38.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.085+0000 7f01da497640 1 Processor -- start 2024-09-19T07:50:38.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.085+0000 7f01da497640 1 -- start start 2024-09-19T07:50:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01da497640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d41019c0 0x7f01d4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01da497640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102360 0x7f01d4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01da497640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d41079c0 0x7f01d419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01da497640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01d4111dc0 con 0x7f01d41019c0 2024-09-19T07:50:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01da497640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01d4111c40 con 0x7f01d4102360 2024-09-19T07:50:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102360 0x7f01d4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d9495640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d41019c0 0x7f01d4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102360 0x7f01d4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54224/0 (socket says 172.21.15.103:54224) 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 -- 172.21.15.103:0/3902854565 learned_addr learned my addr 172.21.15.103:0/3902854565 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d41079c0 0x7f01d419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01da497640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01d4111f40 con 0x7f01d41079c0 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 -- 172.21.15.103:0/3902854565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d41079c0 msgr2=0x7f01d419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d41079c0 0x7f01d419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 -- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d41019c0 msgr2=0x7f01d4198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d41019c0 0x7f01d4198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d8c94640 1 -- 172.21.15.103:0/3902854565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d419d1c0 con 0x7f01d4102360 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.086+0000 7f01d9495640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d41019c0 0x7f01d4198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.087+0000 7f01d8c94640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102360 0x7f01d4198750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01c400ea60 tx=0x7f01c400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:38.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.087+0000 7f01ca7fc640 1 -- 172.21.15.103:0/3902854565 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c400cde0 con 0x7f01d4102360 2024-09-19T07:50:38.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.087+0000 7f01ca7fc640 1 -- 172.21.15.103:0/3902854565 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c40035f0 con 0x7f01d4102360 2024-09-19T07:50:38.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.087+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d419d4b0 con 0x7f01d4102360 2024-09-19T07:50:38.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.087+0000 7f01ca7fc640 1 -- 172.21.15.103:0/3902854565 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4010670 con 0x7f01d4102360 2024-09-19T07:50:38.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.087+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d41a4d90 con 0x7f01d4102360 2024-09-19T07:50:38.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.092+0000 7f01ddad2740 1 Processor -- start 2024-09-19T07:50:38.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.092+0000 7f01ca7fc640 1 -- 172.21.15.103:0/3902854565 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f01c40037a0 con 0x7f01d4102360 2024-09-19T07:50:38.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.092+0000 7f01ddad2740 1 -- start start 2024-09-19T07:50:38.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.092+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e6148a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.092+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e61498e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.092+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61499360 0x558e6149b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01ddad2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e613ef280 con 0x558e61499360 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01ddad2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e61396b10 con 0x558e6148a550 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01ddad2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e613eb240 con 0x558e61489c30 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d8c94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61499360 0x558e6149b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e61498e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e61498e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54236/0 (socket says 172.21.15.103:54236) 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 -- 172.21.15.103:0/3101919944 learned_addr learned my addr 172.21.15.103:0/3101919944 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e6148a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 -- 172.21.15.103:0/3101919944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 msgr2=0x558e6148a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e6148a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 -- 172.21.15.103:0/3101919944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61499360 msgr2=0x558e6149b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61499360 0x558e6149b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 -- 172.21.15.103:0/3101919944 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e613a31f0 con 0x558e6148a550 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01ca7fc640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01c00a33b0 0x7f01c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.093+0000 7f01d9495640 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e61498e20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01d00114f0 tx=0x7f01d00119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4df621bb7c9c8e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.094+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01c00a33b0 0x7f01c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.094+0000 7f01ca7fc640 1 -- 172.21.15.103:0/3902854565 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f01c40cffb0 con 0x7f01d4102360 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.094+0000 7f01bf7fe640 1 -- 172.21.15.103:0/3101919944 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d001e070 con 0x558e6148a550 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.094+0000 7f01bf7fe640 1 -- 172.21.15.103:0/3101919944 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d000faa0 con 0x558e6148a550 2024-09-19T07:50:38.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.094+0000 7f01bf7fe640 1 -- 172.21.15.103:0/3101919944 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0019b90 con 0x558e6148a550 2024-09-19T07:50:38.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 -- 172.21.15.103:0/3101919944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 msgr2=0x558e61498e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01c00a33b0 0x7f01c00a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01cc004ab0 tx=0x7f01cc004a40 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e61498e20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f01d00114f0 tx=0x7f01d00119b0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 -- 172.21.15.103:0/3101919944 shutdown_connections 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61499360 0x558e6149b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e61498e20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3101919944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e6148a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 -- 172.21.15.103:0/3101919944 >> 172.21.15.103:0/3101919944 conn(0x558e61486300 msgr2=0x558e614866e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 -- 172.21.15.103:0/3101919944 shutdown_connections 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 -- 172.21.15.103:0/3101919944 wait complete. 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 -- start start 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.095+0000 7f01ddad2740 1 Processor -- start 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e61496800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e61496d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01ddad2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e6149bf30 con 0x558e6148a550 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01ddad2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e613a9f10 con 0x558e61499360 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01ddad2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e6149c0b0 con 0x558e61489c30 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01ddad2740 10 client.0 _ll_register_callbacks cb 0x558e61372a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01d9495640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01d9495640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:38228/0 (socket says 172.21.15.103:38228) 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01d9495640 1 -- 172.21.15.103:0/3001453888 learned_addr learned my addr 172.21.15.103:0/3001453888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e61496d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01d9495640 1 -- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 msgr2=0x558e61496d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01d9495640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e61496d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.096+0000 7f01d9495640 1 -- 172.21.15.103:0/3001453888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 msgr2=0x558e61496800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:38.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.097+0000 7f01d8c94640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e61496800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.097+0000 7f01d9495640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e61496800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.097+0000 7f01d9495640 1 -- 172.21.15.103:0/3001453888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e611606a0 con 0x558e61489c30 2024-09-19T07:50:38.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.097+0000 7f01d8c94640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e61496800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:38.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.097+0000 7f01d9495640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614962c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000c00 tx=0x7f01d0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a50ef7d05f25c89f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:38.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.097+0000 7f01bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:50:38.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.100+0000 7f01bcff9640 1 -- 172.21.15.103:0/3001453888 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d001e040 con 0x558e61489c30 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.100+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d000faa0 con 0x558e61489c30 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.100+0000 7f01bcff9640 1 -- 172.21.15.103:0/3001453888 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f01d000f5c0 con 0x558e61489c30 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01bcff9640 1 -- 172.21.15.103:0/3001453888 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f01d002f050 con 0x558e61489c30 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01ddad2740 10 client.27271 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558e614a7bf0 con 0x558e61489c30 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01bcff9640 1 -- 172.21.15.103:0/3001453888 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d000faa0 con 0x558e61489c30 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01bcff9640 1 -- 172.21.15.103:0/3001453888 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f01d002a030 con 0x558e61489c30 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01ddad2740 10 client.27271 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01ddad2740 10 client.27271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01ddad2740 20 client.27271 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e614a93d0 0x558e614ab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.101+0000 7f01ddad2740 4 client.27271 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:38.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.102+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e613fc7f0 con 0x558e614a93d0 2024-09-19T07:50:38.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.102+0000 7f01d8c94640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e614a93d0 0x558e614ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.102+0000 7f01bcff9640 10 client.27271 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:38.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.102+0000 7f01d8c94640 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e614a93d0 0x558e614ab860 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3240a1bd9cb520 server_cookie=dc6de23372872ffb in_seq=0 out_seq=0 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01bcff9640 1 -- 172.21.15.103:0/3001453888 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e613fc7f0 con 0x558e614a93d0 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01bcff9640 10 client.27271 handle_command_reply: tid=0 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 client.27271 shutdown 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 msgr2=0x558e614962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614962c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000c00 tx=0x7f01d0005dc0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 shutdown_connections 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e614a93d0 0x558e614ab860 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e61496d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e61496800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3001453888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614962c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 >> 172.21.15.103:0/3001453888 conn(0x558e61486300 msgr2=0x558e6148c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01bcff9640 0 client.27271 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 shutdown_connections 2024-09-19T07:50:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 1 -- 172.21.15.103:0/3001453888 wait complete. 2024-09-19T07:50:38.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.103+0000 7f01ddad2740 20 client.27271 trim_cache size 0 max 16384 2024-09-19T07:50:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.126+0000 7f01ddad2740 1 Processor -- start 2024-09-19T07:50:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01ddad2740 1 -- start start 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61489c30 0x558e614b1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e614b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61499360 0x558e61484bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01ddad2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41ab6e0 con 0x558e61489c30 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01ddad2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41ad250 con 0x558e6148a550 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01ddad2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41abdb0 con 0x558e61499360 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01d8c94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e614b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01d8c94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e614b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54254/0 (socket says 172.21.15.103:54254) 2024-09-19T07:50:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.127+0000 7f01d8c94640 1 -- 172.21.15.103:0/2285437776 learned_addr learned my addr 172.21.15.103:0/2285437776 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.128+0000 7f01d8c94640 1 -- 172.21.15.103:0/2285437776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61499360 msgr2=0x558e61484bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:50:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.128+0000 7f01d8c94640 1 --2- 172.21.15.103:0/2285437776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61499360 0x558e61484bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.128+0000 7f01d8c94640 1 -- 172.21.15.103:0/2285437776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61489c30 msgr2=0x558e614b1140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:50:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.128+0000 7f01d8c94640 1 --2- 172.21.15.103:0/2285437776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61489c30 0x558e614b1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.128+0000 7f01d8c94640 1 -- 172.21.15.103:0/2285437776 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6148a340 con 0x558e6148a550 2024-09-19T07:50:38.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.129+0000 7f01d8c94640 1 --2- 172.21.15.103:0/2285437776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e614b1680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f01c40986e0 tx=0x7f01c40d3b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ed3892d8885e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:38.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.129+0000 7f01beffd640 1 -- 172.21.15.103:0/2285437776 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c407ae60 con 0x558e6148a550 2024-09-19T07:50:38.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.129+0000 7f01beffd640 1 -- 172.21.15.103:0/2285437776 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c409c980 con 0x558e6148a550 2024-09-19T07:50:38.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.129+0000 7f01beffd640 1 -- 172.21.15.103:0/2285437776 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c40a4ef0 con 0x558e6148a550 2024-09-19T07:50:38.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 -- 172.21.15.103:0/2285437776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 msgr2=0x558e614b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 --2- 172.21.15.103:0/2285437776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e614b1680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f01c40986e0 tx=0x7f01c40d3b00 comp rx=0 tx=0).stop 2024-09-19T07:50:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 -- 172.21.15.103:0/2285437776 shutdown_connections 2024-09-19T07:50:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 --2- 172.21.15.103:0/2285437776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61499360 0x558e61484bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 --2- 172.21.15.103:0/2285437776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6148a550 0x558e614b1680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 --2- 172.21.15.103:0/2285437776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e61489c30 0x558e614b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 -- 172.21.15.103:0/2285437776 >> 172.21.15.103:0/2285437776 conn(0x558e61486300 msgr2=0x558e61494b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 -- 172.21.15.103:0/2285437776 shutdown_connections 2024-09-19T07:50:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.130+0000 7f01ddad2740 1 -- 172.21.15.103:0/2285437776 wait complete. 2024-09-19T07:50:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 -- start start 2024-09-19T07:50:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 Processor -- start 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e614a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e614928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41acb50 con 0x558e6148a550 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41ac480 con 0x558e61499360 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01d41a96e0 con 0x558e61489c30 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.131+0000 7f01ddad2740 10 client.0 _ll_register_callbacks cb 0x7f01d001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e614928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d8c94640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e614a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e614928f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:54270/0 (socket says 172.21.15.103:54270) 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 -- 172.21.15.103:0/3732736096 learned_addr learned my addr 172.21.15.103:0/3732736096 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9495640 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 -- 172.21.15.103:0/3732736096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 msgr2=0x558e614a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 -- 172.21.15.103:0/3732736096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 msgr2=0x558e614a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e614a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 -- 172.21.15.103:0/3732736096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e61492ee0 con 0x558e61499360 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01d9c96640 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e614928f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f01cc035000 tx=0x7f01cc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbaa73aa78787c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.132+0000 7f01bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:50:38.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.133+0000 7f01bcff9640 1 -- 172.21.15.103:0/3732736096 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0316c0 con 0x558e61499360 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.133+0000 7f01bcff9640 1 -- 172.21.15.103:0/3732736096 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f01cc004250 con 0x558e61499360 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.133+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c409c980 con 0x558e61499360 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.133+0000 7f01bcff9640 1 -- 172.21.15.103:0/3732736096 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f01cc004450 con 0x558e61499360 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.133+0000 7f01ddad2740 10 client.27311 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.133+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558e6148c9b0 con 0x558e61499360 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.133+0000 7f01bcff9640 1 -- 172.21.15.103:0/3732736096 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c409c980 con 0x558e61499360 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01bcff9640 1 -- 172.21.15.103:0/3732736096 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f01cc030070 con 0x558e61499360 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01ddad2740 10 client.27311 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01ddad2740 10 client.27311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01ddad2740 20 client.27311 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e6148d1e0 0x558e6148f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01ddad2740 4 client.27311 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558e613f6d00 con 0x558e6148d1e0 2024-09-19T07:50:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.134+0000 7f01d9495640 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e6148d1e0 0x558e6148f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.135+0000 7f01bcff9640 10 client.27311 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.135+0000 7f01d9495640 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e6148d1e0 0x558e6148f600 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95d842a4faa017f1 server_cookie=fa14f04b8107eafb in_seq=0 out_seq=0 2024-09-19T07:50:38.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.135+0000 7f01bcff9640 1 -- 172.21.15.103:0/3732736096 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f01d0015080 con 0x558e6148d1e0 2024-09-19T07:50:38.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.135+0000 7f01bcff9640 10 client.27311 handle_command_reply: tid=0 2024-09-19T07:50:38.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.135+0000 7f01ddad2740 1 client.27311 shutdown 2024-09-19T07:50:38.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.135+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 msgr2=0x558e614928f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.135+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e614928f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f01cc035000 tx=0x7f01cc0095c0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 shutdown_connections 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558e6148d1e0 0x558e6148f600 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e61499360 0x558e614928f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e6148a550 0x558e614a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 --2- 172.21.15.103:0/3732736096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e61489c30 0x558e614a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 >> 172.21.15.103:0/3732736096 conn(0x558e61486300 msgr2=0x558e61497900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01bcff9640 0 client.27311 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 shutdown_connections 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 1 -- 172.21.15.103:0/3732736096 wait complete. 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01ddad2740 20 client.27311 trim_cache size 0 max 16384 2024-09-19T07:50:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:50:38.141 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:50:38.141 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "c4387de3-7b8c-4500-936e-2b02f76c96f7", 2024-09-19T07:50:38.141 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:50:38.141 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:50:38.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01c00a33b0 msgr2=0x7f01c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.136+0000 7f01da497640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01c00a33b0 0x7f01c00a5870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01cc004ab0 tx=0x7f01cc004a40 comp rx=0 tx=0).stop 2024-09-19T07:50:38.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.137+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102360 msgr2=0x7f01d4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:38.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.137+0000 7f01da497640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102360 0x7f01d4198750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01c400ea60 tx=0x7f01c400ef20 comp rx=0 tx=0).stop 2024-09-19T07:50:38.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.137+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 shutdown_connections 2024-09-19T07:50:38.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.137+0000 7f01da497640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01c00a33b0 0x7f01c00a5870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.137+0000 7f01da497640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d41079c0 0x7f01d419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.137+0000 7f01da497640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102360 0x7f01d4198750 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.137+0000 7f01da497640 1 --2- 172.21.15.103:0/3902854565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01d41019c0 0x7f01d4198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:38.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.138+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 >> 172.21.15.103:0/3902854565 conn(0x7f01d40fd740 msgr2=0x7f01d40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:38.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.138+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 shutdown_connections 2024-09-19T07:50:38.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:38.138+0000 7f01da497640 1 -- 172.21.15.103:0/3902854565 wait complete. 2024-09-19T07:50:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:38 smithi174 ceph-mon[35370]: pgmap v394: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 823 op/s 2024-09-19T07:50:38.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:38 smithi103 ceph-mon[31201]: pgmap v394: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 823 op/s 2024-09-19T07:50:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:38 smithi134 ceph-mon[35414]: pgmap v394: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 823 op/s 2024-09-19T07:50:38.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:50:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:50:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.221 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:39 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.370 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:50:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:39 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:50:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:39 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:50:39.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.932+0000 7f057f8ec640 1 -- 172.21.15.103:0/866575547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0578106f20 msgr2=0x7f057810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.932+0000 7f057f8ec640 1 --2- 172.21.15.103:0/866575547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0578106f20 0x7f057810eaa0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0570009ca0 tx=0x7f057002a8c0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 -- 172.21.15.103:0/866575547 shutdown_connections 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 --2- 172.21.15.103:0/866575547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0578106f20 0x7f057810eaa0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 --2- 172.21.15.103:0/866575547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05781027a0 0x7f05781067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 --2- 172.21.15.103:0/866575547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0578101df0 0x7f05781021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 -- 172.21.15.103:0/866575547 >> 172.21.15.103:0/866575547 conn(0x7f05780fd8b0 msgr2=0x7f05780ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 -- 172.21.15.103:0/866575547 shutdown_connections 2024-09-19T07:50:39.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 -- 172.21.15.103:0/866575547 wait complete. 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 Processor -- start 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.933+0000 7f057f8ec640 1 -- start start 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057f8ec640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0578101df0 0x7f05780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057f8ec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05781027a0 0x7f0578074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057f8ec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0578106f20 0x7f0578074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057f8ec640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0578111f80 con 0x7f0578101df0 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057f8ec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0578111e00 con 0x7f05781027a0 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057f8ec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0578112100 con 0x7f0578106f20 2024-09-19T07:50:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057de89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0578106f20 0x7f0578074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057de89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0578106f20 0x7f0578074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40668/0 (socket says 172.21.15.103:40668) 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.934+0000 7f057de89640 1 -- 172.21.15.103:0/3986711130 learned_addr learned my addr 172.21.15.103:0/3986711130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057de89640 1 -- 172.21.15.103:0/3986711130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05781027a0 msgr2=0x7f0578074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057ce87640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05781027a0 0x7f0578074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057de89640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05781027a0 0x7f0578074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057de89640 1 -- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0578101df0 msgr2=0x7f05780744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057de89640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0578101df0 0x7f05780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057de89640 1 -- 172.21.15.103:0/3986711130 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057819c530 con 0x7f0578106f20 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057ce87640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05781027a0 0x7f0578074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.935+0000 7f057de89640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0578106f20 0x7f0578074f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0570031a20 tx=0x7f0570031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.936+0000 7f056e7fc640 1 -- 172.21.15.103:0/3986711130 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570003b70 con 0x7f0578106f20 2024-09-19T07:50:39.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.936+0000 7f056e7fc640 1 -- 172.21.15.103:0/3986711130 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0570003d10 con 0x7f0578106f20 2024-09-19T07:50:39.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.936+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057819c7c0 con 0x7f0578106f20 2024-09-19T07:50:39.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.936+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057819cd50 con 0x7f0578106f20 2024-09-19T07:50:39.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.936+0000 7f056e7fc640 1 -- 172.21.15.103:0/3986711130 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057002fc40 con 0x7f0578106f20 2024-09-19T07:50:39.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.939+0000 7f056e7fc640 1 -- 172.21.15.103:0/3986711130 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f0570004050 con 0x7f0578106f20 2024-09-19T07:50:39.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.939+0000 7f0581cc5740 1 Processor -- start 2024-09-19T07:50:39.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.939+0000 7f0581cc5740 1 -- start start 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c452880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c452dc0 0x55895c45d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f0581cc5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55895c3b82e0 con 0x55895c452dc0 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f0581cc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55895c35fce0 con 0x55895c45d680 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f0581cc5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55895c3b42a0 con 0x55895c4524a0 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f056e7fc640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f05600a34b0 0x7f05600a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c452880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057de89640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f05600a34b0 0x7f05600a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c452880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40680/0 (socket says 172.21.15.103:40680) 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 -- 172.21.15.103:0/1343583110 learned_addr learned my addr 172.21.15.103:0/1343583110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057d688640 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c452dc0 0x55895c45d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 -- 172.21.15.103:0/1343583110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 msgr2=0x55895c45fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 -- 172.21.15.103:0/1343583110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c452dc0 msgr2=0x55895c45d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c452dc0 0x55895c45d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 -- 172.21.15.103:0/1343583110 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895c344440 con 0x55895c4524a0 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f056e7fc640 1 -- 172.21.15.103:0/3986711130 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f05700f0ef0 con 0x7f0578106f20 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.940+0000 7f057ce87640 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c452880 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0574009f90 tx=0x7f0574009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af6ca1cb4c3b11b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:39.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f055f7fe640 1 -- 172.21.15.103:0/1343583110 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574019030 con 0x55895c4524a0 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f057de89640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f05600a34b0 0x7f05600a5970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f05700b9cd0 tx=0x7f05700b8e70 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f055f7fe640 1 -- 172.21.15.103:0/1343583110 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574005500 con 0x55895c4524a0 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f055f7fe640 1 -- 172.21.15.103:0/1343583110 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574011760 con 0x55895c4524a0 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f0581cc5740 1 -- 172.21.15.103:0/1343583110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 msgr2=0x55895c452880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f0581cc5740 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c452880 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0574009f90 tx=0x7f0574009670 comp rx=0 tx=0).stop 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f0581cc5740 1 -- 172.21.15.103:0/1343583110 shutdown_connections 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f0581cc5740 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f0581cc5740 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c452dc0 0x55895c45d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f0581cc5740 1 --2- 172.21.15.103:0/1343583110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c452880 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.941+0000 7f0581cc5740 1 -- 172.21.15.103:0/1343583110 >> 172.21.15.103:0/1343583110 conn(0x55895c44ea30 msgr2=0x55895c44ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:39.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.942+0000 7f0581cc5740 1 -- 172.21.15.103:0/1343583110 shutdown_connections 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.942+0000 7f0581cc5740 1 -- 172.21.15.103:0/1343583110 wait complete. 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.943+0000 7f0581cc5740 1 -- start start 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.943+0000 7f0581cc5740 1 Processor -- start 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.943+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c45cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c452dc0 0x55895c45a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f0581cc5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55895c4602a0 con 0x55895c4524a0 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f0581cc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55895c373050 con 0x55895c45d680 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f0581cc5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55895c460420 con 0x55895c452dc0 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057ce87640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c45cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f0581cc5740 10 client.0 _ll_register_callbacks cb 0x55895c3b3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057ce87640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c45cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58472/0 (socket says 172.21.15.103:58472) 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057ce87640 1 -- 172.21.15.103:0/2467201583 learned_addr learned my addr 172.21.15.103:0/2467201583 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057d688640 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c452dc0 0x55895c45a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057ce87640 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057d688640 1 -- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 msgr2=0x55895c45a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057d688640 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057d688640 1 -- 172.21.15.103:0/2467201583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 msgr2=0x55895c45cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057d688640 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c45cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057d688640 1 -- 172.21.15.103:0/2467201583 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895c1293c0 con 0x55895c452dc0 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f057d688640 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c452dc0 0x55895c45a1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55895c44f890 tx=0x7f0568011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f9a9eb9a1ed2cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f055cff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:50:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.944+0000 7f055cff9640 1 -- 172.21.15.103:0/2467201583 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568012920 con 0x55895c452dc0 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f055cff9640 1 -- 172.21.15.103:0/2467201583 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f0568012ac0 con 0x55895c452dc0 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574005500 con 0x55895c452dc0 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f055cff9640 1 -- 172.21.15.103:0/2467201583 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0568012cc0 con 0x55895c452dc0 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 10 client.27286 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55895c46bcb0 con 0x55895c452dc0 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f055cff9640 1 -- 172.21.15.103:0/2467201583 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0568023950 con 0x55895c452dc0 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f055cff9640 1 -- 172.21.15.103:0/2467201583 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f056800f7e0 con 0x55895c452dc0 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 10 client.27286 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 10 client.27286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 20 client.27286 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c46d210 0x55895c46f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 4 client.27286 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:39.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.945+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0574005500 con 0x55895c46d210 2024-09-19T07:50:39.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.946+0000 7f057ce87640 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c46d210 0x55895c46f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.946+0000 7f057ce87640 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c46d210 0x55895c46f650 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade240600a5a7b4d server_cookie=3413b089a1081cf6 in_seq=0 out_seq=0 2024-09-19T07:50:39.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.946+0000 7f055cff9640 10 client.27286 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:39.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f055cff9640 1 -- 172.21.15.103:0/2467201583 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0574005500 con 0x55895c46d210 2024-09-19T07:50:39.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f055cff9640 10 client.27286 handle_command_reply: tid=0 2024-09-19T07:50:39.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 client.27286 shutdown 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c452dc0 msgr2=0x55895c45a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c452dc0 0x55895c45a1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55895c44f890 tx=0x7f0568011af0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 shutdown_connections 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c46d210 0x55895c46f650 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c45d680 0x55895c45a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c452dc0 0x55895c45a1a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 --2- 172.21.15.103:0/2467201583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c45cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 >> 172.21.15.103:0/2467201583 conn(0x55895c44ea30 msgr2=0x55895c44db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f055cff9640 0 client.27286 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 shutdown_connections 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 1 -- 172.21.15.103:0/2467201583 wait complete. 2024-09-19T07:50:39.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.947+0000 7f0581cc5740 20 client.27286 trim_cache size 0 max 16384 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 Processor -- start 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 -- start start 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c44ea30 0x55895c44ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c474f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c475460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f056400e7e0 con 0x55895c4524a0 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0564010350 con 0x55895c452dc0 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f0581cc5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f056400eeb0 con 0x55895c44ea30 2024-09-19T07:50:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f057d688640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c474f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f057ce87640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c475460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f057d688640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c474f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58480/0 (socket says 172.21.15.103:58480) 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.973+0000 7f057d688640 1 -- 172.21.15.103:0/3144638703 learned_addr learned my addr 172.21.15.103:0/3144638703 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f057ce87640 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c44ea30 0x55895c44ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f057ce87640 1 -- 172.21.15.103:0/3144638703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c44ea30 msgr2=0x55895c44ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f057ce87640 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c44ea30 0x55895c44ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f057ce87640 1 -- 172.21.15.103:0/3144638703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 msgr2=0x55895c474f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f057ce87640 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c474f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f057ce87640 1 -- 172.21.15.103:0/3144638703 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895c45d260 con 0x55895c452dc0 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f057ce87640 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c475460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55895c46fec0 tx=0x7f0574020040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c0244054176d879 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f055effd640 1 -- 172.21.15.103:0/3144638703 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574019040 con 0x55895c452dc0 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f055effd640 1 -- 172.21.15.103:0/3144638703 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574004520 con 0x55895c452dc0 2024-09-19T07:50:39.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.974+0000 7f055effd640 1 -- 172.21.15.103:0/3144638703 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574005280 con 0x55895c452dc0 2024-09-19T07:50:39.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 -- 172.21.15.103:0/3144638703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 msgr2=0x55895c475460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c475460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55895c46fec0 tx=0x7f0574020040 comp rx=0 tx=0).stop 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 -- 172.21.15.103:0/3144638703 shutdown_connections 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c475460 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c4524a0 0x55895c474f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3144638703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c44ea30 0x55895c44ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 -- 172.21.15.103:0/3144638703 >> 172.21.15.103:0/3144638703 conn(0x55895c45a840 msgr2=0x55895c455850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 -- 172.21.15.103:0/3144638703 shutdown_connections 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 -- 172.21.15.103:0/3144638703 wait complete. 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 -- start start 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.975+0000 7f0581cc5740 1 Processor -- start 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c44ea30 0x55895c461470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c4619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f0581cc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c461ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f0581cc5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f056400fc50 con 0x55895c44ea30 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f0581cc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f056400f580 con 0x55895c452dc0 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f0581cc5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f056400c7e0 con 0x55895c4524a0 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f0581cc5740 10 client.0 _ll_register_callbacks cb 0x7f05680094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f057d688640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c4619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f057d688640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c4619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40702/0 (socket says 172.21.15.103:40702) 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.976+0000 7f057d688640 1 -- 172.21.15.103:0/3002963058 learned_addr learned my addr 172.21.15.103:0/3002963058 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057ce87640 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c461ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057d688640 1 -- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 msgr2=0x55895c461ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057d688640 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c461ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057d688640 1 -- 172.21.15.103:0/3002963058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c44ea30 msgr2=0x55895c461470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:39.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057d688640 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c44ea30 0x55895c461470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057d688640 1 -- 172.21.15.103:0/3002963058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895c4d6f20 con 0x55895c4524a0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057ce87640 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c461ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f057d688640 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c4619b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f0568009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ff5adc6551402 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f055cff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f055cff9640 1 -- 172.21.15.103:0/3002963058 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568031060 con 0x55895c4524a0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.977+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574004520 con 0x55895c4524a0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f055cff9640 1 -- 172.21.15.103:0/3002963058 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f0568004440 con 0x55895c4524a0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f055cff9640 1 -- 172.21.15.103:0/3002963058 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0568012e60 con 0x55895c4524a0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 10 client.27292 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55895c4d8220 con 0x55895c4524a0 2024-09-19T07:50:39.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f055cff9640 1 -- 172.21.15.103:0/3002963058 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574004520 con 0x55895c4524a0 2024-09-19T07:50:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f055cff9640 1 -- 172.21.15.103:0/3002963058 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f0568003ab0 con 0x55895c4524a0 2024-09-19T07:50:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 10 client.27292 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 10 client.27292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:50:39.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 20 client.27292 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c45d680 0x55895c4daa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 4 client.27292 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.978+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55895c3c5850 con 0x55895c45d680 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f057ce87640 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c45d680 0x55895c4daa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f055cff9640 10 client.27292 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f057ce87640 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c45d680 0x55895c4daa40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b913952504fdd6ba server_cookie=3573d7cfbaec8c41 in_seq=0 out_seq=0 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f055cff9640 1 -- 172.21.15.103:0/3002963058 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55895c3c5850 con 0x55895c45d680 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f055cff9640 10 client.27292 handle_command_reply: tid=0 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f0581cc5740 1 client.27292 shutdown 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 msgr2=0x55895c4619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c4619b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f0568009d20 comp rx=0 tx=0).stop 2024-09-19T07:50:39.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 shutdown_connections 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.979+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55895c45d680 0x55895c4daa40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895c452dc0 0x55895c461ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55895c4524a0 0x55895c4619b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f0581cc5740 1 --2- 172.21.15.103:0/3002963058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55895c44ea30 0x55895c461470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stdout: "c4387de3-7b8c-4500-936e-2b02f76c96f7": { 2024-09-19T07:50:39.985 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:50:39.986 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "c4387de3-7b8c-4500-936e-2b02f76c96f7", 2024-09-19T07:50:39.986 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:50:39.986 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:50:39.986 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:50:39.986 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 >> 172.21.15.103:0/3002963058 conn(0x55895c45a840 msgr2=0x55895c45ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f055cff9640 0 client.27292 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 shutdown_connections 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f0581cc5740 1 -- 172.21.15.103:0/3002963058 wait complete. 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f0581cc5740 20 client.27292 trim_cache size 0 max 16384 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f05600a34b0 msgr2=0x7f05600a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f05600a34b0 0x7f05600a5970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f05700b9cd0 tx=0x7f05700b8e70 comp rx=0 tx=0).stop 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0578106f20 msgr2=0x7f0578074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0578106f20 0x7f0578074f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0570031a20 tx=0x7f0570031a50 comp rx=0 tx=0).stop 2024-09-19T07:50:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 shutdown_connections 2024-09-19T07:50:39.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f05600a34b0 0x7f05600a5970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0578106f20 0x7f0578074f70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05781027a0 0x7f0578074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 --2- 172.21.15.103:0/3986711130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0578101df0 0x7f05780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:39.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.980+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 >> 172.21.15.103:0/3986711130 conn(0x7f05780fd8b0 msgr2=0x7f05780feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:39.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.981+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 shutdown_connections 2024-09-19T07:50:39.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:39.981+0000 7f057f8ec640 1 -- 172.21.15.103:0/3986711130 wait complete. 2024-09-19T07:50:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:40 smithi174 ceph-mon[35370]: pgmap v395: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 586 op/s 2024-09-19T07:50:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 725.0 s, 1910.4 tps, lat 52.322 ms stddev 58.501 2024-09-19T07:50:40.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:40 smithi103 ceph-mon[31201]: pgmap v395: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 586 op/s 2024-09-19T07:50:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:40 smithi134 ceph-mon[35414]: pgmap v395: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 586 op/s 2024-09-19T07:50:40.789 INFO:tasks.cephfs.filesystem:scrub status for tag:c4387de3-7b8c-4500-936e-2b02f76c96f7 - {'path': '/', 'tag': 'c4387de3-7b8c-4500-936e-2b02f76c96f7', 'options': 'recursive,force'} 2024-09-19T07:50:42.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:42 smithi174 ceph-mon[35370]: pgmap v396: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 855 op/s 2024-09-19T07:50:42.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:42 smithi103 ceph-mon[31201]: pgmap v396: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 855 op/s 2024-09-19T07:50:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:42 smithi134 ceph-mon[35414]: pgmap v396: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 855 op/s 2024-09-19T07:50:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:44 smithi174 ceph-mon[35370]: pgmap v397: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 600 op/s 2024-09-19T07:50:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:50:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:50:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:50:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:44 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:50:44.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:44 smithi103 ceph-mon[31201]: pgmap v397: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 600 op/s 2024-09-19T07:50:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:50:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:50:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:50:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:44 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:50:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:44 smithi134 ceph-mon[35414]: pgmap v397: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 600 op/s 2024-09-19T07:50:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:50:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:50:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:50:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:44 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:50:45.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:45 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:50:45.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 730.0 s, 2214.4 tps, lat 45.190 ms stddev 50.195 2024-09-19T07:50:45.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:45 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:50:45.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:45 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:50:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:46 smithi174 ceph-mon[35370]: pgmap v398: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 610 op/s 2024-09-19T07:50:46.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:46 smithi103 ceph-mon[31201]: pgmap v398: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 610 op/s 2024-09-19T07:50:46.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:46 smithi134 ceph-mon[35414]: pgmap v398: 97 pgs: 97 active+clean; 3.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 610 op/s 2024-09-19T07:50:48.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:48 smithi174 ceph-mon[35370]: pgmap v399: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 801 op/s 2024-09-19T07:50:48.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:48 smithi103 ceph-mon[31201]: pgmap v399: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 801 op/s 2024-09-19T07:50:48.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:48 smithi134 ceph-mon[35414]: pgmap v399: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 801 op/s 2024-09-19T07:50:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:50 smithi174 ceph-mon[35370]: pgmap v400: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-09-19T07:50:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 735.0 s, 2285.2 tps, lat 43.743 ms stddev 48.623 2024-09-19T07:50:50.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:50 smithi103 ceph-mon[31201]: pgmap v400: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-09-19T07:50:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:50 smithi134 ceph-mon[35414]: pgmap v400: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 542 op/s 2024-09-19T07:50:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:52 smithi174 ceph-mon[35370]: pgmap v401: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.6 MiB/s wr, 813 op/s 2024-09-19T07:50:52.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:52 smithi103 ceph-mon[31201]: pgmap v401: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.6 MiB/s wr, 813 op/s 2024-09-19T07:50:52.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:52 smithi134 ceph-mon[35414]: pgmap v401: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.6 MiB/s wr, 813 op/s 2024-09-19T07:50:52.633 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:50:52.633 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T07:50:52.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:50:53.005 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:54.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.041+0000 7f34eaf08640 1 -- 172.21.15.103:0/4239066560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e41049a0 msgr2=0x7f34e41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:54.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.041+0000 7f34eaf08640 1 --2- 172.21.15.103:0/4239066560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e41049a0 0x7f34e41089f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009900 tx=0x7f34d002a410 comp rx=0 tx=0).stop 2024-09-19T07:50:54.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.042+0000 7f34eaf08640 1 -- 172.21.15.103:0/4239066560 shutdown_connections 2024-09-19T07:50:54.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.042+0000 7f34eaf08640 1 --2- 172.21.15.103:0/4239066560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4109120 0x7f34e4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.042+0000 7f34eaf08640 1 --2- 172.21.15.103:0/4239066560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e41049a0 0x7f34e41089f0 secure :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009900 tx=0x7f34d002a410 comp rx=0 tx=0).stop 2024-09-19T07:50:54.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.042+0000 7f34eaf08640 1 --2- 172.21.15.103:0/4239066560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 0x7f34e41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.042+0000 7f34eaf08640 1 -- 172.21.15.103:0/4239066560 >> 172.21.15.103:0/4239066560 conn(0x7f34e40ffa70 msgr2=0x7f34e4101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:54.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.043+0000 7f34eaf08640 1 -- 172.21.15.103:0/4239066560 shutdown_connections 2024-09-19T07:50:54.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.043+0000 7f34eaf08640 1 -- 172.21.15.103:0/4239066560 wait complete. 2024-09-19T07:50:54.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 Processor -- start 2024-09-19T07:50:54.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 -- start start 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 0x7f34e41a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4109120 0x7f34e41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e419a670 0x7f34e419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4114140 con 0x7f34e419a670 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4113fc0 con 0x7f34e4103ff0 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34eaf08640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34e41142c0 con 0x7f34e4109120 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34e8ca4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 0x7f34e41a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34e8ca4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 0x7f34e41a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56890/0 (socket says 172.21.15.103:56890) 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34e8ca4640 1 -- 172.21.15.103:0/3453529923 learned_addr learned my addr 172.21.15.103:0/3453529923 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34e3fff640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4109120 0x7f34e41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.044+0000 7f34e94a5640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e419a670 0x7f34e419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:54.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e8ca4640 1 -- 172.21.15.103:0/3453529923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4109120 msgr2=0x7f34e41a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e8ca4640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4109120 0x7f34e41a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e8ca4640 1 -- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e419a670 msgr2=0x7f34e419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e8ca4640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e419a670 0x7f34e419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e8ca4640 1 -- 172.21.15.103:0/3453529923 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e419b220 con 0x7f34e4103ff0 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e8ca4640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 0x7f34e41a05a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9f0 tx=0x7f34d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e1ffb640 1 -- 172.21.15.103:0/3453529923 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800cd80 con 0x7f34e4103ff0 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e41a7350 con 0x7f34e4103ff0 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e41a76f0 con 0x7f34e4103ff0 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e1ffb640 1 -- 172.21.15.103:0/3453529923 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d8010910 con 0x7f34e4103ff0 2024-09-19T07:50:54.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.045+0000 7f34e1ffb640 1 -- 172.21.15.103:0/3453529923 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8018980 con 0x7f34e4103ff0 2024-09-19T07:50:54.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.046+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c8005290 con 0x7f34e4103ff0 2024-09-19T07:50:54.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.048+0000 7f34e1ffb640 1 -- 172.21.15.103:0/3453529923 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f34d8010ac0 con 0x7f34e4103ff0 2024-09-19T07:50:54.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.048+0000 7f34e1ffb640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34d40a33b0 0x7f34d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:54.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.048+0000 7f34e3fff640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34d40a33b0 0x7f34d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:54.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.049+0000 7f34e1ffb640 1 -- 172.21.15.103:0/3453529923 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f34d80d0060 con 0x7f34e4103ff0 2024-09-19T07:50:54.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.049+0000 7f34e3fff640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34d40a33b0 0x7f34d40a5870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005940 tx=0x7f34d00058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:54.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.051+0000 7f34e1ffb640 1 -- 172.21.15.103:0/3453529923 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f34d809c380 con 0x7f34e4103ff0 2024-09-19T07:50:54.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:54 smithi103 ceph-mon[31201]: pgmap v402: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 544 op/s 2024-09-19T07:50:54.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.376+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34c8005580 con 0x7f34e4103ff0 2024-09-19T07:50:54.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.376+0000 7f34e1ffb640 1 -- 172.21.15.103:0/3453529923 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11992 (secure 0 0 0) 0x7f34d80a0680 con 0x7f34e4103ff0 2024-09-19T07:50:54.378 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:50:54.378 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":51,"btime":"2024-09-19T07:50:10:429478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:50:09.866710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:50:54.378 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 51 2024-09-19T07:50:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.379+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34d40a33b0 msgr2=0x7f34d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34d40a33b0 0x7f34d40a5870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005940 tx=0x7f34d00058d0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 msgr2=0x7f34e41a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:54.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 0x7f34e41a05a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9f0 tx=0x7f34d800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 shutdown_connections 2024-09-19T07:50:54.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f34d40a33b0 0x7f34d40a5870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e419a670 0x7f34e419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e4109120 0x7f34e41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 --2- 172.21.15.103:0/3453529923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34e4103ff0 0x7f34e41a05a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.380+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 >> 172.21.15.103:0/3453529923 conn(0x7f34e40ffa70 msgr2=0x7f34e41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.381+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 shutdown_connections 2024-09-19T07:50:54.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:54.381+0000 7f34eaf08640 1 -- 172.21.15.103:0/3453529923 wait complete. 2024-09-19T07:50:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:54 smithi174 ceph-mon[35370]: pgmap v402: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 544 op/s 2024-09-19T07:50:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:54 smithi134 ceph-mon[35414]: pgmap v402: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 544 op/s 2024-09-19T07:50:55.196 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T07:50:55.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T07:50:55.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:55 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3453529923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:50:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 740.0 s, 2434.8 tps, lat 40.915 ms stddev 43.564 2024-09-19T07:50:55.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:55 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3453529923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:50:55.560 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:50:55.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:55 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3453529923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:50:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:56 smithi174 ceph-mon[35370]: pgmap v403: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 555 op/s 2024-09-19T07:50:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:56 smithi103 ceph-mon[31201]: pgmap v403: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 555 op/s 2024-09-19T07:50:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:56 smithi134 ceph-mon[35414]: pgmap v403: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 555 op/s 2024-09-19T07:50:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 -- 172.21.15.103:0/3798193838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1508100a90 msgr2=0x7f1508100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 --2- 172.21.15.103:0/3798193838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1508100a90 0x7f1508100ef0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009930 tx=0x7f14f802a420 comp rx=0 tx=0).stop 2024-09-19T07:50:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 -- 172.21.15.103:0/3798193838 shutdown_connections 2024-09-19T07:50:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 --2- 172.21.15.103:0/3798193838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1508101430 0x7f150810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 --2- 172.21.15.103:0/3798193838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1508100a90 0x7f1508100ef0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 --2- 172.21.15.103:0/3798193838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508106a90 0x7f1508106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 -- 172.21.15.103:0/3798193838 >> 172.21.15.103:0/3798193838 conn(0x7f15080fc7d0 msgr2=0x7f15080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 -- 172.21.15.103:0/3798193838 shutdown_connections 2024-09-19T07:50:56.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.812+0000 7f150fd55640 1 -- 172.21.15.103:0/3798193838 wait complete. 2024-09-19T07:50:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 Processor -- start 2024-09-19T07:50:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 -- start start 2024-09-19T07:50:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508100a90 0x7f1508107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1508101430 0x7f1508107f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1508106a90 0x7f1508108480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1508110ea0 con 0x7f1508101430 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1508110d20 con 0x7f1508100a90 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150fd55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1508111020 con 0x7f1508106a90 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150daf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508100a90 0x7f1508107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150daf1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508100a90 0x7f1508107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56902/0 (socket says 172.21.15.103:56902) 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.813+0000 7f150daf1640 1 -- 172.21.15.103:0/879440291 learned_addr learned my addr 172.21.15.103:0/879440291 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.814+0000 7f150daf1640 1 -- 172.21.15.103:0/879440291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1508106a90 msgr2=0x7f1508108480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.814+0000 7f150daf1640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1508106a90 0x7f1508108480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.814+0000 7f150daf1640 1 -- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1508101430 msgr2=0x7f1508107f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:56.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.814+0000 7f150daf1640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1508101430 0x7f1508107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.814+0000 7f150daf1640 1 -- 172.21.15.103:0/879440291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15081aae90 con 0x7f1508100a90 2024-09-19T07:50:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.814+0000 7f150daf1640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508100a90 0x7f1508107a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f150400e9f0 tx=0x7f150400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:56.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.814+0000 7f14feffd640 1 -- 172.21.15.103:0/879440291 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150400cd80 con 0x7f1508100a90 2024-09-19T07:50:56.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.815+0000 7f14feffd640 1 -- 172.21.15.103:0/879440291 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1504010910 con 0x7f1508100a90 2024-09-19T07:50:56.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.815+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15081ab0c0 con 0x7f1508100a90 2024-09-19T07:50:56.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.815+0000 7f14feffd640 1 -- 172.21.15.103:0/879440291 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040188f0 con 0x7f1508100a90 2024-09-19T07:50:56.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.815+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15081ab650 con 0x7f1508100a90 2024-09-19T07:50:56.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.818+0000 7f14feffd640 1 -- 172.21.15.103:0/879440291 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1504018b90 con 0x7f1508100a90 2024-09-19T07:50:56.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 Processor -- start 2024-09-19T07:50:56.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f14feffd640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f14f40a3430 0x7f14f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 -- start start 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f14feffd640 1 -- 172.21.15.103:0/879440291 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f15040d0270 con 0x7f1508100a90 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c0897b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5605c07fd2a0 con 0x5605c08a2940 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605c07a4b10 con 0x5605c0898080 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f151212e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5605c07f9260 con 0x5605c0897760 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f150e2f2640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f14f40a3430 0x7f14f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f150d2f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f150d2f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56906/0 (socket says 172.21.15.103:56906) 2024-09-19T07:50:56.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.819+0000 7f150d2f0640 1 -- 172.21.15.103:0/3817878619 learned_addr learned my addr 172.21.15.103:0/3817878619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150daf1640 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c0897b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150e2f2640 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150d2f0640 1 -- 172.21.15.103:0/3817878619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 msgr2=0x5605c0897b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150d2f0640 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c0897b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150d2f0640 1 -- 172.21.15.103:0/3817878619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 msgr2=0x5605c08a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150d2f0640 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150d2f0640 1 -- 172.21.15.103:0/3817878619 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c056eb90 con 0x5605c0898080 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150daf1640 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c0897b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150e2f2640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f14f40a3430 0x7f14f40a58f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5605c0895070 tx=0x7f1500009480 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f150d2f0640 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a2400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14f802a940 tx=0x7f14f80314d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade165b01a33346c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.820+0000 7f14f3fff640 1 -- 172.21.15.103:0/3817878619 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f803e030 con 0x5605c0898080 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f14f3fff640 1 -- 172.21.15.103:0/3817878619 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f80035f0 con 0x5605c0898080 2024-09-19T07:50:56.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f14f3fff640 1 -- 172.21.15.103:0/3817878619 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8035040 con 0x5605c0898080 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f151212e740 1 -- 172.21.15.103:0/3817878619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 msgr2=0x5605c08a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f151212e740 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a2400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14f802a940 tx=0x7f14f80314d0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f151212e740 1 -- 172.21.15.103:0/3817878619 shutdown_connections 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f151212e740 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f151212e740 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a2400 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.821+0000 7f151212e740 1 --2- 172.21.15.103:0/3817878619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c0897b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.822+0000 7f151212e740 1 -- 172.21.15.103:0/3817878619 >> 172.21.15.103:0/3817878619 conn(0x5605c0893cf0 msgr2=0x5605c08940d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:56.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.822+0000 7f151212e740 1 -- 172.21.15.103:0/3817878619 shutdown_connections 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.823+0000 7f151212e740 1 -- 172.21.15.103:0/3817878619 wait complete. 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.824+0000 7f151212e740 1 -- start start 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.824+0000 7f151212e740 1 Processor -- start 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.824+0000 7f151212e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c0897760 0x5605c089f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.824+0000 7f151212e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c089fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.824+0000 7f151212e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c08a2940 0x5605c08a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f151212e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5605c08a5510 con 0x5605c0897760 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f151212e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605c07856e0 con 0x5605c08a2940 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f151212e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5605c08a5690 con 0x5605c0898080 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c08a2940 0x5605c08a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150daf1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c0897760 0x5605c089f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c08a2940 0x5605c08a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56916/0 (socket says 172.21.15.103:56916) 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 -- 172.21.15.103:0/3697507825 learned_addr learned my addr 172.21.15.103:0/3697507825 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150d2f0640 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c089fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f151212e740 10 client.0 _ll_register_callbacks cb 0x5605c07a9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 -- 172.21.15.103:0/3697507825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 msgr2=0x5605c089fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c089fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 -- 172.21.15.103:0/3697507825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c0897760 msgr2=0x5605c089f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c0897760 0x5605c089f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 -- 172.21.15.103:0/3697507825 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c08ae9e0 con 0x5605c08a2940 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f150e2f2640 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c08a2940 0x5605c08a0210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1500013c10 tx=0x7f1500005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3384dbbc60d9b222 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f14f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:50:56.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f14f17fa640 1 -- 172.21.15.103:0/3697507825 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500027030 con 0x5605c08a2940 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f14f17fa640 1 -- 172.21.15.103:0/3697507825 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f1500013c40 con 0x5605c08a2940 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.825+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f80035f0 con 0x5605c08a2940 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f14f17fa640 1 -- 172.21.15.103:0/3697507825 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f15000095f0 con 0x5605c08a2940 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 10 client.27341 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5605c08b1390 con 0x5605c08a2940 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f14f17fa640 1 -- 172.21.15.103:0/3697507825 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f80035f0 con 0x5605c08a2940 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f14f17fa640 1 -- 172.21.15.103:0/3697507825 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f1500022c60 con 0x5605c08a2940 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 10 client.27341 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:56.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 10 client.27341 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 20 client.27341 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c08b28a0 0x5605c08b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 4 client.27341 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605c080a9d0 con 0x5605c08b28a0 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.826+0000 7f150daf1640 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c08b28a0 0x5605c08b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.827+0000 7f150daf1640 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c08b28a0 0x5605c08b4d40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e6bc1312f1d59f server_cookie=72fcf1cfacdcd2ac in_seq=0 out_seq=0 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.827+0000 7f14f17fa640 10 client.27341 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.827+0000 7f14f17fa640 1 -- 172.21.15.103:0/3697507825 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605c080a9d0 con 0x5605c08b28a0 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.827+0000 7f14f17fa640 10 client.27341 handle_command_reply: tid=0 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.827+0000 7f151212e740 1 client.27341 shutdown 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c08a2940 msgr2=0x5605c08a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:56.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c08a2940 0x5605c08a0210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1500013c10 tx=0x7f1500005bd0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 shutdown_connections 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c08b28a0 0x5605c08b4d40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c08a2940 0x5605c08a0210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c089fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 --2- 172.21.15.103:0/3697507825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c0897760 0x5605c089f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f14f17fa640 0 client.27341 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 >> 172.21.15.103:0/3697507825 conn(0x5605c0893cf0 msgr2=0x5605c0892e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 shutdown_connections 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 1 -- 172.21.15.103:0/3697507825 wait complete. 2024-09-19T07:50:56.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.828+0000 7f151212e740 20 client.27341 trim_cache size 0 max 16384 2024-09-19T07:50:56.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 Processor -- start 2024-09-19T07:50:56.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 -- start start 2024-09-19T07:50:56.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0897760 0x5605c0892d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c08ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15081b3f40 con 0x5605c08a2940 2024-09-19T07:50:56.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15081b5ab0 con 0x5605c0897760 2024-09-19T07:50:56.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.852+0000 7f151212e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15081b4610 con 0x5605c0898080 2024-09-19T07:50:56.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.854+0000 7f150e2f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.854+0000 7f150e2f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08bab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56068/0 (socket says 172.21.15.103:56068) 2024-09-19T07:50:56.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.854+0000 7f150e2f2640 1 -- 172.21.15.103:0/822355147 learned_addr learned my addr 172.21.15.103:0/822355147 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:56.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.860+0000 7f150d2f0640 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c08ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.865+0000 7f150e2f2640 1 -- 172.21.15.103:0/822355147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 msgr2=0x5605c08ba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.865+0000 7f150e2f2640 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c08ba610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.865+0000 7f150e2f2640 1 -- 172.21.15.103:0/822355147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0897760 msgr2=0x5605c0892d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.865+0000 7f150e2f2640 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0897760 0x5605c0892d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.865+0000 7f150e2f2640 1 -- 172.21.15.103:0/822355147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c089e740 con 0x5605c08a2940 2024-09-19T07:50:56.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.873+0000 7f150e2f2640 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08bab50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f150002c520 tx=0x7f1500015a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea3dd6215a9822f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:56.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.878+0000 7f14f37fe640 1 -- 172.21.15.103:0/822355147 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500027070 con 0x5605c08a2940 2024-09-19T07:50:56.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.878+0000 7f14f37fe640 1 -- 172.21.15.103:0/822355147 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1500028040 con 0x5605c08a2940 2024-09-19T07:50:56.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.878+0000 7f14f37fe640 1 -- 172.21.15.103:0/822355147 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15000052a0 con 0x5605c08a2940 2024-09-19T07:50:56.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.879+0000 7f151212e740 1 -- 172.21.15.103:0/822355147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 msgr2=0x5605c08bab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:56.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.880+0000 7f151212e740 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08bab50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f150002c520 tx=0x7f1500015a00 comp rx=0 tx=0).stop 2024-09-19T07:50:56.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.883+0000 7f151212e740 1 -- 172.21.15.103:0/822355147 shutdown_connections 2024-09-19T07:50:56.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.883+0000 7f151212e740 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08bab50 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.883+0000 7f151212e740 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0898080 0x5605c08ba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.883+0000 7f151212e740 1 --2- 172.21.15.103:0/822355147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0897760 0x5605c0892d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.883+0000 7f151212e740 1 -- 172.21.15.103:0/822355147 >> 172.21.15.103:0/822355147 conn(0x5605c0893cf0 msgr2=0x5605c0892650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:56.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.885+0000 7f151212e740 1 -- 172.21.15.103:0/822355147 shutdown_connections 2024-09-19T07:50:56.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.886+0000 7f151212e740 1 -- 172.21.15.103:0/822355147 wait complete. 2024-09-19T07:50:56.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.886+0000 7f151212e740 1 -- start start 2024-09-19T07:50:56.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.886+0000 7f151212e740 1 Processor -- start 2024-09-19T07:50:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.887+0000 7f151212e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c08aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.887+0000 7f151212e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.887+0000 7f151212e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.887+0000 7f151212e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15081b53b0 con 0x5605c08a2940 2024-09-19T07:50:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.887+0000 7f151212e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15081b4ce0 con 0x5605c0898080 2024-09-19T07:50:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.887+0000 7f151212e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15081b1f40 con 0x5605c0897760 2024-09-19T07:50:56.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.887+0000 7f151212e740 10 client.0 _ll_register_callbacks cb 0x7f1500013790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150d2f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150daf1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c08aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56072/0 (socket says 172.21.15.103:56072) 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 -- 172.21.15.103:0/2259310837 learned_addr learned my addr 172.21.15.103:0/2259310837 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 -- 172.21.15.103:0/2259310837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 msgr2=0x5605c08aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c08aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 -- 172.21.15.103:0/2259310837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 msgr2=0x5605c08a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:56.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.888+0000 7f150e2f2640 1 -- 172.21.15.103:0/2259310837 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c08a8560 con 0x5605c08a2940 2024-09-19T07:50:56.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.889+0000 7f150e2f2640 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a7e50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f150002c690 tx=0x7f150002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f984c19d0f8120c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:50:56.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.889+0000 7f14f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:50:56.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.889+0000 7f14f17fa640 1 -- 172.21.15.103:0/2259310837 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500027070 con 0x5605c08a2940 2024-09-19T07:50:56.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.889+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1500028040 con 0x5605c08a2940 2024-09-19T07:50:56.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.889+0000 7f14f17fa640 1 -- 172.21.15.103:0/2259310837 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f1500004b20 con 0x5605c08a2940 2024-09-19T07:50:56.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.890+0000 7f14f17fa640 1 -- 172.21.15.103:0/2259310837 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f150004e050 con 0x5605c08a2940 2024-09-19T07:50:56.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.890+0000 7f151212e740 10 client.18009 fetch_fsmap learned FSMap version 51 2024-09-19T07:50:56.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.890+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5605c091e510 con 0x5605c08a2940 2024-09-19T07:50:56.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.891+0000 7f14f17fa640 1 -- 172.21.15.103:0/2259310837 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1500028040 con 0x5605c08a2940 2024-09-19T07:50:56.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f14f17fa640 1 -- 172.21.15.103:0/2259310837 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f1500035440 con 0x5605c08a2940 2024-09-19T07:50:56.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f151212e740 10 client.18009 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:50:56.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f151212e740 10 client.18009 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:50:56.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f151212e740 20 client.18009 populate_metadata read hostname 'smithi103' 2024-09-19T07:50:56.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f151212e740 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c091ebb0 0x5605c0920fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:50:56.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f151212e740 4 client.18009 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T07:50:56.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x5605c0804d20 con 0x5605c091ebb0 2024-09-19T07:50:56.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.892+0000 7f150daf1640 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c091ebb0 0x5605c0920fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:50:56.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.893+0000 7f150daf1640 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c091ebb0 0x5605c0920fd0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff1eed90928b9cb server_cookie=9ee2c943fa8e682d in_seq=0 out_seq=0 2024-09-19T07:50:56.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:56.893+0000 7f14f17fa640 10 client.18009 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:57.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f14f17fa640 1 -- 172.21.15.103:0/2259310837 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5605c0804d20 con 0x5605c091ebb0 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f14f17fa640 10 client.18009 handle_command_reply: tid=0 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 client.18009 shutdown 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 msgr2=0x5605c08a7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a7e50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f150002c690 tx=0x7f150002c940 comp rx=0 tx=0).stop 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 shutdown_connections 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5605c091ebb0 0x5605c0920fd0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5605c08a2940 0x5605c08a7e50 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605c0898080 0x5605c08a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f14f17fa640 0 client.18009 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:50:57.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 --2- 172.21.15.103:0/2259310837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605c0897760 0x5605c08aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 >> 172.21.15.103:0/2259310837 conn(0x5605c0893cf0 msgr2=0x5605c089ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 shutdown_connections 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.070+0000 7f151212e740 1 -- 172.21.15.103:0/2259310837 wait complete. 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f151212e740 20 client.18009 trim_cache size 0 max 16384 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 51, 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 42, 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:50:57.073 INFO:teuthology.orchestra.run.smithi103.stdout: "ac701e75": { 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 42, 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:50:57.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:50:57.075 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:50:57.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:50:57.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:50:57.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:50:57.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:50:57.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:50:57.075 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f14f40a3430 msgr2=0x7f14f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f14f40a3430 0x7f14f40a58f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5605c0895070 tx=0x7f1500009480 comp rx=0 tx=0).stop 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508100a90 msgr2=0x7f1508107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508100a90 0x7f1508107a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f150400e9f0 tx=0x7f150400eeb0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 shutdown_connections 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f14f40a3430 0x7f14f40a58f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1508106a90 0x7f1508108480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1508101430 0x7f1508107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 --2- 172.21.15.103:0/879440291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1508100a90 0x7f1508107a00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 >> 172.21.15.103:0/879440291 conn(0x7f15080fc7d0 msgr2=0x7f15080fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:50:57.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 shutdown_connections 2024-09-19T07:50:57.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:50:57.071+0000 7f150fd55640 1 -- 172.21.15.103:0/879440291 wait complete. 2024-09-19T07:50:58.057 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ac701e75', quiesce took 5.4 seconds. Will release after: 16.799999999999997 2024-09-19T07:50:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:50:58 smithi174 ceph-mon[35370]: pgmap v404: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-09-19T07:50:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:50:58 smithi103 ceph-mon[31201]: pgmap v404: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-09-19T07:50:58.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:50:58 smithi134 ceph-mon[35414]: pgmap v404: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-09-19T07:51:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:00 smithi174 ceph-mon[35370]: pgmap v405: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.2 MiB/s wr, 552 op/s 2024-09-19T07:51:00.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 745.0 s, 509.6 tps, lat 57.349 ms stddev 56.857 2024-09-19T07:51:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:00 smithi103 ceph-mon[31201]: pgmap v405: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.2 MiB/s wr, 552 op/s 2024-09-19T07:51:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:00 smithi134 ceph-mon[35414]: pgmap v405: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.2 MiB/s wr, 552 op/s 2024-09-19T07:51:02.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:02 smithi174 ceph-mon[35370]: pgmap v406: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.3 MiB/s wr, 678 op/s 2024-09-19T07:51:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:02 smithi103 ceph-mon[31201]: pgmap v406: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.3 MiB/s wr, 678 op/s 2024-09-19T07:51:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:02 smithi134 ceph-mon[35414]: pgmap v406: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.3 MiB/s wr, 678 op/s 2024-09-19T07:51:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:04 smithi174 ceph-mon[35370]: pgmap v407: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.8 MiB/s wr, 407 op/s 2024-09-19T07:51:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:04 smithi103 ceph-mon[31201]: pgmap v407: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.8 MiB/s wr, 407 op/s 2024-09-19T07:51:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:04 smithi134 ceph-mon[35414]: pgmap v407: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.8 MiB/s wr, 407 op/s 2024-09-19T07:51:05.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:05 smithi103 ceph-mon[31201]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:51:05.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 750.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:51:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:05 smithi134 ceph-mon[35414]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:51:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:05 smithi174 ceph-mon[35370]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T07:51:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:06 smithi103 ceph-mon[31201]: pgmap v409: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.4 MiB/s wr, 493 op/s 2024-09-19T07:51:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:06 smithi134 ceph-mon[35414]: pgmap v409: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.4 MiB/s wr, 493 op/s 2024-09-19T07:51:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:06 smithi174 ceph-mon[35370]: pgmap v409: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.4 MiB/s wr, 493 op/s 2024-09-19T07:51:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:08 smithi103 ceph-mon[31201]: pgmap v410: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-09-19T07:51:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:08 smithi134 ceph-mon[35414]: pgmap v410: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-09-19T07:51:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:08 smithi174 ceph-mon[35370]: pgmap v410: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-09-19T07:51:10.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:10 smithi103 ceph-mon[31201]: pgmap v411: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-09-19T07:51:10.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 755.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:51:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:10 smithi134 ceph-mon[35414]: pgmap v411: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-09-19T07:51:10.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:10 smithi174 ceph-mon[35370]: pgmap v411: 97 pgs: 97 active+clean; 3.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-09-19T07:51:10.790 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:51:11.054 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 -- 172.21.15.103:0/4205868466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 msgr2=0x7fc4701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 --2- 172.21.15.103:0/4205868466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc4701029b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c02a480 comp rx=0 tx=0).stop 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 -- 172.21.15.103:0/4205868466 shutdown_connections 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 --2- 172.21.15.103:0/4205868466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102ef0 0x7fc47010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 --2- 172.21.15.103:0/4205868466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc4701029b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 --2- 172.21.15.103:0/4205868466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc470100e20 0x7fc470101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 -- 172.21.15.103:0/4205868466 >> 172.21.15.103:0/4205868466 conn(0x7fc4700fc7d0 msgr2=0x7fc4700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 -- 172.21.15.103:0/4205868466 shutdown_connections 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.542+0000 7fc476daf640 1 -- 172.21.15.103:0/4205868466 wait complete. 2024-09-19T07:51:11.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 Processor -- start 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 -- start start 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc470100e20 0x7fc4701973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc470197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102ef0 0x7fc47019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc470110fa0 con 0x7fc470102570 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc470110e20 con 0x7fc470100e20 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc476daf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc470111120 con 0x7fc470102ef0 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc470197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc470197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35720/0 (socket says 172.21.15.103:35720) 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 -- 172.21.15.103:0/204105844 learned_addr learned my addr 172.21.15.103:0/204105844 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:11.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc474b4b640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc470100e20 0x7fc4701973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc47534c640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102ef0 0x7fc47019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 -- 172.21.15.103:0/204105844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102ef0 msgr2=0x7fc47019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102ef0 0x7fc47019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 -- 172.21.15.103:0/204105844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc470100e20 msgr2=0x7fc4701973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc470100e20 0x7fc4701973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc46ffff640 1 -- 172.21.15.103:0/204105844 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47019c390 con 0x7fc470102570 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.543+0000 7fc474b4b640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc470100e20 0x7fc4701973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.544+0000 7fc46ffff640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc470197920 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc45c006010 tx=0x7fc45c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.544+0000 7fc46dffb640 1 -- 172.21.15.103:0/204105844 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c03f070 con 0x7fc470102570 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.544+0000 7fc46dffb640 1 -- 172.21.15.103:0/204105844 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c031e30 con 0x7fc470102570 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.544+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47019c620 con 0x7fc470102570 2024-09-19T07:51:11.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.544+0000 7fc46dffb640 1 -- 172.21.15.103:0/204105844 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c035030 con 0x7fc470102570 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.544+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4701a3f60 con 0x7fc470102570 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc46dffb640 1 -- 172.21.15.103:0/204105844 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc45c042070 con 0x7fc470102570 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 Processor -- start 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 -- start start 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 0x56094149f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149fdc0 0x5609414aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609414aa680 0x5609414aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5609414052e0 con 0x5609414aa680 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609413acce0 con 0x56094149f4a0 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc479188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5609414012a0 con 0x56094149fdc0 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609414aa680 0x5609414aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609414aa680 0x5609414aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35734/0 (socket says 172.21.15.103:35734) 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc46ffff640 1 -- 172.21.15.103:0/3972750519 learned_addr learned my addr 172.21.15.103:0/3972750519 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc47534c640 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 0x56094149f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.547+0000 7fc474b4b640 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149fdc0 0x5609414aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46dffb640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4600a33b0 0x7fc4600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46ffff640 1 -- 172.21.15.103:0/3972750519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149fdc0 msgr2=0x5609414aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46ffff640 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149fdc0 0x5609414aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46ffff640 1 -- 172.21.15.103:0/3972750519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 msgr2=0x56094149f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46ffff640 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 0x56094149f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46ffff640 1 -- 172.21.15.103:0/3972750519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560941391440 con 0x5609414aa680 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc47534c640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4600a33b0 0x7fc4600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46dffb640 1 -- 172.21.15.103:0/204105844 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc45c0f0730 con 0x7fc470102570 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc46ffff640 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609414aa680 0x5609414aca70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0edde0 tx=0x7fc45c0b8ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905bd99cbaea5c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:11.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc45affd640 1 -- 172.21.15.103:0/3972750519 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c049070 con 0x5609414aa680 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc45affd640 1 -- 172.21.15.103:0/3972750519 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c085e50 con 0x5609414aa680 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.548+0000 7fc45affd640 1 -- 172.21.15.103:0/3972750519 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0bc540 con 0x5609414aa680 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 -- 172.21.15.103:0/3972750519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609414aa680 msgr2=0x5609414aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609414aa680 0x5609414aca70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0edde0 tx=0x7fc45c0b8ad0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc47534c640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4600a33b0 0x7fc4600a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc46800a4e0 tx=0x7fc46800c040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 -- 172.21.15.103:0/3972750519 shutdown_connections 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609414aa680 0x5609414aca70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149fdc0 0x5609414aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 --2- 172.21.15.103:0/3972750519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 0x56094149f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 -- 172.21.15.103:0/3972750519 >> 172.21.15.103:0/3972750519 conn(0x56094149ba30 msgr2=0x56094149be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 -- 172.21.15.103:0/3972750519 shutdown_connections 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 -- 172.21.15.103:0/3972750519 wait complete. 2024-09-19T07:51:11.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 -- start start 2024-09-19T07:51:11.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.549+0000 7fc479188740 1 Processor -- start 2024-09-19T07:51:11.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc479188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149f4a0 0x5609414a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc479188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc479188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609414aa680 0x5609414a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc479188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5609414ad010 con 0x56094149fdc0 2024-09-19T07:51:11.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc479188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609413c0050 con 0x5609414aa680 2024-09-19T07:51:11.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc479188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5609414ad190 con 0x56094149f4a0 2024-09-19T07:51:11.593 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:51:11.593 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:51:11.593 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:51:11.593 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:51:11.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc479188740 10 client.0 _ll_register_callbacks cb 0x560941400160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:11.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35748/0 (socket says 172.21.15.103:35748) 2024-09-19T07:51:11.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 -- 172.21.15.103:0/3956130052 learned_addr learned my addr 172.21.15.103:0/3956130052 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:11.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc474b4b640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149f4a0 0x5609414a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc47534c640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609414aa680 0x5609414a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 -- 172.21.15.103:0/3956130052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149f4a0 msgr2=0x5609414a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149f4a0 0x5609414a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 -- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609414aa680 msgr2=0x5609414a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609414aa680 0x5609414a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 -- 172.21.15.103:0/3956130052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609411763c0 con 0x56094149fdc0 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc47534c640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609414aa680 0x5609414a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc46ffff640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414a7240 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0c5140 tx=0x7fc45c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e89290243a5349 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc44ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:11.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc44ffff640 1 -- 172.21.15.103:0/3956130052 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c049070 con 0x56094149fdc0 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.550+0000 7fc44ffff640 1 -- 172.21.15.103:0/3956130052 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc45c0a70d0 con 0x56094149fdc0 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45c085e50 con 0x56094149fdc0 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc44ffff640 1 -- 172.21.15.103:0/3956130052 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc45c0bc360 con 0x56094149fdc0 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 10 client.18021 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5609414ad010 con 0x56094149fdc0 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc44ffff640 1 -- 172.21.15.103:0/3956130052 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c085e50 con 0x56094149fdc0 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc44ffff640 1 -- 172.21.15.103:0/3956130052 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fc45c0ee0f0 con 0x56094149fdc0 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 10 client.18021 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 20 client.18021 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x5609414bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 4 client.18021 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560941412850 con 0x5609414b9a80 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.551+0000 7fc474b4b640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x5609414bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.552+0000 7fc44ffff640 10 client.18021 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.552+0000 7fc474b4b640 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x5609414bbe70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e115db06bf695c65 server_cookie=37a33d5a9d95c807 in_seq=0 out_seq=0 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.552+0000 7fc44ffff640 1 -- 172.21.15.103:0/3956130052 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560941412850 con 0x5609414b9a80 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.552+0000 7fc44ffff640 10 client.18021 handle_command_reply: tid=0 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.552+0000 7fc479188740 1 client.18021 shutdown 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 msgr2=0x5609414a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414a7240 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0c5140 tx=0x7fc45c034000 comp rx=0 tx=0).stop 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 shutdown_connections 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x5609414bbe70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609414aa680 0x5609414a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc44ffff640 0 client.18021 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414a7240 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 --2- 172.21.15.103:0/3956130052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56094149f4a0 0x5609414a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 >> 172.21.15.103:0/3956130052 conn(0x56094149ba30 msgr2=0x5609414a1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 shutdown_connections 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 1 -- 172.21.15.103:0/3956130052 wait complete. 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.553+0000 7fc479188740 20 client.18021 trim_cache size 0 max 16384 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 Processor -- start 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 -- start start 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 0x5609414b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x5609414c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701ab710 con 0x56094149fdc0 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701ad280 con 0x56094149f4a0 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc479188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701abde0 con 0x5609414aa680 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35752/0 (socket says 172.21.15.103:35752) 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 -- 172.21.15.103:0/1067789860 learned_addr learned my addr 172.21.15.103:0/1067789860 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 -- 172.21.15.103:0/1067789860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 msgr2=0x5609414c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 --2- 172.21.15.103:0/1067789860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x5609414c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 -- 172.21.15.103:0/1067789860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 msgr2=0x5609414b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 --2- 172.21.15.103:0/1067789860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 0x5609414b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 -- 172.21.15.103:0/1067789860 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609414aa2c0 con 0x56094149fdc0 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc46ffff640 1 --2- 172.21.15.103:0/1067789860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414c16d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0c0e70 tx=0x7fc45c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bfa381b8143866 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:11.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc45a7fc640 1 -- 172.21.15.103:0/1067789860 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c049030 con 0x56094149fdc0 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc45a7fc640 1 -- 172.21.15.103:0/1067789860 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c0bd100 con 0x56094149fdc0 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.570+0000 7fc45a7fc640 1 -- 172.21.15.103:0/1067789860 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0ee0f0 con 0x56094149fdc0 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 -- 172.21.15.103:0/1067789860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 msgr2=0x5609414c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 --2- 172.21.15.103:0/1067789860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414c16d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0c0e70 tx=0x7fc45c02a480 comp rx=0 tx=0).stop 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 -- 172.21.15.103:0/1067789860 shutdown_connections 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 --2- 172.21.15.103:0/1067789860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x5609414c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 --2- 172.21.15.103:0/1067789860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149fdc0 0x5609414c16d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 --2- 172.21.15.103:0/1067789860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149f4a0 0x5609414b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 -- 172.21.15.103:0/1067789860 >> 172.21.15.103:0/1067789860 conn(0x56094149ba30 msgr2=0x5609414a6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 -- 172.21.15.103:0/1067789860 shutdown_connections 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 -- 172.21.15.103:0/1067789860 wait complete. 2024-09-19T07:51:11.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.571+0000 7fc479188740 1 -- start start 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 1 Processor -- start 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149f4a0 0x5609414ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149fdc0 0x5609414adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x560941522690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701acb80 con 0x56094149f4a0 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701ac4b0 con 0x56094149fdc0 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701a9710 con 0x5609414aa680 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc479188740 10 client.0 _ll_register_callbacks cb 0x7fc468009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149f4a0 0x5609414ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149f4a0 0x5609414ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35764/0 (socket says 172.21.15.103:35764) 2024-09-19T07:51:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 -- 172.21.15.103:0/1483221390 learned_addr learned my addr 172.21.15.103:0/1483221390 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc47534c640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x560941522690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 -- 172.21.15.103:0/1483221390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 msgr2=0x560941522690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc46ffff640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149fdc0 0x5609414adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x560941522690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 -- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149fdc0 msgr2=0x5609414adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149fdc0 0x5609414adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 -- 172.21.15.103:0/1483221390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560941522cd0 con 0x56094149f4a0 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc47534c640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x560941522690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.572+0000 7fc474b4b640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149f4a0 0x5609414ad9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc464012700 tx=0x7fc464012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192c2f0962d7d0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc44ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc44ffff640 1 -- 172.21.15.103:0/1483221390 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464004030 con 0x56094149f4a0 2024-09-19T07:51:11.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc44ffff640 1 -- 172.21.15.103:0/1483221390 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc4640041d0 con 0x56094149f4a0 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45c0bd100 con 0x56094149f4a0 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc44ffff640 1 -- 172.21.15.103:0/1483221390 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc46402b050 con 0x56094149f4a0 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 10 client.18027 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x560941524000 con 0x56094149f4a0 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc44ffff640 1 -- 172.21.15.103:0/1483221390 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c0bd100 con 0x56094149f4a0 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc44ffff640 1 -- 172.21.15.103:0/1483221390 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fc464003a20 con 0x56094149f4a0 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 10 client.18027 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 20 client.18027 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x560941526850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 4 client.18027 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.573+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56094140cd60 con 0x5609414b9a80 2024-09-19T07:51:11.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc46ffff640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x560941526850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc46ffff640 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x560941526850 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab878a83f112534b server_cookie=ad0b4112c7900589 in_seq=0 out_seq=0 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc44ffff640 10 client.18027 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc44ffff640 1 -- 172.21.15.103:0/1483221390 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56094140cd60 con 0x5609414b9a80 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc44ffff640 10 client.18027 handle_command_reply: tid=0 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 client.18027 shutdown 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149f4a0 msgr2=0x5609414ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149f4a0 0x5609414ad9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc464012700 tx=0x7fc464012bc0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 shutdown_connections 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5609414b9a80 0x560941526850 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609414aa680 0x560941522690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56094149fdc0 0x5609414adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc44ffff640 0 client.18027 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 --2- 172.21.15.103:0/1483221390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56094149f4a0 0x5609414ad9c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 >> 172.21.15.103:0/1483221390 conn(0x56094149ba30 msgr2=0x56094149a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.574+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 shutdown_connections 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc479188740 1 -- 172.21.15.103:0/1483221390 wait complete. 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc479188740 20 client.18027 trim_cache size 0 max 16384 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4600a33b0 msgr2=0x7fc4600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4600a33b0 0x7fc4600a5870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc46800a4e0 tx=0x7fc46800c040 comp rx=0 tx=0).stop 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 msgr2=0x7fc470197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc470197920 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc45c006010 tx=0x7fc45c003c80 comp rx=0 tx=0).stop 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 shutdown_connections 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4600a33b0 0x7fc4600a5870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc470102ef0 0x7fc47019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc470102570 0x7fc470197920 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 --2- 172.21.15.103:0/204105844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc470100e20 0x7fc4701973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:11.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 >> 172.21.15.103:0/204105844 conn(0x7fc4700fc7d0 msgr2=0x7fc470106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:11.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 shutdown_connections 2024-09-19T07:51:11.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:11.575+0000 7fc476daf640 1 -- 172.21.15.103:0/204105844 wait complete. 2024-09-19T07:51:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:11 smithi134 ceph-mon[35414]: pgmap v412: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-09-19T07:51:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:11 smithi134 ceph-mon[35414]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:51:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:11 smithi174 ceph-mon[35370]: pgmap v412: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-09-19T07:51:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:11 smithi174 ceph-mon[35370]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:51:12.425 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:11 smithi103 ceph-mon[31201]: pgmap v412: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-09-19T07:51:12.425 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:11 smithi103 ceph-mon[31201]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T07:51:12.454 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:51:12.455 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:51:12.705 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:13.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 -- 172.21.15.103:0/1027907375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8102570 msgr2=0x7f3fa81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1027907375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8102570 0x7f3fa81029b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009900 tx=0x7f3f9402a480 comp rx=0 tx=0).stop 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 -- 172.21.15.103:0/1027907375 shutdown_connections 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1027907375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fa8102ef0 0x7f3fa810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1027907375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8102570 0x7f3fa81029b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1027907375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fa8100e20 0x7f3fa8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 -- 172.21.15.103:0/1027907375 >> 172.21.15.103:0/1027907375 conn(0x7f3fa80fc7d0 msgr2=0x7f3fa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.201+0000 7f3faec1e640 1 -- 172.21.15.103:0/1027907375 shutdown_connections 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 -- 172.21.15.103:0/1027907375 wait complete. 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 Processor -- start 2024-09-19T07:51:13.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 -- start start 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8100e20 0x7f3fa810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fa8102570 0x7f3fa810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fa8102ef0 0x7f3fa810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81120a0 con 0x7f3fa8100e20 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8111f20 con 0x7f3fa8102ef0 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3faec1e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8112220 con 0x7f3fa8102570 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3fac9ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8100e20 0x7f3fa810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3fac9ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8100e20 0x7f3fa810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35780/0 (socket says 172.21.15.103:35780) 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.202+0000 7f3fac9ba640 1 -- 172.21.15.103:0/1924237573 learned_addr learned my addr 172.21.15.103:0/1924237573 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fad1bb640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fa8102ef0 0x7f3fa810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fa7fff640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fa8102570 0x7f3fa810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fac9ba640 1 -- 172.21.15.103:0/1924237573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fa8102570 msgr2=0x7f3fa810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fac9ba640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fa8102570 0x7f3fa810ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fac9ba640 1 -- 172.21.15.103:0/1924237573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fa8102ef0 msgr2=0x7f3fa810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fac9ba640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fa8102ef0 0x7f3fa810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fac9ba640 1 -- 172.21.15.103:0/1924237573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa81a69b0 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fac9ba640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8100e20 0x7f3fa810e4d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e9f0 tx=0x7f3f9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fa5ffb640 1 -- 172.21.15.103:0/1924237573 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00cd80 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fa5ffb640 1 -- 172.21.15.103:0/1924237573 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f9c005500 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3fa5ffb640 1 -- 172.21.15.103:0/1924237573 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c010690 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa81a6ca0 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.203+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa81a7190 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.204+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa8101200 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.205+0000 7f3fa5ffb640 1 -- 172.21.15.103:0/1924237573 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3f9c0046c0 con 0x7f3fa8100e20 2024-09-19T07:51:13.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.206+0000 7f3fa5ffb640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3f980a3430 0x7f3f980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.206+0000 7f3fa7fff640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3f980a3430 0x7f3f980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.206+0000 7f3fa5ffb640 1 -- 172.21.15.103:0/1924237573 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f3f9c0d0160 con 0x7f3fa8100e20 2024-09-19T07:51:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.207+0000 7f3fa7fff640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3f980a3430 0x7f3f980a58f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f940096c0 tx=0x7f3f94000bc0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:13.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.210+0000 7f3fa5ffb640 1 -- 172.21.15.103:0/1924237573 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f9c09cb30 con 0x7f3fa8100e20 2024-09-19T07:51:13.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.411+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fa8109090 con 0x7f3fa8100e20 2024-09-19T07:51:13.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.412+0000 7f3fa5ffb640 1 -- 172.21.15.103:0/1924237573 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11992 (secure 0 0 0) 0x7f3f9c09c430 con 0x7f3fa8100e20 2024-09-19T07:51:13.412 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:51:13.412 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":51,"btime":"2024-09-19T07:50:10:429478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:50:09.866710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:51:13.412 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 51 2024-09-19T07:51:13.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3f980a3430 msgr2=0x7f3f980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3f980a3430 0x7f3f980a58f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f940096c0 tx=0x7f3f94000bc0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8100e20 msgr2=0x7f3fa810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:13.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8100e20 0x7f3fa810e4d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e9f0 tx=0x7f3f9c00edf0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 shutdown_connections 2024-09-19T07:51:13.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f3f980a3430 0x7f3f980a58f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fa8102ef0 0x7f3fa810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fa8102570 0x7f3fa810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 --2- 172.21.15.103:0/1924237573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fa8100e20 0x7f3fa810e4d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:13.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 >> 172.21.15.103:0/1924237573 conn(0x7f3fa80fc7d0 msgr2=0x7f3fa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:13.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 shutdown_connections 2024-09-19T07:51:13.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:13.414+0000 7f3faec1e640 1 -- 172.21.15.103:0/1924237573 wait complete. 2024-09-19T07:51:14.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:13 smithi103 ceph-mon[31201]: pgmap v414: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T07:51:14.128 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:13 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1924237573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:14.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:51:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:13 smithi134 ceph-mon[35414]: pgmap v414: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T07:51:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:13 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1924237573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:14.404 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:13 smithi174 ceph-mon[35370]: pgmap v414: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-19T07:51:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:13 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1924237573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:14.857 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:51:14.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 -- 172.21.15.103:0/544607310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 msgr2=0x7f79a4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/544607310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4103110 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7994009930 tx=0x7f799402a420 comp rx=0 tx=0).stop 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 -- 172.21.15.103:0/544607310 shutdown_connections 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/544607310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a41082a0 0x7f79a410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/544607310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79a41036e0 0x7f79a4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/544607310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4103110 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 -- 172.21.15.103:0/544607310 >> 172.21.15.103:0/544607310 conn(0x7f79a40fc7d0 msgr2=0x7f79a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 -- 172.21.15.103:0/544607310 shutdown_connections 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.892+0000 7f79aa3b7640 1 -- 172.21.15.103:0/544607310 wait complete. 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 Processor -- start 2024-09-19T07:51:14.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 -- start start 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79a41036e0 0x7f79a4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a41082a0 0x7f79a4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41120a0 con 0x7f79a4102d30 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4111f20 con 0x7f79a41082a0 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79aa3b7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4112220 con 0x7f79a41036e0 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35800/0 (socket says 172.21.15.103:35800) 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.893+0000 7f79a3fff640 1 -- 172.21.15.103:0/256519403 learned_addr learned my addr 172.21.15.103:0/256519403 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a37fe640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79a41036e0 0x7f79a4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a8954640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a41082a0 0x7f79a4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a3fff640 1 -- 172.21.15.103:0/256519403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79a41036e0 msgr2=0x7f79a4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a3fff640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79a41036e0 0x7f79a4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a3fff640 1 -- 172.21.15.103:0/256519403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a41082a0 msgr2=0x7f79a4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a3fff640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a41082a0 0x7f79a4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a3fff640 1 -- 172.21.15.103:0/256519403 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a41a6aa0 con 0x7f79a4102d30 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a8954640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a41082a0 0x7f79a4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a3fff640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4100990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7994009230 tx=0x7f7994004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a17fa640 1 -- 172.21.15.103:0/256519403 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799403f030 con 0x7f79a4102d30 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a41a6d30 con 0x7f79a4102d30 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a17fa640 1 -- 172.21.15.103:0/256519403 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7994004440 con 0x7f79a4102d30 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79a17fa640 1 -- 172.21.15.103:0/256519403 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994035070 con 0x7f79a4102d30 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.894+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a71b0 con 0x7f79a4102d30 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79a17fa640 1 -- 172.21.15.103:0/256519403 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f7994042030 con 0x7f79a4102d30 2024-09-19T07:51:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 Processor -- start 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 -- start start 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd62a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd62aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd637760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd5902f0 con 0x55f6cd62bb40 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79a17fa640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f798c0a33b0 0x7f798c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd537cf0 con 0x55f6cd635370 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.897+0000 7f79ac790740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd58c2b0 con 0x55f6cd62a5d0 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd62a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd62a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35810/0 (socket says 172.21.15.103:35810) 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 -- 172.21.15.103:0/1747675637 learned_addr learned my addr 172.21.15.103:0/1747675637 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:14.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f798c0a33b0 0x7f798c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a37fe640 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd637760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a17fa640 1 -- 172.21.15.103:0/256519403 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f79940f00a0 con 0x7f79a4102d30 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 -- 172.21.15.103:0/1747675637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 msgr2=0x55f6cd62aa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd62aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 -- 172.21.15.103:0/1747675637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 msgr2=0x55f6cd637760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd637760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 -- 172.21.15.103:0/1747675637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cd51c3a0 con 0x55f6cd62bb40 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a37fe640 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd637760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd62a090 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f799c012d90 tx=0x7f799c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb37740fde6f8009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f79a8954640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f798c0a33b0 0x7f798c0a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f799c010ca0 tx=0x7f799c01d000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f798a7fc640 1 -- 172.21.15.103:0/1747675637 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c012580 con 0x55f6cd62bb40 2024-09-19T07:51:14.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f798a7fc640 1 -- 172.21.15.103:0/1747675637 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c012870 con 0x55f6cd62bb40 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.898+0000 7f798a7fc640 1 -- 172.21.15.103:0/1747675637 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c005460 con 0x55f6cd62bb40 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 -- 172.21.15.103:0/1747675637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 msgr2=0x55f6cd62a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd62a090 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f799c012d90 tx=0x7f799c0107a0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 -- 172.21.15.103:0/1747675637 shutdown_connections 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd637760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd62aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 --2- 172.21.15.103:0/1747675637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd62a090 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 -- 172.21.15.103:0/1747675637 >> 172.21.15.103:0/1747675637 conn(0x55f6cd5f3310 msgr2=0x55f6cd626c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 -- 172.21.15.103:0/1747675637 shutdown_connections 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.899+0000 7f79ac790740 1 -- 172.21.15.103:0/1747675637 wait complete. 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 -- start start 2024-09-19T07:51:14.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 Processor -- start 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd632100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd632640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd632b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd637fb0 con 0x55f6cd62bb40 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd54b060 con 0x55f6cd635370 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd638130 con 0x55f6cd62a5d0 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79ac790740 10 client.0 _ll_register_callbacks cb 0x55f6cd29e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd632640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd632640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35826/0 (socket says 172.21.15.103:35826) 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.900+0000 7f79a37fe640 1 -- 172.21.15.103:0/1761880745 learned_addr learned my addr 172.21.15.103:0/1761880745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a3fff640 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd632100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a8954640 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd632b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a37fe640 1 -- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 msgr2=0x55f6cd632100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a37fe640 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd632100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a37fe640 1 -- 172.21.15.103:0/1761880745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 msgr2=0x55f6cd632b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a37fe640 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd632b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a37fe640 1 -- 172.21.15.103:0/1761880745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cd3013c0 con 0x55f6cd62bb40 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79a37fe640 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd632640 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7998009ca0 tx=0x7f799800e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8f728709159257 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79837fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79837fe640 1 -- 172.21.15.103:0/1761880745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998027060 con 0x55f6cd62bb40 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79837fe640 1 -- 172.21.15.103:0/1761880745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f7998004250 con 0x55f6cd62bb40 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c012870 con 0x55f6cd62bb40 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79837fe640 1 -- 172.21.15.103:0/1761880745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7998004450 con 0x55f6cd62bb40 2024-09-19T07:51:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79ac790740 10 client.18045 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.901+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f6cd643ba0 con 0x55f6cd62bb40 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79837fe640 1 -- 172.21.15.103:0/1761880745 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c012870 con 0x55f6cd62bb40 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79837fe640 1 -- 172.21.15.103:0/1761880745 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f799802a830 con 0x55f6cd62bb40 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79ac790740 10 client.18045 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79ac790740 10 client.18045 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79ac790740 20 client.18045 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79ac790740 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd645060 0x55f6cd6474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79ac790740 4 client.18045 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6cd59d860 con 0x55f6cd645060 2024-09-19T07:51:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79a3fff640 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd645060 0x55f6cd6474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79837fe640 10 client.18045 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.902+0000 7f79a3fff640 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd645060 0x55f6cd6474b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3f7e1eabe4502aa server_cookie=d21c8670d50e17ad in_seq=0 out_seq=0 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.903+0000 7f79837fe640 1 -- 172.21.15.103:0/1761880745 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6cd59d860 con 0x55f6cd645060 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.903+0000 7f79837fe640 10 client.18045 handle_command_reply: tid=0 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.903+0000 7f79ac790740 1 client.18045 shutdown 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.903+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 msgr2=0x55f6cd632640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.903+0000 7f79ac790740 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd632640 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7998009ca0 tx=0x7f799800e780 comp rx=0 tx=0).stop 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.903+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 shutdown_connections 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.903+0000 7f79ac790740 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd645060 0x55f6cd6474b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79ac790740 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd635370 0x55f6cd632b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79ac790740 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd62bb40 0x55f6cd632640 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79ac790740 1 --2- 172.21.15.103:0/1761880745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd632100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 >> 172.21.15.103:0/1761880745 conn(0x55f6cd5f3310 msgr2=0x55f6cd62c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:14.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79837fe640 0 client.18045 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:51:14.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 shutdown_connections 2024-09-19T07:51:14.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79ac790740 1 -- 172.21.15.103:0/1761880745 wait complete. 2024-09-19T07:51:14.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.904+0000 7f79ac790740 20 client.18045 trim_cache size 0 max 16384 2024-09-19T07:51:14.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 Processor -- start 2024-09-19T07:51:14.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 -- start start 2024-09-19T07:51:14.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62a5d0 0x55f6cd626e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62bb40 0x55f6cd64cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd64d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41ab730 con 0x55f6cd635370 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41ad2a0 con 0x55f6cd62a5d0 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79ac790740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41abe00 con 0x55f6cd62bb40 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd64d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd64d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35830/0 (socket says 172.21.15.103:35830) 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62bb40 0x55f6cd64cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 -- 172.21.15.103:0/3660607629 learned_addr learned my addr 172.21.15.103:0/3660607629 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a3fff640 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62a5d0 0x55f6cd626e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 -- 172.21.15.103:0/3660607629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62bb40 msgr2=0x55f6cd64cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:14.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62bb40 0x55f6cd64cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 -- 172.21.15.103:0/3660607629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62a5d0 msgr2=0x55f6cd626e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62a5d0 0x55f6cd626e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a8954640 1 -- 172.21.15.103:0/3660607629 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cd637ad0 con 0x55f6cd635370 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.917+0000 7f79a3fff640 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62a5d0 0x55f6cd626e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.918+0000 7f79a8954640 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd64d300 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f799c028040 tx=0x7f799c0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22eb9d40a99772c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.918+0000 7f7989ffb640 1 -- 172.21.15.103:0/3660607629 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c012870 con 0x55f6cd635370 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.918+0000 7f7989ffb640 1 -- 172.21.15.103:0/3660607629 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c033070 con 0x55f6cd635370 2024-09-19T07:51:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.918+0000 7f7989ffb640 1 -- 172.21.15.103:0/3660607629 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0266b0 con 0x55f6cd635370 2024-09-19T07:51:14.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 -- 172.21.15.103:0/3660607629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 msgr2=0x55f6cd64d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd64d300 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f799c028040 tx=0x7f799c0095b0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 -- 172.21.15.103:0/3660607629 shutdown_connections 2024-09-19T07:51:14.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd64d300 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62bb40 0x55f6cd64cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 --2- 172.21.15.103:0/3660607629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62a5d0 0x55f6cd626e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 -- 172.21.15.103:0/3660607629 >> 172.21.15.103:0/3660607629 conn(0x55f6cd5f3310 msgr2=0x55f6cd626700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 -- 172.21.15.103:0/3660607629 shutdown_connections 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 -- 172.21.15.103:0/3660607629 wait complete. 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 -- start start 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.919+0000 7f79ac790740 1 Processor -- start 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79ac790740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd63d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79ac790740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62bb40 0x55f6cd638c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79ac790740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd639170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79ac790740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41acba0 con 0x55f6cd635370 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79ac790740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41ac4d0 con 0x55f6cd62bb40 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79ac790740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41a9730 con 0x55f6cd62a5d0 2024-09-19T07:51:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79ac790740 10 client.0 _ll_register_callbacks cb 0x7f7998017bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd639170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd639170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35840/0 (socket says 172.21.15.103:35840) 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 -- 172.21.15.103:0/1301507868 learned_addr learned my addr 172.21.15.103:0/1301507868 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a3fff640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd63d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 -- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 msgr2=0x55f6cd63d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd63d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 -- 172.21.15.103:0/1301507868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62bb40 msgr2=0x55f6cd638c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a37fe640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62bb40 0x55f6cd638c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62bb40 0x55f6cd638c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 -- 172.21.15.103:0/1301507868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cd639760 con 0x55f6cd635370 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a37fe640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62bb40 0x55f6cd638c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79a8954640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd639170 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f799c012d90 tx=0x7f799c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183579955cad0033 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:14.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.920+0000 7f79837fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79837fe640 1 -- 172.21.15.103:0/1301507868 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0056d0 con 0x55f6cd635370 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79837fe640 1 -- 172.21.15.103:0/1301507868 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f799c005870 con 0x55f6cd635370 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c033070 con 0x55f6cd635370 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79837fe640 1 -- 172.21.15.103:0/1301507868 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f799c026450 con 0x55f6cd635370 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 10 client.18051 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f6cd6afb90 con 0x55f6cd635370 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79837fe640 1 -- 172.21.15.103:0/1301507868 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c033070 con 0x55f6cd635370 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79837fe640 1 -- 172.21.15.103:0/1301507868 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f799c039710 con 0x55f6cd635370 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 10 client.18051 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 10 client.18051 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 20 client.18051 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:14.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd6b00a0 0x55f6cd6b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 4 client.18051 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.921+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6cd597d70 con 0x55f6cd6b00a0 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79a3fff640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd6b00a0 0x55f6cd6b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79a3fff640 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd6b00a0 0x55f6cd6b2490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b58b9c9beb81e081 server_cookie=81a2275e7306cbaf in_seq=0 out_seq=0 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79837fe640 10 client.18051 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79837fe640 1 -- 172.21.15.103:0/1301507868 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6cd597d70 con 0x55f6cd6b00a0 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79837fe640 10 client.18051 handle_command_reply: tid=0 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 client.18051 shutdown 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 msgr2=0x55f6cd639170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd639170 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f799c012d90 tx=0x7f799c0107a0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 shutdown_connections 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55f6cd6b00a0 0x55f6cd6b2490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd635370 0x55f6cd639170 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6cd62bb40 0x55f6cd638c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 --2- 172.21.15.103:0/1301507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd62a5d0 0x55f6cd63d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 >> 172.21.15.103:0/1301507868 conn(0x55f6cd5f3310 msgr2=0x55f6cd633960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:14.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.922+0000 7f79837fe640 0 client.18051 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:51:14.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 shutdown_connections 2024-09-19T07:51:14.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79ac790740 1 -- 172.21.15.103:0/1301507868 wait complete. 2024-09-19T07:51:14.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79ac790740 20 client.18051 trim_cache size 0 max 16384 2024-09-19T07:51:14.928 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f798c0a33b0 msgr2=0x7f798c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f798c0a33b0 0x7f798c0a5870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f799c010ca0 tx=0x7f799c01d000 comp rx=0 tx=0).stop 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 msgr2=0x7f79a4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4100990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7994009230 tx=0x7f7994004040 comp rx=0 tx=0).stop 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 shutdown_connections 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f798c0a33b0 0x7f798c0a5870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79a41082a0 0x7f79a4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79a41036e0 0x7f79a4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 --2- 172.21.15.103:0/256519403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79a4102d30 0x7f79a4100990 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:14.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.923+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 >> 172.21.15.103:0/256519403 conn(0x7f79a40fc7d0 msgr2=0x7f79a4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:14.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.924+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 shutdown_connections 2024-09-19T07:51:14.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:14.924+0000 7f79aa3b7640 1 -- 172.21.15.103:0/256519403 wait complete. 2024-09-19T07:51:15.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 760.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:51:15.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:51:15.807 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:15 smithi103 ceph-mon[31201]: pgmap v415: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 82 op/s 2024-09-19T07:51:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:15 smithi134 ceph-mon[35414]: pgmap v415: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 82 op/s 2024-09-19T07:51:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3974740286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 msgr2=0x7f5c8c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3974740286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c10ea90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009ca0 tx=0x7f5c8402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3974740286 shutdown_connections 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3974740286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c10ea90 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3974740286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c8c102790 0x7f5c8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3974740286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c8c101de0 0x7f5c8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3974740286 >> 172.21.15.103:0/3974740286 conn(0x7f5c8c0fd880 msgr2=0x7f5c8c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3974740286 shutdown_connections 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.327+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3974740286 wait complete. 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 Processor -- start 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 -- start start 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c8c101de0 0x7f5c8c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c8c102790 0x7f5c8c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:16.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c111f70 con 0x7f5c8c106f10 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c111df0 con 0x7f5c8c101de0 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c92e5c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c1120f0 con 0x7f5c8c102790 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35858/0 (socket says 172.21.15.103:35858) 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 -- 172.21.15.103:0/3005974479 learned_addr learned my addr 172.21.15.103:0/3005974479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c90bf8640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c8c101de0 0x7f5c8c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 -- 172.21.15.103:0/3005974479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c8c102790 msgr2=0x7f5c8c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c8bfff640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c8c102790 0x7f5c8c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c8c102790 0x7f5c8c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 -- 172.21.15.103:0/3005974479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c8c101de0 msgr2=0x7f5c8c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c8c101de0 0x7f5c8c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 -- 172.21.15.103:0/3005974479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8c19d300 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c90bf8640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c8c101de0 0x7f5c8c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.328+0000 7f5c913f9640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c19cc20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5c84031a60 tx=0x7f5c84031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.329+0000 7f5c89ffb640 1 -- 172.21.15.103:0/3005974479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84003c30 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.329+0000 7f5c89ffb640 1 -- 172.21.15.103:0/3005974479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c84003dd0 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.329+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8c19d590 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.329+0000 7f5c89ffb640 1 -- 172.21.15.103:0/3005974479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840047e0 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.329+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8c1a4ed0 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.330+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c8c1021c0 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.331+0000 7f5c89ffb640 1 -- 172.21.15.103:0/3005974479 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5c84004b40 con 0x7f5c8c106f10 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.332+0000 7f5c89ffb640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5c7c0a3430 0x7f5c7c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:16.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.332+0000 7f5c90bf8640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5c7c0a3430 0x7f5c7c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.332+0000 7f5c89ffb640 1 -- 172.21.15.103:0/3005974479 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f5c840f1240 con 0x7f5c8c106f10 2024-09-19T07:51:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.332+0000 7f5c90bf8640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5c7c0a3430 0x7f5c7c0a58f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c800058c0 tx=0x7f5c80005d30 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:16.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.335+0000 7f5c89ffb640 1 -- 172.21.15.103:0/3005974479 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c840bdcc0 con 0x7f5c8c106f10 2024-09-19T07:51:16.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:15 smithi174 ceph-mon[35370]: pgmap v415: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 82 op/s 2024-09-19T07:51:16.498 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:16.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.545+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c8c1a5100 con 0x7f5c8c106f10 2024-09-19T07:51:16.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.546+0000 7f5c89ffb640 1 -- 172.21.15.103:0/3005974479 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11992 (secure 0 0 0) 0x7f5c840bd5c0 con 0x7f5c8c106f10 2024-09-19T07:51:16.546 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:51:16.546 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":51,"btime":"2024-09-19T07:50:10:429478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:50:09.866710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:51:16.546 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 51 2024-09-19T07:51:16.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5c7c0a3430 msgr2=0x7f5c7c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:16.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5c7c0a3430 0x7f5c7c0a58f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c800058c0 tx=0x7f5c80005d30 comp rx=0 tx=0).stop 2024-09-19T07:51:16.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 msgr2=0x7f5c8c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:16.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c19cc20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f5c84031a60 tx=0x7f5c84031a90 comp rx=0 tx=0).stop 2024-09-19T07:51:16.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 shutdown_connections 2024-09-19T07:51:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5c7c0a3430 0x7f5c7c0a58f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5c8c106f10 0x7f5c8c19cc20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c8c102790 0x7f5c8c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 --2- 172.21.15.103:0/3005974479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c8c101de0 0x7f5c8c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.548+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 >> 172.21.15.103:0/3005974479 conn(0x7f5c8c0fd880 msgr2=0x7f5c8c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.549+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 shutdown_connections 2024-09-19T07:51:16.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:16.549+0000 7f5c92e5c640 1 -- 172.21.15.103:0/3005974479 wait complete. 2024-09-19T07:51:17.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 -- 172.21.15.103:0/3829426601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 msgr2=0x7f751c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 --2- 172.21.15.103:0/3829426601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c100ee0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7510009930 tx=0x7f751002a420 comp rx=0 tx=0).stop 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 -- 172.21.15.103:0/3829426601 shutdown_connections 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 --2- 172.21.15.103:0/3829426601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f751c101420 0x7f751c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 --2- 172.21.15.103:0/3829426601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c100ee0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 --2- 172.21.15.103:0/3829426601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f751c106a80 0x7f751c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 -- 172.21.15.103:0/3829426601 >> 172.21.15.103:0/3829426601 conn(0x7f751c0fc7a0 msgr2=0x7f751c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 -- 172.21.15.103:0/3829426601 shutdown_connections 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.199+0000 7f752194e640 1 -- 172.21.15.103:0/3829426601 wait complete. 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 Processor -- start 2024-09-19T07:51:17.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 -- start start 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f751c101420 0x7f751c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f751c106a80 0x7f751c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f751c110e40 con 0x7f751c100a80 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f751c110cc0 con 0x7f751c101420 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f752194e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f751c110fc0 con 0x7f751c106a80 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35878/0 (socket says 172.21.15.103:35878) 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f751affd640 1 -- 172.21.15.103:0/819476963 learned_addr learned my addr 172.21.15.103:0/819476963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f751b7fe640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f751c106a80 0x7f751c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.200+0000 7f751a7fc640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f751c101420 0x7f751c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f751affd640 1 -- 172.21.15.103:0/819476963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f751c106a80 msgr2=0x7f751c19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f751affd640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f751c106a80 0x7f751c19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f751affd640 1 -- 172.21.15.103:0/819476963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f751c101420 msgr2=0x7f751c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f751affd640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f751c101420 0x7f751c1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f751affd640 1 -- 172.21.15.103:0/819476963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c19c330 con 0x7f751c100a80 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f751affd640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c197380 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c890 tx=0x7f750c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f752094c640 1 -- 172.21.15.103:0/819476963 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c004220 con 0x7f751c100a80 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c19c620 con 0x7f751c100a80 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f752094c640 1 -- 172.21.15.103:0/819476963 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c0043c0 con 0x7f751c100a80 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f752094c640 1 -- 172.21.15.103:0/819476963 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c004a00 con 0x7f751c100a80 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.201+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c06a790 con 0x7f751c100a80 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.203+0000 7f752094c640 1 -- 172.21.15.103:0/819476963 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f750c0079b0 con 0x7f751c100a80 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 Processor -- start 2024-09-19T07:51:17.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 -- start start 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8c0b40 0x558e1c8bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f752094c640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f75040a55c0 0x7f75040a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8bf5d0 0x558e1c8bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c8cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e1c8252f0 con 0x558e1c8c0b40 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e1c7cccf0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f7523d27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e1c8212b0 con 0x558e1c8ca370 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8c0b40 0x558e1c8bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751b7fe640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f75040a55c0 0x7f75040a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8c0b40 0x558e1c8bf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35892/0 (socket says 172.21.15.103:35892) 2024-09-19T07:51:17.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 -- 172.21.15.103:0/2373163908 learned_addr learned my addr 172.21.15.103:0/2373163908 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f752094c640 1 -- 172.21.15.103:0/819476963 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f750c0cf8d0 con 0x7f751c100a80 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751a7fc640 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8bf5d0 0x558e1c8bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751b7fe640 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c8cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 -- 172.21.15.103:0/2373163908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 msgr2=0x558e1c8cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c8cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 -- 172.21.15.103:0/2373163908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8bf5d0 msgr2=0x558e1c8bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8bf5d0 0x558e1c8bfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.204+0000 7f751affd640 1 -- 172.21.15.103:0/2373163908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e1c7b13a0 con 0x558e1c8c0b40 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f751b7fe640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f75040a55c0 0x7f75040a7a80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7514005fd0 tx=0x7f7514014bf0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f751affd640 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8c0b40 0x558e1c8bf090 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f750c098620 tx=0x7f750c066ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bacf83d2307a8bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7501ffb640 1 -- 172.21.15.103:0/2373163908 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c01d040 con 0x558e1c8c0b40 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7501ffb640 1 -- 172.21.15.103:0/2373163908 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c09bc50 con 0x558e1c8c0b40 2024-09-19T07:51:17.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7501ffb640 1 -- 172.21.15.103:0/2373163908 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c0cd290 con 0x558e1c8c0b40 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7523d27740 1 -- 172.21.15.103:0/2373163908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8c0b40 msgr2=0x558e1c8bf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7523d27740 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8c0b40 0x558e1c8bf090 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f750c098620 tx=0x7f750c066ae0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7523d27740 1 -- 172.21.15.103:0/2373163908 shutdown_connections 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7523d27740 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c8cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7523d27740 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8bf5d0 0x558e1c8bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.205+0000 7f7523d27740 1 --2- 172.21.15.103:0/2373163908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8c0b40 0x558e1c8bf090 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 -- 172.21.15.103:0/2373163908 >> 172.21.15.103:0/2373163908 conn(0x558e1c888310 msgr2=0x558e1c8bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 -- 172.21.15.103:0/2373163908 shutdown_connections 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 -- 172.21.15.103:0/2373163908 wait complete. 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 -- start start 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 Processor -- start 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8c0b40 0x558e1c8c7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8ca370 0x558e1c8c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e1c8cd010 con 0x558e1c8bf5d0 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e1c7e0060 con 0x558e1c8ca370 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e1c8cd190 con 0x558e1c8c0b40 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f7523d27740 10 client.0 _ll_register_callbacks cb 0x558e1c5338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.206+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35906/0 (socket says 172.21.15.103:35906) 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 -- 172.21.15.103:0/1701520583 learned_addr learned my addr 172.21.15.103:0/1701520583 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751b7fe640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8ca370 0x558e1c8c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751a7fc640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8c0b40 0x558e1c8c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 -- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8c0b40 msgr2=0x558e1c8c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8c0b40 0x558e1c8c7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 -- 172.21.15.103:0/1701520583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8ca370 msgr2=0x558e1c8c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8ca370 0x558e1c8c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 -- 172.21.15.103:0/1701520583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e1c5963c0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751b7fe640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8ca370 0x558e1c8c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f751affd640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c9d70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f750c09b950 tx=0x7f750c096680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab74d9255ae7b267 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f74faffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f74faffd640 1 -- 172.21.15.103:0/1701520583 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c01d040 con 0x558e1c8bf5d0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f74faffd640 1 -- 172.21.15.103:0/1701520583 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f750c0a4380 con 0x558e1c8bf5d0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f750c09bc50 con 0x558e1c8bf5d0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f74faffd640 1 -- 172.21.15.103:0/1701520583 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f750c09b5b0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f7523d27740 10 client.18069 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558e1c8d8ab0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.207+0000 7f74faffd640 1 -- 172.21.15.103:0/1701520583 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750c09bc50 con 0x558e1c8bf5d0 2024-09-19T07:51:17.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f74faffd640 1 -- 172.21.15.103:0/1701520583 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f750c09b0a0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f7523d27740 10 client.18069 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f7523d27740 10 client.18069 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f7523d27740 20 client.18069 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f7523d27740 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c8d9e00 0x558e1c8dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f7523d27740 4 client.18069 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e1c832860 con 0x558e1c8d9e00 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f751a7fc640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c8d9e00 0x558e1c8dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f74faffd640 10 client.18069 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.208+0000 7f751a7fc640 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c8d9e00 0x558e1c8dc1f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10987967d233a2bd server_cookie=20586eee9a5502a in_seq=0 out_seq=0 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.209+0000 7f74faffd640 1 -- 172.21.15.103:0/1701520583 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e1c832860 con 0x558e1c8d9e00 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.209+0000 7f74faffd640 10 client.18069 handle_command_reply: tid=0 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.209+0000 7f7523d27740 1 client.18069 shutdown 2024-09-19T07:51:17.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.209+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 msgr2=0x558e1c8c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.209+0000 7f7523d27740 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c9d70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f750c09b950 tx=0x7f750c096680 comp rx=0 tx=0).stop 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.209+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 shutdown_connections 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.209+0000 7f7523d27740 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c8d9e00 0x558e1c8dc1f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f7523d27740 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8ca370 0x558e1c8c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f7523d27740 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8c0b40 0x558e1c8c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f74faffd640 0 client.18069 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f7523d27740 1 --2- 172.21.15.103:0/1701520583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c9d70 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 >> 172.21.15.103:0/1701520583 conn(0x558e1c888310 msgr2=0x558e1c8c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 shutdown_connections 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f7523d27740 1 -- 172.21.15.103:0/1701520583 wait complete. 2024-09-19T07:51:17.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.210+0000 7f7523d27740 20 client.18069 trim_cache size 0 max 16384 2024-09-19T07:51:17.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.222+0000 7f7523d27740 1 Processor -- start 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.222+0000 7f7523d27740 1 -- start start 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f7523d27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f7523d27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c8df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f7523d27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8ca370 0x558e1c8dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f7523d27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1ab780 con 0x558e1c8ca370 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f7523d27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1ad2f0 con 0x558e1c8c0b40 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f7523d27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1abe50 con 0x558e1c8bf5d0 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8ca370 0x558e1c8dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8ca370 0x558e1c8dfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35908/0 (socket says 172.21.15.103:35908) 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 -- 172.21.15.103:0/3035135330 learned_addr learned my addr 172.21.15.103:0/3035135330 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751affd640 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751a7fc640 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c8df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 -- 172.21.15.103:0/3035135330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8bf5d0 msgr2=0x558e1c8c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 -- 172.21.15.103:0/3035135330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 msgr2=0x558e1c8df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c8df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 -- 172.21.15.103:0/3035135330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e1c8c7980 con 0x558e1c8ca370 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751affd640 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f751b7fe640 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8ca370 0x558e1c8dfb40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7514012e60 tx=0x7f751401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8d637ebacc42d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:17.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f75017fa640 1 -- 172.21.15.103:0/3035135330 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514016e60 con 0x558e1c8ca370 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f75017fa640 1 -- 172.21.15.103:0/3035135330 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75140045c0 con 0x558e1c8ca370 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.223+0000 7f75017fa640 1 -- 172.21.15.103:0/3035135330 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514022660 con 0x558e1c8ca370 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 -- 172.21.15.103:0/3035135330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8ca370 msgr2=0x558e1c8dfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8ca370 0x558e1c8dfb40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7514012e60 tx=0x7f751401d000 comp rx=0 tx=0).stop 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 -- 172.21.15.103:0/3035135330 shutdown_connections 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8ca370 0x558e1c8dfb40 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c8df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 --2- 172.21.15.103:0/3035135330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8bf5d0 0x558e1c8c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 -- 172.21.15.103:0/3035135330 >> 172.21.15.103:0/3035135330 conn(0x558e1c888310 msgr2=0x558e1c8c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 -- 172.21.15.103:0/3035135330 shutdown_connections 2024-09-19T07:51:17.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 -- 172.21.15.103:0/3035135330 wait complete. 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.224+0000 7f7523d27740 1 -- start start 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 1 Processor -- start 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c93b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c93bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c93fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1acbf0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1ac520 con 0x558e1c8c0b40 2024-09-19T07:51:17.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1a9780 con 0x558e1c8ca370 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f7523d27740 10 client.0 _ll_register_callbacks cb 0x7f750c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c93b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c93b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35912/0 (socket says 172.21.15.103:35912) 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 -- 172.21.15.103:0/1376410853 learned_addr learned my addr 172.21.15.103:0/1376410853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751b7fe640 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c93fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751a7fc640 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c93bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 -- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 msgr2=0x558e1c93fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c93fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 -- 172.21.15.103:0/1376410853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 msgr2=0x558e1c93bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c93bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.225+0000 7f751affd640 1 -- 172.21.15.103:0/1376410853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e1c940370 con 0x558e1c8bf5d0 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f751affd640 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c93b6c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f750c0980b0 tx=0x7f750c097f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ababd7508ee625c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:17.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f74faffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f74faffd640 1 -- 172.21.15.103:0/1376410853 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c01d070 con 0x558e1c8bf5d0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f74faffd640 1 -- 172.21.15.103:0/1376410853 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f750c0a4380 con 0x558e1c8bf5d0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75140045c0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f74faffd640 1 -- 172.21.15.103:0/1376410853 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f750c066e60 con 0x558e1c8bf5d0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 10 client.18075 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558e1c940370 con 0x558e1c8bf5d0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f74faffd640 1 -- 172.21.15.103:0/1376410853 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75140045c0 con 0x558e1c8bf5d0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f74faffd640 1 -- 172.21.15.103:0/1376410853 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f750c0ba210 con 0x558e1c8bf5d0 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 10 client.18075 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 10 client.18075 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 20 client.18075 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c948c60 0x558e1c94b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:17.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 4 client.18075 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.226+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e1c82cd70 con 0x558e1c948c60 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f751a7fc640 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c948c60 0x558e1c94b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f74faffd640 10 client.18075 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f751a7fc640 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c948c60 0x558e1c94b050 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c90ecb03302c9697 server_cookie=83b80067bb676712 in_seq=0 out_seq=0 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f74faffd640 1 -- 172.21.15.103:0/1376410853 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e1c82cd70 con 0x558e1c948c60 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f74faffd640 10 client.18075 handle_command_reply: tid=0 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 client.18075 shutdown 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 msgr2=0x558e1c93b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c93b6c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f750c0980b0 tx=0x7f750c097f60 comp rx=0 tx=0).stop 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 shutdown_connections 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x558e1c948c60 0x558e1c94b050 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e1c8ca370 0x558e1c93fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e1c8c0b40 0x558e1c93bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f74faffd640 0 client.18075 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:51:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 --2- 172.21.15.103:0/1376410853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e1c8bf5d0 0x558e1c93b6c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.227+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 >> 172.21.15.103:0/1376410853 conn(0x558e1c888310 msgr2=0x558e1c8ba4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 shutdown_connections 2024-09-19T07:51:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f7523d27740 1 -- 172.21.15.103:0/1376410853 wait complete. 2024-09-19T07:51:17.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f7523d27740 20 client.18075 trim_cache size 0 max 16384 2024-09-19T07:51:17.234 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:51:17.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f75040a55c0 msgr2=0x7f75040a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f75040a55c0 0x7f75040a7a80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7514005fd0 tx=0x7f7514014bf0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 msgr2=0x7f751c197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:17.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c197380 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c890 tx=0x7f750c00cd50 comp rx=0 tx=0).stop 2024-09-19T07:51:17.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 shutdown_connections 2024-09-19T07:51:17.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f75040a55c0 0x7f75040a7a80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f751c106a80 0x7f751c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f751c101420 0x7f751c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 --2- 172.21.15.103:0/819476963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c100a80 0x7f751c197380 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 >> 172.21.15.103:0/819476963 conn(0x7f751c0fc7a0 msgr2=0x7f751c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 shutdown_connections 2024-09-19T07:51:17.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:17.228+0000 7f752194e640 1 -- 172.21.15.103:0/819476963 wait complete. 2024-09-19T07:51:17.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:17 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3005974479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:17.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:17 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3005974479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:17.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:17 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3005974479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:18.068 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:51:18.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:18 smithi134 ceph-mon[35414]: pgmap v416: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 402 KiB/s wr, 17 op/s 2024-09-19T07:51:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:18 smithi174 ceph-mon[35370]: pgmap v416: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 402 KiB/s wr, 17 op/s 2024-09-19T07:51:18.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:18 smithi103 ceph-mon[31201]: pgmap v416: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 402 KiB/s wr, 17 op/s 2024-09-19T07:51:18.633 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id ac701e75 --release --await' 2024-09-19T07:51:18.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id ac701e75 --release --await 2024-09-19T07:51:18.693 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:19.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 -- 172.21.15.103:0/4239672779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 msgr2=0x7f3030100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 --2- 172.21.15.103:0/4239672779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f3030100ee0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f3020009930 tx=0x7f302002a420 comp rx=0 tx=0).stop 2024-09-19T07:51:19.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 -- 172.21.15.103:0/4239672779 shutdown_connections 2024-09-19T07:51:19.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 --2- 172.21.15.103:0/4239672779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3030101420 0x7f303010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 --2- 172.21.15.103:0/4239672779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f3030100ee0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 --2- 172.21.15.103:0/4239672779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030106a80 0x7f3030106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 -- 172.21.15.103:0/4239672779 >> 172.21.15.103:0/4239672779 conn(0x7f30300fc7a0 msgr2=0x7f30300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 -- 172.21.15.103:0/4239672779 shutdown_connections 2024-09-19T07:51:19.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.168+0000 7f3037cc8640 1 -- 172.21.15.103:0/4239672779 wait complete. 2024-09-19T07:51:19.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 Processor -- start 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 -- start start 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f30300fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3030101420 0x7f30300fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030106a80 0x7f30300fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3030110e40 con 0x7f3030100a80 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3030110cc0 con 0x7f3030101420 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3037cc8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3030110fc0 con 0x7f3030106a80 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3035a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f30300fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3035a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f30300fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52832/0 (socket says 172.21.15.103:52832) 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3035a64640 1 -- 172.21.15.103:0/3959373743 learned_addr learned my addr 172.21.15.103:0/3959373743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3035263640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3030101420 0x7f30300fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.169+0000 7f3036265640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030106a80 0x7f30300fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3035a64640 1 -- 172.21.15.103:0/3959373743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030106a80 msgr2=0x7f30300fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3035a64640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030106a80 0x7f30300fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3035a64640 1 -- 172.21.15.103:0/3959373743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3030101420 msgr2=0x7f30300fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3035a64640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3030101420 0x7f30300fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3035a64640 1 -- 172.21.15.103:0/3959373743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30301a68d0 con 0x7f3030100a80 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3036265640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030106a80 0x7f30300fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3035a64640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f30300fcdc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f302c00c920 tx=0x7f302c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3026ffd640 1 -- 172.21.15.103:0/3959373743 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c007c70 con 0x7f3030100a80 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30301a6bc0 con 0x7f3030100a80 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3026ffd640 1 -- 172.21.15.103:0/3959373743 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c007e10 con 0x7f3030100a80 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3026ffd640 1 -- 172.21.15.103:0/3959373743 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c018890 con 0x7f3030100a80 2024-09-19T07:51:19.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.170+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30301a7120 con 0x7f3030100a80 2024-09-19T07:51:19.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.172+0000 7f3026ffd640 1 -- 172.21.15.103:0/3959373743 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f302c018a70 con 0x7f3030100a80 2024-09-19T07:51:19.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 Processor -- start 2024-09-19T07:51:19.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 -- start start 2024-09-19T07:51:19.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb4b40 0x556c2dbb3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbb3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 0x556c2dbc0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c2db192f0 con 0x556c2dbb35d0 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c2dac0cf0 con 0x556c2dbbe370 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f303a0a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c2db152b0 con 0x556c2dbb4b40 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f3026ffd640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f301c0a33b0 0x7f301c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.173+0000 7f3035a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbb3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbb3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52840/0 (socket says 172.21.15.103:52840) 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 -- 172.21.15.103:0/2504920848 learned_addr learned my addr 172.21.15.103:0/2504920848 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3036265640 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb4b40 0x556c2dbb3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3036265640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f301c0a33b0 0x7f301c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035263640 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 0x556c2dbc0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3026ffd640 1 -- 172.21.15.103:0/3959373743 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f302c0cfd60 con 0x7f3030100a80 2024-09-19T07:51:19.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 -- 172.21.15.103:0/2504920848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb4b40 msgr2=0x556c2dbb3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb4b40 0x556c2dbb3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 -- 172.21.15.103:0/2504920848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 msgr2=0x556c2dbc0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 0x556c2dbc0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 -- 172.21.15.103:0/2504920848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2daa53a0 con 0x556c2dbb35d0 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3036265640 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb4b40 0x556c2dbb3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3035a64640 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbb3a10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f302c01af50 tx=0x7f302c00fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e241b17e84d4c449 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f3036265640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f301c0a33b0 0x7f301c0a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3028013d50 tx=0x7f302800a370 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f301bfff640 1 -- 172.21.15.103:0/2504920848 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c09c730 con 0x556c2dbb35d0 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f301bfff640 1 -- 172.21.15.103:0/2504920848 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c09c030 con 0x556c2dbb35d0 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.174+0000 7f301bfff640 1 -- 172.21.15.103:0/2504920848 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0cd720 con 0x556c2dbb35d0 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 -- 172.21.15.103:0/2504920848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 msgr2=0x556c2dbb3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbb3a10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f302c01af50 tx=0x7f302c00fe30 comp rx=0 tx=0).stop 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 -- 172.21.15.103:0/2504920848 shutdown_connections 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 0x556c2dbc0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbb3a10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2504920848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb4b40 0x556c2dbb3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 -- 172.21.15.103:0/2504920848 >> 172.21.15.103:0/2504920848 conn(0x556c2db7c310 msgr2=0x556c2dbafc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 -- 172.21.15.103:0/2504920848 shutdown_connections 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.175+0000 7f303a0a1740 1 -- 172.21.15.103:0/2504920848 wait complete. 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 -- start start 2024-09-19T07:51:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 Processor -- start 2024-09-19T07:51:19.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbbdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbbb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbbe370 0x556c2dbbb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c2dbc1010 con 0x556c2dbb35d0 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c2dad4060 con 0x556c2dbb4b40 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c2dbc1190 con 0x556c2dbbe370 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f303a0a1740 10 client.0 _ll_register_callbacks cb 0x556c2d8278c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f3035a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbbdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f3035a64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbbdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52848/0 (socket says 172.21.15.103:52848) 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f3035a64640 1 -- 172.21.15.103:0/293901610 learned_addr learned my addr 172.21.15.103:0/293901610 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.176+0000 7f3035263640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbbb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3036265640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbbe370 0x556c2dbbb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3035a64640 1 -- 172.21.15.103:0/293901610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbbe370 msgr2=0x556c2dbbb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3035a64640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbbe370 0x556c2dbbb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3035a64640 1 -- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 msgr2=0x556c2dbbb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:19.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3035a64640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbbb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3035a64640 1 -- 172.21.15.103:0/293901610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c2d88a3c0 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3036265640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbbe370 0x556c2dbbb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f3035a64640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbbdd50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f302c09bd30 tx=0x7f302c018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1f93bd67bd7e83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f30197fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f30197fa640 1 -- 172.21.15.103:0/293901610 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0a03e0 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f30197fa640 1 -- 172.21.15.103:0/293901610 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f302c0a4810 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302c09c030 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f30197fa640 1 -- 172.21.15.103:0/293901610 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f302c0a4a90 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f303a0a1740 10 client.18087 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556c2dbccaf0 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.177+0000 7f30197fa640 1 -- 172.21.15.103:0/293901610 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f302c09c030 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f30197fa640 1 -- 172.21.15.103:0/293901610 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f302c062030 con 0x556c2dbb35d0 2024-09-19T07:51:19.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f303a0a1740 10 client.18087 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f303a0a1740 10 client.18087 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f303a0a1740 20 client.18087 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f303a0a1740 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dbcde40 0x556c2dbd0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f303a0a1740 4 client.18087 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c2db26860 con 0x556c2dbcde40 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.178+0000 7f3035263640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dbcde40 0x556c2dbd0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.179+0000 7f3035263640 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dbcde40 0x556c2dbd0230 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91af0ccf302a01d0 server_cookie=da6b2b1cfa0dbdea in_seq=0 out_seq=0 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.179+0000 7f30197fa640 10 client.18087 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.179+0000 7f30197fa640 1 -- 172.21.15.103:0/293901610 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c2db26860 con 0x556c2dbcde40 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.179+0000 7f30197fa640 10 client.18087 handle_command_reply: tid=0 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.179+0000 7f303a0a1740 1 client.18087 shutdown 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.179+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 msgr2=0x556c2dbbdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.179+0000 7f303a0a1740 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbbdd50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f302c09bd30 tx=0x7f302c018f90 comp rx=0 tx=0).stop 2024-09-19T07:51:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 shutdown_connections 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dbcde40 0x556c2dbd0230 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbbe370 0x556c2dbbb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f30197fa640 0 client.18087 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbbb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 --2- 172.21.15.103:0/293901610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb35d0 0x556c2dbbdd50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 >> 172.21.15.103:0/293901610 conn(0x556c2db7c310 msgr2=0x556c2dbb5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 shutdown_connections 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 1 -- 172.21.15.103:0/293901610 wait complete. 2024-09-19T07:51:19.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.180+0000 7f303a0a1740 20 client.18087 trim_cache size 0 max 16384 2024-09-19T07:51:19.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.201+0000 7f303a0a1740 1 Processor -- start 2024-09-19T07:51:19.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.201+0000 7f303a0a1740 1 -- start start 2024-09-19T07:51:19.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbb5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbb5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbbe370 0x556c2dbb8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f303a0a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f301400e7e0 con 0x556c2dbbe370 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f303a0a1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3014010350 con 0x556c2dbb4b40 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f303a0a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f301400eeb0 con 0x556c2dbb35d0 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbbe370 0x556c2dbb8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbbe370 0x556c2dbb8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52854/0 (socket says 172.21.15.103:52854) 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 -- 172.21.15.103:0/2494593851 learned_addr learned my addr 172.21.15.103:0/2494593851 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3035a64640 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbb5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3035263640 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbb5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 -- 172.21.15.103:0/2494593851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 msgr2=0x556c2dbb5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbb5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 -- 172.21.15.103:0/2494593851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 msgr2=0x556c2dbb5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:19.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbb5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 -- 172.21.15.103:0/2494593851 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2dbaf4f0 con 0x556c2dbbe370 2024-09-19T07:51:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3035a64640 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbb5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3035263640 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbb5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.202+0000 7f3036265640 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbbe370 0x556c2dbb8260 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f302800ac20 tx=0x7f302800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d3b91db0f3fae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.203+0000 7f301b7fe640 1 -- 172.21.15.103:0/2494593851 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028004b20 con 0x556c2dbbe370 2024-09-19T07:51:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.203+0000 7f301b7fe640 1 -- 172.21.15.103:0/2494593851 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3028004cc0 con 0x556c2dbbe370 2024-09-19T07:51:19.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.203+0000 7f301b7fe640 1 -- 172.21.15.103:0/2494593851 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028029690 con 0x556c2dbbe370 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 -- 172.21.15.103:0/2494593851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbbe370 msgr2=0x556c2dbb8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbbe370 0x556c2dbb8260 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f302800ac20 tx=0x7f302800a620 comp rx=0 tx=0).stop 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 -- 172.21.15.103:0/2494593851 shutdown_connections 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbbe370 0x556c2dbb8260 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbb4b40 0x556c2dbb5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 --2- 172.21.15.103:0/2494593851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbb5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 -- 172.21.15.103:0/2494593851 >> 172.21.15.103:0/2494593851 conn(0x556c2db7c310 msgr2=0x556c2dbb5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 -- 172.21.15.103:0/2494593851 shutdown_connections 2024-09-19T07:51:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 -- 172.21.15.103:0/2494593851 wait complete. 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 -- start start 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.204+0000 7f303a0a1740 1 Processor -- start 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbd13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb4b40 0x556c2dbd1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f303a0a1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 0x556c2dc37780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f303a0a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f301400fc50 con 0x556c2dbb4b40 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f303a0a1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f301400f580 con 0x556c2dbbe370 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f303a0a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f301400c7e0 con 0x556c2dbb35d0 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f303a0a1740 10 client.0 _ll_register_callbacks cb 0x7f3020005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f3035263640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb4b40 0x556c2dbd1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f3035263640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb4b40 0x556c2dbd1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52862/0 (socket says 172.21.15.103:52862) 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.205+0000 7f3035263640 1 -- 172.21.15.103:0/422031825 learned_addr learned my addr 172.21.15.103:0/422031825 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:19.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f3035263640 1 -- 172.21.15.103:0/422031825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 msgr2=0x556c2dbd13f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f3035263640 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbd13f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f3035263640 1 -- 172.21.15.103:0/422031825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 msgr2=0x556c2dc37780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f3035263640 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 0x556c2dc37780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f3035263640 1 -- 172.21.15.103:0/422031825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c2dc37cc0 con 0x556c2dbb4b40 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f3035263640 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb4b40 0x556c2dbd1930 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3020009930 tx=0x7f302002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14825d782383c360 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f30197fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f30197fa640 1 -- 172.21.15.103:0/422031825 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020035040 con 0x556c2dbb4b40 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f30197fa640 1 -- 172.21.15.103:0/422031825 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f30200035f0 con 0x556c2dbb4b40 2024-09-19T07:51:19.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.206+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3028004cc0 con 0x556c2dbb4b40 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f30197fa640 1 -- 172.21.15.103:0/422031825 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f30200037f0 con 0x556c2dbb4b40 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f303a0a1740 10 client.18093 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556c2dc38f60 con 0x556c2dbb4b40 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f30197fa640 1 -- 172.21.15.103:0/422031825 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3028004cc0 con 0x556c2dbb4b40 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f30197fa640 1 -- 172.21.15.103:0/422031825 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f302004f070 con 0x556c2dbb4b40 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f303a0a1740 10 client.18093 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f303a0a1740 10 client.18093 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:51:19.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f303a0a1740 20 client.18093 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f303a0a1740 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dc396c0 0x556c2dc3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.207+0000 7f303a0a1740 4 client.18093 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.208+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c2db20d70 con 0x556c2dc396c0 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.208+0000 7f3035a64640 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dc396c0 0x556c2dc3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.208+0000 7f3035a64640 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dc396c0 0x556c2dc3bae0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38c6b112160c22bd server_cookie=e02a344fe4fb858d in_seq=0 out_seq=0 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.208+0000 7f30197fa640 10 client.18093 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.208+0000 7f30197fa640 1 -- 172.21.15.103:0/422031825 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c2db20d70 con 0x556c2dc396c0 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.208+0000 7f30197fa640 10 client.18093 handle_command_reply: tid=0 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.208+0000 7f303a0a1740 1 client.18093 shutdown 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb4b40 msgr2=0x556c2dbd1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb4b40 0x556c2dbd1930 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3020009930 tx=0x7f302002ab90 comp rx=0 tx=0).stop 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 shutdown_connections 2024-09-19T07:51:19.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x556c2dc396c0 0x556c2dc3bae0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c2dbbe370 0x556c2dc37780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c2dbb4b40 0x556c2dbd1930 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 --2- 172.21.15.103:0/422031825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c2dbb35d0 0x556c2dbd13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 >> 172.21.15.103:0/422031825 conn(0x556c2db7c310 msgr2=0x556c2dbaed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f30197fa640 0 client.18093 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 shutdown_connections 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 1 -- 172.21.15.103:0/422031825 wait complete. 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.209+0000 7f303a0a1740 20 client.18093 trim_cache size 0 max 16384 2024-09-19T07:51:19.212 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:51:19.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f301c0a33b0 msgr2=0x7f301c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f301c0a33b0 0x7f301c0a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3028013d50 tx=0x7f302800a370 comp rx=0 tx=0).stop 2024-09-19T07:51:19.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 msgr2=0x7f30300fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:19.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f30300fcdc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f302c00c920 tx=0x7f302c00cde0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 shutdown_connections 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f301c0a33b0 0x7f301c0a5870 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030106a80 0x7f30300fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3030101420 0x7f30300fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 --2- 172.21.15.103:0/3959373743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3030100a80 0x7f30300fcdc0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 >> 172.21.15.103:0/3959373743 conn(0x7f30300fc7a0 msgr2=0x7f303006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 shutdown_connections 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:19.210+0000 7f3037cc8640 1 -- 172.21.15.103:0/3959373743 wait complete. 2024-09-19T07:51:19.214 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:19.858 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:51:20.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:20 smithi103 ceph-mon[31201]: pgmap v417: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 402 KiB/s wr, 17 op/s 2024-09-19T07:51:20.108 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:20 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:51:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:20 smithi134 ceph-mon[35414]: pgmap v417: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 402 KiB/s wr, 17 op/s 2024-09-19T07:51:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:20 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:51:20.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 -- 172.21.15.103:0/3981563245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c100a50 msgr2=0x7f340c1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 --2- 172.21.15.103:0/3981563245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c100a50 0x7f340c1057e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3404009ca0 tx=0x7f340402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 -- 172.21.15.103:0/3981563245 shutdown_connections 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 --2- 172.21.15.103:0/3981563245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c100a50 0x7f340c1057e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 --2- 172.21.15.103:0/3981563245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f340c100090 0x7f340c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 --2- 172.21.15.103:0/3981563245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c106b70 0x7f340c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 -- 172.21.15.103:0/3981563245 >> 172.21.15.103:0/3981563245 conn(0x7f340c077210 msgr2=0x7f340c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 -- 172.21.15.103:0/3981563245 shutdown_connections 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 -- 172.21.15.103:0/3981563245 wait complete. 2024-09-19T07:51:20.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.368+0000 7f3412e1d640 1 Processor -- start 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3412e1d640 1 -- start start 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3412e1d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c100090 0x7f340c1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3412e1d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f340c100a50 0x7f340c1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3412e1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c106b70 0x7f340c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3412e1d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f340c1185c0 con 0x7f340c106b70 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3412e1d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f340c118440 con 0x7f340c100090 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3412e1d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f340c118740 con 0x7f340c100a50 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c100090 0x7f340c1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f34113ba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c106b70 0x7f340c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c100090 0x7f340c1a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38890/0 (socket says 172.21.15.103:38890) 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 -- 172.21.15.103:0/95552470 learned_addr learned my addr 172.21.15.103:0/95552470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f340bfff640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f340c100a50 0x7f340c1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 -- 172.21.15.103:0/95552470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f340c100a50 msgr2=0x7f340c1a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f340c100a50 0x7f340c1a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 -- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c106b70 msgr2=0x7f340c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c106b70 0x7f340c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f3410bb9640 1 -- 172.21.15.103:0/95552470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c19ef70 con 0x7f340c100090 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.369+0000 7f34113ba640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c106b70 0x7f340c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.370+0000 7f3410bb9640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c100090 0x7f340c1a4650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f340000e9f0 tx=0x7f340000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.370+0000 7f3409ffb640 1 -- 172.21.15.103:0/95552470 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340000cd80 con 0x7f340c100090 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.370+0000 7f3409ffb640 1 -- 172.21.15.103:0/95552470 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3400010910 con 0x7f340c100090 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.370+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c19f260 con 0x7f340c100090 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.370+0000 7f3409ffb640 1 -- 172.21.15.103:0/95552470 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000188f0 con 0x7f340c100090 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.370+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c1ab4b0 con 0x7f340c100090 2024-09-19T07:51:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.372+0000 7f3409ffb640 1 -- 172.21.15.103:0/95552470 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f3400010ac0 con 0x7f340c100090 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 Processor -- start 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f3409ffb640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33fc0a33b0 0x7f33fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 -- start start 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f340bfff640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33fc0a33b0 0x7f33fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f3409ffb640 1 -- 172.21.15.103:0/95552470 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f34000d0120 con 0x7f340c100090 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499060 0x55aee3499440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aee33fe620 con 0x55aee3499980 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aee33a5e50 con 0x55aee3499060 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34151f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aee33fa5e0 con 0x55aee34a4240 2024-09-19T07:51:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499060 0x55aee3499440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f34113ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499060 0x55aee3499440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38898/0 (socket says 172.21.15.103:38898) 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f3410bb9640 1 -- 172.21.15.103:0/2230049501 learned_addr learned my addr 172.21.15.103:0/2230049501 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.373+0000 7f340bfff640 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f3410bb9640 1 -- 172.21.15.103:0/2230049501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 msgr2=0x55aee34a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f340bfff640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33fc0a33b0 0x7f33fc0a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f340c0fdda0 tx=0x7f33f8011440 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f3410bb9640 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f3410bb9640 1 -- 172.21.15.103:0/2230049501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 msgr2=0x55aee34a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f3410bb9640 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f3410bb9640 1 -- 172.21.15.103:0/2230049501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee3396d40 con 0x55aee3499060 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f3410bb9640 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499060 0x55aee3499440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f34000988a0 tx=0x7f340009c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36179afa3721b6b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f33f6ffd640 1 -- 172.21.15.103:0/2230049501 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400098ab0 con 0x55aee3499060 2024-09-19T07:51:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f33f6ffd640 1 -- 172.21.15.103:0/2230049501 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340009b890 con 0x55aee3499060 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.374+0000 7f33f6ffd640 1 -- 172.21.15.103:0/2230049501 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000984b0 con 0x55aee3499060 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 -- 172.21.15.103:0/2230049501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499060 msgr2=0x55aee3499440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499060 0x55aee3499440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f34000988a0 tx=0x7f340009c580 comp rx=0 tx=0).stop 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 -- 172.21.15.103:0/2230049501 shutdown_connections 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 --2- 172.21.15.103:0/2230049501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499060 0x55aee3499440 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 -- 172.21.15.103:0/2230049501 >> 172.21.15.103:0/2230049501 conn(0x55aee34955f0 msgr2=0x55aee34959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 -- 172.21.15.103:0/2230049501 shutdown_connections 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 -- 172.21.15.103:0/2230049501 wait complete. 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 -- start start 2024-09-19T07:51:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.375+0000 7f34151f6740 1 Processor -- start 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee3499060 0x55aee34a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee34a4240 0x55aee34a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aee34a6e10 con 0x55aee3499980 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55aee3386f10 con 0x55aee34a4240 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aee34a6f90 con 0x55aee3499060 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 10 client.0 _ll_register_callbacks cb 0x55aee33aadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52880/0 (socket says 172.21.15.103:52880) 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 -- 172.21.15.103:0/960273313 learned_addr learned my addr 172.21.15.103:0/960273313 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f3410bb9640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee3499060 0x55aee34a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34113ba640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee34a4240 0x55aee34a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 -- 172.21.15.103:0/960273313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee3499060 msgr2=0x55aee34a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee3499060 0x55aee34a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 -- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee34a4240 msgr2=0x55aee34a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee34a4240 0x55aee34a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 -- 172.21.15.103:0/960273313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee310d580 con 0x55aee3499980 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34113ba640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee34a4240 0x55aee34a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f340bfff640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a1700 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f33f800fa90 tx=0x7f33f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b172041fa31412cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f33ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f33ebfff640 1 -- 172.21.15.103:0/960273313 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8017040 con 0x55aee3499980 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f33ebfff640 1 -- 172.21.15.103:0/960273313 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f33f801e8e0 con 0x55aee3499980 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.376+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f340009b890 con 0x55aee3499980 2024-09-19T07:51:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f33ebfff640 1 -- 172.21.15.103:0/960273313 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f33f801eae0 con 0x55aee3499980 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 10 client.18105 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55aee31d7960 con 0x55aee3499980 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f33ebfff640 1 -- 172.21.15.103:0/960273313 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340009b890 con 0x55aee3499980 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f33ebfff640 1 -- 172.21.15.103:0/960273313 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f33f8025070 con 0x55aee3499980 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 10 client.18105 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 10 client.18105 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 20 client.18105 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee34b3f50 0x55aee34b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 4 client.18105 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aee340bb90 con 0x55aee34b3f50 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.377+0000 7f3410bb9640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee34b3f50 0x55aee34b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.378+0000 7f3410bb9640 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee34b3f50 0x55aee34b6390 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc284eeca40d37c4 server_cookie=b81b8aa24fea0483 in_seq=0 out_seq=0 2024-09-19T07:51:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.378+0000 7f33ebfff640 10 client.18105 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.378+0000 7f33ebfff640 1 -- 172.21.15.103:0/960273313 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aee340bb90 con 0x55aee34b3f50 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.378+0000 7f33ebfff640 10 client.18105 handle_command_reply: tid=0 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.378+0000 7f34151f6740 1 client.18105 shutdown 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 msgr2=0x55aee34a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a1700 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f33f800fa90 tx=0x7f33f8004730 comp rx=0 tx=0).stop 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 shutdown_connections 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee34b3f50 0x55aee34b6390 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee34a4240 0x55aee34a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499980 0x55aee34a1700 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 --2- 172.21.15.103:0/960273313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee3499060 0x55aee34a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f33ebfff640 0 client.18105 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:20.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 >> 172.21.15.103:0/960273313 conn(0x55aee34955f0 msgr2=0x55aee3494750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 shutdown_connections 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 1 -- 172.21.15.103:0/960273313 wait complete. 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.379+0000 7f34151f6740 20 client.18105 trim_cache size 0 max 16384 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 Processor -- start 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 -- start start 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee349bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f340c1b3f50 con 0x55aee3499060 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f340c1b5ac0 con 0x55aee3499980 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34151f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f340c1b4620 con 0x55aee34a4240 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee349bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee349bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52886/0 (socket says 172.21.15.103:52886) 2024-09-19T07:51:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 -- 172.21.15.103:0/3336000630 learned_addr learned my addr 172.21.15.103:0/3336000630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34113ba640 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f340bfff640 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 -- 172.21.15.103:0/3336000630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 msgr2=0x55aee34afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 -- 172.21.15.103:0/3336000630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 msgr2=0x55aee34af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 -- 172.21.15.103:0/3336000630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee34966a0 con 0x55aee3499060 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f34113ba640 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.396+0000 7f3410bb9640 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee349bac0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f34000a0900 tx=0x7f340009a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31413c229ed54ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f33f67fc640 1 -- 172.21.15.103:0/3336000630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400098ab0 con 0x55aee3499060 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f33f67fc640 1 -- 172.21.15.103:0/3336000630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34000cda80 con 0x55aee3499060 2024-09-19T07:51:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f33f67fc640 1 -- 172.21.15.103:0/3336000630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340009b560 con 0x55aee3499060 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 -- 172.21.15.103:0/3336000630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 msgr2=0x55aee349bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee349bac0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f34000a0900 tx=0x7f340009a950 comp rx=0 tx=0).stop 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 -- 172.21.15.103:0/3336000630 shutdown_connections 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee34afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 --2- 172.21.15.103:0/3336000630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee349bac0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 -- 172.21.15.103:0/3336000630 >> 172.21.15.103:0/3336000630 conn(0x55aee34955f0 msgr2=0x55aee34af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 -- 172.21.15.103:0/3336000630 shutdown_connections 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.397+0000 7f34151f6740 1 -- 172.21.15.103:0/3336000630 wait complete. 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 -- start start 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 Processor -- start 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee34a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee351dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f340c1b53c0 con 0x55aee3499060 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f340c1b4cf0 con 0x55aee3499980 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f340c1b1f50 con 0x55aee34a4240 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34151f6740 10 client.0 _ll_register_callbacks cb 0x7f33f800f970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee34a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f3410bb9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee34a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52890/0 (socket says 172.21.15.103:52890) 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f3410bb9640 1 -- 172.21.15.103:0/1780567292 learned_addr learned my addr 172.21.15.103:0/1780567292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f340bfff640 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.398+0000 7f34113ba640 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee351dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f3410bb9640 1 -- 172.21.15.103:0/1780567292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 msgr2=0x55aee351dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f3410bb9640 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee351dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f3410bb9640 1 -- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 msgr2=0x55aee34a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f3410bb9640 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f3410bb9640 1 -- 172.21.15.103:0/1780567292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee351e170 con 0x55aee3499060 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f3410bb9640 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee34a8570 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3400023040 tx=0x7f3400009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520f2cdffc5af385 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f33ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f33ebfff640 1 -- 172.21.15.103:0/1780567292 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000cdc30 con 0x55aee3499060 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f33ebfff640 1 -- 172.21.15.103:0/1780567292 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f3400098ab0 con 0x55aee3499060 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34000cda80 con 0x55aee3499060 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f33ebfff640 1 -- 172.21.15.103:0/1780567292 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3400010430 con 0x55aee3499060 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f34151f6740 10 client.18111 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55aee351f3b0 con 0x55aee3499060 2024-09-19T07:51:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f33ebfff640 1 -- 172.21.15.103:0/1780567292 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34000cda80 con 0x55aee3499060 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f33ebfff640 1 -- 172.21.15.103:0/1780567292 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f340009b500 con 0x55aee3499060 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f34151f6740 10 client.18111 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f34151f6740 10 client.18111 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f34151f6740 20 client.18111 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.399+0000 7f34151f6740 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee351f580 0x55aee3521970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.400+0000 7f34151f6740 4 client.18111 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ac701e75", "release": true, "await": true}] 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.400+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ac701e75", "release": true, "await": true}) -- 0x55aee34060a0 con 0x55aee351f580 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.400+0000 7f340bfff640 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee351f580 0x55aee3521970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.400+0000 7f340bfff640 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee351f580 0x55aee3521970 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7cf9503bb7c3c9 server_cookie=540e1230ff61b1fb in_seq=0 out_seq=0 2024-09-19T07:51:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.400+0000 7f33ebfff640 10 client.18111 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:20.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:20 smithi174 ceph-mon[35370]: pgmap v417: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 402 KiB/s wr, 17 op/s 2024-09-19T07:51:20.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:20 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f33ebfff640 1 -- 172.21.15.103:0/1780567292 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55aee34060a0 con 0x55aee351f580 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f33ebfff640 10 client.18111 handle_command_reply: tid=0 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 client.18111 shutdown 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 msgr2=0x55aee34a8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee34a8570 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3400023040 tx=0x7f3400009310 comp rx=0 tx=0).stop 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 shutdown_connections 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55aee351f580 0x55aee3521970 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f33ebfff640 0 client.18111 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aee34a4240 0x55aee351dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aee3499980 0x55aee34a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 --2- 172.21.15.103:0/1780567292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aee3499060 0x55aee34a8570 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 >> 172.21.15.103:0/1780567292 conn(0x55aee34955f0 msgr2=0x55aee34af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 shutdown_connections 2024-09-19T07:51:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 1 -- 172.21.15.103:0/1780567292 wait complete. 2024-09-19T07:51:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.427+0000 7f34151f6740 20 client.18111 trim_cache size 0 max 16384 2024-09-19T07:51:20.428 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:51:20.428 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 51, 2024-09-19T07:51:20.428 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 44, 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ac701e75": { 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 44, 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:51:20.429 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:20.430 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:51:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33fc0a33b0 msgr2=0x7f33fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33fc0a33b0 0x7f33fc0a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f340c0fdda0 tx=0x7f33f8011440 comp rx=0 tx=0).stop 2024-09-19T07:51:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c100090 msgr2=0x7f340c1a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c100090 0x7f340c1a4650 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f340000e9f0 tx=0x7f340000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 shutdown_connections 2024-09-19T07:51:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33fc0a33b0 0x7f33fc0a5870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f340c106b70 0x7f340c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f340c100a50 0x7f340c1a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 --2- 172.21.15.103:0/95552470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f340c100090 0x7f340c1a4650 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f340000e9f0 tx=0x7f340000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:51:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 >> 172.21.15.103:0/95552470 conn(0x7f340c077210 msgr2=0x7f340c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.430+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 shutdown_connections 2024-09-19T07:51:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:20.431+0000 7f3412e1d640 1 -- 172.21.15.103:0/95552470 wait complete. 2024-09-19T07:51:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 765.0 s, 24.6 tps, lat 16293.147 ms stddev 10877.772 2024-09-19T07:51:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:21 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:51:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:21 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:51:21.153 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:21 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:51:21.184 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ac701e75', total seconds elapsed: 28.6 2024-09-19T07:51:21.184 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T07:51:21.184 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:51:21.185 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:21 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:51:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:21 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:51:21.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:21 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:51:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:21 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:51:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:21 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:51:21.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:21 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:51:21.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 -- 172.21.15.103:0/1737337301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b41036b0 msgr2=0x7f10b410bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 --2- 172.21.15.103:0/1737337301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b41036b0 0x7f10b410bba0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009900 tx=0x7f10a802a480 comp rx=0 tx=0).stop 2024-09-19T07:51:21.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 -- 172.21.15.103:0/1737337301 shutdown_connections 2024-09-19T07:51:21.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 --2- 172.21.15.103:0/1737337301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b410c170 0x7f10b410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 --2- 172.21.15.103:0/1737337301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b41036b0 0x7f10b410bba0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 --2- 172.21.15.103:0/1737337301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b4102d90 0x7f10b4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 -- 172.21.15.103:0/1737337301 >> 172.21.15.103:0/1737337301 conn(0x7f10b40fc790 msgr2=0x7f10b40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 -- 172.21.15.103:0/1737337301 shutdown_connections 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.771+0000 7f10b8e50640 1 -- 172.21.15.103:0/1737337301 wait complete. 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.772+0000 7f10b8e50640 1 Processor -- start 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.772+0000 7f10b8e50640 1 -- start start 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.772+0000 7f10b8e50640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b4102d90 0x7f10b4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.772+0000 7f10b8e50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b41036b0 0x7f10b41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.772+0000 7f10b8e50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b410c170 0x7f10b419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.772+0000 7f10b8e50640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10b4111fa0 con 0x7f10b4102d90 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.772+0000 7f10b8e50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10b4111e20 con 0x7f10b410c170 2024-09-19T07:51:21.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b8e50640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10b4112120 con 0x7f10b41036b0 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b4102d90 0x7f10b4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b41036b0 0x7f10b41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b410c170 0x7f10b419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b4102d90 0x7f10b4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52900/0 (socket says 172.21.15.103:52900) 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b41036b0 0x7f10b41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48342/0 (socket says 172.21.15.103:48342) 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b259c640 1 -- 172.21.15.103:0/2851748980 learned_addr learned my addr 172.21.15.103:0/2851748980 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b410c170 0x7f10b419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38950/0 (socket says 172.21.15.103:38950) 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b2d9d640 1 -- 172.21.15.103:0/2851748980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b41036b0 msgr2=0x7f10b41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b2d9d640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b41036b0 0x7f10b41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b2d9d640 1 -- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b4102d90 msgr2=0x7f10b4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b2d9d640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b4102d90 0x7f10b4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b2d9d640 1 -- 172.21.15.103:0/2851748980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b419d430 con 0x7f10b410c170 2024-09-19T07:51:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.773+0000 7f10b259c640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b4102d90 0x7f10b4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:51:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.774+0000 7f10b2d9d640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b410c170 0x7f10b419cd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009fd0 tx=0x7f10ac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.774+0000 7f10a37fe640 1 -- 172.21.15.103:0/2851748980 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00ce20 con 0x7f10b410c170 2024-09-19T07:51:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.774+0000 7f10a37fe640 1 -- 172.21.15.103:0/2851748980 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac005020 con 0x7f10b410c170 2024-09-19T07:51:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.774+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b419d720 con 0x7f10b410c170 2024-09-19T07:51:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.774+0000 7f10a37fe640 1 -- 172.21.15.103:0/2851748980 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac010720 con 0x7f10b410c170 2024-09-19T07:51:21.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.774+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b41a5000 con 0x7f10b410c170 2024-09-19T07:51:21.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.777+0000 7f10a37fe640 1 -- 172.21.15.103:0/2851748980 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f10ac005540 con 0x7f10b410c170 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.778+0000 7f10a37fe640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f109c0a3430 0x7f109c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.778+0000 7f10b259c640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f109c0a3430 0x7f109c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.778+0000 7f10bb229740 1 Processor -- start 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.778+0000 7f10a37fe640 1 -- 172.21.15.103:0/2851748980 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f10ac0d0280 con 0x7f10b410c170 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.778+0000 7f10bb229740 1 -- start start 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10bb229740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e95b40 0x5628c9e94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10bb229740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9e94a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10bb229740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9ea3780 0x5628c9ea5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10bb229740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5628c9dfa2f0 con 0x5628c9e945d0 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10bb229740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5628c9da1cf0 con 0x5628c9ea3780 2024-09-19T07:51:21.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10bb229740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628c9df62b0 con 0x5628c9e95b40 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9ea3780 0x5628c9ea5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9ea3780 0x5628c9ea5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:38966/0 (socket says 172.21.15.103:38966) 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b2d9d640 1 -- 172.21.15.103:0/2869123927 learned_addr learned my addr 172.21.15.103:0/2869123927 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b259c640 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e95b40 0x5628c9e94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b2d9d640 1 -- 172.21.15.103:0/2869123927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e95b40 msgr2=0x5628c9e94090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b2d9d640 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e95b40 0x5628c9e94090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b2d9d640 1 -- 172.21.15.103:0/2869123927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 msgr2=0x5628c9e94a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.779+0000 7f10b2d9d640 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9e94a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.780+0000 7f10b2d9d640 1 -- 172.21.15.103:0/2869123927 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c9d863a0 con 0x5628c9ea3780 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.780+0000 7f10b259c640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f109c0a3430 0x7f109c0a58f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f10a4011550 tx=0x7f10a4005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:21.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.780+0000 7f10b2d9d640 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9ea3780 0x5628c9ea5b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0a4c30 tx=0x7f10ac0058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c34dc5294192f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:21.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.781+0000 7f10a0ff9640 1 -- 172.21.15.103:0/2869123927 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0a5160 con 0x5628c9ea3780 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.781+0000 7f10a0ff9640 1 -- 172.21.15.103:0/2869123927 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac0051d0 con 0x5628c9ea3780 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.782+0000 7f10a0ff9640 1 -- 172.21.15.103:0/2869123927 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac09c9b0 con 0x5628c9ea3780 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.782+0000 7f10bb229740 1 -- 172.21.15.103:0/2869123927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9ea3780 msgr2=0x5628c9ea5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.782+0000 7f10bb229740 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9ea3780 0x5628c9ea5b70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0a4c30 tx=0x7f10ac0058b0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.783+0000 7f10bb229740 1 -- 172.21.15.103:0/2869123927 shutdown_connections 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.783+0000 7f10bb229740 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9ea3780 0x5628c9ea5b70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.783+0000 7f10bb229740 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9e94a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.783+0000 7f10bb229740 1 --2- 172.21.15.103:0/2869123927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e95b40 0x5628c9e94090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.783+0000 7f10bb229740 1 -- 172.21.15.103:0/2869123927 >> 172.21.15.103:0/2869123927 conn(0x5628c9e5d310 msgr2=0x5628c9e90c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.783+0000 7f10bb229740 1 -- 172.21.15.103:0/2869123927 shutdown_connections 2024-09-19T07:51:21.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.783+0000 7f10bb229740 1 -- 172.21.15.103:0/2869123927 wait complete. 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 -- start start 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 Processor -- start 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9ea3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 0x5628c9ea0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5628c9ea6480 con 0x5628c9e945d0 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5628c9db5060 con 0x5628c9e95b40 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628c9ea6600 con 0x5628c9ea3780 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10bb229740 10 client.0 _ll_register_callbacks cb 0x5628c9b088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:21.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10b259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9ea3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10b259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9ea3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52914/0 (socket says 172.21.15.103:52914) 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.784+0000 7f10b259c640 1 -- 172.21.15.103:0/2697009050 learned_addr learned my addr 172.21.15.103:0/2697009050 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f10b259c640 1 -- 172.21.15.103:0/2697009050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 msgr2=0x5628c9ea09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f10b259c640 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f10b259c640 1 -- 172.21.15.103:0/2697009050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 msgr2=0x5628c9ea0410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f10b259c640 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 0x5628c9ea0410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f10b259c640 1 -- 172.21.15.103:0/2697009050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c9b6b3c0 con 0x5628c9e945d0 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f10b259c640 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9ea3160 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f10a400fb60 tx=0x7f10a4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b336dcaf60cc5e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f1099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:21.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f1099ffb640 1 -- 172.21.15.103:0/2697009050 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a40263f0 con 0x5628c9e945d0 2024-09-19T07:51:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f1099ffb640 1 -- 172.21.15.103:0/2697009050 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f10a4026590 con 0x5628c9e945d0 2024-09-19T07:51:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.785+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac0051d0 con 0x5628c9e945d0 2024-09-19T07:51:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.786+0000 7f1099ffb640 1 -- 172.21.15.103:0/2697009050 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f10a4026790 con 0x5628c9e945d0 2024-09-19T07:51:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.786+0000 7f10bb229740 10 client.18120 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.786+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5628c9eb1f60 con 0x5628c9e945d0 2024-09-19T07:51:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.786+0000 7f1099ffb640 1 -- 172.21.15.103:0/2697009050 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a4026e10 con 0x5628c9e945d0 2024-09-19T07:51:21.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f1099ffb640 1 -- 172.21.15.103:0/2697009050 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f10a4025070 con 0x5628c9e945d0 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f10bb229740 10 client.18120 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f10bb229740 10 client.18120 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f10bb229740 20 client.18120 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f10bb229740 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9eb5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f10bb229740 4 client.18120 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f10ac0051d0 con 0x5628c9eb3420 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.787+0000 7f10b1d9b640 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9eb5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.788+0000 7f1099ffb640 10 client.18120 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.788+0000 7f10b1d9b640 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9eb5870 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f95e8d2777abf0d server_cookie=36a0aebc03dd71ad in_seq=0 out_seq=0 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.788+0000 7f1099ffb640 1 -- 172.21.15.103:0/2697009050 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f10ac0051d0 con 0x5628c9eb3420 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.788+0000 7f1099ffb640 10 client.18120 handle_command_reply: tid=0 2024-09-19T07:51:21.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.788+0000 7f10bb229740 1 client.18120 shutdown 2024-09-19T07:51:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 msgr2=0x5628c9ea3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9ea3160 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f10a400fb60 tx=0x7f10a4013040 comp rx=0 tx=0).stop 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 shutdown_connections 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9eb5870 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f1099ffb640 0 client.18120 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 0x5628c9ea0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 --2- 172.21.15.103:0/2697009050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e945d0 0x5628c9ea3160 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 >> 172.21.15.103:0/2697009050 conn(0x5628c9e5d310 msgr2=0x5628c9e969c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 shutdown_connections 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 1 -- 172.21.15.103:0/2697009050 wait complete. 2024-09-19T07:51:21.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.789+0000 7f10bb229740 20 client.18120 trim_cache size 0 max 16384 2024-09-19T07:51:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.809+0000 7f10bb229740 1 Processor -- start 2024-09-19T07:51:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.809+0000 7f10bb229740 1 -- start start 2024-09-19T07:51:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10bb229740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e945d0 0x5628c9e90e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10bb229740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 0x5628c9eaf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10bb229740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9ea3780 0x5628c9ebb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10bb229740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41ab7e0 con 0x5628c9ea3780 2024-09-19T07:51:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10bb229740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41ad350 con 0x5628c9e95b40 2024-09-19T07:51:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10bb229740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41abeb0 con 0x5628c9e945d0 2024-09-19T07:51:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e945d0 0x5628c9e90e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e945d0 0x5628c9e90e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48368/0 (socket says 172.21.15.103:48368) 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 -- 172.21.15.103:0/3538012703 learned_addr learned my addr 172.21.15.103:0/3538012703 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b1d9b640 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 0x5628c9eaf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 -- 172.21.15.103:0/3538012703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 msgr2=0x5628c9eaf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 0x5628c9eaf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 -- 172.21.15.103:0/3538012703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9ea3780 msgr2=0x5628c9ebb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9ea3780 0x5628c9ebb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.810+0000 7f10b259c640 1 -- 172.21.15.103:0/3538012703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c9ea34d0 con 0x5628c9e945d0 2024-09-19T07:51:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.811+0000 7f10b259c640 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e945d0 0x5628c9e90e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f10a4000c00 tx=0x7f10a4005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=303fe066224bc446 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.811+0000 7f109bfff640 1 -- 172.21.15.103:0/3538012703 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a40263f0 con 0x5628c9e945d0 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.811+0000 7f109bfff640 1 -- 172.21.15.103:0/3538012703 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a402fe50 con 0x5628c9e945d0 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.811+0000 7f109bfff640 1 -- 172.21.15.103:0/3538012703 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a400e680 con 0x5628c9e945d0 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 -- 172.21.15.103:0/3538012703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e945d0 msgr2=0x5628c9e90e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e945d0 0x5628c9e90e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f10a4000c00 tx=0x7f10a4005630 comp rx=0 tx=0).stop 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 -- 172.21.15.103:0/3538012703 shutdown_connections 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9ea3780 0x5628c9ebb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e95b40 0x5628c9eaf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 --2- 172.21.15.103:0/3538012703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9e945d0 0x5628c9e90e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 -- 172.21.15.103:0/3538012703 >> 172.21.15.103:0/3538012703 conn(0x5628c9e5d310 msgr2=0x5628c9e90700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 -- 172.21.15.103:0/3538012703 shutdown_connections 2024-09-19T07:51:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.812+0000 7f10bb229740 1 -- 172.21.15.103:0/3538012703 wait complete. 2024-09-19T07:51:21.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.813+0000 7f10bb229740 1 -- start start 2024-09-19T07:51:21.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.813+0000 7f10bb229740 1 Processor -- start 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.813+0000 7f10bb229740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e945d0 0x5628c9eab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.813+0000 7f10bb229740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e95b40 0x5628c9eabdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10bb229740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10b1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e95b40 0x5628c9eabdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10b1d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e95b40 0x5628c9eabdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52936/0 (socket says 172.21.15.103:52936) 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10b259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e945d0 0x5628c9eab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10b259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e945d0 0x5628c9eab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39000/0 (socket says 172.21.15.103:39000) 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10b2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10b2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48378/0 (socket says 172.21.15.103:48378) 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10b1d9b640 1 -- 172.21.15.103:0/99547832 learned_addr learned my addr 172.21.15.103:0/99547832 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41acc50 con 0x5628c9e95b40 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41ac580 con 0x5628c9e945d0 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41a97e0 con 0x5628c9ea3780 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.814+0000 7f10bb229740 10 client.0 _ll_register_callbacks cb 0x7f10a4009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10b259c640 1 -- 172.21.15.103:0/99547832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 msgr2=0x5628c9ea8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10b259c640 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10b259c640 1 -- 172.21.15.103:0/99547832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e95b40 msgr2=0x5628c9eabdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10b259c640 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e95b40 0x5628c9eabdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10b259c640 1 -- 172.21.15.103:0/99547832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c9ea8da0 con 0x5628c9e945d0 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10b259c640 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e945d0 0x5628c9eab890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009f40 tx=0x7f10a40266a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f995024566cb79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f1099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:51:21.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f1099ffb640 1 -- 172.21.15.103:0/99547832 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4026b30 con 0x5628c9e945d0 2024-09-19T07:51:21.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f1099ffb640 1 -- 172.21.15.103:0/99547832 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f10a4026cd0 con 0x5628c9e945d0 2024-09-19T07:51:21.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a402fe50 con 0x5628c9e945d0 2024-09-19T07:51:21.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f1099ffb640 1 -- 172.21.15.103:0/99547832 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f10a4038430 con 0x5628c9e945d0 2024-09-19T07:51:21.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10bb229740 10 client.27434 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:21.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.815+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5628c9f1e8e0 con 0x5628c9e945d0 2024-09-19T07:51:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f1099ffb640 1 -- 172.21.15.103:0/99547832 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a4038ab0 con 0x5628c9e945d0 2024-09-19T07:51:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f1099ffb640 1 -- 172.21.15.103:0/99547832 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f10a4025040 con 0x5628c9e945d0 2024-09-19T07:51:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f10bb229740 10 client.27434 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f10bb229740 10 client.27434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:51:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f10bb229740 20 client.27434 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f10bb229740 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9f20fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f10bb229740 4 client.27434 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f10a402fe50 con 0x5628c9eb3420 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.816+0000 7f10b1d9b640 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9f20fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10b1d9b640 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9f20fa0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b474965dd2a85777 server_cookie=52c32ce34c64bfc1 in_seq=0 out_seq=0 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f1099ffb640 10 client.27434 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f1099ffb640 1 -- 172.21.15.103:0/99547832 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f10a402fe50 con 0x5628c9eb3420 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f1099ffb640 10 client.27434 handle_command_reply: tid=0 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 client.27434 shutdown 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e945d0 msgr2=0x5628c9eab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e945d0 0x5628c9eab890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009f40 tx=0x7f10a40266a0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 shutdown_connections 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5628c9eb3420 0x5628c9f20fa0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628c9ea3780 0x5628c9ea8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5628c9e95b40 0x5628c9eabdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 --2- 172.21.15.103:0/99547832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628c9e945d0 0x5628c9eab890 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f1099ffb640 0 client.27434 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:51:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.817+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 >> 172.21.15.103:0/99547832 conn(0x5628c9e5d310 msgr2=0x5628c9e961a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 shutdown_connections 2024-09-19T07:51:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10bb229740 1 -- 172.21.15.103:0/99547832 wait complete. 2024-09-19T07:51:21.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10bb229740 20 client.27434 trim_cache size 0 max 16384 2024-09-19T07:51:21.820 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f109c0a3430 msgr2=0x7f109c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10b8e50640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f109c0a3430 0x7f109c0a58f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f10a4011550 tx=0x7f10a4005f90 comp rx=0 tx=0).stop 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b410c170 msgr2=0x7f10b419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10b8e50640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b410c170 0x7f10b419cd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009fd0 tx=0x7f10ac00ef10 comp rx=0 tx=0).stop 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.818+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 shutdown_connections 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.819+0000 7f10b8e50640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f109c0a3430 0x7f109c0a58f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.819+0000 7f10b8e50640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b410c170 0x7f10b419cd50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.819+0000 7f10b8e50640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b41036b0 0x7f10b41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.819+0000 7f10b8e50640 1 --2- 172.21.15.103:0/2851748980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b4102d90 0x7f10b4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.819+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 >> 172.21.15.103:0/2851748980 conn(0x7f10b40fc790 msgr2=0x7f10b40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:21.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.819+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 shutdown_connections 2024-09-19T07:51:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:21.819+0000 7f10b8e50640 1 -- 172.21.15.103:0/2851748980 wait complete. 2024-09-19T07:51:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:22.440 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:22 smithi103 ceph-mon[31201]: pgmap v418: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-19T07:51:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:22 smithi134 ceph-mon[35414]: pgmap v418: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-19T07:51:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:22 smithi174 ceph-mon[35370]: pgmap v418: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-19T07:51:22.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 -- 172.21.15.103:0/230299061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 msgr2=0x7f239c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:22.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 --2- 172.21.15.103:0/230299061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c101200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2398009930 tx=0x7f239802a4d0 comp rx=0 tx=0).stop 2024-09-19T07:51:22.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 -- 172.21.15.103:0/230299061 shutdown_connections 2024-09-19T07:51:22.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 --2- 172.21.15.103:0/230299061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f239c102ef0 0x7f239c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:22.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 --2- 172.21.15.103:0/230299061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 0x7f239c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 --2- 172.21.15.103:0/230299061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c101200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 -- 172.21.15.103:0/230299061 >> 172.21.15.103:0/230299061 conn(0x7f239c0fc7d0 msgr2=0x7f239c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 -- 172.21.15.103:0/230299061 shutdown_connections 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.890+0000 7f23a36d0640 1 -- 172.21.15.103:0/230299061 wait complete. 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.891+0000 7f23a36d0640 1 Processor -- start 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.891+0000 7f23a36d0640 1 -- start start 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a36d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a36d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 0x7f239c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a36d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f239c102ef0 0x7f239c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a36d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f239c110fd0 con 0x7f239c102ef0 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:22.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a0c6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 0x7f239c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48382/0 (socket says 172.21.15.103:48382) 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a0c6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 0x7f239c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39022/0 (socket says 172.21.15.103:39022) 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 -- 172.21.15.103:0/880665866 learned_addr learned my addr 172.21.15.103:0/880665866 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a36d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f239c110e50 con 0x7f239c102570 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f239c111150 con 0x7f239c100e20 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a1c6d640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f239c102ef0 0x7f239c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 -- 172.21.15.103:0/880665866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 msgr2=0x7f239c197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 0x7f239c197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 -- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f239c102ef0 msgr2=0x7f239c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f239c102ef0 0x7f239c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a146c640 1 -- 172.21.15.103:0/880665866 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c19c3d0 con 0x7f239c100e20 2024-09-19T07:51:22.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.892+0000 7f23a0c6b640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 0x7f239c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:51:22.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.893+0000 7f23a146c640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c197420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f239802a9f0 tx=0x7f2398003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:22.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.893+0000 7f23927fc640 1 -- 172.21.15.103:0/880665866 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239803f070 con 0x7f239c100e20 2024-09-19T07:51:22.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.893+0000 7f23927fc640 1 -- 172.21.15.103:0/880665866 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2398031c90 con 0x7f239c100e20 2024-09-19T07:51:22.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.893+0000 7f23927fc640 1 -- 172.21.15.103:0/880665866 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398035030 con 0x7f239c100e20 2024-09-19T07:51:22.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.893+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c19c660 con 0x7f239c100e20 2024-09-19T07:51:22.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.894+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c19caf0 con 0x7f239c100e20 2024-09-19T07:51:22.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.895+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239c101200 con 0x7f239c100e20 2024-09-19T07:51:22.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.896+0000 7f23927fc640 1 -- 172.21.15.103:0/880665866 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2398042070 con 0x7f239c100e20 2024-09-19T07:51:22.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.897+0000 7f23927fc640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f23880a55c0 0x7f23880a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:22.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.897+0000 7f23a0c6b640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f23880a55c0 0x7f23880a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:22.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.897+0000 7f23927fc640 1 -- 172.21.15.103:0/880665866 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f23980f0710 con 0x7f239c100e20 2024-09-19T07:51:22.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.897+0000 7f23a0c6b640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f23880a55c0 0x7f23880a7a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f239c0fed80 tx=0x7f238c005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:22.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:22.900+0000 7f23927fc640 1 -- 172.21.15.103:0/880665866 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f23980bca30 con 0x7f239c100e20 2024-09-19T07:51:23.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.150+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f239c108b00 con 0x7f239c100e20 2024-09-19T07:51:23.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.151+0000 7f23927fc640 1 -- 172.21.15.103:0/880665866 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11992 (secure 0 0 0) 0x7f23980c0d30 con 0x7f239c100e20 2024-09-19T07:51:23.152 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:51:23.152 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":51,"btime":"2024-09-19T07:50:10:429478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:50:09.866710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:51:23.152 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 51 2024-09-19T07:51:23.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.154+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f23880a55c0 msgr2=0x7f23880a7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:23.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.154+0000 7f23a36d0640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f23880a55c0 0x7f23880a7a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f239c0fed80 tx=0x7f238c005f90 comp rx=0 tx=0).stop 2024-09-19T07:51:23.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.154+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 msgr2=0x7f239c197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:23.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.154+0000 7f23a36d0640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c197420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f239802a9f0 tx=0x7f2398003b20 comp rx=0 tx=0).stop 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 shutdown_connections 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f23880a55c0 0x7f23880a7a80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f239c102ef0 0x7f239c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f239c102570 0x7f239c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 --2- 172.21.15.103:0/880665866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f239c100e20 0x7f239c197420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 >> 172.21.15.103:0/880665866 conn(0x7f239c0fc7d0 msgr2=0x7f239c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 shutdown_connections 2024-09-19T07:51:23.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:23.155+0000 7f23a36d0640 1 -- 172.21.15.103:0/880665866 wait complete. 2024-09-19T07:51:23.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:23 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/880665866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:23.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:23 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/880665866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:23.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:51:23.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:23 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/880665866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:24.317 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T07:51:24.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T07:51:24.383 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:24.531 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:24 smithi103 ceph-mon[31201]: pgmap v419: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-19T07:51:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:24 smithi134 ceph-mon[35414]: pgmap v419: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-19T07:51:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:24 smithi174 ceph-mon[35370]: pgmap v419: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-19T07:51:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 770.0 s, 388.2 tps, lat 438.037 ms stddev 2076.015 2024-09-19T07:51:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2342284932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014100100 msgr2=0x7fb014100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2342284932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014100100 0x7fb014100560 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb004009860 tx=0x7fb00402a340 comp rx=0 tx=0).stop 2024-09-19T07:51:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2342284932 shutdown_connections 2024-09-19T07:51:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2342284932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100aa0 0x7fb01410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2342284932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014100100 0x7fb014100560 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2342284932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014106b80 0x7fb014106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2342284932 >> 172.21.15.103:0/2342284932 conn(0x7fb014077230 msgr2=0x7fb0140fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2342284932 shutdown_connections 2024-09-19T07:51:25.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.703+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2342284932 wait complete. 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 Processor -- start 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 -- start start 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100100 0x7fb0141a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014100aa0 0x7fb0141a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014106b80 0x7fb01410dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb014112090 con 0x7fb014106b80 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb014111f10 con 0x7fb014100100 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.704+0000 7fb01b9d3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb014112210 con 0x7fb014100aa0 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100100 0x7fb0141a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100100 0x7fb0141a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39040/0 (socket says 172.21.15.103:39040) 2024-09-19T07:51:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 -- 172.21.15.103:0/2540042724 learned_addr learned my addr 172.21.15.103:0/2540042724 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 -- 172.21.15.103:0/2540042724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014100aa0 msgr2=0x7fb0141a4d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:51:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014100aa0 0x7fb0141a4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 -- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014106b80 msgr2=0x7fb01410dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014106b80 0x7fb01410dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 -- 172.21.15.103:0/2540042724 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01410e310 con 0x7fb014100100 2024-09-19T07:51:25.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb01976f640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100100 0x7fb0141a4830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b4a0 tx=0x7fb01000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:25.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.705+0000 7fb00a7fc640 1 -- 172.21.15.103:0/2540042724 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010014070 con 0x7fb014100100 2024-09-19T07:51:25.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.706+0000 7fb00a7fc640 1 -- 172.21.15.103:0/2540042724 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb010010af0 con 0x7fb014100100 2024-09-19T07:51:25.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.706+0000 7fb00a7fc640 1 -- 172.21.15.103:0/2540042724 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010017d10 con 0x7fb014100100 2024-09-19T07:51:25.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.706+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01410e600 con 0x7fb014100100 2024-09-19T07:51:25.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.706+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0141ab700 con 0x7fb014100100 2024-09-19T07:51:25.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.714+0000 7fb01ddac740 1 Processor -- start 2024-09-19T07:51:25.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.715+0000 7fb00a7fc640 1 -- 172.21.15.103:0/2540042724 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb010017070 con 0x7fb014100100 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.716+0000 7fb00a7fc640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0000a34b0 0x7fb0000a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.716+0000 7fb00a7fc640 1 -- 172.21.15.103:0/2540042724 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fb0100cf080 con 0x7fb014100100 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01ddac740 1 -- start start 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383e5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383f0340 0x561a383f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01ddac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561a3834b2f0 con 0x561a383e55a0 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01ddac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561a382f2cf0 con 0x561a383e6b10 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01ddac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a383472b0 con 0x561a383f0340 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01976f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01976f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383e5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39054/0 (socket says 172.21.15.103:39054) 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.717+0000 7fb01976f640 1 -- 172.21.15.103:0/2915506320 learned_addr learned my addr 172.21.15.103:0/2915506320 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.718+0000 7fb01976f640 1 -- 172.21.15.103:0/2915506320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383f0340 msgr2=0x561a383f2730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.718+0000 7fb01976f640 1 --2- 172.21.15.103:0/2915506320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383f0340 0x561a383f2730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.718+0000 7fb01976f640 1 -- 172.21.15.103:0/2915506320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 msgr2=0x561a383e59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.718+0000 7fb01976f640 1 --2- 172.21.15.103:0/2915506320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.718+0000 7fb01976f640 1 -- 172.21.15.103:0/2915506320 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a382d73a0 con 0x561a383e6b10 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.718+0000 7fb01976f640 1 --2- 172.21.15.103:0/2915506320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383e5060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb010097dd0 tx=0x7fb01000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac4f355257591ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.719+0000 7fafff7fe640 1 -- 172.21.15.103:0/2915506320 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100b4070 con 0x561a383e6b10 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.719+0000 7fafff7fe640 1 -- 172.21.15.103:0/2915506320 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0100a4070 con 0x561a383e6b10 2024-09-19T07:51:25.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.719+0000 7fafff7fe640 1 -- 172.21.15.103:0/2915506320 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01001e070 con 0x561a383e6b10 2024-09-19T07:51:25.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.720+0000 7fb01ddac740 1 -- 172.21.15.103:0/2915506320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 msgr2=0x561a383e5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.720+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2915506320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383e5060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb010097dd0 tx=0x7fb01000bf90 comp rx=0 tx=0).stop 2024-09-19T07:51:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.723+0000 7fb01ddac740 1 -- 172.21.15.103:0/2915506320 shutdown_connections 2024-09-19T07:51:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.723+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2915506320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383f0340 0x561a383f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.723+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2915506320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.723+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2915506320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383e5060 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.723+0000 7fb01ddac740 1 -- 172.21.15.103:0/2915506320 >> 172.21.15.103:0/2915506320 conn(0x561a383ae310 msgr2=0x561a383e1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:25.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.726+0000 7fb019f70640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0000a34b0 0x7fb0000a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.726+0000 7fb01ddac740 1 -- 172.21.15.103:0/2915506320 shutdown_connections 2024-09-19T07:51:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.727+0000 7fb01ddac740 1 -- 172.21.15.103:0/2915506320 wait complete. 2024-09-19T07:51:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.727+0000 7fb01ddac740 1 -- start start 2024-09-19T07:51:25.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.727+0000 7fb01ddac740 1 Processor -- start 2024-09-19T07:51:25.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.728+0000 7fb019f70640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0000a34b0 0x7fb0000a5970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb00c005c40 tx=0x7fb00c005bd0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.732+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e55a0 0x561a383efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383f0340 0x561a383ed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01ddac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561a383f2fe0 con 0x561a383f0340 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01ddac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561a38306060 con 0x561a383e6b10 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01ddac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a383f3160 con 0x561a383e55a0 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01ddac740 10 client.0 _ll_register_callbacks cb 0x561a380598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01976f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e55a0 0x561a383efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01976f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e55a0 0x561a383efdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48404/0 (socket says 172.21.15.103:48404) 2024-09-19T07:51:25.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.733+0000 7fb01976f640 1 -- 172.21.15.103:0/2042634507 learned_addr learned my addr 172.21.15.103:0/2042634507 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7fb01976f640 1 -- 172.21.15.103:0/2042634507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 msgr2=0x561a383ed0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:51:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7fb01976f640 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383ed0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7fb01976f640 1 -- 172.21.15.103:0/2042634507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383f0340 msgr2=0x561a383ed830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7fb01976f640 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383f0340 0x561a383ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7fb01976f640 1 -- 172.21.15.103:0/2042634507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a380bc3c0 con 0x561a383e55a0 2024-09-19T07:51:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7fb01976f640 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e55a0 0x561a383efdf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb01009b390 tx=0x7fb010097ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ac8210c06fb68e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:25.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7faffcff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:51:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7faffcff9640 1 -- 172.21.15.103:0/2042634507 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100b9060 con 0x561a383e55a0 2024-09-19T07:51:25.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.734+0000 7faffcff9640 1 -- 172.21.15.103:0/2042634507 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fb01009a850 con 0x561a383e55a0 2024-09-19T07:51:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.737+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0100a4070 con 0x561a383e55a0 2024-09-19T07:51:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.738+0000 7faffcff9640 1 -- 172.21.15.103:0/2042634507 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb01007e050 con 0x561a383e55a0 2024-09-19T07:51:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.738+0000 7fb01ddac740 10 client.27415 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.738+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x561a383f2fe0 con 0x561a383e55a0 2024-09-19T07:51:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.738+0000 7faffcff9640 1 -- 172.21.15.103:0/2042634507 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01009aa70 con 0x561a383e55a0 2024-09-19T07:51:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.738+0000 7faffcff9640 1 -- 172.21.15.103:0/2042634507 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fb01009a2c0 con 0x561a383e55a0 2024-09-19T07:51:25.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.739+0000 7fb01ddac740 10 client.27415 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.739+0000 7fb01ddac740 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.739+0000 7fb01ddac740 20 client.27415 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.739+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a383ffcd0 0x561a384020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.739+0000 7fb01ddac740 4 client.27415 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:25.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.739+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb0100a4070 con 0x561a383ffcd0 2024-09-19T07:51:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.740+0000 7fb018f6e640 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a383ffcd0 0x561a384020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.740+0000 7faffcff9640 10 client.27415 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.740+0000 7fb018f6e640 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a383ffcd0 0x561a384020c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6a5abe8d317c3b server_cookie=161fdc80330bd6fe in_seq=0 out_seq=0 2024-09-19T07:51:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.741+0000 7faffcff9640 1 -- 172.21.15.103:0/2042634507 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb0100a4070 con 0x561a383ffcd0 2024-09-19T07:51:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.741+0000 7faffcff9640 10 client.27415 handle_command_reply: tid=0 2024-09-19T07:51:25.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.742+0000 7fb01ddac740 1 client.27415 shutdown 2024-09-19T07:51:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.744+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e55a0 msgr2=0x561a383efdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.744+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e55a0 0x561a383efdf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb01009b390 tx=0x7fb010097ca0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.746+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 shutdown_connections 2024-09-19T07:51:25.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.746+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a383ffcd0 0x561a384020c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.746+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383f0340 0x561a383ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.746+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383e6b10 0x561a383ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.746+0000 7fb01ddac740 1 --2- 172.21.15.103:0/2042634507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e55a0 0x561a383efdf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.746+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 >> 172.21.15.103:0/2042634507 conn(0x561a383ae310 msgr2=0x561a383e7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:25.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.747+0000 7faffcff9640 0 client.27415 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:25.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.747+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 shutdown_connections 2024-09-19T07:51:25.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.747+0000 7fb01ddac740 1 -- 172.21.15.103:0/2042634507 wait complete. 2024-09-19T07:51:25.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.747+0000 7fb01ddac740 20 client.27415 trim_cache size 0 max 16384 2024-09-19T07:51:25.763 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 Processor -- start 2024-09-19T07:51:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 -- start start 2024-09-19T07:51:25.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141b41e0 con 0x561a383e55a0 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141b5d50 con 0x561a383f0340 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01ddac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141b48b0 con 0x561a383e6b10 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01976f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01976f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52988/0 (socket says 172.21.15.103:52988) 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb01976f640 1 -- 172.21.15.103:0/595319434 learned_addr learned my addr 172.21.15.103:0/595319434 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb019f70640 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.768+0000 7fb018f6e640 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7fb01976f640 1 -- 172.21.15.103:0/595319434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 msgr2=0x561a383e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:25.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7fb01976f640 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7fb01976f640 1 -- 172.21.15.103:0/595319434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 msgr2=0x561a383e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:25.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7fb01976f640 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7fb01976f640 1 -- 172.21.15.103:0/595319434 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a383e1ea0 con 0x561a383e55a0 2024-09-19T07:51:25.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7fb01976f640 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e0bf0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb0100996b0 tx=0x7fb0100047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a17d3c31c49c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:25.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7faffeffd640 1 -- 172.21.15.103:0/595319434 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100b9070 con 0x561a383e55a0 2024-09-19T07:51:25.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7faffeffd640 1 -- 172.21.15.103:0/595319434 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0100a0ce0 con 0x561a383e55a0 2024-09-19T07:51:25.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.769+0000 7faffeffd640 1 -- 172.21.15.103:0/595319434 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01001e070 con 0x561a383e55a0 2024-09-19T07:51:25.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 -- 172.21.15.103:0/595319434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 msgr2=0x561a383e0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e0bf0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb0100996b0 tx=0x7fb0100047d0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 -- 172.21.15.103:0/595319434 shutdown_connections 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 --2- 172.21.15.103:0/595319434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383e0bf0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 -- 172.21.15.103:0/595319434 >> 172.21.15.103:0/595319434 conn(0x561a383ae310 msgr2=0x561a383e04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 -- 172.21.15.103:0/595319434 shutdown_connections 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 -- 172.21.15.103:0/595319434 wait complete. 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.770+0000 7fb01ddac740 1 -- start start 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 1 Processor -- start 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141b5650 con 0x561a383e55a0 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141b4f80 con 0x561a383f0340 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0141b21e0 con 0x561a383e6b10 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01ddac740 10 client.0 _ll_register_callbacks cb 0x7fb0100b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01976f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01976f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52996/0 (socket says 172.21.15.103:52996) 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb01976f640 1 -- 172.21.15.103:0/387581282 learned_addr learned my addr 172.21.15.103:0/387581282 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.771+0000 7fb019f70640 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.772+0000 7fb018f6e640 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.772+0000 7fb019f70640 1 -- 172.21.15.103:0/387581282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 msgr2=0x561a383f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.772+0000 7fb019f70640 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.772+0000 7fb019f70640 1 -- 172.21.15.103:0/387581282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 msgr2=0x561a383f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.772+0000 7fb019f70640 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.772+0000 7fb019f70640 1 -- 172.21.15.103:0/387581282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a383f4bc0 con 0x561a383f0340 2024-09-19T07:51:25.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.772+0000 7fb019f70640 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383f4390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x561a38347390 tx=0x7fb00c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f22292f5b19bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:25.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.773+0000 7faffcff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:51:25.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.774+0000 7faffcff9640 1 -- 172.21.15.103:0/387581282 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c028060 con 0x561a383f0340 2024-09-19T07:51:25.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.774+0000 7faffcff9640 1 -- 172.21.15.103:0/387581282 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fb00c019ae0 con 0x561a383f0340 2024-09-19T07:51:25.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.774+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0100a0ce0 con 0x561a383f0340 2024-09-19T07:51:25.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.776+0000 7faffcff9640 1 -- 172.21.15.103:0/387581282 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb00c019ce0 con 0x561a383f0340 2024-09-19T07:51:25.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.776+0000 7fb01ddac740 10 client.27455 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:25.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.776+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x561a3846a710 con 0x561a383f0340 2024-09-19T07:51:25.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.776+0000 7faffcff9640 1 -- 172.21.15.103:0/387581282 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0100a0ce0 con 0x561a383f0340 2024-09-19T07:51:25.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7faffcff9640 1 -- 172.21.15.103:0/387581282 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fb00c00b040 con 0x561a383f0340 2024-09-19T07:51:25.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01ddac740 10 client.27455 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:25.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01ddac740 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:25.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01ddac740 20 client.27455 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:25.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01ddac740 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a3846ae40 0x561a3846d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:25.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01ddac740 4 client.27455 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:51:25.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a383e5cf0 con 0x561a3846ae40 2024-09-19T07:51:25.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01976f640 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a3846ae40 0x561a3846d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:25.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7faffcff9640 10 client.27455 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:25.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.777+0000 7fb01976f640 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a3846ae40 0x561a3846d260 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e3b7b758d980eb server_cookie=fa173a1fad78ffeb in_seq=0 out_seq=0 2024-09-19T07:51:25.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7faffcff9640 1 -- 172.21.15.103:0/387581282 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a383e5cf0 con 0x561a3846ae40 2024-09-19T07:51:25.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7faffcff9640 10 client.27455 handle_command_reply: tid=0 2024-09-19T07:51:25.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 client.27455 shutdown 2024-09-19T07:51:25.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 msgr2=0x561a383f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383f4390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x561a38347390 tx=0x7fb00c004a40 comp rx=0 tx=0).stop 2024-09-19T07:51:25.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 shutdown_connections 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561a3846ae40 0x561a3846d260 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a383f0340 0x561a383f4390 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a383e6b10 0x561a383f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 --2- 172.21.15.103:0/387581282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561a383e55a0 0x561a383f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7faffcff9640 0 client.27455 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 >> 172.21.15.103:0/387581282 conn(0x561a383ae310 msgr2=0x561a383e0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 shutdown_connections 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 1 -- 172.21.15.103:0/387581282 wait complete. 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.778+0000 7fb01ddac740 20 client.27455 trim_cache size 0 max 16384 2024-09-19T07:51:25.781 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:51:25.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0000a34b0 msgr2=0x7fb0000a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0000a34b0 0x7fb0000a5970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb00c005c40 tx=0x7fb00c005bd0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100100 msgr2=0x7fb0141a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:25.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100100 0x7fb0141a4830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b4a0 tx=0x7fb01000b960 comp rx=0 tx=0).stop 2024-09-19T07:51:25.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 shutdown_connections 2024-09-19T07:51:25.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0000a34b0 0x7fb0000a5970 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb014106b80 0x7fb01410dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb014100aa0 0x7fb0141a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 --2- 172.21.15.103:0/2540042724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb014100100 0x7fb0141a4830 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:25.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 >> 172.21.15.103:0/2540042724 conn(0x7fb014077230 msgr2=0x7fb014104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:25.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 shutdown_connections 2024-09-19T07:51:25.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:25.779+0000 7fb01b9d3640 1 -- 172.21.15.103:0/2540042724 wait complete. 2024-09-19T07:51:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:26 smithi103 ceph-mon[31201]: pgmap v420: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-09-19T07:51:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.552+0000 7fd694dda640 1 -- 172.21.15.103:0/1128304631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690074eb0 msgr2=0x7fd690112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.552+0000 7fd694dda640 1 --2- 172.21.15.103:0/1128304631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690074eb0 0x7fd690112650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd688009ca0 tx=0x7fd68802a980 comp rx=0 tx=0).stop 2024-09-19T07:51:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.553+0000 7fd694dda640 1 -- 172.21.15.103:0/1128304631 shutdown_connections 2024-09-19T07:51:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.553+0000 7fd694dda640 1 --2- 172.21.15.103:0/1128304631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690112b90 0x7fd690114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.553+0000 7fd694dda640 1 --2- 172.21.15.103:0/1128304631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690074eb0 0x7fd690112650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.553+0000 7fd694dda640 1 --2- 172.21.15.103:0/1128304631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd690074590 0x7fd690074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.553+0000 7fd694dda640 1 -- 172.21.15.103:0/1128304631 >> 172.21.15.103:0/1128304631 conn(0x7fd6900fd8e0 msgr2=0x7fd6900ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:26.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.560+0000 7fd694dda640 1 -- 172.21.15.103:0/1128304631 shutdown_connections 2024-09-19T07:51:26.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.560+0000 7fd694dda640 1 -- 172.21.15.103:0/1128304631 wait complete. 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.561+0000 7fd694dda640 1 Processor -- start 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.561+0000 7fd694dda640 1 -- start start 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.562+0000 7fd694dda640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd690074590 0x7fd69010c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.562+0000 7fd694dda640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690074eb0 0x7fd69010c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.562+0000 7fd694dda640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690112b90 0x7fd690111b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.562+0000 7fd694dda640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd690106430 con 0x7fd690074590 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.562+0000 7fd694dda640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901062b0 con 0x7fd690112b90 2024-09-19T07:51:26.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.562+0000 7fd694dda640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901065b0 con 0x7fd690074eb0 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.563+0000 7fd68dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690074eb0 0x7fd69010c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.563+0000 7fd68dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690074eb0 0x7fd69010c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:48418/0 (socket says 172.21.15.103:48418) 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.563+0000 7fd68dd9b640 1 -- 172.21.15.103:0/3189652639 learned_addr learned my addr 172.21.15.103:0/3189652639 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.563+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690112b90 0x7fd690111b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.563+0000 7fd68e59c640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd690074590 0x7fd69010c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd68dd9b640 1 -- 172.21.15.103:0/3189652639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690112b90 msgr2=0x7fd690111b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690112b90 0x7fd690111b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd68dd9b640 1 -- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd690074590 msgr2=0x7fd69010c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd690074590 0x7fd69010c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd68dd9b640 1 -- 172.21.15.103:0/3189652639 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6901123d0 con 0x7fd690074eb0 2024-09-19T07:51:26.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690074eb0 0x7fd69010c860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd6880308b0 tx=0x7fd6880308e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd67f7fe640 1 -- 172.21.15.103:0/3189652639 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688047070 con 0x7fd690074eb0 2024-09-19T07:51:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd67f7fe640 1 -- 172.21.15.103:0/3189652639 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd688004040 con 0x7fd690074eb0 2024-09-19T07:51:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.564+0000 7fd67f7fe640 1 -- 172.21.15.103:0/3189652639 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688034070 con 0x7fd690074eb0 2024-09-19T07:51:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.565+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd690112600 con 0x7fd690074eb0 2024-09-19T07:51:26.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.566+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69011e0e0 con 0x7fd690074eb0 2024-09-19T07:51:26.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.570+0000 7fd67f7fe640 1 -- 172.21.15.103:0/3189652639 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd68804a070 con 0x7fd690074eb0 2024-09-19T07:51:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.570+0000 7fd6971b3740 1 Processor -- start 2024-09-19T07:51:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd6971b3740 1 -- start start 2024-09-19T07:51:26.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd5c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd5ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd6971b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55945dcb72a0 con 0x55945dd51760 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd6971b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55945dc5eb10 con 0x55945dd5c940 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd6971b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55945dcb3260 con 0x55945dd52080 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd68ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd51b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53026/0 (socket says 172.21.15.103:53026) 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3681056118 learned_addr learned my addr 172.21.15.103:0/3681056118 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.571+0000 7fd68e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd5c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd67f7fe640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6780e7500 0x7fd6780e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd5ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3681056118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 msgr2=0x55945dd5c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd5c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3681056118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 msgr2=0x55945dd5ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd5ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3681056118 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945da28b90 con 0x55945dd51760 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6780e7500 0x7fd6780e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.572+0000 7fd67f7fe640 1 -- 172.21.15.103:0/3189652639 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fd6880f0a40 con 0x7fd690074eb0 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.573+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd51b40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd680009ad0 tx=0x7fd680009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744428e1fca136d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.573+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6780e7500 0x7fd6780e99c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd68000e660 tx=0x7fd68001b000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.573+0000 7fd67cff9640 1 -- 172.21.15.103:0/3681056118 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680022070 con 0x55945dd51760 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.573+0000 7fd67cff9640 1 -- 172.21.15.103:0/3681056118 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68001d900 con 0x55945dd51760 2024-09-19T07:51:26.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.574+0000 7fd67cff9640 1 -- 172.21.15.103:0/3681056118 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68001c670 con 0x55945dd51760 2024-09-19T07:51:26.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.574+0000 7fd6971b3740 1 -- 172.21.15.103:0/3681056118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 msgr2=0x55945dd51b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.574+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd51b40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd680009ad0 tx=0x7fd680009f90 comp rx=0 tx=0).stop 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.575+0000 7fd6971b3740 1 -- 172.21.15.103:0/3681056118 shutdown_connections 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.575+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd5ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.575+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd5c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.575+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3681056118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd51b40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.575+0000 7fd6971b3740 1 -- 172.21.15.103:0/3681056118 >> 172.21.15.103:0/3681056118 conn(0x55945dd4dcf0 msgr2=0x55945dd4e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.575+0000 7fd6971b3740 1 -- 172.21.15.103:0/3681056118 shutdown_connections 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.575+0000 7fd6971b3740 1 -- 172.21.15.103:0/3681056118 wait complete. 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.576+0000 7fd6971b3740 1 -- start start 2024-09-19T07:51:26.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.576+0000 7fd6971b3740 1 Processor -- start 2024-09-19T07:51:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.576+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd52080 0x55945dd59d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd5a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd6971b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55945dd5f510 con 0x55945dd51760 2024-09-19T07:51:26.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd6971b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55945dc3f6e0 con 0x55945dd52080 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd6971b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55945dd5f690 con 0x55945dd5c940 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd59830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53028/0 (socket says 172.21.15.103:53028) 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 -- 172.21.15.103:0/2082421542 learned_addr learned my addr 172.21.15.103:0/2082421542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd52080 0x55945dd59d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd6971b3740 10 client.0 _ll_register_callbacks cb 0x55945dc63a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd5a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 -- 172.21.15.103:0/2082421542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 msgr2=0x55945dd5a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd5a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 -- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd52080 msgr2=0x55945dd59d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd52080 0x55945dd59d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.577+0000 7fd68e59c640 1 -- 172.21.15.103:0/2082421542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945dd68a60 con 0x55945dd51760 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.578+0000 7fd68e59c640 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd59830 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd684011570 tx=0x7fd684011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4e1810fb1f947 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:26.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.578+0000 7fd675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:26.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:26 smithi134 ceph-mon[35414]: pgmap v420: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-09-19T07:51:26.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.581+0000 7fd675ffb640 1 -- 172.21.15.103:0/2082421542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68400f6d0 con 0x55945dd51760 2024-09-19T07:51:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.581+0000 7fd675ffb640 1 -- 172.21.15.103:0/2082421542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fd68400f870 con 0x55945dd51760 2024-09-19T07:51:26.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.581+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68001d900 con 0x55945dd51760 2024-09-19T07:51:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.582+0000 7fd675ffb640 1 -- 172.21.15.103:0/2082421542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd68400fa70 con 0x55945dd51760 2024-09-19T07:51:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.582+0000 7fd6971b3740 10 client.18144 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:26.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.582+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55945dd6b480 con 0x55945dd51760 2024-09-19T07:51:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd675ffb640 1 -- 172.21.15.103:0/2082421542 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68400fc10 con 0x55945dd51760 2024-09-19T07:51:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd675ffb640 1 -- 172.21.15.103:0/2082421542 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fd6840275c0 con 0x55945dd51760 2024-09-19T07:51:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd6971b3740 10 client.18144 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd6971b3740 10 client.18144 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:51:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd6971b3740 20 client.18144 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:26.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd6971b3740 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945dd6c890 0x55945dd6eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd6971b3740 4 client.18144 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:26.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.583+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd68001d900 con 0x55945dd6c890 2024-09-19T07:51:26.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.584+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945dd6c890 0x55945dd6eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.584+0000 7fd675ffb640 10 client.18144 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:26.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.584+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945dd6c890 0x55945dd6eca0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a5875e1e21ec8f server_cookie=36884651ff6104f0 in_seq=0 out_seq=0 2024-09-19T07:51:26.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd675ffb640 1 -- 172.21.15.103:0/2082421542 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd68001d900 con 0x55945dd6c890 2024-09-19T07:51:26.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd675ffb640 10 client.18144 handle_command_reply: tid=0 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 client.18144 shutdown 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 msgr2=0x55945dd59830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd59830 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd684011570 tx=0x7fd684011a30 comp rx=0 tx=0).stop 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 shutdown_connections 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945dd6c890 0x55945dd6eca0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd5a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd52080 0x55945dd59d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 --2- 172.21.15.103:0/2082421542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd59830 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 >> 172.21.15.103:0/2082421542 conn(0x55945dd4dcf0 msgr2=0x55945dd539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.585+0000 7fd675ffb640 0 client.18144 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.586+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 shutdown_connections 2024-09-19T07:51:26.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.586+0000 7fd6971b3740 1 -- 172.21.15.103:0/2082421542 wait complete. 2024-09-19T07:51:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.586+0000 7fd6971b3740 20 client.18144 trim_cache size 0 max 16384 2024-09-19T07:51:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.613+0000 7fd6971b3740 1 Processor -- start 2024-09-19T07:51:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.613+0000 7fd6971b3740 1 -- start start 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd51760 0x55945dd5b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd52080 0x55945dd74570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd6971b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd690126ad0 con 0x55945dd52080 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd51760 0x55945dd5b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd51760 0x55945dd5b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39106/0 (socket says 172.21.15.103:39106) 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 -- 172.21.15.103:0/4127403350 learned_addr learned my addr 172.21.15.103:0/4127403350 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd6971b3740 1 -- 172.21.15.103:0/4127403350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd690128640 con 0x55945dd51760 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd6971b3740 1 -- 172.21.15.103:0/4127403350 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901271a0 con 0x55945dd5c940 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68dd9b640 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd52080 0x55945dd74570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd74ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 -- 172.21.15.103:0/4127403350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 msgr2=0x55945dd74ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd74ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 -- 172.21.15.103:0/4127403350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd52080 msgr2=0x55945dd74570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd52080 0x55945dd74570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.614+0000 7fd68e59c640 1 -- 172.21.15.103:0/4127403350 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945dd49260 con 0x55945dd51760 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.615+0000 7fd68e59c640 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd51760 0x55945dd5b7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd684027990 tx=0x7fd684004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c813ddcc04297f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.615+0000 7fd677fff640 1 -- 172.21.15.103:0/4127403350 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68400f6d0 con 0x55945dd51760 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.615+0000 7fd677fff640 1 -- 172.21.15.103:0/4127403350 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68400f870 con 0x55945dd51760 2024-09-19T07:51:26.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.615+0000 7fd677fff640 1 -- 172.21.15.103:0/4127403350 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840148d0 con 0x55945dd51760 2024-09-19T07:51:26.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 -- 172.21.15.103:0/4127403350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd51760 msgr2=0x55945dd5b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd51760 0x55945dd5b7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd684027990 tx=0x7fd684004770 comp rx=0 tx=0).stop 2024-09-19T07:51:26.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 -- 172.21.15.103:0/4127403350 shutdown_connections 2024-09-19T07:51:26.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd5c940 0x55945dd74ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd52080 0x55945dd74570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 --2- 172.21.15.103:0/4127403350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd51760 0x55945dd5b7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 -- 172.21.15.103:0/4127403350 >> 172.21.15.103:0/4127403350 conn(0x55945dd4dcf0 msgr2=0x55945dd5b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 -- 172.21.15.103:0/4127403350 shutdown_connections 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 -- 172.21.15.103:0/4127403350 wait complete. 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 -- start start 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.616+0000 7fd6971b3740 1 Processor -- start 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.617+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd60970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.617+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd60eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.617+0000 7fd6971b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd6971b3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd690127f40 con 0x55945dd51760 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd60970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd60eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:39112/0 (socket says 172.21.15.103:39112) 2024-09-19T07:51:26.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3804330991 learned_addr learned my addr 172.21.15.103:0/3804330991 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd690127870 con 0x55945dd5c940 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd690124ad0 con 0x55945dd52080 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd6971b3740 10 client.0 _ll_register_callbacks cb 0x7fd684009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3804330991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 msgr2=0x55945dd60eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd60eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3804330991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 msgr2=0x55945dd60970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd60970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.618+0000 7fd68ed9d640 1 -- 172.21.15.103:0/3804330991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945ddd6830 con 0x55945dd5c940 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.619+0000 7fd68ed9d640 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd613f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd680009ad0 tx=0x7fd680009210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d438d7146e81139a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.619+0000 7fd675ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.619+0000 7fd675ffb640 1 -- 172.21.15.103:0/3804330991 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680022030 con 0x55945dd5c940 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.619+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68400f870 con 0x55945dd5c940 2024-09-19T07:51:26.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.619+0000 7fd675ffb640 1 -- 172.21.15.103:0/3804330991 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fd680003aa0 con 0x55945dd5c940 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd675ffb640 1 -- 172.21.15.103:0/3804330991 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd680039050 con 0x55945dd5c940 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 10 client.27470 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=51}) -- 0x55945ddd7a00 con 0x55945dd5c940 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd675ffb640 1 -- 172.21.15.103:0/3804330991 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fd680003cc0 con 0x55945dd5c940 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 10 client.27470 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 10 client.27470 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 20 client.27470 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945ddd7d50 0x55945ddda140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 4 client.27470 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T07:51:26.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.620+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x55945dcc49d0 con 0x55945ddd7d50 2024-09-19T07:51:26.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.621+0000 7fd68e59c640 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945ddd7d50 0x55945ddda140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:26.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.622+0000 7fd675ffb640 10 client.27470 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:26.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.622+0000 7fd68e59c640 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945ddd7d50 0x55945ddda140 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129142c23326e6c6 server_cookie=ee2d217a7131235b in_seq=0 out_seq=0 2024-09-19T07:51:26.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:26 smithi174 ceph-mon[35370]: pgmap v420: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-09-19T07:51:26.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd675ffb640 1 -- 172.21.15.103:0/3804330991 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55945dcc49d0 con 0x55945ddd7d50 2024-09-19T07:51:26.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd675ffb640 10 client.27470 handle_command_reply: tid=0 2024-09-19T07:51:26.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 client.27470 shutdown 2024-09-19T07:51:26.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 msgr2=0x55945dd613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd613f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd680009ad0 tx=0x7fd680009210 comp rx=0 tx=0).stop 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 shutdown_connections 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55945ddd7d50 0x55945ddda140 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55945dd5c940 0x55945dd613f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55945dd52080 0x55945dd60eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 --2- 172.21.15.103:0/3804330991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55945dd51760 0x55945dd60970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd675ffb640 0 client.27470 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 >> 172.21.15.103:0/3804330991 conn(0x55945dd4dcf0 msgr2=0x55945dd5b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 shutdown_connections 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 1 -- 172.21.15.103:0/3804330991 wait complete. 2024-09-19T07:51:26.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.801+0000 7fd6971b3740 20 client.27470 trim_cache size 0 max 16384 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 51, 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 46, 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "58797124": { 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 46, 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:51:26.804 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:51:26.805 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:51:26.806 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:26.806 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:26.806 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:26.806 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:26.806 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:26.806 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:51:26.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6780e7500 msgr2=0x7fd6780e99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6780e7500 0x7fd6780e99c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd68000e660 tx=0x7fd68001b000 comp rx=0 tx=0).stop 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690074eb0 msgr2=0x7fd69010c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690074eb0 0x7fd69010c860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd6880308b0 tx=0x7fd6880308e0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 shutdown_connections 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6780e7500 0x7fd6780e99c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd690112b90 0x7fd690111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd690074eb0 0x7fd69010c860 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 --2- 172.21.15.103:0/3189652639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd690074590 0x7fd69010c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 >> 172.21.15.103:0/3189652639 conn(0x7fd6900fd8e0 msgr2=0x7fd6900ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 shutdown_connections 2024-09-19T07:51:26.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:26.802+0000 7fd694dda640 1 -- 172.21.15.103:0/3189652639 wait complete. 2024-09-19T07:51:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:28 smithi134 ceph-mon[35414]: pgmap v421: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-19T07:51:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:28 smithi174 ceph-mon[35370]: pgmap v421: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-19T07:51:28.672 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:51:28.673 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:51:28.701 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:28 smithi103 ceph-mon[31201]: pgmap v421: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-19T07:51:28.853 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '58797124', quiesce took 7.7 seconds. Will release after: 14.5 2024-09-19T07:51:28.937 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:29.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.403+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1336927623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 msgr2=0x7fc1cc10d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.403+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1336927623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10d9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009ca0 tx=0x7fc1c402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.403+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1336927623 shutdown_connections 2024-09-19T07:51:29.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.403+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1336927623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10d9c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.403+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1336927623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1cc101fc0 0x7fc1cc105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.403+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1336927623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1cc101610 0x7fc1cc1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.403+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1336927623 >> 172.21.15.103:0/1336927623 conn(0x7fc1cc0fc5c0 msgr2=0x7fc1cc0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1336927623 shutdown_connections 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1336927623 wait complete. 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 Processor -- start 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 -- start start 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1cc101610 0x7fc1cc10e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1cc101fc0 0x7fc1cc10e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc111fe0 con 0x7fc1cc105e40 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc111e60 con 0x7fc1cc101fc0 2024-09-19T07:51:29.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d2ee0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc112160 con 0x7fc1cc101610 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d147d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d147d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48292/0 (socket says 172.21.15.103:48292) 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.404+0000 7fc1d147d640 1 -- 172.21.15.103:0/1562136216 learned_addr learned my addr 172.21.15.103:0/1562136216 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1cc101fc0 0x7fc1cc10e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1cc101610 0x7fc1cc10e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d147d640 1 -- 172.21.15.103:0/1562136216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1cc101610 msgr2=0x7fc1cc10e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d147d640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1cc101610 0x7fc1cc10e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d147d640 1 -- 172.21.15.103:0/1562136216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1cc101fc0 msgr2=0x7fc1cc10e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d147d640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1cc101fc0 0x7fc1cc10e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d147d640 1 -- 172.21.15.103:0/1562136216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc1a6860 con 0x7fc1cc105e40 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1cc101610 0x7fc1cc10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:29.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d147d640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10edd0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4031a60 tx=0x7fc1c4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1c9ffb640 1 -- 172.21.15.103:0/1562136216 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4003c30 con 0x7fc1cc105e40 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc1a6af0 con 0x7fc1cc105e40 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1c9ffb640 1 -- 172.21.15.103:0/1562136216 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c4003dd0 con 0x7fc1cc105e40 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1c9ffb640 1 -- 172.21.15.103:0/1562136216 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40047e0 con 0x7fc1cc105e40 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.405+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc1a7030 con 0x7fc1cc105e40 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1c9ffb640 1 -- 172.21.15.103:0/1562136216 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc1c402f790 con 0x7fc1cc105e40 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 Processor -- start 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 -- start start 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232ef480 0x55e0232f9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232f9d40 0x55e0232fc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e023254280 con 0x55e0232eeb60 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e0231fbb10 con 0x55e0232f9d40 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e023250240 con 0x55e0232ef480 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48298/0 (socket says 172.21.15.103:48298) 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.408+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3345707421 learned_addr learned my addr 172.21.15.103:0/3345707421 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1c9ffb640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1b40a33b0 0x7fc1b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232f9d40 0x55e0232fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232ef480 0x55e0232f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3345707421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232ef480 msgr2=0x55e0232f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232ef480 0x55e0232f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3345707421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232f9d40 msgr2=0x55e0232fc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:29.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232f9d40 0x55e0232fc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3345707421 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0232081f0 con 0x55e0232eeb60 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1d147d640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1b40a33b0 0x7fc1b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1c9ffb640 1 -- 172.21.15.103:0/1562136216 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc1c40f10b0 con 0x7fc1cc105e40 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232eef40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006e10 tx=0x7fc1c000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8eb410336480e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1b2ffd640 1 -- 172.21.15.103:0/3345707421 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c001a070 con 0x55e0232eeb60 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1b2ffd640 1 -- 172.21.15.103:0/3345707421 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c0010040 con 0x55e0232eeb60 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1d147d640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1b40a33b0 0x7fc1b40a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40ee9b0 tx=0x7fc1c403d040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.409+0000 7fc1b2ffd640 1 -- 172.21.15.103:0/3345707421 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0013570 con 0x55e0232eeb60 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3345707421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 msgr2=0x55e0232eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232eef40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006e10 tx=0x7fc1c000b400 comp rx=0 tx=0).stop 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3345707421 shutdown_connections 2024-09-19T07:51:29.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232f9d40 0x55e0232fc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232ef480 0x55e0232f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3345707421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232eef40 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3345707421 >> 172.21.15.103:0/3345707421 conn(0x55e0232eb0f0 msgr2=0x55e0232eb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3345707421 shutdown_connections 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3345707421 wait complete. 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 -- start start 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.410+0000 7fc1d52b9740 1 Processor -- start 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e0232f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e0232fc910 con 0x55e0232eeb60 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e02320ef10 con 0x55e0232ef480 2024-09-19T07:51:29.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e0232fca90 con 0x55e0232f9d40 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d52b9740 10 client.0 _ll_register_callbacks cb 0x55e0231d7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48310/0 (socket says 172.21.15.103:48310) 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 -- 172.21.15.103:0/1791407887 learned_addr learned my addr 172.21.15.103:0/1791407887 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e0232f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 -- 172.21.15.103:0/1791407887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 msgr2=0x55e0232f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e0232f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 -- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 msgr2=0x55e0232f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.411+0000 7fc1cbfff640 1 -- 172.21.15.103:0/1791407887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e022fc56a0 con 0x55e0232eeb60 2024-09-19T07:51:29.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232f6c40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000c010 tx=0x7fc1c0015f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c325d4a2c4c1f722 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1abfff640 1 -- 172.21.15.103:0/1791407887 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c001a040 con 0x55e0232eeb60 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1abfff640 1 -- 172.21.15.103:0/1791407887 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc1c0023420 con 0x55e0232eeb60 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c0010040 con 0x55e0232eeb60 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1abfff640 1 -- 172.21.15.103:0/1791407887 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc1c0031050 con 0x55e0232eeb60 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 10 client.18162 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e023308540 con 0x55e0232eeb60 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1abfff640 1 -- 172.21.15.103:0/1791407887 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1c0010040 con 0x55e0232eeb60 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1abfff640 1 -- 172.21.15.103:0/1791407887 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fc1c0020070 con 0x55e0232eeb60 2024-09-19T07:51:29.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 10 client.18162 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 20 client.18162 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023309940 0x55e02330bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 4 client.18162 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.412+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0232617f0 con 0x55e023309940 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.413+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023309940 0x55e02330bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.413+0000 7fc1abfff640 10 client.18162 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.413+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023309940 0x55e02330bd30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8b839bb0b0602b server_cookie=5d3b04c4a681962d in_seq=0 out_seq=0 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1abfff640 1 -- 172.21.15.103:0/1791407887 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0232617f0 con 0x55e023309940 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1abfff640 10 client.18162 handle_command_reply: tid=0 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 client.18162 shutdown 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 msgr2=0x55e0232f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232f6c40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000c010 tx=0x7fc1c0015f20 comp rx=0 tx=0).stop 2024-09-19T07:51:29.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 shutdown_connections 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023309940 0x55e02330bd30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e0232f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/1791407887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e0232f6c40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1abfff640 0 client.18162 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 >> 172.21.15.103:0/1791407887 conn(0x55e0232eb0f0 msgr2=0x55e0232f0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 shutdown_connections 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 1 -- 172.21.15.103:0/1791407887 wait complete. 2024-09-19T07:51:29.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.414+0000 7fc1d52b9740 20 client.18162 trim_cache size 0 max 16384 2024-09-19T07:51:29.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 Processor -- start 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 -- start start 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232eeb60 0x55e0232f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232ef480 0x55e0233114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e023311a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1afa30 con 0x55e0232ef480 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b15a0 con 0x55e0232eeb60 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b0100 con 0x55e0232f9d40 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d0c7c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232ef480 0x55e0233114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d0c7c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232ef480 0x55e0233114c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48324/0 (socket says 172.21.15.103:48324) 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1d0c7c640 1 -- 172.21.15.103:0/2899363679 learned_addr learned my addr 172.21.15.103:0/2899363679 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.431+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232eeb60 0x55e0232f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1d0c7c640 1 -- 172.21.15.103:0/2899363679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 msgr2=0x55e023311a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:29.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e023311a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1d0c7c640 1 -- 172.21.15.103:0/2899363679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232eeb60 msgr2=0x55e0232f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232eeb60 0x55e0232f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1d0c7c640 1 -- 172.21.15.103:0/2899363679 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0232f5bf0 con 0x55e0232ef480 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232ef480 0x55e0233114c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55e0232f64d0 tx=0x7fc1bc005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3037040096f41e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1b27fc640 1 -- 172.21.15.103:0/2899363679 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc004250 con 0x55e0232ef480 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1b27fc640 1 -- 172.21.15.103:0/2899363679 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc020070 con 0x55e0232ef480 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.432+0000 7fc1b27fc640 1 -- 172.21.15.103:0/2899363679 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc01b690 con 0x55e0232ef480 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- 172.21.15.103:0/2899363679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232ef480 msgr2=0x55e0233114c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232ef480 0x55e0233114c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55e0232f64d0 tx=0x7fc1bc005f40 comp rx=0 tx=0).stop 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- 172.21.15.103:0/2899363679 shutdown_connections 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e023311a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232ef480 0x55e0233114c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/2899363679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232eeb60 0x55e0232f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- 172.21.15.103:0/2899363679 >> 172.21.15.103:0/2899363679 conn(0x55e0232eb0f0 msgr2=0x55e0232f8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- 172.21.15.103:0/2899363679 shutdown_connections 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- 172.21.15.103:0/2899363679 wait complete. 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- start start 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 Processor -- start 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e023302400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e0232fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b0ea0 con 0x55e0232eeb60 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b07d0 con 0x55e0232ef480 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.433+0000 7fc1d52b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1ada30 con 0x55e0232f9d40 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1d52b9740 10 client.0 _ll_register_callbacks cb 0x7fc1c0018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:29.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e023302400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e023302400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48336/0 (socket says 172.21.15.103:48336) 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3836542506 learned_addr learned my addr 172.21.15.103:0/3836542506 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3836542506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 msgr2=0x55e0232fdc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e0232fdc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 msgr2=0x55e0232fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 -- 172.21.15.103:0/3836542506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0232fe470 con 0x55e0232eeb60 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:51:29.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1cbfff640 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e023302400 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0015ef0 tx=0x7fc1c0021620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7575a5fea7f4f7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1abfff640 1 -- 172.21.15.103:0/3836542506 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0021e50 con 0x55e0232eeb60 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1abfff640 1 -- 172.21.15.103:0/3836542506 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fc1c0015690 con 0x55e0232eeb60 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1bc020070 con 0x55e0232eeb60 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1abfff640 1 -- 172.21.15.103:0/3836542506 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fc1c000b950 con 0x55e0232eeb60 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1d52b9740 10 client.18168 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.434+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e0233744a0 con 0x55e0232eeb60 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1abfff640 1 -- 172.21.15.103:0/3836542506 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc020070 con 0x55e0232eeb60 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1abfff640 1 -- 172.21.15.103:0/3836542506 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fc1c0020040 con 0x55e0232eeb60 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1d52b9740 10 client.18168 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1d52b9740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1d52b9740 20 client.18168 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:29.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023374c20 0x55e023377040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1d52b9740 4 client.18168 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e02325bd00 con 0x55e023374c20 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.435+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023374c20 0x55e023377040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.436+0000 7fc1abfff640 10 client.18168 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.436+0000 7fc1d0c7c640 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023374c20 0x55e023377040 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d45a0763754cbe server_cookie=beefcc8566fc4022 in_seq=0 out_seq=0 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1abfff640 1 -- 172.21.15.103:0/3836542506 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e02325bd00 con 0x55e023374c20 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1abfff640 10 client.18168 handle_command_reply: tid=0 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 client.18168 shutdown 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 msgr2=0x55e023302400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e023302400 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0015ef0 tx=0x7fc1c0021620 comp rx=0 tx=0).stop 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 shutdown_connections 2024-09-19T07:51:29.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e023374c20 0x55e023377040 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e0232f9d40 0x55e0232fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0232ef480 0x55e0232fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 --2- 172.21.15.103:0/3836542506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e0232eeb60 0x55e023302400 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 >> 172.21.15.103:0/3836542506 conn(0x55e0232eb0f0 msgr2=0x55e0232f8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1abfff640 0 client.18168 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 shutdown_connections 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 1 -- 172.21.15.103:0/3836542506 wait complete. 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.437+0000 7fc1d52b9740 20 client.18168 trim_cache size 0 max 16384 2024-09-19T07:51:29.450 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:51:29.451 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:51:29.451 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "29c1164e-2208-4fb8-89d1-2a09cb2ea61c", 2024-09-19T07:51:29.451 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:51:29.451 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:51:29.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1b40a33b0 msgr2=0x7fc1b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1b40a33b0 0x7fc1b40a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40ee9b0 tx=0x7fc1c403d040 comp rx=0 tx=0).stop 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 msgr2=0x7fc1cc10edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10edd0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4031a60 tx=0x7fc1c4031a90 comp rx=0 tx=0).stop 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 shutdown_connections 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc1b40a33b0 0x7fc1b40a5870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1cc105e40 0x7fc1cc10edd0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1cc101fc0 0x7fc1cc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 --2- 172.21.15.103:0/1562136216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1cc101610 0x7fc1cc10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 >> 172.21.15.103:0/1562136216 conn(0x7fc1cc0fc5c0 msgr2=0x7fc1cc104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:29.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 shutdown_connections 2024-09-19T07:51:29.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:29.438+0000 7fc1d2ee0640 1 -- 172.21.15.103:0/1562136216 wait complete. 2024-09-19T07:51:30.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:51:30.437 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:30.469 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 775.0 s, 401.2 tps, lat 58.354 ms stddev 66.588 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: pgmap v422: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:30 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: pgmap v422: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-19T07:51:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:30.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:30 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: pgmap v422: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-19T07:51:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:51:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:30.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:30 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:31.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 -- 172.21.15.103:0/846455467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98100a80 msgr2=0x7fcf98100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 --2- 172.21.15.103:0/846455467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98100a80 0x7fcf98100ee0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009930 tx=0x7fcf8802a420 comp rx=0 tx=0).stop 2024-09-19T07:51:31.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 -- 172.21.15.103:0/846455467 shutdown_connections 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 --2- 172.21.15.103:0/846455467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf98101420 0x7fcf9810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 --2- 172.21.15.103:0/846455467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98100a80 0x7fcf98100ee0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 --2- 172.21.15.103:0/846455467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf98106a80 0x7fcf98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 -- 172.21.15.103:0/846455467 >> 172.21.15.103:0/846455467 conn(0x7fcf980fc7a0 msgr2=0x7fcf980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 -- 172.21.15.103:0/846455467 shutdown_connections 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.014+0000 7fcfa047a640 1 -- 172.21.15.103:0/846455467 wait complete. 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 Processor -- start 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 -- start start 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf98100a80 0x7fcf981991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98101420 0x7fcf98199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf98106a80 0x7fcf981a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98110e80 con 0x7fcf98101420 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98110d00 con 0x7fcf98106a80 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcfa047a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98111000 con 0x7fcf98100a80 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcf9da15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98101420 0x7fcf98199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcf9da15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98101420 0x7fcf98199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48356/0 (socket says 172.21.15.103:48356) 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcf9da15640 1 -- 172.21.15.103:0/3434827831 learned_addr learned my addr 172.21.15.103:0/3434827831 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcf9e216640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf98100a80 0x7fcf981991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.015+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf98106a80 0x7fcf981a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf9da15640 1 -- 172.21.15.103:0/3434827831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf98100a80 msgr2=0x7fcf981991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf9da15640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf98100a80 0x7fcf981991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf9da15640 1 -- 172.21.15.103:0/3434827831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf98106a80 msgr2=0x7fcf981a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf9da15640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf98106a80 0x7fcf981a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf9da15640 1 -- 172.21.15.103:0/3434827831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf981a47b0 con 0x7fcf98101420 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf9da15640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98101420 0x7fcf98199730 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009230 tx=0x7fcf88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf8f7fe640 1 -- 172.21.15.103:0/3434827831 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8803f030 con 0x7fcf98101420 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf8f7fe640 1 -- 172.21.15.103:0/3434827831 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf88003ad0 con 0x7fcf98101420 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981a4980 con 0x7fcf98101420 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcf8f7fe640 1 -- 172.21.15.103:0/3434827831 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88035070 con 0x7fcf98101420 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.016+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981a4e30 con 0x7fcf98101420 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.018+0000 7fcf8f7fe640 1 -- 172.21.15.103:0/3434827831 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcf88042030 con 0x7fcf98101420 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.018+0000 7fcfa2853740 1 Processor -- start 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcfa2853740 1 -- start start 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba19140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba1ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcfa2853740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559beb9742e0 con 0x559beba19680 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcfa2853740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559beb91bce0 con 0x559beba0e4a0 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcfa2853740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559beb9702a0 con 0x559beba0edc0 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba1ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba1ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48358/0 (socket says 172.21.15.103:48358) 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 -- 172.21.15.103:0/324789766 learned_addr learned my addr 172.21.15.103:0/324789766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf8f7fe640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf840a33b0 0x7fcf840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9e216640 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba19140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf840a33b0 0x7fcf840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf8f7fe640 1 -- 172.21.15.103:0/3434827831 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fcf880f02d0 con 0x7fcf98101420 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 -- 172.21.15.103:0/324789766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 msgr2=0x559beba19140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba19140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 -- 172.21.15.103:0/324789766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 msgr2=0x559beba0e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba0e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.019+0000 7fcf9da15640 1 -- 172.21.15.103:0/324789766 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559beb900440 con 0x559beba19680 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.020+0000 7fcf9da15640 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba1ba70 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcf880b8f70 tx=0x7fcf88005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752972dd13c40b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.020+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf840a33b0 0x7fcf840a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005c40 tx=0x7fcf90005bd0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.020+0000 7fcf8cff9640 1 -- 172.21.15.103:0/324789766 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88049070 con 0x559beba19680 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.020+0000 7fcf8cff9640 1 -- 172.21.15.103:0/324789766 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf880bc5a0 con 0x559beba19680 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.020+0000 7fcf8cff9640 1 -- 172.21.15.103:0/324789766 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf880edc90 con 0x559beba19680 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.020+0000 7fcfa2853740 1 -- 172.21.15.103:0/324789766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 msgr2=0x559beba1ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.020+0000 7fcfa2853740 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba1ba70 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcf880b8f70 tx=0x7fcf88005e50 comp rx=0 tx=0).stop 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- 172.21.15.103:0/324789766 shutdown_connections 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba1ba70 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba19140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 --2- 172.21.15.103:0/324789766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba0e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- 172.21.15.103:0/324789766 >> 172.21.15.103:0/324789766 conn(0x559beba0aa30 msgr2=0x559beba0ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- 172.21.15.103:0/324789766 shutdown_connections 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- 172.21.15.103:0/324789766 wait complete. 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- start start 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 Processor -- start 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba16740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559beba1c010 con 0x559beba19680 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559beb92f050 con 0x559beba0e4a0 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559beba1c190 con 0x559beba0edc0 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcfa2853740 10 client.0 _ll_register_callbacks cb 0x559beb96f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:31.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcf9ea17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba16740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcf9ea17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba16740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48370/0 (socket says 172.21.15.103:48370) 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcf9e216640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcf9ea17640 1 -- 172.21.15.103:0/2685315597 learned_addr learned my addr 172.21.15.103:0/2685315597 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.021+0000 7fcf9da15640 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba16200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf9ea17640 1 -- 172.21.15.103:0/2685315597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 msgr2=0x559beba16200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba16200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf9ea17640 1 -- 172.21.15.103:0/2685315597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 msgr2=0x559beba18f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba18f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf9ea17640 1 -- 172.21.15.103:0/2685315597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559beb6e53c0 con 0x559beba19680 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba16740 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000d930 tx=0x7fcf90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2060b090d8f4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf81ffb640 1 -- 172.21.15.103:0/2685315597 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90025070 con 0x559beba19680 2024-09-19T07:51:31.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf880bc5a0 con 0x559beba19680 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf81ffb640 1 -- 172.21.15.103:0/2685315597 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fcf90010b90 con 0x559beba19680 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf81ffb640 1 -- 172.21.15.103:0/2685315597 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fcf90031050 con 0x559beba19680 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcfa2853740 10 client.18180 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559beba27940 con 0x559beba19680 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.022+0000 7fcf81ffb640 1 -- 172.21.15.103:0/2685315597 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf880bc5a0 con 0x559beba19680 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcf81ffb640 1 -- 172.21.15.103:0/2685315597 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fcf900203e0 con 0x559beba19680 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcfa2853740 10 client.18180 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcfa2853740 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcfa2853740 20 client.18180 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcfa2853740 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba28da0 0x559beba2b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcfa2853740 4 client.18180 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559beb981850 con 0x559beba28da0 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcf9e216640 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba28da0 0x559beba2b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcf81ffb640 10 client.18180 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.023+0000 7fcf9e216640 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba28da0 0x559beba2b1d0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b21730075afb07 server_cookie=d4118a1a46c126e in_seq=0 out_seq=0 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcf81ffb640 1 -- 172.21.15.103:0/2685315597 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559beb981850 con 0x559beba28da0 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcf81ffb640 10 client.18180 handle_command_reply: tid=0 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 client.18180 shutdown 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 msgr2=0x559beba16740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba16740 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000d930 tx=0x7fcf90004a40 comp rx=0 tx=0).stop 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 shutdown_connections 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba28da0 0x559beba2b1d0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba16740 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba16200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcf81ffb640 0 client.18180 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 --2- 172.21.15.103:0/2685315597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba18f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 >> 172.21.15.103:0/2685315597 conn(0x559beba0aa30 msgr2=0x559beba10710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 shutdown_connections 2024-09-19T07:51:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 1 -- 172.21.15.103:0/2685315597 wait complete. 2024-09-19T07:51:31.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.024+0000 7fcfa2853740 20 client.18180 trim_cache size 0 max 16384 2024-09-19T07:51:31.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 Processor -- start 2024-09-19T07:51:31.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 -- start start 2024-09-19T07:51:31.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0e4a0 0x559beba251e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba0edc0 0x559beba14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba19680 0x559beba30aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ad880 con 0x559beba0edc0 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981af3f0 con 0x559beba19680 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.041+0000 7fcfa2853740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981adf50 con 0x559beba0e4a0 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba0edc0 0x559beba14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba0edc0 0x559beba14ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48372/0 (socket says 172.21.15.103:48372) 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 -- 172.21.15.103:0/3387216830 learned_addr learned my addr 172.21.15.103:0/3387216830 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9e216640 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0e4a0 0x559beba251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba19680 0x559beba30aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 -- 172.21.15.103:0/3387216830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0e4a0 msgr2=0x559beba251e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0e4a0 0x559beba251e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 -- 172.21.15.103:0/3387216830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba19680 msgr2=0x559beba30aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba19680 0x559beba30aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 -- 172.21.15.103:0/3387216830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559beba151b0 con 0x559beba0edc0 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf9da15640 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba0edc0 0x559beba14ad0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcf880b8280 tx=0x7fcf880f3b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1467f2cf1a335b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf83fff640 1 -- 172.21.15.103:0/3387216830 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88049040 con 0x559beba0edc0 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf83fff640 1 -- 172.21.15.103:0/3387216830 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf880c0950 con 0x559beba0edc0 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.042+0000 7fcf83fff640 1 -- 172.21.15.103:0/3387216830 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88003c80 con 0x559beba0edc0 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 -- 172.21.15.103:0/3387216830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba0edc0 msgr2=0x559beba14ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba0edc0 0x559beba14ad0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcf880b8280 tx=0x7fcf880f3b50 comp rx=0 tx=0).stop 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 -- 172.21.15.103:0/3387216830 shutdown_connections 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba19680 0x559beba30aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba0edc0 0x559beba14ad0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3387216830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0e4a0 0x559beba251e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 -- 172.21.15.103:0/3387216830 >> 172.21.15.103:0/3387216830 conn(0x559beba0aa30 msgr2=0x559beba118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 -- 172.21.15.103:0/3387216830 shutdown_connections 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 -- 172.21.15.103:0/3387216830 wait complete. 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 -- start start 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.043+0000 7fcfa2853740 1 Processor -- start 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba8ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba8bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcfa2853740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba96ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcfa2853740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981aecf0 con 0x559beba19680 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcfa2853740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ae620 con 0x559beba0e4a0 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcfa2853740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ab880 con 0x559beba0edc0 2024-09-19T07:51:31.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcfa2853740 10 client.0 _ll_register_callbacks cb 0x7fcf90012440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba96ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba96ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48388/0 (socket says 172.21.15.103:48388) 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 -- 172.21.15.103:0/3480074286 learned_addr learned my addr 172.21.15.103:0/3480074286 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9e216640 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba8ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9da15640 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba8bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 -- 172.21.15.103:0/3480074286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 msgr2=0x559beba8bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba8bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 -- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 msgr2=0x559beba8ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba8ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 -- 172.21.15.103:0/3480074286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559beba8c860 con 0x559beba19680 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.044+0000 7fcf9ea17640 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba96ab0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x559beba0b7d0 tx=0x7fcf900107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960348e58b42e842 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:31.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcf81ffb640 1 -- 172.21.15.103:0/3480074286 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9003a060 con 0x559beba19680 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcf81ffb640 1 -- 172.21.15.103:0/3480074286 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fcf90010b90 con 0x559beba19680 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf880c0950 con 0x559beba19680 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcf81ffb640 1 -- 172.21.15.103:0/3480074286 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fcf90025040 con 0x559beba19680 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 10 client.18186 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559beba98100 con 0x559beba19680 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcf81ffb640 1 -- 172.21.15.103:0/3480074286 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf880c0950 con 0x559beba19680 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcf81ffb640 1 -- 172.21.15.103:0/3480074286 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fcf90005040 con 0x559beba19680 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 10 client.18186 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 20 client.18186 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:31.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba987f0 0x559beba9ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 4 client.18186 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.045+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559beb97bd60 con 0x559beba987f0 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcf9e216640 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba987f0 0x559beba9ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcf81ffb640 10 client.18186 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcf9e216640 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba987f0 0x559beba9ac10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec913760b170e0d server_cookie=de8c90bbc2b57f1d in_seq=0 out_seq=0 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcf81ffb640 1 -- 172.21.15.103:0/3480074286 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559beb97bd60 con 0x559beba987f0 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcf81ffb640 10 client.18186 handle_command_reply: tid=0 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcfa2853740 1 client.18186 shutdown 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 msgr2=0x559beba96ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba96ab0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x559beba0b7d0 tx=0x7fcf900107d0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 shutdown_connections 2024-09-19T07:51:31.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559beba987f0 0x559beba9ac10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559beba19680 0x559beba96ab0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcf81ffb640 0 client.18186 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.046+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559beba0edc0 0x559beba8bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa2853740 1 --2- 172.21.15.103:0/3480074286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559beba0e4a0 0x559beba8ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 >> 172.21.15.103:0/3480074286 conn(0x559beba0aa30 msgr2=0x559beba11890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 shutdown_connections 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa2853740 1 -- 172.21.15.103:0/3480074286 wait complete. 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa2853740 20 client.18186 trim_cache size 0 max 16384 2024-09-19T07:51:31.054 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: "29c1164e-2208-4fb8-89d1-2a09cb2ea61c": { 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "29c1164e-2208-4fb8-89d1-2a09cb2ea61c", 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:31.055 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:51:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf840a33b0 msgr2=0x7fcf840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa047a640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf840a33b0 0x7fcf840a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005c40 tx=0x7fcf90005bd0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98101420 msgr2=0x7fcf98199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.047+0000 7fcfa047a640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98101420 0x7fcf98199730 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009230 tx=0x7fcf88004270 comp rx=0 tx=0).stop 2024-09-19T07:51:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 shutdown_connections 2024-09-19T07:51:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf840a33b0 0x7fcf840a5870 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf98106a80 0x7fcf981a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf98101420 0x7fcf98199730 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 --2- 172.21.15.103:0/3434827831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf98100a80 0x7fcf981991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 >> 172.21.15.103:0/3434827831 conn(0x7fcf980fc7a0 msgr2=0x7fcf980fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 shutdown_connections 2024-09-19T07:51:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:31.048+0000 7fcfa047a640 1 -- 172.21.15.103:0/3434827831 wait complete. 2024-09-19T07:51:31.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:31 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:51:31.645 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:31 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:31 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:51:31.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:31 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:31.676 INFO:tasks.cephfs.filesystem:scrub status for tag:29c1164e-2208-4fb8-89d1-2a09cb2ea61c - {'path': '/', 'tag': '29c1164e-2208-4fb8-89d1-2a09cb2ea61c', 'options': 'recursive,force'} 2024-09-19T07:51:31.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:31 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:51:31.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:31 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:51:32.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:32 smithi174 ceph-mon[35370]: pgmap v423: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:32.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:32 smithi103 ceph-mon[31201]: pgmap v423: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:32.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:32 smithi134 ceph-mon[35414]: pgmap v423: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:34 smithi174 ceph-mon[35370]: pgmap v424: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:34.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:34 smithi103 ceph-mon[31201]: pgmap v424: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:34.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:34 smithi134 ceph-mon[35414]: pgmap v424: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:35.468 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 780.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:51:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:36 smithi174 ceph-mon[35370]: pgmap v425: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:36.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:36 smithi103 ceph-mon[31201]: pgmap v425: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:36.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:36 smithi134 ceph-mon[35414]: pgmap v425: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-09-19T07:51:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:38 smithi174 ceph-mon[35370]: pgmap v426: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.32k op/s 2024-09-19T07:51:38.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:38 smithi103 ceph-mon[31201]: pgmap v426: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.32k op/s 2024-09-19T07:51:38.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:38 smithi134 ceph-mon[35414]: pgmap v426: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 25 MiB/s wr, 1.32k op/s 2024-09-19T07:51:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:39 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:51:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:39 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:51:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:39 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:51:39.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:39 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:51:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:39 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:51:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:39 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:51:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:39 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:51:39.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:39 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:51:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:39 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:51:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:39 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:51:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:39 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:51:39.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:39 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:51:40.471 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 785.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:51:40.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:40 smithi174 ceph-mon[35370]: pgmap v427: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-19T07:51:40.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:40 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:51:40.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:40 smithi103 ceph-mon[31201]: pgmap v427: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-19T07:51:40.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:40 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:51:40.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:40 smithi134 ceph-mon[35414]: pgmap v427: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-19T07:51:40.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:40 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:51:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:42 smithi174 ceph-mon[35370]: pgmap v428: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-09-19T07:51:42.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:42 smithi103 ceph-mon[31201]: pgmap v428: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-09-19T07:51:42.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:42 smithi134 ceph-mon[35414]: pgmap v428: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.7 MiB/s wr, 328 op/s 2024-09-19T07:51:43.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:51:43.615 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:44.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 -- 172.21.15.103:0/2947569664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 msgr2=0x7feccc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:44.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 --2- 172.21.15.103:0/2947569664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc10bb50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009900 tx=0x7fecc002a480 comp rx=0 tx=0).stop 2024-09-19T07:51:44.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 -- 172.21.15.103:0/2947569664 shutdown_connections 2024-09-19T07:51:44.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 --2- 172.21.15.103:0/2947569664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc10c170 0x7feccc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 --2- 172.21.15.103:0/2947569664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc10bb50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 --2- 172.21.15.103:0/2947569664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102d40 0x7feccc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 -- 172.21.15.103:0/2947569664 >> 172.21.15.103:0/2947569664 conn(0x7feccc0fc7a0 msgr2=0x7feccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 -- 172.21.15.103:0/2947569664 shutdown_connections 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.088+0000 7fecd2783640 1 -- 172.21.15.103:0/2947569664 wait complete. 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 Processor -- start 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 -- start start 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102d40 0x7feccc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc10c170 0x7feccc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feccc111f70 con 0x7feccc103660 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feccc111df0 con 0x7feccc10c170 2024-09-19T07:51:44.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7fecd2783640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feccc1120f0 con 0x7feccc102d40 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7feccb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55188/0 (socket says 172.21.15.103:55188) 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.089+0000 7feccb7fe640 1 -- 172.21.15.103:0/2941598583 learned_addr learned my addr 172.21.15.103:0/2941598583 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7fecd0d20640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc10c170 0x7feccc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7feccbfff640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102d40 0x7feccc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7feccb7fe640 1 -- 172.21.15.103:0/2941598583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102d40 msgr2=0x7feccc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7feccb7fe640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102d40 0x7feccc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7feccb7fe640 1 -- 172.21.15.103:0/2941598583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc10c170 msgr2=0x7feccc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7feccb7fe640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc10c170 0x7feccc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7feccb7fe640 1 -- 172.21.15.103:0/2941598583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc1a68d0 con 0x7feccc103660 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7fecd0d20640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc10c170 0x7feccc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:51:44.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7feccb7fe640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc100ee0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006010 tx=0x7fecc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7fecc97fa640 1 -- 172.21.15.103:0/2941598583 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc003f030 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1a6b60 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7fecc97fa640 1 -- 172.21.15.103:0/2941598583 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecc0003ce0 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7fecc97fa640 1 -- 172.21.15.103:0/2941598583 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0035070 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.090+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a7040 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.091+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc103120 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.092+0000 7fecc97fa640 1 -- 172.21.15.103:0/2941598583 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fecc0042030 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.093+0000 7fecc97fa640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fecb40a33b0 0x7fecb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.093+0000 7feccbfff640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fecb40a33b0 0x7fecb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.093+0000 7fecc97fa640 1 -- 172.21.15.103:0/2941598583 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fecc00f05a0 con 0x7feccc103660 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.093+0000 7feccbfff640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fecb40a33b0 0x7fecb40a5870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005fd0 tx=0x7fecbc005850 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:44.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.096+0000 7fecc97fa640 1 -- 172.21.15.103:0/2941598583 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fecc00bd030 con 0x7feccc103660 2024-09-19T07:51:44.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.295+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feccc101ed0 con 0x7feccc103660 2024-09-19T07:51:44.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.296+0000 7fecc97fa640 1 -- 172.21.15.103:0/2941598583 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11992 (secure 0 0 0) 0x7feccc101ed0 con 0x7feccc103660 2024-09-19T07:51:44.297 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:51:44.297 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":51,"btime":"2024-09-19T07:50:10:429478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:50:09.866710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:51:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 51 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fecb40a33b0 msgr2=0x7fecb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fecb40a33b0 0x7fecb40a5870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fecbc005fd0 tx=0x7fecbc005850 comp rx=0 tx=0).stop 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 msgr2=0x7feccc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc100ee0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fecc0006010 tx=0x7fecc00042f0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 shutdown_connections 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fecb40a33b0 0x7fecb40a5870 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feccc10c170 0x7feccc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feccc103660 0x7feccc100ee0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 --2- 172.21.15.103:0/2941598583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feccc102d40 0x7feccc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:44.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 >> 172.21.15.103:0/2941598583 conn(0x7feccc0fc7a0 msgr2=0x7feccc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:44.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 shutdown_connections 2024-09-19T07:51:44.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:44.298+0000 7fecd2783640 1 -- 172.21.15.103:0/2941598583 wait complete. 2024-09-19T07:51:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:44 smithi174 ceph-mon[35370]: pgmap v429: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-19T07:51:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:44 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2941598583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:44.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:44 smithi103 ceph-mon[31201]: pgmap v429: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-19T07:51:44.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:44 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2941598583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:44.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:44 smithi134 ceph-mon[35414]: pgmap v429: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-19T07:51:44.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:44 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2941598583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:51:45.100 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 58797124 --release --await' 2024-09-19T07:51:45.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 58797124 --release --await 2024-09-19T07:51:45.355 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:51:45.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 790.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 -- 172.21.15.103:0/4275180943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 msgr2=0x7f01c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 --2- 172.21.15.103:0/4275180943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c81021c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009900 tx=0x7f01b802a410 comp rx=0 tx=0).stop 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 -- 172.21.15.103:0/4275180943 shutdown_connections 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 --2- 172.21.15.103:0/4275180943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c8106f10 0x7f01c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 --2- 172.21.15.103:0/4275180943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c8102790 0x7f01c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 --2- 172.21.15.103:0/4275180943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c81021c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 -- 172.21.15.103:0/4275180943 >> 172.21.15.103:0/4275180943 conn(0x7f01c80fd8c0 msgr2=0x7f01c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 -- 172.21.15.103:0/4275180943 shutdown_connections 2024-09-19T07:51:45.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.819+0000 7f01cc92d640 1 -- 172.21.15.103:0/4275180943 wait complete. 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 Processor -- start 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 -- start start 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c8102790 0x7f01c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c8106f10 0x7f01c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8111f70 con 0x7f01c8101de0 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8111df0 con 0x7f01c8102790 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01cc92d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81120f0 con 0x7f01c8106f10 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01c659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01c659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55218/0 (socket says 172.21.15.103:55218) 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01c659c640 1 -- 172.21.15.103:0/380372168 learned_addr learned my addr 172.21.15.103:0/380372168 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:45.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c8106f10 0x7f01c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.820+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c8102790 0x7f01c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01c659c640 1 -- 172.21.15.103:0/380372168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c8106f10 msgr2=0x7f01c8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01c659c640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c8106f10 0x7f01c8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01c659c640 1 -- 172.21.15.103:0/380372168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c8102790 msgr2=0x7f01c8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01c659c640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c8102790 0x7f01c8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01c659c640 1 -- 172.21.15.103:0/380372168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c81a68e0 con 0x7f01c8101de0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01c659c640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c80744f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005ab0 tx=0x7f01b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01b77fe640 1 -- 172.21.15.103:0/380372168 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b803f030 con 0x7f01c8101de0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c81a6b70 con 0x7f01c8101de0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01b77fe640 1 -- 172.21.15.103:0/380372168 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b8003ce0 con 0x7f01c8101de0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01b77fe640 1 -- 172.21.15.103:0/380372168 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8035070 con 0x7f01c8101de0 2024-09-19T07:51:45.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.821+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c81a70a0 con 0x7f01c8101de0 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.823+0000 7f01b77fe640 1 -- 172.21.15.103:0/380372168 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f01b8042030 con 0x7f01c8101de0 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 Processor -- start 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 -- start start 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d248440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248980 0x55e29d252d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d255630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e29d1ad620 con 0x55e29d248060 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e29d154e50 con 0x55e29d253240 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01ced06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e29d1a95e0 con 0x55e29d248980 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01c6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d248440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01b77fe640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b00a33b0 0x7f01b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01c6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d248440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55222/0 (socket says 172.21.15.103:55222) 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01c6d9d640 1 -- 172.21.15.103:0/3551694272 learned_addr learned my addr 172.21.15.103:0/3551694272 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01c659c640 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248980 0x55e29d252d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d255630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.824+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b00a33b0 0x7f01b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01c6d9d640 1 -- 172.21.15.103:0/3551694272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248980 msgr2=0x55e29d252d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248980 0x55e29d252d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01c6d9d640 1 -- 172.21.15.103:0/3551694272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 msgr2=0x55e29d255630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d255630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01c6d9d640 1 -- 172.21.15.103:0/3551694272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29d145d40 con 0x55e29d248060 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01b77fe640 1 -- 172.21.15.103:0/380372168 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f01b80f05f0 con 0x7f01c8101de0 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d248440 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f01c0013940 tx=0x7f01c0013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d54f4ac33d5736 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01b4ff9640 1 -- 172.21.15.103:0/3551694272 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0021030 con 0x55e29d248060 2024-09-19T07:51:45.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01b4ff9640 1 -- 172.21.15.103:0/3551694272 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c00043a0 con 0x55e29d248060 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01b4ff9640 1 -- 172.21.15.103:0/3551694272 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c001dad0 con 0x55e29d248060 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.825+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b00a33b0 0x7f01b00a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f01c0021000 tx=0x7f01c0020000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 -- 172.21.15.103:0/3551694272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 msgr2=0x55e29d248440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d248440 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f01c0013940 tx=0x7f01c0013e00 comp rx=0 tx=0).stop 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 -- 172.21.15.103:0/3551694272 shutdown_connections 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d255630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248980 0x55e29d252d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 --2- 172.21.15.103:0/3551694272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d248440 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 -- 172.21.15.103:0/3551694272 >> 172.21.15.103:0/3551694272 conn(0x55e29d2445f0 msgr2=0x55e29d2449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 -- 172.21.15.103:0/3551694272 shutdown_connections 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 -- 172.21.15.103:0/3551694272 wait complete. 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 -- start start 2024-09-19T07:51:45.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.826+0000 7f01ced06740 1 Processor -- start 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248060 0x55e29d250170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d2506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d250bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e29d255e10 con 0x55e29d248980 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e29d135f10 con 0x55e29d253240 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e29d255f90 con 0x55e29d248060 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 10 client.0 _ll_register_callbacks cb 0x55e29d159db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d2506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d2506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55238/0 (socket says 172.21.15.103:55238) 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 -- 172.21.15.103:0/2575223841 learned_addr learned my addr 172.21.15.103:0/2575223841 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d250bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c659c640 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248060 0x55e29d250170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 -- 172.21.15.103:0/2575223841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248060 msgr2=0x55e29d250170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248060 0x55e29d250170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 -- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 msgr2=0x55e29d250bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d250bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 -- 172.21.15.103:0/2575223841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29cebc580 con 0x55e29d248980 2024-09-19T07:51:45.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d2506b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0107a0 tx=0x7f01bc010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5769b9dedeae37b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:45.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01adffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:45.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01adffb640 1 -- 172.21.15.103:0/2575223841 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00fb90 con 0x55e29d248980 2024-09-19T07:51:45.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01adffb640 1 -- 172.21.15.103:0/2575223841 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f01bc00fd30 con 0x55e29d248980 2024-09-19T07:51:45.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.827+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c00043a0 con 0x55e29d248980 2024-09-19T07:51:45.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01adffb640 1 -- 172.21.15.103:0/2575223841 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f01bc004020 con 0x55e29d248980 2024-09-19T07:51:45.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 10 client.18204 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e29cf86960 con 0x55e29d248980 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01adffb640 1 -- 172.21.15.103:0/2575223841 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c00043a0 con 0x55e29d248980 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01adffb640 1 -- 172.21.15.103:0/2575223841 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f01bc0194a0 con 0x55e29d248980 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 10 client.18204 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 10 client.18204 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 20 client.18204 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d262f20 0x55e29d265350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 4 client.18204 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e29d1bab90 con 0x55e29d262f20 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.828+0000 7f01c659c640 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d262f20 0x55e29d265350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.829+0000 7f01adffb640 10 client.18204 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.829+0000 7f01c659c640 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d262f20 0x55e29d265350 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768e6b4ba614a9d3 server_cookie=188825a7429ff3bc in_seq=0 out_seq=0 2024-09-19T07:51:45.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.829+0000 7f01adffb640 1 -- 172.21.15.103:0/2575223841 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e29d1bab90 con 0x55e29d262f20 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.829+0000 7f01adffb640 10 client.18204 handle_command_reply: tid=0 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.829+0000 7f01ced06740 1 client.18204 shutdown 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 msgr2=0x55e29d2506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d2506b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0107a0 tx=0x7f01bc010c60 comp rx=0 tx=0).stop 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 shutdown_connections 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d262f20 0x55e29d265350 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d253240 0x55e29d250bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01adffb640 0 client.18204 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d2506b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 --2- 172.21.15.103:0/2575223841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d248060 0x55e29d250170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 >> 172.21.15.103:0/2575223841 conn(0x55e29d2445f0 msgr2=0x55e29d243750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 shutdown_connections 2024-09-19T07:51:45.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 1 -- 172.21.15.103:0/2575223841 wait complete. 2024-09-19T07:51:45.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.830+0000 7f01ced06740 20 client.18204 trim_cache size 0 max 16384 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.846+0000 7f01ced06740 1 Processor -- start 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.846+0000 7f01ced06740 1 -- start start 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.846+0000 7f01ced06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d26ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.846+0000 7f01ced06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248980 0x55e29d26b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.846+0000 7f01ced06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 0x55e29d24d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.846+0000 7f01ced06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81afb70 con 0x55e29d248060 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01ced06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81b16e0 con 0x55e29d248980 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01ced06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81b0240 con 0x55e29d253240 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d26ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d26ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55254/0 (socket says 172.21.15.103:55254) 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 -- 172.21.15.103:0/3370484365 learned_addr learned my addr 172.21.15.103:0/3370484365 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 -- 172.21.15.103:0/3370484365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 msgr2=0x55e29d24d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 --2- 172.21.15.103:0/3370484365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 0x55e29d24d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 -- 172.21.15.103:0/3370484365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248980 msgr2=0x55e29d26b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 --2- 172.21.15.103:0/3370484365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248980 0x55e29d26b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 -- 172.21.15.103:0/3370484365 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29d245280 con 0x55e29d248060 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01c659c640 1 --2- 172.21.15.103:0/3370484365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d26ac10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f01b80b8d20 tx=0x7f01b80b8bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a712f198fd413d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01affff640 1 -- 172.21.15.103:0/3370484365 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8049040 con 0x55e29d248060 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01affff640 1 -- 172.21.15.103:0/3370484365 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b80c54d0 con 0x55e29d248060 2024-09-19T07:51:45.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.847+0000 7f01affff640 1 -- 172.21.15.103:0/3370484365 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b803b460 con 0x55e29d248060 2024-09-19T07:51:45.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- 172.21.15.103:0/3370484365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 msgr2=0x55e29d26ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 --2- 172.21.15.103:0/3370484365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d26ac10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f01b80b8d20 tx=0x7f01b80b8bb0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- 172.21.15.103:0/3370484365 shutdown_connections 2024-09-19T07:51:45.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 --2- 172.21.15.103:0/3370484365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 0x55e29d24d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 --2- 172.21.15.103:0/3370484365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248980 0x55e29d26b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 --2- 172.21.15.103:0/3370484365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248060 0x55e29d26ac10 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- 172.21.15.103:0/3370484365 >> 172.21.15.103:0/3370484365 conn(0x55e29d2445f0 msgr2=0x55e29d24ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- 172.21.15.103:0/3370484365 shutdown_connections 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- 172.21.15.103:0/3370484365 wait complete. 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- start start 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 Processor -- start 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248060 0x55e29d257660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d257ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 0x55e29d2ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81b0fe0 con 0x55e29d248980 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81b0910 con 0x55e29d248060 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81adb70 con 0x55e29d253240 2024-09-19T07:51:45.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.848+0000 7f01ced06740 10 client.0 _ll_register_callbacks cb 0x7f01bc00abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d257ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d257ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55264/0 (socket says 172.21.15.103:55264) 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 -- 172.21.15.103:0/1125935117 learned_addr learned my addr 172.21.15.103:0/1125935117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c6d9d640 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 0x55e29d2ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c659c640 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248060 0x55e29d257660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 -- 172.21.15.103:0/1125935117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 msgr2=0x55e29d2ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 0x55e29d2ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 -- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248060 msgr2=0x55e29d257660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248060 0x55e29d257660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 -- 172.21.15.103:0/1125935117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29d2cd120 con 0x55e29d248980 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01c5d9b640 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d257ba0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f01bc019a50 tx=0x7f01bc019de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0bc9186f50fd22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01adffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01adffb640 1 -- 172.21.15.103:0/1125935117 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0041c0 con 0x55e29d248980 2024-09-19T07:51:45.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01adffb640 1 -- 172.21.15.103:0/1125935117 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f01bc004360 con 0x55e29d248980 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b80c54d0 con 0x55e29d248980 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01adffb640 1 -- 172.21.15.103:0/1125935117 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f01bc00fb90 con 0x55e29d248980 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01ced06740 10 client.18210 fetch_fsmap learned FSMap version 51 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.849+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e29d2ce430 con 0x55e29d248980 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01adffb640 1 -- 172.21.15.103:0/1125935117 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b80c54d0 con 0x55e29d248980 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01adffb640 1 -- 172.21.15.103:0/1125935117 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f01bc035470 con 0x55e29d248980 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01ced06740 10 client.18210 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01ced06740 10 client.18210 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01ced06740 20 client.18210 populate_metadata read hostname 'smithi103' 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01ced06740 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d2cec10 0x55e29d2d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01ced06740 4 client.18210 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "58797124", "release": true, "await": true}] 2024-09-19T07:51:45.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "58797124", "release": true, "await": true}) -- 0x55e29d24bd90 con 0x55e29d2cec10 2024-09-19T07:51:45.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01c659c640 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d2cec10 0x55e29d2d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:51:45.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01c659c640 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d2cec10 0x55e29d2d1030 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7805ed473a23f087 server_cookie=c2561b663cec7550 in_seq=0 out_seq=0 2024-09-19T07:51:45.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.850+0000 7f01adffb640 10 client.18210 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:45.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.893+0000 7f01adffb640 1 -- 172.21.15.103:0/1125935117 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e29d24bd90 con 0x55e29d2cec10 2024-09-19T07:51:45.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.893+0000 7f01adffb640 10 client.18210 handle_command_reply: tid=0 2024-09-19T07:51:45.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.893+0000 7f01ced06740 1 client.18210 shutdown 2024-09-19T07:51:45.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.894+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 msgr2=0x55e29d257ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.894+0000 7f01ced06740 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d257ba0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f01bc019a50 tx=0x7f01bc019de0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 shutdown_connections 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e29d2cec10 0x55e29d2d1030 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e29d253240 0x55e29d2ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01adffb640 0 client.18210 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e29d248980 0x55e29d257ba0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 --2- 172.21.15.103:0/1125935117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e29d248060 0x55e29d257660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 >> 172.21.15.103:0/1125935117 conn(0x55e29d2445f0 msgr2=0x55e29d242f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 shutdown_connections 2024-09-19T07:51:45.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 1 -- 172.21.15.103:0/1125935117 wait complete. 2024-09-19T07:51:45.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.897+0000 7f01ced06740 20 client.18210 trim_cache size 0 max 16384 2024-09-19T07:51:45.899 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:51:45.899 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 51, 2024-09-19T07:51:45.899 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:51:45.899 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 48, 2024-09-19T07:51:45.899 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "58797124": { 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 48, 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 142.2, 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:51:45.900 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:51:45.901 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b00a33b0 msgr2=0x7f01b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b00a33b0 0x7f01b00a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f01c0021000 tx=0x7f01c0020000 comp rx=0 tx=0).stop 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 msgr2=0x7f01c80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c80744f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005ab0 tx=0x7f01b80042f0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 shutdown_connections 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f01b00a33b0 0x7f01b00a5870 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c8106f10 0x7f01c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01c8102790 0x7f01c8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 --2- 172.21.15.103:0/380372168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01c8101de0 0x7f01c80744f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:51:45.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 >> 172.21.15.103:0/380372168 conn(0x7f01c80fd8c0 msgr2=0x7f01c80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:51:45.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 shutdown_connections 2024-09-19T07:51:45.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:51:45.898+0000 7f01cc92d640 1 -- 172.21.15.103:0/380372168 wait complete. 2024-09-19T07:51:46.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:46 smithi174 ceph-mon[35370]: pgmap v430: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-19T07:51:46.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:46 smithi103 ceph-mon[31201]: pgmap v430: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-19T07:51:46.793 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '58797124', total seconds elapsed: 25.6 2024-09-19T07:51:46.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:46 smithi134 ceph-mon[35414]: pgmap v430: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-19T07:51:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:48 smithi174 ceph-mon[35370]: pgmap v431: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-09-19T07:51:48.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:48 smithi103 ceph-mon[31201]: pgmap v431: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-09-19T07:51:48.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:48 smithi134 ceph-mon[35414]: pgmap v431: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-09-19T07:51:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 795.0 s, 1969.8 tps, lat 242.181 ms stddev 1938.308 2024-09-19T07:51:50.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:50 smithi174 ceph-mon[35370]: pgmap v432: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-09-19T07:51:50.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:50 smithi103 ceph-mon[31201]: pgmap v432: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-09-19T07:51:50.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:50 smithi134 ceph-mon[35414]: pgmap v432: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-09-19T07:51:52.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:52 smithi174 ceph-mon[35370]: pgmap v433: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.3 MiB/s wr, 388 op/s 2024-09-19T07:51:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:52 smithi103 ceph-mon[31201]: pgmap v433: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.3 MiB/s wr, 388 op/s 2024-09-19T07:51:52.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:52 smithi134 ceph-mon[35414]: pgmap v433: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.3 MiB/s wr, 388 op/s 2024-09-19T07:51:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:54 smithi174 ceph-mon[35370]: pgmap v434: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-19T07:51:54.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:54 smithi103 ceph-mon[31201]: pgmap v434: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-19T07:51:54.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:54 smithi134 ceph-mon[35414]: pgmap v434: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-19T07:51:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 800.0 s, 2002.8 tps, lat 49.742 ms stddev 63.455 2024-09-19T07:51:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:56 smithi174 ceph-mon[35370]: pgmap v435: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-19T07:51:56.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:56 smithi103 ceph-mon[31201]: pgmap v435: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-19T07:51:56.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:56 smithi134 ceph-mon[35414]: pgmap v435: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-19T07:51:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:51:58 smithi174 ceph-mon[35370]: pgmap v436: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.5 MiB/s wr, 489 op/s 2024-09-19T07:51:58.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:51:58 smithi103 ceph-mon[31201]: pgmap v436: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.5 MiB/s wr, 489 op/s 2024-09-19T07:51:58.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:51:58 smithi134 ceph-mon[35414]: pgmap v436: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.5 MiB/s wr, 489 op/s 2024-09-19T07:52:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 805.0 s, 1995.2 tps, lat 50.439 ms stddev 54.870 2024-09-19T07:52:00.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:00 smithi134 ceph-mon[35414]: pgmap v437: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.5 MiB/s wr, 484 op/s 2024-09-19T07:52:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:00 smithi174 ceph-mon[35370]: pgmap v437: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.5 MiB/s wr, 484 op/s 2024-09-19T07:52:00.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:00 smithi103 ceph-mon[31201]: pgmap v437: 97 pgs: 97 active+clean; 4.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.5 MiB/s wr, 484 op/s 2024-09-19T07:52:01.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:52:02.097 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:02 smithi174 ceph-mon[35370]: pgmap v438: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 8.2 MiB/s wr, 752 op/s 2024-09-19T07:52:02.712 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:02 smithi103 ceph-mon[31201]: pgmap v438: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 8.2 MiB/s wr, 752 op/s 2024-09-19T07:52:02.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:02 smithi134 ceph-mon[35414]: pgmap v438: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 8.2 MiB/s wr, 752 op/s 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 -- 172.21.15.103:0/243253079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 msgr2=0x7f26dc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 --2- 172.21.15.103:0/243253079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 0x7f26dc10da40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009ca0 tx=0x7f26d402a950 comp rx=0 tx=0).stop 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 -- 172.21.15.103:0/243253079 shutdown_connections 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 --2- 172.21.15.103:0/243253079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 0x7f26dc10da40 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 --2- 172.21.15.103:0/243253079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26dc102570 0x7f26dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 --2- 172.21.15.103:0/243253079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 0x7f26dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 -- 172.21.15.103:0/243253079 >> 172.21.15.103:0/243253079 conn(0x7f26dc0fc7d0 msgr2=0x7f26dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 -- 172.21.15.103:0/243253079 shutdown_connections 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.372+0000 7f26e15aa640 1 -- 172.21.15.103:0/243253079 wait complete. 2024-09-19T07:52:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.373+0000 7f26e15aa640 1 Processor -- start 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.373+0000 7f26e15aa640 1 -- start start 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.373+0000 7f26e15aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 0x7f26dc197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26e15aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26dc102570 0x7f26dc197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26e15aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 0x7f26dc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26e15aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc110f80 con 0x7f26dc102ef0 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26e15aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc110e00 con 0x7f26dc102570 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26e15aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc111100 con 0x7f26dc100e20 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 0x7f26dc197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 0x7f26dc197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40550/0 (socket says 172.21.15.103:40550) 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 -- 172.21.15.103:0/1334700233 learned_addr learned my addr 172.21.15.103:0/1334700233 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26da7fc640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26dc102570 0x7f26dc197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26db7fe640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 0x7f26dc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 -- 172.21.15.103:0/1334700233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26dc102570 msgr2=0x7f26dc197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26dc102570 0x7f26dc197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 -- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 msgr2=0x7f26dc19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 0x7f26dc19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.374+0000 7f26daffd640 1 -- 172.21.15.103:0/1334700233 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc19c3f0 con 0x7f26dc100e20 2024-09-19T07:52:03.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.375+0000 7f26daffd640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 0x7f26dc197440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00e9f0 tx=0x7f26cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:03.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.376+0000 7f26c3fff640 1 -- 172.21.15.103:0/1334700233 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00cd80 con 0x7f26dc100e20 2024-09-19T07:52:03.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.376+0000 7f26c3fff640 1 -- 172.21.15.103:0/1334700233 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26cc010910 con 0x7f26dc100e20 2024-09-19T07:52:03.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.376+0000 7f26c3fff640 1 -- 172.21.15.103:0/1334700233 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0188f0 con 0x7f26dc100e20 2024-09-19T07:52:03.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.376+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc19c6e0 con 0x7f26dc100e20 2024-09-19T07:52:03.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.376+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc19cb10 con 0x7f26dc100e20 2024-09-19T07:52:03.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26c3fff640 1 -- 172.21.15.103:0/1334700233 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f26cc010430 con 0x7f26dc100e20 2024-09-19T07:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 Processor -- start 2024-09-19T07:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 -- start start 2024-09-19T07:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1f9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cec1ef32e0 con 0x55cec1f8ddc0 2024-09-19T07:52:03.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cec1e9ace0 con 0x55cec1f8d4a0 2024-09-19T07:52:03.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.379+0000 7f26e3983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec1eef2a0 con 0x55cec1f98680 2024-09-19T07:52:03.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26c3fff640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26c40a7900 0x7f26c40a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26c3fff640 1 -- 172.21.15.103:0/1334700233 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f26cc0d00e0 con 0x7f26dc100e20 2024-09-19T07:52:03.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26daffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26db7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26db7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f8d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48626/0 (socket says 172.21.15.103:48626) 2024-09-19T07:52:03.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26db7fe640 1 -- 172.21.15.103:0/4287055106 learned_addr learned my addr 172.21.15.103:0/4287055106 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:03.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26db7fe640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26c40a7900 0x7f26c40a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26da7fc640 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1f9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26db7fe640 1 -- 172.21.15.103:0/4287055106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 msgr2=0x55cec1f9aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.380+0000 7f26db7fe640 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1f9aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26db7fe640 1 -- 172.21.15.103:0/4287055106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 msgr2=0x55cec1f98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26db7fe640 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26db7fe640 1 -- 172.21.15.103:0/4287055106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec1e7f440 con 0x55cec1f8d4a0 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26db7fe640 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f8d880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26d4030040 tx=0x7f26d402fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf3996de5eef954 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26db7fe640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26c40a7900 0x7f26c40a9dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f26d402f4a0 tx=0x7f26d4042000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26c17fa640 1 -- 172.21.15.103:0/4287055106 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4034040 con 0x55cec1f8d4a0 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26c17fa640 1 -- 172.21.15.103:0/4287055106 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d402fad0 con 0x55cec1f8d4a0 2024-09-19T07:52:03.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.381+0000 7f26c17fa640 1 -- 172.21.15.103:0/4287055106 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4005030 con 0x55cec1f8d4a0 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 -- 172.21.15.103:0/4287055106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 msgr2=0x55cec1f8d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f8d880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26d4030040 tx=0x7f26d402fd40 comp rx=0 tx=0).stop 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 -- 172.21.15.103:0/4287055106 shutdown_connections 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1f9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 --2- 172.21.15.103:0/4287055106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f8d880 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 -- 172.21.15.103:0/4287055106 >> 172.21.15.103:0/4287055106 conn(0x55cec1f89a30 msgr2=0x55cec1f89e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:03.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 -- 172.21.15.103:0/4287055106 shutdown_connections 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.382+0000 7f26e3983740 1 -- 172.21.15.103:0/4287055106 wait complete. 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 -- start start 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 Processor -- start 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f8ddc0 0x55cec1f95360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f98680 0x55cec1f95af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cec1f9b250 con 0x55cec1f98680 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cec1eae050 con 0x55cec1f8d4a0 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec1f9b3d0 con 0x55cec1f8ddc0 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26e3983740 10 client.0 _ll_register_callbacks cb 0x55cec1eee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26daffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26daffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48640/0 (socket says 172.21.15.103:48640) 2024-09-19T07:52:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26daffd640 1 -- 172.21.15.103:0/3370966208 learned_addr learned my addr 172.21.15.103:0/3370966208 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26db7fe640 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f98680 0x55cec1f95af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.383+0000 7f26da7fc640 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f8ddc0 0x55cec1f95360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26daffd640 1 -- 172.21.15.103:0/3370966208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f8ddc0 msgr2=0x55cec1f95360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26daffd640 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f8ddc0 0x55cec1f95360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26daffd640 1 -- 172.21.15.103:0/3370966208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f98680 msgr2=0x55cec1f95af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26daffd640 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f98680 0x55cec1f95af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26daffd640 1 -- 172.21.15.103:0/3370966208 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec1c643c0 con 0x55cec1f8d4a0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26daffd640 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f980b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f26cc098e20 tx=0x7f26cc09c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed14e1cb3797eb00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26beff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26beff5640 1 -- 172.21.15.103:0/3370966208 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc09c450 con 0x55cec1f8d4a0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d402fad0 con 0x55cec1f8d4a0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26beff5640 1 -- 172.21.15.103:0/3370966208 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f26cc0b5220 con 0x55cec1f8d4a0 2024-09-19T07:52:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26beff5640 1 -- 172.21.15.103:0/3370966208 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f26cc0b5420 con 0x55cec1f8d4a0 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26e3983740 10 client.27518 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55cec1fa6d40 con 0x55cec1f8d4a0 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.384+0000 7f26beff5640 1 -- 172.21.15.103:0/3370966208 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d402fad0 con 0x55cec1f8d4a0 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26beff5640 1 -- 172.21.15.103:0/3370966208 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f26cc09b9c0 con 0x55cec1f8d4a0 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26e3983740 10 client.27518 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26e3983740 10 client.27518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26e3983740 20 client.27518 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26e3983740 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec1fa8730 0x55cec1faab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26e3983740 4 client.27518 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec1f00850 con 0x55cec1fa8730 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26da7fc640 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec1fa8730 0x55cec1faab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.385+0000 7f26beff5640 10 client.27518 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.387+0000 7f26da7fc640 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec1fa8730 0x55cec1faab90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b55cf41efdd5ed4 server_cookie=e1bd93c5c36e8147 in_seq=0 out_seq=0 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.387+0000 7f26beff5640 1 -- 172.21.15.103:0/3370966208 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cec1f00850 con 0x55cec1fa8730 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.387+0000 7f26beff5640 10 client.27518 handle_command_reply: tid=0 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 client.27518 shutdown 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 msgr2=0x55cec1f980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f980b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f26cc098e20 tx=0x7f26cc09c330 comp rx=0 tx=0).stop 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 shutdown_connections 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec1fa8730 0x55cec1faab90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f98680 0x55cec1f95af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f8ddc0 0x55cec1f95360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 --2- 172.21.15.103:0/3370966208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1f980b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 >> 172.21.15.103:0/3370966208 conn(0x55cec1f89a30 msgr2=0x55cec1f8f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26beff5640 0 client.27518 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 shutdown_connections 2024-09-19T07:52:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 1 -- 172.21.15.103:0/3370966208 wait complete. 2024-09-19T07:52:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.388+0000 7f26e3983740 20 client.27518 trim_cache size 0 max 16384 2024-09-19T07:52:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 Processor -- start 2024-09-19T07:52:03.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 -- start start 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8d4a0 0x55cec1faa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8ddc0 0x55cec1fb04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1fb09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1ab720 con 0x55cec1f8d4a0 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1ad290 con 0x55cec1f8ddc0 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.418+0000 7f26e3983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1abdf0 con 0x55cec1f98680 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26da7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8ddc0 0x55cec1fb04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26db7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1fb09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26db7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1fb09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40574/0 (socket says 172.21.15.103:40574) 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26db7fe640 1 -- 172.21.15.103:0/270443117 learned_addr learned my addr 172.21.15.103:0/270443117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:03.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26daffd640 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8d4a0 0x55cec1faa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26db7fe640 1 -- 172.21.15.103:0/270443117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8ddc0 msgr2=0x55cec1fb04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26db7fe640 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8ddc0 0x55cec1fb04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26db7fe640 1 -- 172.21.15.103:0/270443117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8d4a0 msgr2=0x55cec1faa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.420+0000 7f26db7fe640 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8d4a0 0x55cec1faa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.421+0000 7f26db7fe640 1 -- 172.21.15.103:0/270443117 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec1f94310 con 0x55cec1f98680 2024-09-19T07:52:03.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.421+0000 7f26db7fe640 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1fb09e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004a80 tx=0x7f26d403d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e070d5ba9f37c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:03.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.422+0000 7f26c0ff9640 1 -- 172.21.15.103:0/270443117 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d402fad0 con 0x55cec1f98680 2024-09-19T07:52:03.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.422+0000 7f26c0ff9640 1 -- 172.21.15.103:0/270443117 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4004b40 con 0x55cec1f98680 2024-09-19T07:52:03.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.422+0000 7f26c0ff9640 1 -- 172.21.15.103:0/270443117 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d40306a0 con 0x55cec1f98680 2024-09-19T07:52:03.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.424+0000 7f26e3983740 1 -- 172.21.15.103:0/270443117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 msgr2=0x55cec1fb09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.424+0000 7f26e3983740 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1fb09e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26d4004a80 tx=0x7f26d403d040 comp rx=0 tx=0).stop 2024-09-19T07:52:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.424+0000 7f26e3983740 1 -- 172.21.15.103:0/270443117 shutdown_connections 2024-09-19T07:52:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.424+0000 7f26e3983740 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1fb09e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.424+0000 7f26e3983740 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8ddc0 0x55cec1fb04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.424+0000 7f26e3983740 1 --2- 172.21.15.103:0/270443117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8d4a0 0x55cec1faa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.424+0000 7f26e3983740 1 -- 172.21.15.103:0/270443117 >> 172.21.15.103:0/270443117 conn(0x55cec1f89a30 msgr2=0x55cec1f88b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:03.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.425+0000 7f26e3983740 1 -- 172.21.15.103:0/270443117 shutdown_connections 2024-09-19T07:52:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.425+0000 7f26e3983740 1 -- 172.21.15.103:0/270443117 wait complete. 2024-09-19T07:52:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 -- start start 2024-09-19T07:52:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 Processor -- start 2024-09-19T07:52:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1fa0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f9e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1f9e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1acb90 con 0x55cec1f8ddc0 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1ac4c0 con 0x55cec1f8d4a0 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1a9720 con 0x55cec1f98680 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.426+0000 7f26e3983740 10 client.0 _ll_register_callbacks cb 0x7f26cc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f9e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f9e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57414/0 (socket says 172.21.15.103:57414) 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 -- 172.21.15.103:0/1852171876 learned_addr learned my addr 172.21.15.103:0/1852171876 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26daffd640 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1fa0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 -- 172.21.15.103:0/1852171876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 msgr2=0x55cec1f9e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1f9e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 -- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 msgr2=0x55cec1fa0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1fa0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.427+0000 7f26da7fc640 1 -- 172.21.15.103:0/1852171876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec1f8dbf0 con 0x55cec1f8ddc0 2024-09-19T07:52:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.428+0000 7f26da7fc640 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f9e1f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x55cec1f94bf0 tx=0x7f26d0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c731a1c1fa48dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:03.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.428+0000 7f26beff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:52:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.428+0000 7f26beff5640 1 -- 172.21.15.103:0/1852171876 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0003d20 con 0x55cec1f8ddc0 2024-09-19T07:52:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.428+0000 7f26beff5640 1 -- 172.21.15.103:0/1852171876 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f26d0010430 con 0x55cec1f8ddc0 2024-09-19T07:52:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.428+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d4004b40 con 0x55cec1f8ddc0 2024-09-19T07:52:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.429+0000 7f26beff5640 1 -- 172.21.15.103:0/1852171876 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f26d0010630 con 0x55cec1f8ddc0 2024-09-19T07:52:03.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.429+0000 7f26e3983740 10 client.18228 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.429+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55cec2011480 con 0x55cec1f8ddc0 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26beff5640 1 -- 172.21.15.103:0/1852171876 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4004b40 con 0x55cec1f8ddc0 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26beff5640 1 -- 172.21.15.103:0/1852171876 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f26d0027800 con 0x55cec1f8ddc0 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26e3983740 10 client.18228 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26e3983740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26e3983740 20 client.18228 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26e3983740 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec2035cb0 0x55cec20380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26e3983740 4 client.18228 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cec1efad60 con 0x55cec2035cb0 2024-09-19T07:52:03.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.430+0000 7f26daffd640 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec2035cb0 0x55cec20380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.431+0000 7f26daffd640 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec2035cb0 0x55cec20380d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e255f40f6b6ce7 server_cookie=69adc7c7f978cc75 in_seq=0 out_seq=0 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26beff5640 10 client.18228 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26beff5640 1 -- 172.21.15.103:0/1852171876 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cec1efad60 con 0x55cec2035cb0 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26beff5640 10 client.18228 handle_command_reply: tid=0 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 client.18228 shutdown 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 msgr2=0x55cec1f9e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f9e1f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x55cec1f94bf0 tx=0x7f26d0004410 comp rx=0 tx=0).stop 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 shutdown_connections 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55cec2035cb0 0x55cec20380d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec1f98680 0x55cec1f9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cec1f8ddc0 0x55cec1f9e1f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 --2- 172.21.15.103:0/1852171876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cec1f8d4a0 0x55cec1fa0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 >> 172.21.15.103:0/1852171876 conn(0x55cec1f89a30 msgr2=0x55cec1fa34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26beff5640 0 client.18228 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 shutdown_connections 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 1 -- 172.21.15.103:0/1852171876 wait complete. 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.432+0000 7f26e3983740 20 client.18228 trim_cache size 0 max 16384 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:52:03.434 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:52:03.435 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:52:03.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26c40a7900 msgr2=0x7f26c40a9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26c40a7900 0x7f26c40a9dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f26d402f4a0 tx=0x7f26d4042000 comp rx=0 tx=0).stop 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 msgr2=0x7f26dc197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 0x7f26dc197440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00e9f0 tx=0x7f26cc00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 shutdown_connections 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f26c40a7900 0x7f26c40a9dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc102ef0 0x7f26dc19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26dc102570 0x7f26dc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 --2- 172.21.15.103:0/1334700233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26dc100e20 0x7f26dc197440 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.433+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 >> 172.21.15.103:0/1334700233 conn(0x7f26dc0fc7d0 msgr2=0x7f26dc1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.434+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 shutdown_connections 2024-09-19T07:52:03.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:03.434+0000 7f26e15aa640 1 -- 172.21.15.103:0/1334700233 wait complete. 2024-09-19T07:52:04.293 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:52:04.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:52:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:04 smithi174 ceph-mon[35370]: pgmap v439: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.3 MiB/s wr, 537 op/s 2024-09-19T07:52:04.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:04 smithi103 ceph-mon[31201]: pgmap v439: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.3 MiB/s wr, 537 op/s 2024-09-19T07:52:04.780 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:04.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:04 smithi134 ceph-mon[35414]: pgmap v439: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.3 MiB/s wr, 537 op/s 2024-09-19T07:52:05.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 -- 172.21.15.103:0/786092146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0102570 msgr2=0x7f84d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 --2- 172.21.15.103:0/786092146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0102570 0x7f84d01029b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009900 tx=0x7f84b802a490 comp rx=0 tx=0).stop 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 -- 172.21.15.103:0/786092146 shutdown_connections 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 --2- 172.21.15.103:0/786092146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0102ef0 0x7f84d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 --2- 172.21.15.103:0/786092146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0102570 0x7f84d01029b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 --2- 172.21.15.103:0/786092146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0100e20 0x7f84d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 -- 172.21.15.103:0/786092146 >> 172.21.15.103:0/786092146 conn(0x7f84d00fc7d0 msgr2=0x7f84d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 -- 172.21.15.103:0/786092146 shutdown_connections 2024-09-19T07:52:05.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.363+0000 7f84d4af7640 1 -- 172.21.15.103:0/786092146 wait complete. 2024-09-19T07:52:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.364+0000 7f84d4af7640 1 Processor -- start 2024-09-19T07:52:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.364+0000 7f84d4af7640 1 -- start start 2024-09-19T07:52:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84d4af7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0100e20 0x7f84d0197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84d4af7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0102570 0x7f84d01979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:05.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84d4af7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0102ef0 0x7f84d019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84d4af7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84d0110ff0 con 0x7f84d0100e20 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84d4af7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84d0110e70 con 0x7f84d0102570 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84d4af7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84d0111170 con 0x7f84d0102ef0 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0100e20 0x7f84d0197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0102570 0x7f84d01979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ce59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0100e20 0x7f84d0197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57440/0 (socket says 172.21.15.103:57440) 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0102570 0x7f84d01979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48664/0 (socket says 172.21.15.103:48664) 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ce59c640 1 -- 172.21.15.103:0/379370061 learned_addr learned my addr 172.21.15.103:0/379370061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ced9d640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0102ef0 0x7f84d019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ced9d640 1 -- 172.21.15.103:0/379370061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0102570 msgr2=0x7f84d01979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ced9d640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0102570 0x7f84d01979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ced9d640 1 -- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0100e20 msgr2=0x7f84d0197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:05.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ced9d640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0100e20 0x7f84d0197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ced9d640 1 -- 172.21.15.103:0/379370061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d019c420 con 0x7f84d0102ef0 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84ce59c640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0100e20 0x7f84d0197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.365+0000 7f84c7fff640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0102570 0x7f84d01979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.366+0000 7f84ced9d640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0102ef0 0x7f84d019bd40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e940 tx=0x7f84c800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.366+0000 7f84c77fe640 1 -- 172.21.15.103:0/379370061 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8004d20 con 0x7f84d0102ef0 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.366+0000 7f84c77fe640 1 -- 172.21.15.103:0/379370061 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c8005020 con 0x7f84d0102ef0 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.366+0000 7f84c77fe640 1 -- 172.21.15.103:0/379370061 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c80116b0 con 0x7f84d0102ef0 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.366+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d019c710 con 0x7f84d0102ef0 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.366+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d01a3ff0 con 0x7f84d0102ef0 2024-09-19T07:52:05.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.367+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d0101200 con 0x7f84d0102ef0 2024-09-19T07:52:05.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.368+0000 7f84c77fe640 1 -- 172.21.15.103:0/379370061 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f84c80054d0 con 0x7f84d0102ef0 2024-09-19T07:52:05.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.370+0000 7f84c77fe640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84bc0a3400 0x7f84bc0a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:05.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.371+0000 7f84c77fe640 1 -- 172.21.15.103:0/379370061 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7f84c80d01c0 con 0x7f84d0102ef0 2024-09-19T07:52:05.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.371+0000 7f84ce59c640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84bc0a3400 0x7f84bc0a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:05.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.372+0000 7f84ce59c640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84bc0a3400 0x7f84bc0a58c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009e10 tx=0x7f84c00091c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:05.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.375+0000 7f84c77fe640 1 -- 172.21.15.103:0/379370061 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f84c8016030 con 0x7f84d0102ef0 2024-09-19T07:52:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 810.0 s, 1092.4 tps, lat 91.317 ms stddev 144.994 2024-09-19T07:52:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:05 smithi174 ceph-mon[35370]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:52:05.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.675+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84d0109090 con 0x7f84d0102ef0 2024-09-19T07:52:05.675 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:05 smithi103 ceph-mon[31201]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:52:05.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.676+0000 7f84c77fe640 1 -- 172.21.15.103:0/379370061 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11992 (secure 0 0 0) 0x7f84c809c510 con 0x7f84d0102ef0 2024-09-19T07:52:05.676 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:52:05.677 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":51,"btime":"2024-09-19T07:50:10:429478+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:50:09.866710+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:52:05.677 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 51 2024-09-19T07:52:05.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84bc0a3400 msgr2=0x7f84bc0a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84bc0a3400 0x7f84bc0a58c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009e10 tx=0x7f84c00091c0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0102ef0 msgr2=0x7f84d019bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0102ef0 0x7f84d019bd40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e940 tx=0x7f84c800ee00 comp rx=0 tx=0).stop 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 shutdown_connections 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f84bc0a3400 0x7f84bc0a58c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0102ef0 0x7f84d019bd40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84d0102570 0x7f84d01979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 --2- 172.21.15.103:0/379370061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84d0100e20 0x7f84d0197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 >> 172.21.15.103:0/379370061 conn(0x7f84d00fc7d0 msgr2=0x7f84d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 shutdown_connections 2024-09-19T07:52:05.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:05.680+0000 7f84d4af7640 1 -- 172.21.15.103:0/379370061 wait complete. 2024-09-19T07:52:05.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:05 smithi134 ceph-mon[35414]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T07:52:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:06 smithi174 ceph-mon[35370]: pgmap v441: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.5 MiB/s wr, 645 op/s 2024-09-19T07:52:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:06 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/379370061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:52:06.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:06 smithi103 ceph-mon[31201]: pgmap v441: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.5 MiB/s wr, 645 op/s 2024-09-19T07:52:06.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:06 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/379370061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:52:06.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:06 smithi134 ceph-mon[35414]: pgmap v441: 97 pgs: 97 active+clean; 4.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.5 MiB/s wr, 645 op/s 2024-09-19T07:52:06.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:06 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/379370061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:52:07.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:52:07.356 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 -- 172.21.15.103:0/3271625598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 msgr2=0x7f5d641056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 --2- 172.21.15.103:0/3271625598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641056b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800d910 tx=0x7f5d5802a180 comp rx=0 tx=0).stop 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 -- 172.21.15.103:0/3271625598 shutdown_connections 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 --2- 172.21.15.103:0/3271625598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 0x7f5d6410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 --2- 172.21.15.103:0/3271625598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641056b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 --2- 172.21.15.103:0/3271625598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 0x7f5d64101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 -- 172.21.15.103:0/3271625598 >> 172.21.15.103:0/3271625598 conn(0x7f5d64077230 msgr2=0x7f5d640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 -- 172.21.15.103:0/3271625598 shutdown_connections 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.927+0000 7f5d693bf640 1 -- 172.21.15.103:0/3271625598 wait complete. 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.928+0000 7f5d693bf640 1 Processor -- start 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.928+0000 7f5d693bf640 1 -- start start 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.928+0000 7f5d693bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 0x7f5d641963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.928+0000 7f5d693bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.928+0000 7f5d693bf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 0x7f5d6419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.928+0000 7f5d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 0x7f5d641963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.928+0000 7f5d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 0x7f5d641963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57452/0 (socket says 172.21.15.103:57452) 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d637fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 0x7f5d6419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d637fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 0x7f5d6419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48690/0 (socket says 172.21.15.103:48690) 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40618/0 (socket says 172.21.15.103:40618) 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d62ffd640 1 -- 172.21.15.103:0/1598218374 learned_addr learned my addr 172.21.15.103:0/1598218374 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410fdc0 con 0x7f5d64101470 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410fc40 con 0x7f5d64105de0 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410ff40 con 0x7f5d64101e20 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d62ffd640 1 -- 172.21.15.103:0/1598218374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 msgr2=0x7f5d641968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d62ffd640 1 -- 172.21.15.103:0/1598218374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 msgr2=0x7f5d6419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 0x7f5d6419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d62ffd640 1 -- 172.21.15.103:0/1598218374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6419b350 con 0x7f5d64101470 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d637fe640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 0x7f5d6419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d627fc640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.929+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 0x7f5d641963a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5d54011b20 tx=0x7f5d5400f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.930+0000 7f5d4bfff640 1 -- 172.21.15.103:0/1598218374 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5401e070 con 0x7f5d64101470 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.930+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6419b640 con 0x7f5d64101470 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.930+0000 7f5d4bfff640 1 -- 172.21.15.103:0/1598218374 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5400fc90 con 0x7f5d64101470 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.930+0000 7f5d4bfff640 1 -- 172.21.15.103:0/1598218374 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54018780 con 0x7f5d64101470 2024-09-19T07:52:07.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.930+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d641a2f20 con 0x7f5d64101470 2024-09-19T07:52:07.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.932+0000 7f5d4bfff640 1 -- 172.21.15.103:0/1598218374 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5d54021070 con 0x7f5d64101470 2024-09-19T07:52:07.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.932+0000 7f5d6b798740 1 Processor -- start 2024-09-19T07:52:07.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d4bfff640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5d4c0a33b0 0x7f5d4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d6b798740 1 -- start start 2024-09-19T07:52:07.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d4bfff640 1 -- 172.21.15.103:0/1598218374 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7f5d540d3440 con 0x7f5d64101470 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2be2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be25d0 0x559ab2be2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2bf5b80 0x559ab2bf7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d6b798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ab2b482f0 con 0x559ab2be3b40 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d6b798740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559ab2aefcf0 con 0x559ab2bf5b80 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d6b798740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ab2b442b0 con 0x559ab2be25d0 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d637fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2bf5b80 0x559ab2bf7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d637fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2bf5b80 0x559ab2bf7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:48706/0 (socket says 172.21.15.103:48706) 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d637fe640 1 -- 172.21.15.103:0/2020803532 learned_addr learned my addr 172.21.15.103:0/2020803532 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.933+0000 7f5d627fc640 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be25d0 0x559ab2be2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d637fe640 1 -- 172.21.15.103:0/2020803532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be25d0 msgr2=0x559ab2be2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d637fe640 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be25d0 0x559ab2be2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d637fe640 1 -- 172.21.15.103:0/2020803532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 msgr2=0x559ab2be2090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d637fe640 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2be2090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d637fe640 1 -- 172.21.15.103:0/2020803532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab2ad43a0 con 0x559ab2bf5b80 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d627fc640 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be25d0 0x559ab2be2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d637fe640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5d4c0a33b0 0x7f5d4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.934+0000 7f5d637fe640 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2bf5b80 0x559ab2bf7f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00b930 tx=0x7f5d5c00bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06c7fafe4b76f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.935+0000 7f5d497fa640 1 -- 172.21.15.103:0/2020803532 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c008d30 con 0x559ab2bf5b80 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.935+0000 7f5d497fa640 1 -- 172.21.15.103:0/2020803532 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5c01c020 con 0x559ab2bf5b80 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.935+0000 7f5d497fa640 1 -- 172.21.15.103:0/2020803532 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c016870 con 0x559ab2bf5b80 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 -- 172.21.15.103:0/2020803532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2bf5b80 msgr2=0x559ab2bf7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2bf5b80 0x559ab2bf7f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00b930 tx=0x7f5d5c00bdf0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d637fe640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5d4c0a33b0 0x7f5d4c0a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c008fd0 tx=0x7f5d5c016af0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 -- 172.21.15.103:0/2020803532 shutdown_connections 2024-09-19T07:52:07.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2bf5b80 0x559ab2bf7f70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be25d0 0x559ab2be2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2020803532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2be2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 -- 172.21.15.103:0/2020803532 >> 172.21.15.103:0/2020803532 conn(0x559ab2bab310 msgr2=0x559ab2bdec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.936+0000 7f5d6b798740 1 -- 172.21.15.103:0/2020803532 shutdown_connections 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 -- 172.21.15.103:0/2020803532 wait complete. 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 -- start start 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 Processor -- start 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2bf55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be3b40 0x559ab2bf28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2bf5b80 0x559ab2bf2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ab2bf8880 con 0x559ab2bf5b80 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559ab2b03060 con 0x559ab2be25d0 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.937+0000 7f5d6b798740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ab2bf8a00 con 0x559ab2be3b40 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be3b40 0x559ab2bf28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d62ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2bf55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be3b40 0x559ab2bf28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:40638/0 (socket says 172.21.15.103:40638) 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 -- 172.21.15.103:0/1193230623 learned_addr learned my addr 172.21.15.103:0/1193230623 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d637fe640 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2bf5b80 0x559ab2bf2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d6b798740 10 client.0 _ll_register_callbacks cb 0x559ab28568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 -- 172.21.15.103:0/1193230623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 msgr2=0x559ab2bf55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2bf55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 -- 172.21.15.103:0/1193230623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2bf5b80 msgr2=0x559ab2bf2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2bf5b80 0x559ab2bf2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 -- 172.21.15.103:0/1193230623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab28b93c0 con 0x559ab2be3b40 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5c01c020 con 0x559ab2be3b40 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2bf55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d627fc640 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be3b40 0x559ab2bf28a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802a770 tx=0x7f5d58005220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd98c42cfa8f9245 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.938+0000 7f5d46ff5640 10 client.27505 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d46ff5640 1 -- 172.21.15.103:0/1193230623 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58005620 con 0x559ab2be3b40 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d46ff5640 1 -- 172.21.15.103:0/1193230623 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7f5d580046c0 con 0x559ab2be3b40 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d46ff5640 1 -- 172.21.15.103:0/1193230623 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5d580048c0 con 0x559ab2be3b40 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 10 client.27505 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=51}) -- 0x559ab2c04300 con 0x559ab2be3b40 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d46ff5640 1 -- 172.21.15.103:0/1193230623 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f5d58004a60 con 0x559ab2be3b40 2024-09-19T07:52:07.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 10 client.27505 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 10 client.27505 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 20 client.27505 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c05700 0x559ab2c07af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 4 client.27505 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.939+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ab2b55860 con 0x559ab2c05700 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.940+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c05700 0x559ab2c07af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.940+0000 7f5d46ff5640 10 client.27505 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.940+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c05700 0x559ab2c07af0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dad7783a84d22160 server_cookie=ad8f060bdb72a2b0 in_seq=0 out_seq=0 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.940+0000 7f5d46ff5640 1 -- 172.21.15.103:0/1193230623 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ab2b55860 con 0x559ab2c05700 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.940+0000 7f5d46ff5640 10 client.27505 handle_command_reply: tid=0 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 client.27505 shutdown 2024-09-19T07:52:07.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be3b40 msgr2=0x559ab2bf28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be3b40 0x559ab2bf28a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802a770 tx=0x7f5d58005220 comp rx=0 tx=0).stop 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 shutdown_connections 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c05700 0x559ab2c07af0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2bf5b80 0x559ab2bf2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2be3b40 0x559ab2bf28a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1193230623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2bf55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d46ff5640 0 client.27505 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 >> 172.21.15.103:0/1193230623 conn(0x559ab2bab310 msgr2=0x559ab2be4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 shutdown_connections 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 1 -- 172.21.15.103:0/1193230623 wait complete. 2024-09-19T07:52:07.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.941+0000 7f5d6b798740 20 client.27505 trim_cache size 0 max 16384 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 Processor -- start 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 -- start start 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2c01b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2c0d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2c0d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641ab6c0 con 0x559ab2be3b40 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641ad230 con 0x559ab2be25d0 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d6b798740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641abd90 con 0x559ab2bf5b80 2024-09-19T07:52:07.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2c0d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2c0d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57464/0 (socket says 172.21.15.103:57464) 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 -- 172.21.15.103:0/1742174664 learned_addr learned my addr 172.21.15.103:0/1742174664 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d637fe640 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2c0d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2c01b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 -- 172.21.15.103:0/1742174664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 msgr2=0x559ab2c0d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2c0d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 -- 172.21.15.103:0/1742174664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 msgr2=0x559ab2c01b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2c01b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.957+0000 7f5d627fc640 1 -- 172.21.15.103:0/1742174664 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab2be5120 con 0x559ab2be3b40 2024-09-19T07:52:07.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.958+0000 7f5d627fc640 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2c0d3a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006010 tx=0x7f5d58037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cd5b881133b242 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.958+0000 7f5d48ff9640 1 -- 172.21.15.103:0/1742174664 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58005620 con 0x559ab2be3b40 2024-09-19T07:52:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.958+0000 7f5d48ff9640 1 -- 172.21.15.103:0/1742174664 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5c01c020 con 0x559ab2be3b40 2024-09-19T07:52:07.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.958+0000 7f5d48ff9640 1 -- 172.21.15.103:0/1742174664 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d58032680 con 0x559ab2be3b40 2024-09-19T07:52:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 -- 172.21.15.103:0/1742174664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 msgr2=0x559ab2c0d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2c0d3a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006010 tx=0x7f5d58037000 comp rx=0 tx=0).stop 2024-09-19T07:52:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 -- 172.21.15.103:0/1742174664 shutdown_connections 2024-09-19T07:52:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2c0d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be3b40 0x559ab2c0d3a0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 --2- 172.21.15.103:0/1742174664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be25d0 0x559ab2c01b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 -- 172.21.15.103:0/1742174664 >> 172.21.15.103:0/1742174664 conn(0x559ab2bab310 msgr2=0x559ab2bf1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 -- 172.21.15.103:0/1742174664 shutdown_connections 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 -- 172.21.15.103:0/1742174664 wait complete. 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.959+0000 7f5d6b798740 1 -- start start 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 1 Processor -- start 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be25d0 0x559ab2bfe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be3b40 0x559ab2bef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2befaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641acb30 con 0x559ab2be25d0 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641ac460 con 0x559ab2be3b40 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641a96c0 con 0x559ab2bf5b80 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be25d0 0x559ab2bfe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d6b798740 10 client.0 _ll_register_callbacks cb 0x7f5d58005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be25d0 0x559ab2bfe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57474/0 (socket says 172.21.15.103:57474) 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 -- 172.21.15.103:0/2147666646 learned_addr learned my addr 172.21.15.103:0/2147666646 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d627fc640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be3b40 0x559ab2bef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d637fe640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2befaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 -- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 msgr2=0x559ab2befaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2befaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 -- 172.21.15.103:0/2147666646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be3b40 msgr2=0x559ab2bef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be3b40 0x559ab2bef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 -- 172.21.15.103:0/2147666646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab2bf0280 con 0x559ab2be25d0 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d637fe640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2befaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d62ffd640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be25d0 0x559ab2bfe310 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5d5409f6c0 tx=0x7f5d54019ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303737a83171c351 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.960+0000 7f5d46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:52:07.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d46ff5640 1 -- 172.21.15.103:0/2147666646 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d540b7020 con 0x559ab2be25d0 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d46ff5640 1 -- 172.21.15.103:0/2147666646 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7f5d54019b00 con 0x559ab2be25d0 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5c01c020 con 0x559ab2be25d0 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d46ff5640 1 -- 172.21.15.103:0/2147666646 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5d5409bda0 con 0x559ab2be25d0 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 10 client.18249 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559ab2c70340 con 0x559ab2be25d0 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d46ff5640 1 -- 172.21.15.103:0/2147666646 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5c01c020 con 0x559ab2be25d0 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d46ff5640 1 -- 172.21.15.103:0/2147666646 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f5d540b3ba0 con 0x559ab2be25d0 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 10 client.18249 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 10 client.18249 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 20 client.18249 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c70ab0 0x559ab2c72ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:07.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 4 client.18249 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.961+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ab2b4fd70 con 0x559ab2c70ab0 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.962+0000 7f5d627fc640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c70ab0 0x559ab2c72ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.962+0000 7f5d627fc640 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c70ab0 0x559ab2c72ea0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8c6dab80b6b2218 server_cookie=e57de8e544963ebb in_seq=0 out_seq=0 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.962+0000 7f5d46ff5640 10 client.18249 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d46ff5640 1 -- 172.21.15.103:0/2147666646 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ab2b4fd70 con 0x559ab2c70ab0 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d46ff5640 10 client.18249 handle_command_reply: tid=0 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 client.18249 shutdown 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be25d0 msgr2=0x559ab2bfe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be25d0 0x559ab2bfe310 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5d5409f6c0 tx=0x7f5d54019ed0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 shutdown_connections 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x559ab2c70ab0 0x559ab2c72ea0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ab2bf5b80 0x559ab2befaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ab2be3b40 0x559ab2bef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 --2- 172.21.15.103:0/2147666646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ab2be25d0 0x559ab2bfe310 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 >> 172.21.15.103:0/2147666646 conn(0x559ab2bab310 msgr2=0x559ab2bf1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.963+0000 7f5d46ff5640 0 client.18249 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:52:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.964+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 shutdown_connections 2024-09-19T07:52:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.964+0000 7f5d6b798740 1 -- 172.21.15.103:0/2147666646 wait complete. 2024-09-19T07:52:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.964+0000 7f5d6b798740 20 client.18249 trim_cache size 0 max 16384 2024-09-19T07:52:07.969 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:52:07.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5d4c0a33b0 msgr2=0x7f5d4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5d4c0a33b0 0x7f5d4c0a5870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c008fd0 tx=0x7f5d5c016af0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 msgr2=0x7f5d641963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:07.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 0x7f5d641963a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5d54011b20 tx=0x7f5d5400f050 comp rx=0 tx=0).stop 2024-09-19T07:52:07.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 shutdown_connections 2024-09-19T07:52:07.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5d4c0a33b0 0x7f5d4c0a5870 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d64105de0 0x7f5d6419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d64101e20 0x7f5d641968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 --2- 172.21.15.103:0/1598218374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d64101470 0x7f5d641963a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:07.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 >> 172.21.15.103:0/1598218374 conn(0x7f5d64077230 msgr2=0x7f5d640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:07.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 shutdown_connections 2024-09-19T07:52:07.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:07.965+0000 7f5d693bf640 1 -- 172.21.15.103:0/1598218374 wait complete. 2024-09-19T07:52:08.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:08 smithi103 ceph-mon[31201]: pgmap v442: 97 pgs: 29 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 50 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-09-19T07:52:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:08 smithi174 ceph-mon[35370]: pgmap v442: 97 pgs: 29 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 50 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-09-19T07:52:08.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:08 smithi134 ceph-mon[35414]: pgmap v442: 97 pgs: 29 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 50 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-09-19T07:52:09.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:52:09.595 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:10.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 -- 172.21.15.103:0/3543088277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8100e20 msgr2=0x7f8cb8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/3543088277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8100e20 0x7f8cb8101200 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009900 tx=0x7f8cb402a480 comp rx=0 tx=0).stop 2024-09-19T07:52:10.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 -- 172.21.15.103:0/3543088277 shutdown_connections 2024-09-19T07:52:10.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/3543088277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb8102ef0 0x7f8cb810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/3543088277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb8102570 0x7f8cb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/3543088277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8100e20 0x7f8cb8101200 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 -- 172.21.15.103:0/3543088277 >> 172.21.15.103:0/3543088277 conn(0x7f8cb80fc7d0 msgr2=0x7f8cb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 -- 172.21.15.103:0/3543088277 shutdown_connections 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.099+0000 7f8cbf75f640 1 -- 172.21.15.103:0/3543088277 wait complete. 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 Processor -- start 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 -- start start 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb8100e20 0x7f8cb8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb8102570 0x7f8cb8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8102ef0 0x7f8cb81a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110f80 con 0x7f8cb8102ef0 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8110e00 con 0x7f8cb8100e20 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbf75f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8111100 con 0x7f8cb8102570 2024-09-19T07:52:10.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8102ef0 0x7f8cb81a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8102ef0 0x7f8cb81a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46748/0 (socket says 172.21.15.103:46748) 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1137212052 learned_addr learned my addr 172.21.15.103:0/1137212052 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb8100e20 0x7f8cb8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.100+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb8102570 0x7f8cb8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1137212052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb8102570 msgr2=0x7f8cb8199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb8102570 0x7f8cb8199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1137212052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb8100e20 msgr2=0x7f8cb8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb8100e20 0x7f8cb8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1137212052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb81a48f0 con 0x7f8cb8102ef0 2024-09-19T07:52:10.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb8100e20 0x7f8cb8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:10.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8102ef0 0x7f8cb81a43b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000eb10 tx=0x7f8cb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:10.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cae7fc640 1 -- 172.21.15.103:0/1137212052 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0004b20 con 0x7f8cb8102ef0 2024-09-19T07:52:10.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cae7fc640 1 -- 172.21.15.103:0/1137212052 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb0004cc0 con 0x7f8cb8102ef0 2024-09-19T07:52:10.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cae7fc640 1 -- 172.21.15.103:0/1137212052 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0010760 con 0x7f8cb8102ef0 2024-09-19T07:52:10.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb81a4ac0 con 0x7f8cb8102ef0 2024-09-19T07:52:10.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.101+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb81a4fd0 con 0x7f8cb8102ef0 2024-09-19T07:52:10.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.104+0000 7f8cae7fc640 1 -- 172.21.15.103:0/1137212052 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8cb0005020 con 0x7f8cb8102ef0 2024-09-19T07:52:10.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.105+0000 7f8cae7fc640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8ca40a3430 0x7f8ca40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.105+0000 7f8cae7fc640 1 -- 172.21.15.103:0/1137212052 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7f8cb00d03f0 con 0x7f8cb8102ef0 2024-09-19T07:52:10.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8ca40a3430 0x7f8ca40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 Processor -- start 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 -- start start 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8ca40a3430 0x7f8ca40a58f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009230 tx=0x7f8cb40058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1eab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f68c13f2f0 con 0x55f68c1e8780 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f68c0e6cf0 con 0x55f68c1dab40 2024-09-19T07:52:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f68c13b2b0 con 0x55f68c1d95d0 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1eab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46750/0 (socket says 172.21.15.103:46750) 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1872299598 learned_addr learned my addr 172.21.15.103:0/1872299598 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.106+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1872299598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 msgr2=0x55f68c1d9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1d9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1872299598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 msgr2=0x55f68c1d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/1872299598 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68c0cb3a0 con 0x55f68c1e8780 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1d9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1eab70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00cda40 tx=0x7f8cb0098950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5128321f077a877 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8ca37fe640 1 -- 172.21.15.103:0/1872299598 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb009bf90 con 0x55f68c1e8780 2024-09-19T07:52:10.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8ca37fe640 1 -- 172.21.15.103:0/1872299598 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb009cd30 con 0x55f68c1e8780 2024-09-19T07:52:10.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.107+0000 7f8ca37fe640 1 -- 172.21.15.103:0/1872299598 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0005310 con 0x55f68c1e8780 2024-09-19T07:52:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 -- 172.21.15.103:0/1872299598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 msgr2=0x55f68c1eab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1eab70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00cda40 tx=0x7f8cb0098950 comp rx=0 tx=0).stop 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 -- 172.21.15.103:0/1872299598 shutdown_connections 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1eab70 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1d9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/1872299598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 -- 172.21.15.103:0/1872299598 >> 172.21.15.103:0/1872299598 conn(0x55f68c1a2310 msgr2=0x55f68c1d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 -- 172.21.15.103:0/1872299598 shutdown_connections 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 -- 172.21.15.103:0/1872299598 wait complete. 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 -- start start 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.108+0000 7f8cc1b38740 1 Processor -- start 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 0x55f68c1e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1dab40 0x55f68c1e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1e8780 0x55f68c1e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f68c1eb410 con 0x55f68c1dab40 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f68c0fa060 con 0x55f68c1d95d0 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f68c1eb590 con 0x55f68c1e8780 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cc1b38740 10 client.0 _ll_register_callbacks cb 0x55f68be4d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1e8780 0x55f68c1e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1e8780 0x55f68c1e5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37528/0 (socket says 172.21.15.103:37528) 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/3705709000 learned_addr learned my addr 172.21.15.103:0/3705709000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1dab40 0x55f68c1e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/3705709000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 msgr2=0x55f68c1e80a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 0x55f68c1e80a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/3705709000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1dab40 msgr2=0x55f68c1e5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:10.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1dab40 0x55f68c1e5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.109+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/3705709000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68beb03c0 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1e8780 0x55f68c1e5890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005910 tx=0x7f8cb00055c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3646d6adf453fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8ca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/3705709000 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00b9070 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/3705709000 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7f8cb00b0d60 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb009cd30 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/3705709000 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8cb009bf90 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8cc1b38740 10 client.27523 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f68c1eb410 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.110+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/3705709000 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb009cd30 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/3705709000 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f8cb009b4f0 con 0x55f68c1e8780 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cc1b38740 10 client.27523 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:10.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cc1b38740 10 client.27523 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cc1b38740 20 client.27523 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f7f80 0x55f68c1fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cc1b38740 4 client.27523 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f68c14c860 con 0x55f68c1f7f80 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f7f80 0x55f68c1fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f7f80 0x55f68c1fa370 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83a7ddf9f51f6adb server_cookie=174441f2277a811e in_seq=0 out_seq=0 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.111+0000 7f8ca0ff9640 10 client.27523 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/3705709000 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f68c14c860 con 0x55f68c1f7f80 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8ca0ff9640 10 client.27523 handle_command_reply: tid=0 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 client.27523 shutdown 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1e8780 msgr2=0x55f68c1e5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1e8780 0x55f68c1e5890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005910 tx=0x7f8cb00055c0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 shutdown_connections 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f7f80 0x55f68c1fa370 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1e8780 0x55f68c1e5890 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1dab40 0x55f68c1e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3705709000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 0x55f68c1e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 >> 172.21.15.103:0/3705709000 conn(0x55f68c1a2310 msgr2=0x55f68c1db990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8ca0ff9640 0 client.27523 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.112+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 shutdown_connections 2024-09-19T07:52:10.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.114+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3705709000 wait complete. 2024-09-19T07:52:10.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.114+0000 7f8cc1b38740 20 client.27523 trim_cache size 0 max 16384 2024-09-19T07:52:10.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 Processor -- start 2024-09-19T07:52:10.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 -- start start 2024-09-19T07:52:10.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ab8c0 con 0x55f68c1e8780 2024-09-19T07:52:10.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ad430 con 0x55f68c1dab40 2024-09-19T07:52:10.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.130+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81abf90 con 0x55f68c1d95d0 2024-09-19T07:52:10.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.133+0000 7f8cbccfa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.133+0000 7f8cbccfa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50728/0 (socket says 172.21.15.103:50728) 2024-09-19T07:52:10.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.133+0000 7f8cbccfa640 1 -- 172.21.15.103:0/3118883371 learned_addr learned my addr 172.21.15.103:0/3118883371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:10.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.133+0000 7f8cbccfa640 1 -- 172.21.15.103:0/3118883371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 msgr2=0x55f68c1f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:10.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.133+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/3118883371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.133+0000 7f8cbccfa640 1 -- 172.21.15.103:0/3118883371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 msgr2=0x55f68c1ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:10.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.134+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/3118883371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.134+0000 7f8cbccfa640 1 -- 172.21.15.103:0/3118883371 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68c1e38a0 con 0x55f68c1dab40 2024-09-19T07:52:10.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.134+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/3118883371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1e72c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009c00 tx=0x7f8ca8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dff25979bffc17d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:10.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.137+0000 7f8ca2ffd640 1 -- 172.21.15.103:0/3118883371 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8004030 con 0x55f68c1dab40 2024-09-19T07:52:10.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.137+0000 7f8ca2ffd640 1 -- 172.21.15.103:0/3118883371 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca80041d0 con 0x55f68c1dab40 2024-09-19T07:52:10.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8ca2ffd640 1 -- 172.21.15.103:0/3118883371 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca801c7e0 con 0x55f68c1dab40 2024-09-19T07:52:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3118883371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 msgr2=0x55f68c1e72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3118883371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1e72c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009c00 tx=0x7f8ca8010800 comp rx=0 tx=0).stop 2024-09-19T07:52:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3118883371 shutdown_connections 2024-09-19T07:52:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3118883371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3118883371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1dab40 0x55f68c1e72c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/3118883371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1d95d0 0x55f68c1f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.138+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3118883371 >> 172.21.15.103:0/3118883371 conn(0x55f68c1a2310 msgr2=0x55f68c1e6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:10.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.150+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3118883371 shutdown_connections 2024-09-19T07:52:10.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.154+0000 7f8cc1b38740 1 -- 172.21.15.103:0/3118883371 wait complete. 2024-09-19T07:52:10.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.154+0000 7f8cc1b38740 1 -- start start 2024-09-19T07:52:10.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.154+0000 7f8cc1b38740 1 Processor -- start 2024-09-19T07:52:10.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.158+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 0x55f68c1e0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.158+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1dab40 0x55f68c1dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.158+0000 7f8cc1b38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.158+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81acd30 con 0x55f68c1e8780 2024-09-19T07:52:10.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.158+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81ac660 con 0x55f68c1d95d0 2024-09-19T07:52:10.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.158+0000 7f8cc1b38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81a98c0 con 0x55f68c1dab40 2024-09-19T07:52:10.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.158+0000 7f8cc1b38740 10 client.0 _ll_register_callbacks cb 0x7f8cb0098ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:10.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.160+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.160+0000 7f8cbdcfc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1dc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46766/0 (socket says 172.21.15.103:46766) 2024-09-19T07:52:10.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.160+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/4263220635 learned_addr learned my addr 172.21.15.103:0/4263220635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:10.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.160+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 0x55f68c1e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.160+0000 7f8cbccfa640 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1dab40 0x55f68c1dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.163+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1dab40 msgr2=0x55f68c1dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.163+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1dab40 0x55f68c1dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.163+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/4263220635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 msgr2=0x55f68c1e0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.163+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 0x55f68c1e0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.163+0000 7f8cbdcfc640 1 -- 172.21.15.103:0/4263220635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68c1dd110 con 0x55f68c1e8780 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.163+0000 7f8cbdcfc640 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1dc910 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00049d0 tx=0x7f8cb009bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1238265fffe264e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.164+0000 7f8ca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.164+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/4263220635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00b9070 con 0x55f68c1e8780 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.164+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/4263220635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7f8cb00a1ea0 con 0x55f68c1e8780 2024-09-19T07:52:10.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.165+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca80041d0 con 0x55f68c1e8780 2024-09-19T07:52:10.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.169+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/4263220635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8cb00988d0 con 0x55f68c1e8780 2024-09-19T07:52:10.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.170+0000 7f8cc1b38740 10 client.18264 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:10.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.170+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f68c1dd110 con 0x55f68c1e8780 2024-09-19T07:52:10.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.171+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/4263220635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca80041d0 con 0x55f68c1e8780 2024-09-19T07:52:10.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.172+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/4263220635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7f8cb00ac080 con 0x55f68c1e8780 2024-09-19T07:52:10.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.173+0000 7f8cc1b38740 10 client.18264 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:10.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.173+0000 7f8cc1b38740 10 client.18264 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:52:10.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.173+0000 7f8cc1b38740 20 client.18264 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:10.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.173+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f1210 0x55f68c25ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:10.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.173+0000 7f8cc1b38740 4 client.18264 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:52:10.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.173+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f68c146d70 con 0x55f68c1f1210 2024-09-19T07:52:10.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.177+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f1210 0x55f68c25ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:10.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.180+0000 7f8cbd4fb640 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f1210 0x55f68c25ca00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea7f22c2d7aa10 server_cookie=eb74bf6bcdb71b3a in_seq=0 out_seq=0 2024-09-19T07:52:10.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.180+0000 7f8ca0ff9640 10 client.18264 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:52:10.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.186+0000 7f8ca0ff9640 1 -- 172.21.15.103:0/4263220635 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f68c146d70 con 0x55f68c1f1210 2024-09-19T07:52:10.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.186+0000 7f8ca0ff9640 10 client.18264 handle_command_reply: tid=0 2024-09-19T07:52:10.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.187+0000 7f8cc1b38740 1 client.18264 shutdown 2024-09-19T07:52:10.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.191+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 msgr2=0x55f68c1dc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.191+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1dc910 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00049d0 tx=0x7f8cb009bcb0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 shutdown_connections 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f68c1f1210 0x55f68c25ca00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f68c1e8780 0x55f68c1dc910 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f68c1dab40 0x55f68c1dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 --2- 172.21.15.103:0/4263220635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f68c1d95d0 0x55f68c1e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 >> 172.21.15.103:0/4263220635 conn(0x55f68c1a2310 msgr2=0x55f68c1e73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8ca0ff9640 0 client.18264 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 shutdown_connections 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 1 -- 172.21.15.103:0/4263220635 wait complete. 2024-09-19T07:52:10.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.192+0000 7f8cc1b38740 20 client.18264 trim_cache size 0 max 16384 2024-09-19T07:52:10.194 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8ca40a3430 msgr2=0x7f8ca40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8ca40a3430 0x7f8ca40a58f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009230 tx=0x7f8cb40058d0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8102ef0 msgr2=0x7f8cb81a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8102ef0 0x7f8cb81a43b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000eb10 tx=0x7f8cb000c2d0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 shutdown_connections 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f8ca40a3430 0x7f8ca40a58f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8cb8102ef0 0x7f8cb81a43b0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb8102570 0x7f8cb8199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 --2- 172.21.15.103:0/1137212052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb8100e20 0x7f8cb8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:10.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 >> 172.21.15.103:0/1137212052 conn(0x7f8cb80fc7d0 msgr2=0x7f8cb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:10.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 shutdown_connections 2024-09-19T07:52:10.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:10.193+0000 7f8cbf75f640 1 -- 172.21.15.103:0/1137212052 wait complete. 2024-09-19T07:52:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 815.0 s, 1769.4 tps, lat 56.360 ms stddev 61.576 2024-09-19T07:52:10.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:10 smithi174 ceph-mon[35370]: pgmap v443: 97 pgs: 29 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 50 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-09-19T07:52:10.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:10 smithi103 ceph-mon[31201]: pgmap v443: 97 pgs: 29 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 50 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-09-19T07:52:10.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:10 smithi134 ceph-mon[35414]: pgmap v443: 97 pgs: 29 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 50 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 20 MiB/s wr, 1.78k op/s 2024-09-19T07:52:11.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:52:11.646 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:12.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.206+0000 7fb0c1485640 1 -- 172.21.15.103:0/3360449337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102ef0 msgr2=0x7fb0bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.206+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3360449337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102ef0 0x7fb0bc10da40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009ca0 tx=0x7fb0b402a950 comp rx=0 tx=0).stop 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 -- 172.21.15.103:0/3360449337 shutdown_connections 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3360449337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102ef0 0x7fb0bc10da40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3360449337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0bc102570 0x7fb0bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3360449337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 0x7fb0bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 -- 172.21.15.103:0/3360449337 >> 172.21.15.103:0/3360449337 conn(0x7fb0bc0fc7d0 msgr2=0x7fb0bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 -- 172.21.15.103:0/3360449337 shutdown_connections 2024-09-19T07:52:12.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 -- 172.21.15.103:0/3360449337 wait complete. 2024-09-19T07:52:12.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.207+0000 7fb0c1485640 1 Processor -- start 2024-09-19T07:52:12.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0c1485640 1 -- start start 2024-09-19T07:52:12.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0c1485640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 0x7fb0bc199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0c1485640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102570 0x7fb0bc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0c1485640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0bc102ef0 0x7fb0bc1a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0c1485640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc110fd0 con 0x7fb0bc102570 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0baffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 0x7fb0bc199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0baffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 0x7fb0bc199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50764/0 (socket says 172.21.15.103:50764) 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0baffd640 1 -- 172.21.15.103:0/3586063532 learned_addr learned my addr 172.21.15.103:0/3586063532 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0c1485640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc110e50 con 0x7fb0bc100e20 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc111150 con 0x7fb0bc102ef0 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102570 0x7fb0bc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.208+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0bc102ef0 0x7fb0bc1a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0baffd640 1 -- 172.21.15.103:0/3586063532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0bc102ef0 msgr2=0x7fb0bc1a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0baffd640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0bc102ef0 0x7fb0bc1a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0baffd640 1 -- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102570 msgr2=0x7fb0bc199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0baffd640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102570 0x7fb0bc199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0baffd640 1 -- 172.21.15.103:0/3586063532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc1a48c0 con 0x7fb0bc100e20 2024-09-19T07:52:12.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0baffd640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 0x7fb0bc199300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9f0 tx=0x7fb0ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0a3fff640 1 -- 172.21.15.103:0/3586063532 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00cd80 con 0x7fb0bc100e20 2024-09-19T07:52:12.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0a3fff640 1 -- 172.21.15.103:0/3586063532 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ac010910 con 0x7fb0bc100e20 2024-09-19T07:52:12.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0a3fff640 1 -- 172.21.15.103:0/3586063532 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac018980 con 0x7fb0bc100e20 2024-09-19T07:52:12.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc1a4a90 con 0x7fb0bc100e20 2024-09-19T07:52:12.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.209+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1a4e90 con 0x7fb0bc100e20 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.213+0000 7fb0c385e740 1 Processor -- start 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.213+0000 7fb0c385e740 1 -- start start 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.213+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513ceb40 0x55a7513cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.213+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513cd5d0 0x55a7513cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0c385e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a7513332f0 con 0x55a7513d8370 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0c385e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7512dacf0 con 0x55a7513cd5d0 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0c385e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a75132f2b0 con 0x55a7513ceb40 2024-09-19T07:52:12.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0a3fff640 1 -- 172.21.15.103:0/3586063532 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb0ac010ac0 con 0x7fb0bc100e20 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513ceb40 0x55a7513cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513ceb40 0x55a7513cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37578/0 (socket says 172.21.15.103:37578) 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/1109257085 learned_addr learned my addr 172.21.15.103:0/1109257085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0baffd640 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513cd5d0 0x55a7513cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/1109257085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513cd5d0 msgr2=0x55a7513cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513cd5d0 0x55a7513cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/1109257085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 msgr2=0x55a7513da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/1109257085 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7512bf3a0 con 0x55a7513ceb40 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.214+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513ceb40 0x55a7513cd090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009ca0 tx=0x7fb0b4005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=615976b56b9c9b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.215+0000 7fb0a17fa640 1 -- 172.21.15.103:0/1109257085 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4048070 con 0x55a7513ceb40 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.215+0000 7fb0a3fff640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0a40a33b0 0x7fb0a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.215+0000 7fb0a17fa640 1 -- 172.21.15.103:0/1109257085 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b4003e50 con 0x55a7513ceb40 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.215+0000 7fb0a17fa640 1 -- 172.21.15.103:0/1109257085 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4034040 con 0x55a7513ceb40 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.215+0000 7fb0baffd640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0a40a33b0 0x7fb0a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.215+0000 7fb0a3fff640 1 -- 172.21.15.103:0/3586063532 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7fb0ac0d00b0 con 0x7fb0bc100e20 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 -- 172.21.15.103:0/1109257085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513ceb40 msgr2=0x55a7513cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0baffd640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0a40a33b0 0x7fb0a40a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55a7513caac0 tx=0x7fb0ac098040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513ceb40 0x55a7513cd090 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009ca0 tx=0x7fb0b4005f80 comp rx=0 tx=0).stop 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 -- 172.21.15.103:0/1109257085 shutdown_connections 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513cd5d0 0x55a7513cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 --2- 172.21.15.103:0/1109257085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513ceb40 0x55a7513cd090 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 -- 172.21.15.103:0/1109257085 >> 172.21.15.103:0/1109257085 conn(0x55a751396310 msgr2=0x55a7513c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 -- 172.21.15.103:0/1109257085 shutdown_connections 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 -- 172.21.15.103:0/1109257085 wait complete. 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 -- start start 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.216+0000 7fb0c385e740 1 Processor -- start 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a7513d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513ceb40 0x55a7513d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513d8370 0x55a7513d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0c385e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a7513db010 con 0x55a7513ceb40 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0c385e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7512ee060 con 0x55a7513d8370 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0c385e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a7513db190 con 0x55a7513cd5d0 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0c385e740 10 client.0 _ll_register_callbacks cb 0x55a7510418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a7513d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513d8370 0x55a7513d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513ceb40 0x55a7513d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a7513d7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37580/0 (socket says 172.21.15.103:37580) 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/3333957929 learned_addr learned my addr 172.21.15.103:0/3333957929 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/3333957929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513d8370 msgr2=0x55a7513d55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513d8370 0x55a7513d55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/3333957929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513ceb40 msgr2=0x55a7513d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513ceb40 0x55a7513d4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 -- 172.21.15.103:0/3333957929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7510a43c0 con 0x55a7513cd5d0 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a7513d7d30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb0b402fb60 tx=0x7fb0b402fd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e85520fcb89b2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.217+0000 7fb09eff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb09eff5640 1 -- 172.21.15.103:0/3333957929 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4048070 con 0x55a7513cd5d0 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b4003e50 con 0x55a7513cd5d0 2024-09-19T07:52:12.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb09eff5640 1 -- 172.21.15.103:0/3333957929 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7fb0b4004250 con 0x55a7513cd5d0 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb09eff5640 1 -- 172.21.15.103:0/3333957929 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb0b4004450 con 0x55a7513cd5d0 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 10 client.27538 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a7513db010 con 0x55a7513cd5d0 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb09eff5640 1 -- 172.21.15.103:0/3333957929 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b4003e50 con 0x55a7513cd5d0 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb09eff5640 1 -- 172.21.15.103:0/3333957929 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fb0b4052cc0 con 0x55a7513cd5d0 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 10 client.27538 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 10 client.27538 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 20 client.27538 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a7513e9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 4 client.27538 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.218+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a751340860 con 0x55a7513e7b60 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.219+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a7513e9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.220+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a7513e9f50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60bab2e67f855598 server_cookie=1113ca5910445be5 in_seq=0 out_seq=0 2024-09-19T07:52:12.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.220+0000 7fb09eff5640 10 client.27538 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.221+0000 7fb09eff5640 1 -- 172.21.15.103:0/3333957929 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a751340860 con 0x55a7513e7b60 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.221+0000 7fb09eff5640 10 client.27538 handle_command_reply: tid=0 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.221+0000 7fb0c385e740 1 client.27538 shutdown 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 msgr2=0x55a7513d7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a7513d7d30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb0b402fb60 tx=0x7fb0b402fd00 comp rx=0 tx=0).stop 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 shutdown_connections 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a7513e9f50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513d8370 0x55a7513d55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb09eff5640 0 client.27538 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:52:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513ceb40 0x55a7513d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3333957929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a7513d7d30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 >> 172.21.15.103:0/3333957929 conn(0x55a751396310 msgr2=0x55a7513cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 shutdown_connections 2024-09-19T07:52:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 1 -- 172.21.15.103:0/3333957929 wait complete. 2024-09-19T07:52:12.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.222+0000 7fb0c385e740 20 client.27538 trim_cache size 0 max 16384 2024-09-19T07:52:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.242+0000 7fb0c385e740 1 Processor -- start 2024-09-19T07:52:12.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.245+0000 7fb0c385e740 1 -- start start 2024-09-19T07:52:12.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.246+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513cd5d0 0x55a7513e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.246+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.246+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513d8370 0x55a7513efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.246+0000 7fb0c385e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ab7c0 con 0x55a7513cd5d0 2024-09-19T07:52:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.246+0000 7fb0c385e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ad330 con 0x55a7513ceb40 2024-09-19T07:52:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.246+0000 7fb0c385e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1abe90 con 0x55a7513d8370 2024-09-19T07:52:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513ef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50796/0 (socket says 172.21.15.103:50796) 2024-09-19T07:52:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/3373004231 learned_addr learned my addr 172.21.15.103:0/3373004231 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513d8370 0x55a7513efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/3373004231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513d8370 msgr2=0x55a7513efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513d8370 0x55a7513efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/3373004231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513cd5d0 msgr2=0x55a7513e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513cd5d0 0x55a7513e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.247+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/3373004231 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7513d0120 con 0x55a7513ceb40 2024-09-19T07:52:12.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.248+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513ef5e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc06aac0 tx=0x7fb0b0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0da353a0853777c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.256+0000 7fb0a0ff9640 1 -- 172.21.15.103:0/3373004231 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0015110 con 0x55a7513ceb40 2024-09-19T07:52:12.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.256+0000 7fb0a0ff9640 1 -- 172.21.15.103:0/3373004231 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b0005020 con 0x55a7513ceb40 2024-09-19T07:52:12.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.257+0000 7fb0c385e740 1 -- 172.21.15.103:0/3373004231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 msgr2=0x55a7513ef5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.257+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513ef5e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc06aac0 tx=0x7fb0b0005f90 comp rx=0 tx=0).stop 2024-09-19T07:52:12.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.261+0000 7fb0c385e740 1 -- 172.21.15.103:0/3373004231 shutdown_connections 2024-09-19T07:52:12.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.261+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513d8370 0x55a7513efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.261+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513ef5e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.261+0000 7fb0c385e740 1 --2- 172.21.15.103:0/3373004231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513cd5d0 0x55a7513e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.261+0000 7fb0c385e740 1 -- 172.21.15.103:0/3373004231 >> 172.21.15.103:0/3373004231 conn(0x55a751396310 msgr2=0x55a7513d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:12.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.262+0000 7fb0c385e740 1 -- 172.21.15.103:0/3373004231 shutdown_connections 2024-09-19T07:52:12.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.266+0000 7fb0c385e740 1 -- 172.21.15.103:0/3373004231 wait complete. 2024-09-19T07:52:12.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.266+0000 7fb0c385e740 1 -- start start 2024-09-19T07:52:12.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.266+0000 7fb0c385e740 1 Processor -- start 2024-09-19T07:52:12.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.268+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a75144dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.268+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513e0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.268+0000 7fb0c385e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.268+0000 7fb0c385e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1acc30 con 0x55a7513d8370 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.268+0000 7fb0c385e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1ac560 con 0x55a7513ceb40 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.268+0000 7fb0c385e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1a97c0 con 0x55a7513cd5d0 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.268+0000 7fb0c385e740 10 client.0 _ll_register_callbacks cb 0x7fb0b4046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.269+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.269+0000 7fb0ba7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513e0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50800/0 (socket says 172.21.15.103:50800) 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.269+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2569106319 learned_addr learned my addr 172.21.15.103:0/2569106319 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.269+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.269+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a75144dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.270+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2569106319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 msgr2=0x55a75144dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.270+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a75144dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.270+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2569106319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 msgr2=0x55a7513e0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:12.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.270+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513e0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.270+0000 7fb0ba7fc640 1 -- 172.21.15.103:0/2569106319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7513e0f20 con 0x55a7513ceb40 2024-09-19T07:52:12.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.270+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:12.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.272+0000 7fb0ba7fc640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513e0420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009b50 tx=0x7fb0b0009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd062a0beb4fae43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:12.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.272+0000 7fb09eff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:52:12.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.272+0000 7fb09eff5640 1 -- 172.21.15.103:0/2569106319 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00053d0 con 0x55a7513ceb40 2024-09-19T07:52:12.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.273+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b0005020 con 0x55a7513ceb40 2024-09-19T07:52:12.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.273+0000 7fb09eff5640 1 -- 172.21.15.103:0/2569106319 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7fb0b0005570 con 0x55a7513ceb40 2024-09-19T07:52:12.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.273+0000 7fb09eff5640 1 -- 172.21.15.103:0/2569106319 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb0b0015110 con 0x55a7513ceb40 2024-09-19T07:52:12.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.273+0000 7fb0c385e740 10 client.27575 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:12.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.273+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a75145f5a0 con 0x55a7513ceb40 2024-09-19T07:52:12.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb09eff5640 1 -- 172.21.15.103:0/2569106319 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b0005020 con 0x55a7513ceb40 2024-09-19T07:52:12.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb09eff5640 1 -- 172.21.15.103:0/2569106319 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fb0b0004790 con 0x55a7513ceb40 2024-09-19T07:52:12.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb0c385e740 10 client.27575 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:12.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb0c385e740 10 client.27575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:52:12.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb0c385e740 20 client.27575 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:12.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb0c385e740 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a751461cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:12.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb0c385e740 4 client.27575 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:52:12.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a75133ad70 con 0x55a7513e7b60 2024-09-19T07:52:12.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.274+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a751461cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:12.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.279+0000 7fb0bb7fe640 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a751461cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb775346e01ec8b8 server_cookie=489c80f9133e040e in_seq=0 out_seq=0 2024-09-19T07:52:12.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.280+0000 7fb09eff5640 10 client.27575 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:52:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb09eff5640 1 -- 172.21.15.103:0/2569106319 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a75133ad70 con 0x55a7513e7b60 2024-09-19T07:52:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb09eff5640 10 client.27575 handle_command_reply: tid=0 2024-09-19T07:52:12.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 client.27575 shutdown 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 msgr2=0x55a7513e0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513e0420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009b50 tx=0x7fb0b0009d40 comp rx=0 tx=0).stop 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 shutdown_connections 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55a7513e7b60 0x55a751461cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7513d8370 0x55a7513e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7513ceb40 0x55a7513e0420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 --2- 172.21.15.103:0/2569106319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7513cd5d0 0x55a75144dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb09eff5640 0 client.27575 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:52:12.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.281+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 >> 172.21.15.103:0/2569106319 conn(0x55a751396310 msgr2=0x55a7513c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.282+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 shutdown_connections 2024-09-19T07:52:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.282+0000 7fb0c385e740 1 -- 172.21.15.103:0/2569106319 wait complete. 2024-09-19T07:52:12.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.282+0000 7fb0c385e740 20 client.27575 trim_cache size 0 max 16384 2024-09-19T07:52:12.283 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:52:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0a40a33b0 msgr2=0x7fb0a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0a40a33b0 0x7fb0a40a5870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55a7513caac0 tx=0x7fb0ac098040 comp rx=0 tx=0).stop 2024-09-19T07:52:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 msgr2=0x7fb0bc199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:12.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 0x7fb0bc199300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9f0 tx=0x7fb0ac00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 shutdown_connections 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb0a40a33b0 0x7fb0a40a5870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0bc102ef0 0x7fb0bc1a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0bc102570 0x7fb0bc199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 --2- 172.21.15.103:0/3586063532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0bc100e20 0x7fb0bc199300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.283+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 >> 172.21.15.103:0/3586063532 conn(0x7fb0bc0fc7d0 msgr2=0x7fb0bc106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.284+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 shutdown_connections 2024-09-19T07:52:12.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:12.284+0000 7fb0c1485640 1 -- 172.21.15.103:0/3586063532 wait complete. 2024-09-19T07:52:12.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:12 smithi103 ceph-mon[31201]: pgmap v444: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 19 MiB/s wr, 1.86k op/s 2024-09-19T07:52:12.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:12 smithi134 ceph-mon[35414]: pgmap v444: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 19 MiB/s wr, 1.86k op/s 2024-09-19T07:52:12.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:12 smithi174 ceph-mon[35370]: pgmap v444: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 19 MiB/s wr, 1.86k op/s 2024-09-19T07:52:13.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:52:13.608 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 -- 172.21.15.103:0/1591729847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100e30 msgr2=0x7fe334104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 --2- 172.21.15.103:0/1591729847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100e30 0x7fe334104e80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe32400aa10 tx=0x7fe32402b720 comp rx=0 tx=0).stop 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 -- 172.21.15.103:0/1591729847 shutdown_connections 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 --2- 172.21.15.103:0/1591729847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3341055b0 0x7fe33410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 --2- 172.21.15.103:0/1591729847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100e30 0x7fe334104e80 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 --2- 172.21.15.103:0/1591729847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe334100480 0x7fe334100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 -- 172.21.15.103:0/1591729847 >> 172.21.15.103:0/1591729847 conn(0x7fe334077210 msgr2=0x7fe3340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 -- 172.21.15.103:0/1591729847 shutdown_connections 2024-09-19T07:52:14.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.319+0000 7fe339440640 1 -- 172.21.15.103:0/1591729847 wait complete. 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 Processor -- start 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 -- start start 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100480 0x7fe334198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe334100e30 0x7fe334198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3341055b0 0x7fe33419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe334111fa0 con 0x7fe334100480 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe334111e20 con 0x7fe3341055b0 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe339440640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe334112120 con 0x7fe334100e30 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100480 0x7fe334198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100480 0x7fe334198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46832/0 (socket says 172.21.15.103:46832) 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe332ffd640 1 -- 172.21.15.103:0/2864383966 learned_addr learned my addr 172.21.15.103:0/2864383966 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe334100e30 0x7fe334198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.320+0000 7fe3337fe640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3341055b0 0x7fe33419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe3327fc640 1 -- 172.21.15.103:0/2864383966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3341055b0 msgr2=0x7fe33419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3341055b0 0x7fe33419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe3327fc640 1 -- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100480 msgr2=0x7fe334198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100480 0x7fe334198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe3327fc640 1 -- 172.21.15.103:0/2864383966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33419d3f0 con 0x7fe334100e30 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe334100e30 0x7fe334198980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe324005940 tx=0x7fe3240042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe32bfff640 1 -- 172.21.15.103:0/2864383966 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32403f070 con 0x7fe334100e30 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33419d680 con 0x7fe334100e30 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe32bfff640 1 -- 172.21.15.103:0/2864383966 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe324032400 con 0x7fe334100e30 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3341a4fc0 con 0x7fe334100e30 2024-09-19T07:52:14.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.321+0000 7fe32bfff640 1 -- 172.21.15.103:0/2864383966 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324036040 con 0x7fe334100e30 2024-09-19T07:52:14.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.324+0000 7fe33b819740 1 Processor -- start 2024-09-19T07:52:14.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.324+0000 7fe32bfff640 1 -- 172.21.15.103:0/2864383966 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe324042070 con 0x7fe334100e30 2024-09-19T07:52:14.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.324+0000 7fe33b819740 1 -- start start 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe33b819740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe33b819740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1c5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe33b819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1daf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe33b819740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f80d12b2f0 con 0x55f80d1c5b80 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe33b819740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f80d0d2cf0 con 0x55f80d1c6840 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe33b819740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f80d1272b0 con 0x55f80d1c5120 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37616/0 (socket says 172.21.15.103:37616) 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 -- 172.21.15.103:0/2088705661 learned_addr learned my addr 172.21.15.103:0/2088705661 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1daf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe3337fe640 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 -- 172.21.15.103:0/2088705661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 msgr2=0x55f80d1c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 -- 172.21.15.103:0/2088705661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 msgr2=0x55f80d1daf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1daf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.325+0000 7fe332ffd640 1 -- 172.21.15.103:0/2088705661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80d0b73a0 con 0x55f80d1c5120 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.326+0000 7fe32bfff640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3200a3430 0x7fe3200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.326+0000 7fe332ffd640 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1c5560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009df0 tx=0x7fe31c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6123ac1f58004a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.326+0000 7fe3337fe640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3200a3430 0x7fe3200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.326+0000 7fe32bfff640 1 -- 172.21.15.103:0/2864383966 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7fe3240f0050 con 0x7fe334100e30 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.326+0000 7fe3297fa640 1 -- 172.21.15.103:0/2088705661 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c005220 con 0x55f80d1c5120 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.326+0000 7fe3297fa640 1 -- 172.21.15.103:0/2088705661 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe31c004c80 con 0x55f80d1c5120 2024-09-19T07:52:14.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.326+0000 7fe3297fa640 1 -- 172.21.15.103:0/2088705661 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c019970 con 0x55f80d1c5120 2024-09-19T07:52:14.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 -- 172.21.15.103:0/2088705661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 msgr2=0x55f80d1c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe3337fe640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3200a3430 0x7fe3200a58f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005fd0 tx=0x7fe32c009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:14.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1c5560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009df0 tx=0x7fe31c011810 comp rx=0 tx=0).stop 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 -- 172.21.15.103:0/2088705661 shutdown_connections 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1daf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1c5560 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 --2- 172.21.15.103:0/2088705661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 -- 172.21.15.103:0/2088705661 >> 172.21.15.103:0/2088705661 conn(0x55f80d18e310 msgr2=0x55f80d1c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 -- 172.21.15.103:0/2088705661 shutdown_connections 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.327+0000 7fe33b819740 1 -- 172.21.15.103:0/2088705661 wait complete. 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 -- start start 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 Processor -- start 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1d8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.333 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f80d1db930 con 0x55f80d1c5b80 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f80d0e6060 con 0x55f80d1c6840 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f80d1dbab0 con 0x55f80d1c5120 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe33b819740 10 client.0 _ll_register_callbacks cb 0x55f80ce398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1dadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:37622/0 (socket says 172.21.15.103:37622) 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe332ffd640 1 -- 172.21.15.103:0/2994540596 learned_addr learned my addr 172.21.15.103:0/2994540596 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1d8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe3337fe640 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.328+0000 7fe332ffd640 1 -- 172.21.15.103:0/2994540596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 msgr2=0x55f80d1d85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe332ffd640 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1d85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe332ffd640 1 -- 172.21.15.103:0/2994540596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 msgr2=0x55f80d1d8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe332ffd640 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1d8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe332ffd640 1 -- 172.21.15.103:0/2994540596 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80ce9c3c0 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe332ffd640 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1dadc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005600 tx=0x7fe31c005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8de483a8f4ab95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe31a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe31a7fc640 1 -- 172.21.15.103:0/2994540596 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c027070 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe31a7fc640 1 -- 172.21.15.103:0/2994540596 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7fe31c019a80 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31c004c80 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe31a7fc640 1 -- 172.21.15.103:0/2994540596 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe31c033050 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe33b819740 10 client.27556 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f80d1e72f0 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe31a7fc640 1 -- 172.21.15.103:0/2994540596 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe31c0099d0 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.329+0000 7fe31a7fc640 1 -- 172.21.15.103:0/2994540596 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fe31c023460 con 0x55f80d1c5120 2024-09-19T07:52:14.335 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe33b819740 10 client.27556 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe33b819740 10 client.27556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe33b819740 20 client.27556 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe33b819740 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1e89c0 0x55f80d1eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe33b819740 4 client.27556 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe31c004c80 con 0x55f80d1e89c0 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1e89c0 0x55f80d1eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe3327fc640 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1e89c0 0x55f80d1eadb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d41ca32877c84d45 server_cookie=3032e50179e6abac in_seq=0 out_seq=0 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.330+0000 7fe31a7fc640 10 client.27556 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe31a7fc640 1 -- 172.21.15.103:0/2994540596 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe31c004c80 con 0x55f80d1e89c0 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe31a7fc640 10 client.27556 handle_command_reply: tid=0 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 client.27556 shutdown 2024-09-19T07:52:14.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 msgr2=0x55f80d1dadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1dadc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005600 tx=0x7fe31c005500 comp rx=0 tx=0).stop 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 shutdown_connections 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1e89c0 0x55f80d1eadb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d1c6840 0x55f80d1d85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5b80 0x55f80d1d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 --2- 172.21.15.103:0/2994540596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5120 0x55f80d1dadc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 >> 172.21.15.103:0/2994540596 conn(0x55f80d18e310 msgr2=0x55f80d1c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe31a7fc640 0 client.27556 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 shutdown_connections 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.331+0000 7fe33b819740 1 -- 172.21.15.103:0/2994540596 wait complete. 2024-09-19T07:52:14.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.332+0000 7fe33b819740 20 client.27556 trim_cache size 0 max 16384 2024-09-19T07:52:14.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.357+0000 7fe33b819740 1 Processor -- start 2024-09-19T07:52:14.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.357+0000 7fe33b819740 1 -- start start 2024-09-19T07:52:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe33b819740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe33b819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 0x55f80d1c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe33b819740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d1c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe33b819740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341ab800 con 0x55f80d1c5120 2024-09-19T07:52:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe33b819740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341ad370 con 0x55f80d18e310 2024-09-19T07:52:14.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe33b819740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341abed0 con 0x55f80d1c5b80 2024-09-19T07:52:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50834/0 (socket says 172.21.15.103:50834) 2024-09-19T07:52:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.358+0000 7fe332ffd640 1 -- 172.21.15.103:0/1764897099 learned_addr learned my addr 172.21.15.103:0/1764897099 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:14.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.359+0000 7fe3337fe640 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d1c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.361+0000 7fe3337fe640 1 -- 172.21.15.103:0/1764897099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 msgr2=0x55f80d1cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.361+0000 7fe3337fe640 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.361+0000 7fe3337fe640 1 -- 172.21.15.103:0/1764897099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 msgr2=0x55f80d1c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.361+0000 7fe3337fe640 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 0x55f80d1c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.361+0000 7fe3337fe640 1 -- 172.21.15.103:0/1764897099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80d1dbaa0 con 0x55f80d1c5b80 2024-09-19T07:52:14.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.363+0000 7fe3337fe640 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d1c1c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009f10 tx=0x7fe32c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb54b07f9ccfa469 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:14.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.367+0000 7fe328ff9640 1 -- 172.21.15.103:0/1764897099 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c004b20 con 0x55f80d1c5b80 2024-09-19T07:52:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.367+0000 7fe328ff9640 1 -- 172.21.15.103:0/1764897099 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe32c004cc0 con 0x55f80d1c5b80 2024-09-19T07:52:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.367+0000 7fe328ff9640 1 -- 172.21.15.103:0/1764897099 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c005490 con 0x55f80d1c5b80 2024-09-19T07:52:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.369+0000 7fe33b819740 1 -- 172.21.15.103:0/1764897099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 msgr2=0x55f80d1c1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.369+0000 7fe33b819740 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d1c1c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009f10 tx=0x7fe32c01a000 comp rx=0 tx=0).stop 2024-09-19T07:52:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.371+0000 7fe33b819740 1 -- 172.21.15.103:0/1764897099 shutdown_connections 2024-09-19T07:52:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.371+0000 7fe33b819740 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d1c1c40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.371+0000 7fe33b819740 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 0x55f80d1c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.371+0000 7fe33b819740 1 --2- 172.21.15.103:0/1764897099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.371+0000 7fe33b819740 1 -- 172.21.15.103:0/1764897099 >> 172.21.15.103:0/1764897099 conn(0x55f80d1d8710 msgr2=0x55f80d1cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:14.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.374+0000 7fe33b819740 1 -- 172.21.15.103:0/1764897099 shutdown_connections 2024-09-19T07:52:14.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.377+0000 7fe33b819740 1 -- 172.21.15.103:0/1764897099 wait complete. 2024-09-19T07:52:14.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.377+0000 7fe33b819740 1 -- start start 2024-09-19T07:52:14.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.378+0000 7fe33b819740 1 Processor -- start 2024-09-19T07:52:14.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.378+0000 7fe33b819740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.378+0000 7fe33b819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 0x55f80d1eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.378+0000 7fe33b819740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d262300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.378+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.379+0000 7fe332ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1eaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50840/0 (socket says 172.21.15.103:50840) 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.379+0000 7fe332ffd640 1 -- 172.21.15.103:0/3382117604 learned_addr learned my addr 172.21.15.103:0/3382117604 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.379+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341acc70 con 0x55f80d1c5120 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.379+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341ac5a0 con 0x55f80d18e310 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.379+0000 7fe3337fe640 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d262300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.379+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341a9800 con 0x55f80d1c5b80 2024-09-19T07:52:14.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.379+0000 7fe33b819740 10 client.0 _ll_register_callbacks cb 0x7fe3240f3a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.380+0000 7fe3327fc640 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 0x55f80d1eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.380+0000 7fe3337fe640 1 -- 172.21.15.103:0/3382117604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 msgr2=0x55f80d1eaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.380+0000 7fe3337fe640 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1eaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.380+0000 7fe3337fe640 1 -- 172.21.15.103:0/3382117604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 msgr2=0x55f80d1eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.380+0000 7fe3337fe640 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 0x55f80d1eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.380+0000 7fe3337fe640 1 -- 172.21.15.103:0/3382117604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f80d2628c0 con 0x55f80d1c5b80 2024-09-19T07:52:14.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.381+0000 7fe3337fe640 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d262300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009c30 tx=0x7fe32c00fc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61b269d289edc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:14.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.383+0000 7fe31a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:52:14.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.383+0000 7fe31a7fc640 1 -- 172.21.15.103:0/3382117604 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c004a40 con 0x55f80d1c5b80 2024-09-19T07:52:14.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.383+0000 7fe31a7fc640 1 -- 172.21.15.103:0/3382117604 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8586+0+0 (secure 0 0 0) 0x7fe32c00f540 con 0x55f80d1c5b80 2024-09-19T07:52:14.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.386+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe32c004cc0 con 0x55f80d1c5b80 2024-09-19T07:52:14.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.387+0000 7fe31a7fc640 1 -- 172.21.15.103:0/3382117604 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe32c00f740 con 0x55f80d1c5b80 2024-09-19T07:52:14.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.393+0000 7fe33b819740 10 client.27562 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:14.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.394+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f80d263b80 con 0x55f80d1c5b80 2024-09-19T07:52:14.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.397+0000 7fe31a7fc640 1 -- 172.21.15.103:0/3382117604 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe32c00f8e0 con 0x55f80d1c5b80 2024-09-19T07:52:14.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.397+0000 7fe31a7fc640 1 -- 172.21.15.103:0/3382117604 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fe32c03c470 con 0x55f80d1c5b80 2024-09-19T07:52:14.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.400+0000 7fe33b819740 10 client.27562 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:14.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.400+0000 7fe33b819740 10 client.27562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:52:14.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.400+0000 7fe33b819740 20 client.27562 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:14.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.400+0000 7fe33b819740 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1c6840 0x55f80d2660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:14.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.400+0000 7fe33b819740 4 client.27562 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:52:14.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.400+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe32c004cc0 con 0x55f80d1c6840 2024-09-19T07:52:14.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.400+0000 7fe332ffd640 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1c6840 0x55f80d2660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:14.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.401+0000 7fe332ffd640 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1c6840 0x55f80d2660e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1df544e92b29c861 server_cookie=1636a2b961104e8f in_seq=0 out_seq=0 2024-09-19T07:52:14.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.401+0000 7fe31a7fc640 10 client.27562 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:52:14.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.401+0000 7fe31a7fc640 1 -- 172.21.15.103:0/3382117604 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe32c004cc0 con 0x55f80d1c6840 2024-09-19T07:52:14.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.401+0000 7fe31a7fc640 10 client.27562 handle_command_reply: tid=0 2024-09-19T07:52:14.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.401+0000 7fe33b819740 1 client.27562 shutdown 2024-09-19T07:52:14.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.410+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 msgr2=0x55f80d262300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.410+0000 7fe33b819740 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d262300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009c30 tx=0x7fe32c00fc80 comp rx=0 tx=0).stop 2024-09-19T07:52:14.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.410+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 shutdown_connections 2024-09-19T07:52:14.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.410+0000 7fe33b819740 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55f80d1c6840 0x55f80d2660e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.411+0000 7fe33b819740 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f80d1c5b80 0x55f80d262300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.411+0000 7fe33b819740 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f80d1c5120 0x55f80d1eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.411+0000 7fe33b819740 1 --2- 172.21.15.103:0/3382117604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f80d18e310 0x55f80d1eaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.411+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 >> 172.21.15.103:0/3382117604 conn(0x55f80d1d8710 msgr2=0x55f80d1c1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:14.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.411+0000 7fe31a7fc640 0 client.27562 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:52:14.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.412+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 shutdown_connections 2024-09-19T07:52:14.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.413+0000 7fe33b819740 1 -- 172.21.15.103:0/3382117604 wait complete. 2024-09-19T07:52:14.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.414+0000 7fe33b819740 20 client.27562 trim_cache size 0 max 16384 2024-09-19T07:52:14.414 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:52:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3200a3430 msgr2=0x7fe3200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3200a3430 0x7fe3200a58f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005fd0 tx=0x7fe32c009500 comp rx=0 tx=0).stop 2024-09-19T07:52:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe334100e30 msgr2=0x7fe334198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:14.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe334100e30 0x7fe334198980 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe324005940 tx=0x7fe3240042f0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 shutdown_connections 2024-09-19T07:52:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe3200a3430 0x7fe3200a58f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3341055b0 0x7fe33419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe334100e30 0x7fe334198980 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 --2- 172.21.15.103:0/2864383966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe334100480 0x7fe334198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.418+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 >> 172.21.15.103:0/2864383966 conn(0x7fe334077210 msgr2=0x7fe334103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:14.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.419+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 shutdown_connections 2024-09-19T07:52:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:14.419+0000 7fe339440640 1 -- 172.21.15.103:0/2864383966 wait complete. 2024-09-19T07:52:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:14 smithi103 ceph-mon[31201]: pgmap v445: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 19 MiB/s wr, 1.86k op/s 2024-09-19T07:52:14.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:14 smithi134 ceph-mon[35414]: pgmap v445: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 19 MiB/s wr, 1.86k op/s 2024-09-19T07:52:14.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:14 smithi174 ceph-mon[35370]: pgmap v445: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 19 MiB/s wr, 1.86k op/s 2024-09-19T07:52:15.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:52:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 820.0 s, 1636.7 tps, lat 60.332 ms stddev 73.842 2024-09-19T07:52:15.791 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:16.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:16 smithi103 ceph-mon[31201]: pgmap v446: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 1.83k op/s 2024-09-19T07:52:16.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:16 smithi103 ceph-mon[31201]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:52:16.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:16 smithi134 ceph-mon[35414]: pgmap v446: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 1.83k op/s 2024-09-19T07:52:16.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:16 smithi134 ceph-mon[35414]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:52:16.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:16 smithi174 ceph-mon[35370]: pgmap v446: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 1.83k op/s 2024-09-19T07:52:16.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:16 smithi174 ceph-mon[35370]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T07:52:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.357+0000 7fd68db17640 1 -- 172.21.15.103:0/3065663924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6881027a0 msgr2=0x7fd6881067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.357+0000 7fd68db17640 1 --2- 172.21.15.103:0/3065663924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6881027a0 0x7fd6881067f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009900 tx=0x7fd67c02a410 comp rx=0 tx=0).stop 2024-09-19T07:52:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.357+0000 7fd68db17640 1 -- 172.21.15.103:0/3065663924 shutdown_connections 2024-09-19T07:52:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.357+0000 7fd68db17640 1 --2- 172.21.15.103:0/3065663924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd688106f20 0x7fd68810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.357+0000 7fd68db17640 1 --2- 172.21.15.103:0/3065663924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6881027a0 0x7fd6881067f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.357+0000 7fd68db17640 1 --2- 172.21.15.103:0/3065663924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd688101df0 0x7fd6881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.358+0000 7fd68db17640 1 -- 172.21.15.103:0/3065663924 >> 172.21.15.103:0/3065663924 conn(0x7fd6880fd8f0 msgr2=0x7fd6880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.358+0000 7fd68db17640 1 -- 172.21.15.103:0/3065663924 shutdown_connections 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.358+0000 7fd68db17640 1 -- 172.21.15.103:0/3065663924 wait complete. 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.358+0000 7fd68db17640 1 Processor -- start 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd68db17640 1 -- start start 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd68db17640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd688101df0 0x7fd688198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd68db17640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6881027a0 0x7fd688198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd68db17640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd688106f20 0x7fd68819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd68db17640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd688111f40 con 0x7fd688106f20 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd68db17640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd688111dc0 con 0x7fd6881027a0 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd68db17640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881120c0 con 0x7fd688101df0 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd686ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6881027a0 0x7fd688198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd6877fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd688101df0 0x7fd688198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd686ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6881027a0 0x7fd688198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:50856/0 (socket says 172.21.15.103:50856) 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd686ffd640 1 -- 172.21.15.103:0/1506739013 learned_addr learned my addr 172.21.15.103:0/1506739013 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd687fff640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd688106f20 0x7fd68819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd6877fe640 1 -- 172.21.15.103:0/1506739013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6881027a0 msgr2=0x7fd688198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd6877fe640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6881027a0 0x7fd688198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd6877fe640 1 -- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd688106f20 msgr2=0x7fd68819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd6877fe640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd688106f20 0x7fd68819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.359+0000 7fd6877fe640 1 -- 172.21.15.103:0/1506739013 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68819d400 con 0x7fd688101df0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.360+0000 7fd6877fe640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd688101df0 0x7fd688198450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9f0 tx=0x7fd67800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.360+0000 7fd684ff9640 1 -- 172.21.15.103:0/1506739013 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800cd80 con 0x7fd688101df0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.360+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68819d6f0 con 0x7fd688101df0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.360+0000 7fd684ff9640 1 -- 172.21.15.103:0/1506739013 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6780035f0 con 0x7fd688101df0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.360+0000 7fd684ff9640 1 -- 172.21.15.103:0/1506739013 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678010670 con 0x7fd688101df0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.360+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6881a4fd0 con 0x7fd688101df0 2024-09-19T07:52:17.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 Processor -- start 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd684ff9640 1 -- 172.21.15.103:0/1506739013 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd678010810 con 0x7fd688101df0 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 -- start start 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55756d4582f0 con 0x55756d4f3b40 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55756d3ffcf0 con 0x55756d4fd370 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd68fef0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55756d4542b0 con 0x55756d4f25d0 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46890/0 (socket says 172.21.15.103:46890) 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd687fff640 1 -- 172.21.15.103:0/894504028 learned_addr learned my addr 172.21.15.103:0/894504028 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:17.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd686ffd640 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.363+0000 7fd6877fe640 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd687fff640 1 -- 172.21.15.103:0/894504028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 msgr2=0x55756d4f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd687fff640 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd687fff640 1 -- 172.21.15.103:0/894504028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 msgr2=0x55756d4ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd687fff640 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd687fff640 1 -- 172.21.15.103:0/894504028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756d3e43a0 con 0x55756d4f3b40 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd6877fe640 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd684ff9640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6700a33b0 0x7fd6700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd687fff640 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4f2090 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd680009f70 tx=0x7fd680010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e05566568f38d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd66dffb640 1 -- 172.21.15.103:0/894504028 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680016070 con 0x55756d4f3b40 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd684ff9640 1 -- 172.21.15.103:0/1506739013 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fd6780cffe0 con 0x7fd688101df0 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd66dffb640 1 -- 172.21.15.103:0/894504028 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680004d20 con 0x55756d4f3b40 2024-09-19T07:52:17.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd66dffb640 1 -- 172.21.15.103:0/894504028 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680012910 con 0x55756d4f3b40 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.364+0000 7fd6877fe640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6700a33b0 0x7fd6700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 -- 172.21.15.103:0/894504028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 msgr2=0x55756d4f2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd6877fe640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6700a33b0 0x7fd6700a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cd7d0 tx=0x7fd678003e30 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4f2090 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd680009f70 tx=0x7fd680010800 comp rx=0 tx=0).stop 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 -- 172.21.15.103:0/894504028 shutdown_connections 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 --2- 172.21.15.103:0/894504028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4f2090 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 -- 172.21.15.103:0/894504028 >> 172.21.15.103:0/894504028 conn(0x55756d4bb310 msgr2=0x55756d4eec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 -- 172.21.15.103:0/894504028 shutdown_connections 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.365+0000 7fd68fef0740 1 -- 172.21.15.103:0/894504028 wait complete. 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 -- start start 2024-09-19T07:52:17.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 Processor -- start 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55756d500070 con 0x55756d4f3b40 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55756d413060 con 0x55756d4fd370 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55756d5001f0 con 0x55756d4f25d0 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd68fef0740 10 client.0 _ll_register_callbacks cb 0x55756d1668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd686ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd687fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd686ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4fa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46896/0 (socket says 172.21.15.103:46896) 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd686ffd640 1 -- 172.21.15.103:0/3132804631 learned_addr learned my addr 172.21.15.103:0/3132804631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.366+0000 7fd687fff640 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd686ffd640 1 -- 172.21.15.103:0/3132804631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 msgr2=0x55756d4fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd686ffd640 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4fcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd686ffd640 1 -- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 msgr2=0x55756d4fa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd686ffd640 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4fa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd686ffd640 1 -- 172.21.15.103:0/3132804631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756d1c93c0 con 0x55756d4f3b40 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd687fff640 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd686ffd640 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4fa040 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f19dce3b9b1103 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd666ffd640 1 -- 172.21.15.103:0/3132804631 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c035040 con 0x55756d4f3b40 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd666ffd640 1 -- 172.21.15.103:0/3132804631 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fd67c004240 con 0x55756d4f3b40 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680004d20 con 0x55756d4f3b40 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd666ffd640 1 -- 172.21.15.103:0/3132804631 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd67c031a10 con 0x55756d4f3b40 2024-09-19T07:52:17.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd68fef0740 10 client.18303 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55756d514350 con 0x55756d4f3b40 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.367+0000 7fd666ffd640 1 -- 172.21.15.103:0/3132804631 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680004d20 con 0x55756d4f3b40 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd666ffd640 1 -- 172.21.15.103:0/3132804631 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fd67c004ac0 con 0x55756d4f3b40 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd68fef0740 10 client.18303 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd68fef0740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd68fef0740 20 client.18303 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d515810 0x55756d517c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd68fef0740 4 client.18303 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756d465860 con 0x55756d515810 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd687fff640 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d515810 0x55756d517c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd666ffd640 10 client.18303 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.368+0000 7fd687fff640 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d515810 0x55756d517c30 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd2c2c90c9ad3a3 server_cookie=6825e3a3fe726c7b in_seq=0 out_seq=0 2024-09-19T07:52:17.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd666ffd640 1 -- 172.21.15.103:0/3132804631 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55756d465860 con 0x55756d515810 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd666ffd640 10 client.18303 handle_command_reply: tid=0 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 client.18303 shutdown 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 msgr2=0x55756d4fa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4fa040 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c0317e0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 shutdown_connections 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d515810 0x55756d517c30 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d4fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d4fa040 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3132804631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d4fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 >> 172.21.15.103:0/3132804631 conn(0x55756d4bb310 msgr2=0x55756d4f4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd666ffd640 0 client.18303 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:17.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 shutdown_connections 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 1 -- 172.21.15.103:0/3132804631 wait complete. 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.369+0000 7fd68fef0740 20 client.18303 trim_cache size 0 max 16384 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 Processor -- start 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 -- start start 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d511ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4f3b40 0x55756d4f9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4fd370 0x55756d51d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ab7b0 con 0x55756d4fd370 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ad320 con 0x55756d4f3b40 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd68fef0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881abe80 con 0x55756d4f25d0 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4fd370 0x55756d51d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd687fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4fd370 0x55756d51d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46912/0 (socket says 172.21.15.103:46912) 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd687fff640 1 -- 172.21.15.103:0/4038449374 learned_addr learned my addr 172.21.15.103:0/4038449374 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd687fff640 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d511ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.382+0000 7fd686ffd640 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4f3b40 0x55756d4f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd687fff640 1 -- 172.21.15.103:0/4038449374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 msgr2=0x55756d511ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd687fff640 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d511ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd687fff640 1 -- 172.21.15.103:0/4038449374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4f3b40 msgr2=0x55756d4f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd687fff640 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4f3b40 0x55756d4f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd687fff640 1 -- 172.21.15.103:0/4038449374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756d4ef960 con 0x55756d4fd370 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd687fff640 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4fd370 0x55756d51d4e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd680012560 tx=0x7fd68001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2e801e8702fea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd66d7fa640 1 -- 172.21.15.103:0/4038449374 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680016040 con 0x55756d4fd370 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd66d7fa640 1 -- 172.21.15.103:0/4038449374 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680012d90 con 0x55756d4fd370 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd66d7fa640 1 -- 172.21.15.103:0/4038449374 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680023040 con 0x55756d4fd370 2024-09-19T07:52:17.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd68fef0740 1 -- 172.21.15.103:0/4038449374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4fd370 msgr2=0x55756d51d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.383+0000 7fd68fef0740 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4fd370 0x55756d51d4e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd680012560 tx=0x7fd68001c040 comp rx=0 tx=0).stop 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- 172.21.15.103:0/4038449374 shutdown_connections 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4fd370 0x55756d51d4e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4f3b40 0x55756d4f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 --2- 172.21.15.103:0/4038449374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d511ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- 172.21.15.103:0/4038449374 >> 172.21.15.103:0/4038449374 conn(0x55756d4bb310 msgr2=0x55756d4f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- 172.21.15.103:0/4038449374 shutdown_connections 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- 172.21.15.103:0/4038449374 wait complete. 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- start start 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 Processor -- start 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d509b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d50a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d50a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881acc20 con 0x55756d4f3b40 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ac550 con 0x55756d4fd370 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881a97b0 con 0x55756d4f25d0 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.384+0000 7fd68fef0740 10 client.0 _ll_register_callbacks cb 0x7fd67c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d50a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d50a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46920/0 (socket says 172.21.15.103:46920) 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 -- 172.21.15.103:0/3056675857 learned_addr learned my addr 172.21.15.103:0/3056675857 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd687fff640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d50a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 -- 172.21.15.103:0/3056675857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 msgr2=0x55756d509b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd687fff640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d509b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d509b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 -- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 msgr2=0x55756d50a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:17.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d50a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 -- 172.21.15.103:0/3056675857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756d4fafe0 con 0x55756d4f3b40 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd687fff640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d509b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd686ffd640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d50a090 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd67c005ab0 tx=0x7fd67c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d884dd0493a9ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd666ffd640 1 -- 172.21.15.103:0/3056675857 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c05b060 con 0x55756d4f3b40 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd666ffd640 1 -- 172.21.15.103:0/3056675857 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fd67c0048c0 con 0x55756d4f3b40 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680012d90 con 0x55756d4f3b40 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd666ffd640 1 -- 172.21.15.103:0/3056675857 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd67c035050 con 0x55756d4f3b40 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd68fef0740 10 client.18309 fetch_fsmap learned FSMap version 51 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.385+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55756d5806d0 con 0x55756d4f3b40 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd666ffd640 1 -- 172.21.15.103:0/3056675857 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c0054c0 con 0x55756d4f3b40 2024-09-19T07:52:17.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd666ffd640 1 -- 172.21.15.103:0/3056675857 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8366+0+0 (secure 0 0 0) 0x7fd67c055070 con 0x55756d4f3b40 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd68fef0740 10 client.18309 fetch_fsmap finished waiting for FSMap version 51 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd68fef0740 10 client.18309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd68fef0740 20 client.18309 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d580e50 0x55756d583270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd68fef0740 4 client.18309 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd680012d90 con 0x55756d580e50 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd687fff640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d580e50 0x55756d583270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd666ffd640 10 client.18309 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.386+0000 7fd687fff640 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d580e50 0x55756d583270 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fa0d54371ce3e2 server_cookie=b7c933a9959b7534 in_seq=0 out_seq=0 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd666ffd640 1 -- 172.21.15.103:0/3056675857 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd680012d90 con 0x55756d580e50 2024-09-19T07:52:17.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd666ffd640 10 client.18309 handle_command_reply: tid=0 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 client.18309 shutdown 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 msgr2=0x55756d50a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d50a090 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd67c005ab0 tx=0x7fd67c005fd0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 shutdown_connections 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55756d580e50 0x55756d583270 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55756d4fd370 0x55756d50a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55756d4f3b40 0x55756d50a090 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd666ffd640 0 client.18309 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:17.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 --2- 172.21.15.103:0/3056675857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55756d4f25d0 0x55756d509b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 >> 172.21.15.103:0/3056675857 conn(0x55756d4bb310 msgr2=0x55756d4f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 shutdown_connections 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 1 -- 172.21.15.103:0/3056675857 wait complete. 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68fef0740 20 client.18309 trim_cache size 0 max 16384 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6700a33b0 msgr2=0x7fd6700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68db17640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6700a33b0 0x7fd6700a5870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd6780cd7d0 tx=0x7fd678003e30 comp rx=0 tx=0).stop 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd688101df0 msgr2=0x7fd688198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.387+0000 7fd68db17640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd688101df0 0x7fd688198450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9f0 tx=0x7fd67800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 shutdown_connections 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd6700a33b0 0x7fd6700a5870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd688106f20 0x7fd68819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6881027a0 0x7fd688198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 --2- 172.21.15.103:0/1506739013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd688101df0 0x7fd688198450 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 >> 172.21.15.103:0/1506739013 conn(0x7fd6880fd8f0 msgr2=0x7fd6880fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 shutdown_connections 2024-09-19T07:52:17.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:16.388+0000 7fd68db17640 1 -- 172.21.15.103:0/1506739013 wait complete. 2024-09-19T07:52:18.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:18 smithi103 ceph-mon[31201]: pgmap v448: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 812 op/s 2024-09-19T07:52:18.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:18 smithi134 ceph-mon[35414]: pgmap v448: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 812 op/s 2024-09-19T07:52:18.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:18 smithi174 ceph-mon[35370]: pgmap v448: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 812 op/s 2024-09-19T07:52:19.471 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:52:19.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:52:19.852 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:20.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.408+0000 7f59655b2640 1 -- 172.21.15.103:0/1638904785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102d30 msgr2=0x7f5960103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.408+0000 7f59655b2640 1 --2- 172.21.15.103:0/1638904785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102d30 0x7f5960103110 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5950009930 tx=0x7f595002a420 comp rx=0 tx=0).stop 2024-09-19T07:52:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.408+0000 7f59655b2640 1 -- 172.21.15.103:0/1638904785 shutdown_connections 2024-09-19T07:52:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.408+0000 7f59655b2640 1 --2- 172.21.15.103:0/1638904785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59601082a0 0x7f596010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.408+0000 7f59655b2640 1 --2- 172.21.15.103:0/1638904785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59601036e0 0x7f5960107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.408+0000 7f59655b2640 1 --2- 172.21.15.103:0/1638904785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102d30 0x7f5960103110 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.408+0000 7f59655b2640 1 -- 172.21.15.103:0/1638904785 >> 172.21.15.103:0/1638904785 conn(0x7f59600fc7d0 msgr2=0x7f59600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.410+0000 7f59655b2640 1 -- 172.21.15.103:0/1638904785 shutdown_connections 2024-09-19T07:52:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.410+0000 7f59655b2640 1 -- 172.21.15.103:0/1638904785 wait complete. 2024-09-19T07:52:20.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.411+0000 7f59655b2640 1 Processor -- start 2024-09-19T07:52:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.411+0000 7f59655b2640 1 -- start start 2024-09-19T07:52:20.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f59655b2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102d30 0x7f59601a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f59655b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59601036e0 0x7f59601a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f59655b2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59601082a0 0x7f596010ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f59655b2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59601120a0 con 0x7f59601082a0 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f59655b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5960111f20 con 0x7f59601036e0 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f59655b2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5960112220 con 0x7f5960102d30 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59601082a0 0x7f596010ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59601082a0 0x7f596010ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47012/0 (socket says 172.21.15.103:47012) 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 -- 172.21.15.103:0/1204423591 learned_addr learned my addr 172.21.15.103:0/1204423591 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:20.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f59567fc640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59601036e0 0x7f59601a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595effd640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102d30 0x7f59601a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 -- 172.21.15.103:0/1204423591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102d30 msgr2=0x7f59601a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102d30 0x7f59601a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 -- 172.21.15.103:0/1204423591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59601036e0 msgr2=0x7f59601a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59601036e0 0x7f59601a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595f7fe640 1 -- 172.21.15.103:0/1204423591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596010e4e0 con 0x7f59601082a0 2024-09-19T07:52:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.412+0000 7f595effd640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102d30 0x7f59601a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.413+0000 7f595f7fe640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59601082a0 0x7f596010ddb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f595800eaa0 tx=0x7f595800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.414+0000 7f595cff9640 1 -- 172.21.15.103:0/1204423591 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595800cdb0 con 0x7f59601082a0 2024-09-19T07:52:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.414+0000 7f595cff9640 1 -- 172.21.15.103:0/1204423591 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5958005020 con 0x7f59601082a0 2024-09-19T07:52:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.414+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596010e7d0 con 0x7f59601082a0 2024-09-19T07:52:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.414+0000 7f595cff9640 1 -- 172.21.15.103:0/1204423591 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59580116b0 con 0x7f59601082a0 2024-09-19T07:52:20.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.414+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59601ab6c0 con 0x7f59601082a0 2024-09-19T07:52:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.418+0000 7f596798b740 1 Processor -- start 2024-09-19T07:52:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.418+0000 7f595cff9640 1 -- 172.21.15.103:0/1204423591 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5958004a40 con 0x7f59601082a0 2024-09-19T07:52:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.418+0000 7f596798b740 1 -- start start 2024-09-19T07:52:20.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f596798b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f596798b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f596798b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f596798b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555d2610280 con 0x5555d26ab480 2024-09-19T07:52:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f596798b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5555d25b7b10 con 0x5555d26aab60 2024-09-19T07:52:20.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f596798b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555d260c240 con 0x5555d26b5d40 2024-09-19T07:52:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f595effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f595effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47028/0 (socket says 172.21.15.103:47028) 2024-09-19T07:52:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.419+0000 7f595effd640 1 -- 172.21.15.103:0/3621762117 learned_addr learned my addr 172.21.15.103:0/3621762117 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f59567fc640 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f59567fc640 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f595cff9640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f594c0a3500 0x7f594c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f595cff9640 1 -- 172.21.15.103:0/1204423591 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7f59580cfdd0 con 0x7f59601082a0 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f59567fc640 1 -- 172.21.15.103:0/3621762117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 msgr2=0x5555d26b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f595f7fe640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f594c0a3500 0x7f594c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f59567fc640 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26b8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f59567fc640 1 -- 172.21.15.103:0/3621762117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 msgr2=0x5555d26b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f59567fc640 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.420+0000 7f59567fc640 1 -- 172.21.15.103:0/3621762117 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d25c41f0 con 0x5555d26aab60 2024-09-19T07:52:20.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.421+0000 7f59567fc640 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26aaf40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5948009730 tx=0x7f59480096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378d4b67dc6ec058 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.421+0000 7f595f7fe640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f594c0a3500 0x7f594c0a59c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59580a4780 tx=0x7f595809bf00 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.421+0000 7f59557fa640 1 -- 172.21.15.103:0/3621762117 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948004050 con 0x5555d26aab60 2024-09-19T07:52:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.421+0000 7f59557fa640 1 -- 172.21.15.103:0/3621762117 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5948005500 con 0x5555d26aab60 2024-09-19T07:52:20.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.421+0000 7f59557fa640 1 -- 172.21.15.103:0/3621762117 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948004d90 con 0x5555d26aab60 2024-09-19T07:52:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.422+0000 7f596798b740 1 -- 172.21.15.103:0/3621762117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 msgr2=0x5555d26aaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.422+0000 7f596798b740 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26aaf40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5948009730 tx=0x7f59480096c0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.424+0000 7f596798b740 1 -- 172.21.15.103:0/3621762117 shutdown_connections 2024-09-19T07:52:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.424+0000 7f596798b740 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.424+0000 7f596798b740 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.424+0000 7f596798b740 1 --2- 172.21.15.103:0/3621762117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26aaf40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.424+0000 7f596798b740 1 -- 172.21.15.103:0/3621762117 >> 172.21.15.103:0/3621762117 conn(0x5555d26a70f0 msgr2=0x5555d26a74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:20.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.425+0000 7f596798b740 1 -- 172.21.15.103:0/3621762117 shutdown_connections 2024-09-19T07:52:20.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.425+0000 7f596798b740 1 -- 172.21.15.103:0/3621762117 wait complete. 2024-09-19T07:52:20.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.425+0000 7f596798b740 1 -- start start 2024-09-19T07:52:20.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.425+0000 7f596798b740 1 Processor -- start 2024-09-19T07:52:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f596798b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f596798b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f596798b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26b5d40 0x5555d26b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56176/0 (socket says 172.21.15.103:56176) 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 -- 172.21.15.103:0/4285240911 learned_addr learned my addr 172.21.15.103:0/4285240911 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555d26b8910 con 0x5555d26b5d40 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f595effd640 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5555d25caf10 con 0x5555d26aab60 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555d26b8a90 con 0x5555d26ab480 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26b5d40 0x5555d26b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f596798b740 10 client.0 _ll_register_callbacks cb 0x5555d2593a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 -- 172.21.15.103:0/4285240911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 msgr2=0x5555d26b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 -- 172.21.15.103:0/4285240911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26b5d40 msgr2=0x5555d26b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26b5d40 0x5555d26b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 -- 172.21.15.103:0/4285240911 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d23816a0 con 0x5555d26aab60 2024-09-19T07:52:20.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.426+0000 7f59567fc640 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5948004810 tx=0x7f5948004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de10693c96a0315 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f59467fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:52:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f59467fc640 1 -- 172.21.15.103:0/4285240911 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948004bd0 con 0x5555d26aab60 2024-09-19T07:52:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5948005500 con 0x5555d26aab60 2024-09-19T07:52:20.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f59467fc640 1 -- 172.21.15.103:0/4285240911 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7f5948004d70 con 0x5555d26aab60 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f59467fc640 1 -- 172.21.15.103:0/4285240911 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f594800f790 con 0x5555d26aab60 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f596798b740 10 client.27623 fetch_fsmap learned FSMap version 52 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5555d26c4590 con 0x5555d26aab60 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f59467fc640 1 -- 172.21.15.103:0/4285240911 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5948005500 con 0x5555d26aab60 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.427+0000 7f59467fc640 1 -- 172.21.15.103:0/4285240911 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7f594801f9e0 con 0x5555d26aab60 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.428+0000 7f596798b740 10 client.27623 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.428+0000 7f596798b740 10 client.27623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:20.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.428+0000 7f596798b740 20 client.27623 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.428+0000 7f596798b740 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d26c58e0 0x5555d26c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.428+0000 7f596798b740 4 client.27623 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.428+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d261d7f0 con 0x5555d26c58e0 2024-09-19T07:52:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.428+0000 7f595effd640 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d26c58e0 0x5555d26c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.429+0000 7f595effd640 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d26c58e0 0x5555d26c7cd0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2e25402e68293f server_cookie=43d0a91760dc88f in_seq=0 out_seq=0 2024-09-19T07:52:20.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.429+0000 7f59467fc640 10 client.27623 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.430+0000 7f59467fc640 1 -- 172.21.15.103:0/4285240911 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555d261d7f0 con 0x5555d26c58e0 2024-09-19T07:52:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.430+0000 7f59467fc640 10 client.27623 handle_command_reply: tid=0 2024-09-19T07:52:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.430+0000 7f596798b740 1 client.27623 shutdown 2024-09-19T07:52:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 msgr2=0x5555d26b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5948004810 tx=0x7f5948004970 comp rx=0 tx=0).stop 2024-09-19T07:52:20.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 shutdown_connections 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d26c58e0 0x5555d26c7cd0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26b5d40 0x5555d26b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 --2- 172.21.15.103:0/4285240911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 >> 172.21.15.103:0/4285240911 conn(0x5555d26a70f0 msgr2=0x5555d26a6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f59467fc640 0 client.27623 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 shutdown_connections 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 1 -- 172.21.15.103:0/4285240911 wait complete. 2024-09-19T07:52:20.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.431+0000 7f596798b740 20 client.27623 trim_cache size 0 max 16384 2024-09-19T07:52:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.456+0000 7f596798b740 1 Processor -- start 2024-09-19T07:52:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f596798b740 1 -- start start 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f596798b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f596798b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f596798b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f596798b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f594000e7e0 con 0x5555d26ab480 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f596798b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5940010320 con 0x5555d26aab60 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f596798b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f594000eeb0 con 0x5555d26b5d40 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f59567fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f59567fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56190/0 (socket says 172.21.15.103:56190) 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f59567fc640 1 -- 172.21.15.103:0/1992508696 learned_addr learned my addr 172.21.15.103:0/1992508696 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.457+0000 7f59567fc640 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.458+0000 7f59567fc640 1 -- 172.21.15.103:0/1992508696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 msgr2=0x5555d26cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.458+0000 7f59567fc640 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.458+0000 7f59567fc640 1 -- 172.21.15.103:0/1992508696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 msgr2=0x5555d26cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.458+0000 7f59567fc640 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.458+0000 7f59567fc640 1 -- 172.21.15.103:0/1992508696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d26b52a0 con 0x5555d26aab60 2024-09-19T07:52:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.459+0000 7f59567fc640 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5948009250 tx=0x7f5948001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85b70fd5543d032 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.459+0000 7f5954ff9640 1 -- 172.21.15.103:0/1992508696 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594803a070 con 0x5555d26aab60 2024-09-19T07:52:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.459+0000 7f5954ff9640 1 -- 172.21.15.103:0/1992508696 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f594803b040 con 0x5555d26aab60 2024-09-19T07:52:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.459+0000 7f5954ff9640 1 -- 172.21.15.103:0/1992508696 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948004e30 con 0x5555d26aab60 2024-09-19T07:52:20.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 -- 172.21.15.103:0/1992508696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 msgr2=0x5555d26b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5948009250 tx=0x7f5948001e50 comp rx=0 tx=0).stop 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 -- 172.21.15.103:0/1992508696 shutdown_connections 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26b5d40 0x5555d26cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26ab480 0x5555d26cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 --2- 172.21.15.103:0/1992508696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26aab60 0x5555d26b2580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 -- 172.21.15.103:0/1992508696 >> 172.21.15.103:0/1992508696 conn(0x5555d26a70f0 msgr2=0x5555d26a6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 -- 172.21.15.103:0/1992508696 shutdown_connections 2024-09-19T07:52:20.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.460+0000 7f596798b740 1 -- 172.21.15.103:0/1992508696 wait complete. 2024-09-19T07:52:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.461+0000 7f596798b740 1 -- start start 2024-09-19T07:52:20.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.461+0000 7f596798b740 1 Processor -- start 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.461+0000 7f596798b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26aab60 0x5555d26bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f596798b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26be280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f596798b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26b5d40 0x5555d26bad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f596798b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f594000fc50 con 0x5555d26aab60 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f596798b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f594000f580 con 0x5555d26b5d40 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f596798b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f594000c7e0 con 0x5555d26ab480 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26aab60 0x5555d26bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f596798b740 10 client.0 _ll_register_callbacks cb 0x7f5948005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26aab60 0x5555d26bdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47056/0 (socket says 172.21.15.103:47056) 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 -- 172.21.15.103:0/1468564389 learned_addr learned my addr 172.21.15.103:0/1468564389 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26b5d40 0x5555d26bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f595effd640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26be280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 -- 172.21.15.103:0/1468564389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 msgr2=0x5555d26be280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26be280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 -- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26b5d40 msgr2=0x5555d26bad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:20.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26b5d40 0x5555d26bad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 -- 172.21.15.103:0/1468564389 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d26ab2b0 con 0x5555d26aab60 2024-09-19T07:52:20.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f595effd640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26be280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:20.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59567fc640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26aab60 0x5555d26bdd40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5948013040 tx=0x7f59480096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8514c8601a8d5fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.462+0000 7f59467fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.463+0000 7f59467fc640 1 -- 172.21.15.103:0/1468564389 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594803a070 con 0x5555d26aab60 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.463+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f594803b040 con 0x5555d26aab60 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.463+0000 7f59467fc640 1 -- 172.21.15.103:0/1468564389 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7f5948011040 con 0x5555d26aab60 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.463+0000 7f59467fc640 1 -- 172.21.15.103:0/1468564389 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5948023070 con 0x5555d26aab60 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.463+0000 7f596798b740 10 client.18324 fetch_fsmap learned FSMap version 52 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.463+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5555d2730910 con 0x5555d26aab60 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.463+0000 7f59467fc640 1 -- 172.21.15.103:0/1468564389 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f594803b040 con 0x5555d26aab60 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f59467fc640 1 -- 172.21.15.103:0/1468564389 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7f59480308d0 con 0x5555d26aab60 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f596798b740 10 client.18324 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f596798b740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f596798b740 20 client.18324 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f596798b740 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d2730f70 0x5555d2733390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:20.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f596798b740 4 client.18324 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:52:20.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555d2617d00 con 0x5555d2730f70 2024-09-19T07:52:20.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 825.0 s, 1550.1 tps, lat 65.465 ms stddev 101.008 2024-09-19T07:52:20.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.464+0000 7f595effd640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d2730f70 0x5555d2733390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:20.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.465+0000 7f595effd640 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d2730f70 0x5555d2733390 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7800462cd52bf server_cookie=2c69eaac89b4cc38 in_seq=0 out_seq=0 2024-09-19T07:52:20.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.465+0000 7f59467fc640 10 client.18324 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:20.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.466+0000 7f59467fc640 1 -- 172.21.15.103:0/1468564389 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555d2617d00 con 0x5555d2730f70 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.466+0000 7f59467fc640 10 client.18324 handle_command_reply: tid=0 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.466+0000 7f596798b740 1 client.18324 shutdown 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.466+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26aab60 msgr2=0x5555d26bdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26aab60 0x5555d26bdd40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5948013040 tx=0x7f59480096c0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 shutdown_connections 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5555d2730f70 0x5555d2733390 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555d26b5d40 0x5555d26bad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f59467fc640 0 client.18324 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d26ab480 0x5555d26be280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 --2- 172.21.15.103:0/1468564389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555d26aab60 0x5555d26bdd40 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 >> 172.21.15.103:0/1468564389 conn(0x5555d26a70f0 msgr2=0x5555d26a5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 shutdown_connections 2024-09-19T07:52:20.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 1 -- 172.21.15.103:0/1468564389 wait complete. 2024-09-19T07:52:20.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.467+0000 7f596798b740 20 client.18324 trim_cache size 0 max 16384 2024-09-19T07:52:20.470 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:52:20.470 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:52:20.470 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "208076d4-2ea0-4e1a-8bec-367fb8d8fe3c", 2024-09-19T07:52:20.470 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:52:20.470 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:52:20.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f594c0a3500 msgr2=0x7f594c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f594c0a3500 0x7f594c0a59c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59580a4780 tx=0x7f595809bf00 comp rx=0 tx=0).stop 2024-09-19T07:52:20.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59601082a0 msgr2=0x7f596010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:20.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59601082a0 0x7f596010ddb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f595800eaa0 tx=0x7f595800ef60 comp rx=0 tx=0).stop 2024-09-19T07:52:20.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 shutdown_connections 2024-09-19T07:52:20.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f594c0a3500 0x7f594c0a59c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59601082a0 0x7f596010ddb0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59601036e0 0x7f59601a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 --2- 172.21.15.103:0/1204423591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102d30 0x7f59601a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 >> 172.21.15.103:0/1204423591 conn(0x7f59600fc7d0 msgr2=0x7f5960100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 shutdown_connections 2024-09-19T07:52:20.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:20.468+0000 7f59655b2640 1 -- 172.21.15.103:0/1204423591 wait complete. 2024-09-19T07:52:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:20 smithi103 ceph-mon[31201]: pgmap v449: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 812 op/s 2024-09-19T07:52:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:20 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:52:20.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:20 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:52:20.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:20 smithi134 ceph-mon[35414]: pgmap v449: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 812 op/s 2024-09-19T07:52:20.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:20 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:52:20.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:20 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:52:20.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:20 smithi174 ceph-mon[35370]: pgmap v449: 97 pgs: 97 active+clean; 3.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 12 MiB/s wr, 812 op/s 2024-09-19T07:52:20.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:20 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:52:20.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:20 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:52:21.299 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:52:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:52:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:21 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.727 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:52:21.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:21 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:52:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:21.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:21 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:22.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:22 smithi103 ceph-mon[31201]: pgmap v450: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:22.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:22 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:22.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:22 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:52:22.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:22 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:52:22.497 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:22 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 -- 172.21.15.103:0/3952072664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf41036a0 msgr2=0x7ffaf410bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 --2- 172.21.15.103:0/3952072664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf41036a0 0x7ffaf410bb90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffae400dc70 tx=0x7ffae4032940 comp rx=0 tx=0).stop 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 -- 172.21.15.103:0/3952072664 shutdown_connections 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 --2- 172.21.15.103:0/3952072664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf410c160 0x7ffaf410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 --2- 172.21.15.103:0/3952072664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf41036a0 0x7ffaf410bb90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 --2- 172.21.15.103:0/3952072664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffaf4102d80 0x7ffaf4103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 -- 172.21.15.103:0/3952072664 >> 172.21.15.103:0/3952072664 conn(0x7ffaf40fc7a0 msgr2=0x7ffaf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 -- 172.21.15.103:0/3952072664 shutdown_connections 2024-09-19T07:52:22.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.612+0000 7ffafb565640 1 -- 172.21.15.103:0/3952072664 wait complete. 2024-09-19T07:52:22.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.613+0000 7ffafb565640 1 Processor -- start 2024-09-19T07:52:22.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.613+0000 7ffafb565640 1 -- start start 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.613+0000 7ffafb565640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf4102d80 0x7ffaf4196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.613+0000 7ffafb565640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffaf41036a0 0x7ffaf41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.613+0000 7ffafb565640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf410c160 0x7ffaf419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffafb565640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf410fcf0 con 0x7ffaf41036a0 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffafb565640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf410fb70 con 0x7ffaf4102d80 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffafb565640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf410fe70 con 0x7ffaf410c160 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf8b00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffaf41036a0 0x7ffaf41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf8b00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffaf41036a0 0x7ffaf41966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47060/0 (socket says 172.21.15.103:47060) 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9b02640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf410c160 0x7ffaf419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf8b00640 1 -- 172.21.15.103:0/2905732911 learned_addr learned my addr 172.21.15.103:0/2905732911 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9301640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf4102d80 0x7ffaf4196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9b02640 1 -- 172.21.15.103:0/2905732911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf4102d80 msgr2=0x7ffaf4196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf4102d80 0x7ffaf4196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9b02640 1 -- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffaf41036a0 msgr2=0x7ffaf41966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffaf41036a0 0x7ffaf41966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9b02640 1 -- 172.21.15.103:0/2905732911 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf419b140 con 0x7ffaf410c160 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf410c160 0x7ffaf419aa60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009fd0 tx=0x7ffaec00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.614+0000 7ffaea7fc640 1 -- 172.21.15.103:0/2905732911 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00ce20 con 0x7ffaf410c160 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.615+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf419b430 con 0x7ffaf410c160 2024-09-19T07:52:22.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.615+0000 7ffaea7fc640 1 -- 172.21.15.103:0/2905732911 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaec005020 con 0x7ffaf410c160 2024-09-19T07:52:22.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.615+0000 7ffaea7fc640 1 -- 172.21.15.103:0/2905732911 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec010720 con 0x7ffaf410c160 2024-09-19T07:52:22.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.615+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf41a2d10 con 0x7ffaf410c160 2024-09-19T07:52:22.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.618+0000 7ffaea7fc640 1 -- 172.21.15.103:0/2905732911 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffaec005540 con 0x7ffaf410c160 2024-09-19T07:52:22.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 Processor -- start 2024-09-19T07:52:22.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 -- start start 2024-09-19T07:52:22.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c583880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c583dc0 0x55ad6c58e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c58e680 0x55ad6c590a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad6c4e92e0 con 0x55ad6c583dc0 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ad6c490ce0 con 0x55ad6c5834a0 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffafd93e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad6c4e52a0 con 0x55ad6c58e680 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffaf8b00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c583880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffaf9301640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c583dc0 0x55ad6c58e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffaf8b00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c583880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56226/0 (socket says 172.21.15.103:56226) 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffaf8b00640 1 -- 172.21.15.103:0/3645011907 learned_addr learned my addr 172.21.15.103:0/3645011907 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.619+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c58e680 0x55ad6c590a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf8b00640 1 -- 172.21.15.103:0/3645011907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c58e680 msgr2=0x55ad6c590a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c58e680 0x55ad6c590a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf8b00640 1 -- 172.21.15.103:0/3645011907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c583dc0 msgr2=0x55ad6c58e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c583dc0 0x55ad6c58e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf8b00640 1 -- 172.21.15.103:0/3645011907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6c475440 con 0x55ad6c5834a0 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf9301640 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c583dc0 0x55ad6c58e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaea7fc640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffae00a3430 0x7ffae00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf9301640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffae00a3430 0x7ffae00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaea7fc640 1 -- 172.21.15.103:0/2905732911 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7ffaec0d02f0 con 0x7ffaf410c160 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c583880 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55ad6c580db0 tx=0x7ffae4032940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=396bf3e4d7036d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffadf7fe640 1 -- 172.21.15.103:0/3645011907 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4038610 con 0x55ad6c5834a0 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.620+0000 7ffaf9301640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffae00a3430 0x7ffae00a58f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55ad6c580890 tx=0x7ffaf000b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffadf7fe640 1 -- 172.21.15.103:0/3645011907 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae403d070 con 0x55ad6c5834a0 2024-09-19T07:52:22.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffadf7fe640 1 -- 172.21.15.103:0/3645011907 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4004920 con 0x55ad6c5834a0 2024-09-19T07:52:22.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 -- 172.21.15.103:0/3645011907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 msgr2=0x55ad6c583880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c583880 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55ad6c580db0 tx=0x7ffae4032940 comp rx=0 tx=0).stop 2024-09-19T07:52:22.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 -- 172.21.15.103:0/3645011907 shutdown_connections 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c58e680 0x55ad6c590a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c583dc0 0x55ad6c58e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 --2- 172.21.15.103:0/3645011907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c583880 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 -- 172.21.15.103:0/3645011907 >> 172.21.15.103:0/3645011907 conn(0x55ad6c57fa30 msgr2=0x55ad6c57fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 -- 172.21.15.103:0/3645011907 shutdown_connections 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.621+0000 7ffafd93e740 1 -- 172.21.15.103:0/3645011907 wait complete. 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.622+0000 7ffafd93e740 1 -- start start 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.622+0000 7ffafd93e740 1 Processor -- start 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c58e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c58b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c58e680 0x55ad6c58b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffafd93e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad6c591250 con 0x55ad6c58e680 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffafd93e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ad6c4a4050 con 0x55ad6c5834a0 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffafd93e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad6c5913d0 con 0x55ad6c583dc0 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffafd93e740 10 client.0 _ll_register_callbacks cb 0x55ad6c4e4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c58e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9b02640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c58e680 0x55ad6c58b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c58e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:56242/0 (socket says 172.21.15.103:56242) 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 -- 172.21.15.103:0/158631899 learned_addr learned my addr 172.21.15.103:0/158631899 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c58b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 -- 172.21.15.103:0/158631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 msgr2=0x55ad6c58b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c58b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 -- 172.21.15.103:0/158631899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c58e680 msgr2=0x55ad6c58b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c58e680 0x55ad6c58b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 -- 172.21.15.103:0/158631899 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6c25a3c0 con 0x55ad6c5834a0 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c58b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffaf9301640 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c58e000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b5e0 tx=0x7ffaf0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5080190fd1dbc7cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.623+0000 7ffadcff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.625+0000 7ffadcff9640 1 -- 172.21.15.103:0/158631899 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf002c060 con 0x55ad6c5834a0 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffadcff9640 1 -- 172.21.15.103:0/158631899 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7ffaf0017030 con 0x55ad6c5834a0 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae403d070 con 0x55ad6c5834a0 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffadcff9640 1 -- 172.21.15.103:0/158631899 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ffaf0030420 con 0x55ad6c5834a0 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 10 client.27641 fetch_fsmap learned FSMap version 52 2024-09-19T07:52:22.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ad6c59cca0 con 0x55ad6c5834a0 2024-09-19T07:52:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffadcff9640 1 -- 172.21.15.103:0/158631899 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae403d070 con 0x55ad6c5834a0 2024-09-19T07:52:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffadcff9640 1 -- 172.21.15.103:0/158631899 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7ffaf0011560 con 0x55ad6c5834a0 2024-09-19T07:52:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 10 client.27641 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:52:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 10 client.27641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 20 client.27641 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:22.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c59e160 0x55ad6c5a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 4 client.27641 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.626+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad6c4f6850 con 0x55ad6c59e160 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.627+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c59e160 0x55ad6c5a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.627+0000 7ffadcff9640 10 client.27641 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.627+0000 7ffaf8b00640 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c59e160 0x55ad6c5a0600 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792da2c0617b51b0 server_cookie=830a2a7694ad3bb6 in_seq=0 out_seq=0 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffadcff9640 1 -- 172.21.15.103:0/158631899 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad6c4f6850 con 0x55ad6c59e160 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffadcff9640 10 client.27641 handle_command_reply: tid=0 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 client.27641 shutdown 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 msgr2=0x55ad6c58e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c58e000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b5e0 tx=0x7ffaf0004730 comp rx=0 tx=0).stop 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 shutdown_connections 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c59e160 0x55ad6c5a0600 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c58e680 0x55ad6c58b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c58b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffadcff9640 0 client.27641 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 --2- 172.21.15.103:0/158631899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c5834a0 0x55ad6c58e000 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 >> 172.21.15.103:0/158631899 conn(0x55ad6c57fa30 msgr2=0x55ad6c585760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 shutdown_connections 2024-09-19T07:52:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 1 -- 172.21.15.103:0/158631899 wait complete. 2024-09-19T07:52:22.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.628+0000 7ffafd93e740 20 client.27641 trim_cache size 0 max 16384 2024-09-19T07:52:22.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.661+0000 7ffafd93e740 1 Processor -- start 2024-09-19T07:52:22.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.663+0000 7ffafd93e740 1 -- start start 2024-09-19T07:52:22.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.663+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c589b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.663+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c5a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c589b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c589b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47098/0 (socket says 172.21.15.103:47098) 2024-09-19T07:52:22.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 -- 172.21.15.103:0/2907385919 learned_addr learned my addr 172.21.15.103:0/2907385919 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffafd93e740 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c5a6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf41ab670 con 0x55ad6c5834a0 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf41ad1b0 con 0x55ad6c58e680 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf41abd40 con 0x55ad6c583dc0 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c5a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 -- 172.21.15.103:0/2907385919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 msgr2=0x55ad6c5a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c5a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 -- 172.21.15.103:0/2907385919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 msgr2=0x55ad6c5a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c5a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.664+0000 7ffaf9301640 1 -- 172.21.15.103:0/2907385919 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6c58a260 con 0x55ad6c5834a0 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.665+0000 7ffaf9301640 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c589b80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0012010 tx=0x7ffaf000b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9793f1e685c49e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:22.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.669+0000 7ffadeffd640 1 -- 172.21.15.103:0/2907385919 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0011dc0 con 0x55ad6c5834a0 2024-09-19T07:52:22.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.669+0000 7ffadeffd640 1 -- 172.21.15.103:0/2907385919 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf0011440 con 0x55ad6c5834a0 2024-09-19T07:52:22.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.670+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 msgr2=0x55ad6c589b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.670+0000 7ffafd93e740 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c589b80 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0012010 tx=0x7ffaf000b740 comp rx=0 tx=0).stop 2024-09-19T07:52:22.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.671+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 shutdown_connections 2024-09-19T07:52:22.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.671+0000 7ffafd93e740 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c5a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.671+0000 7ffafd93e740 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c5a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.671+0000 7ffafd93e740 1 --2- 172.21.15.103:0/2907385919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c589b80 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.671+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 >> 172.21.15.103:0/2907385919 conn(0x55ad6c57fa30 msgr2=0x55ad6c57ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:22.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.674+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 shutdown_connections 2024-09-19T07:52:22.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.674+0000 7ffafd93e740 1 -- 172.21.15.103:0/2907385919 wait complete. 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.674+0000 7ffafd93e740 1 -- start start 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.674+0000 7ffafd93e740 1 Processor -- start 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c5945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c594b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffafd93e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c595040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffafd93e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf41acae0 con 0x55ad6c5834a0 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffafd93e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf41ac410 con 0x55ad6c58e680 2024-09-19T07:52:22.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffafd93e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf41a9670 con 0x55ad6c583dc0 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffafd93e740 10 client.0 _ll_register_callbacks cb 0x7ffaf000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffaf8b00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c594b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffaf8b00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c594b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:36954/0 (socket says 172.21.15.103:36954) 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffaf8b00640 1 -- 172.21.15.103:0/1202777963 learned_addr learned my addr 172.21.15.103:0/1202777963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.675+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c595040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.676+0000 7ffaf9b02640 1 -- 172.21.15.103:0/1202777963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 msgr2=0x55ad6c594b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.676+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c594b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.676+0000 7ffaf9b02640 1 -- 172.21.15.103:0/1202777963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 msgr2=0x55ad6c5945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.676+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c5945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.676+0000 7ffaf9b02640 1 -- 172.21.15.103:0/1202777963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6c583bf0 con 0x55ad6c58e680 2024-09-19T07:52:22.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.680+0000 7ffaf9b02640 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c595040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0cda30 tx=0x7ffaec09ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9411794cbe21daaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:22.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.681+0000 7ffadcff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:52:22.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.681+0000 7ffadcff9640 1 -- 172.21.15.103:0/1202777963 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0a5230 con 0x55ad6c58e680 2024-09-19T07:52:22.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.681+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaf0011440 con 0x55ad6c58e680 2024-09-19T07:52:22.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.681+0000 7ffadcff9640 1 -- 172.21.15.103:0/1202777963 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7ffaec09c820 con 0x55ad6c58e680 2024-09-19T07:52:22.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.690+0000 7ffadcff9640 1 -- 172.21.15.103:0/1202777963 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ffaec09ca20 con 0x55ad6c58e680 2024-09-19T07:52:22.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.691+0000 7ffafd93e740 10 client.27647 fetch_fsmap learned FSMap version 52 2024-09-19T07:52:22.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.691+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55ad6c609260 con 0x55ad6c58e680 2024-09-19T07:52:22.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffadcff9640 1 -- 172.21.15.103:0/1202777963 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf0011440 con 0x55ad6c58e680 2024-09-19T07:52:22.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffadcff9640 1 -- 172.21.15.103:0/1202777963 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7ffaec0b5410 con 0x55ad6c58e680 2024-09-19T07:52:22.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffafd93e740 10 client.27647 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:52:22.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffafd93e740 10 client.27647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:22.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffafd93e740 20 client.27647 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:22.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffafd93e740 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c609a20 0x55ad6c60be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:22.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffafd93e740 4 client.27647 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:52:22.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.692+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad6c4f0d60 con 0x55ad6c609a20 2024-09-19T07:52:22.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.694+0000 7ffaf9301640 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c609a20 0x55ad6c60be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:22.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.703+0000 7ffaf9301640 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c609a20 0x55ad6c60be40 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e5df744bcca345 server_cookie=a861a2bda282bd2f in_seq=0 out_seq=0 2024-09-19T07:52:22.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.703+0000 7ffadcff9640 10 client.27647 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:22.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.703+0000 7ffadcff9640 1 -- 172.21.15.103:0/1202777963 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ad6c4f0d60 con 0x55ad6c609a20 2024-09-19T07:52:22.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.703+0000 7ffadcff9640 10 client.27647 handle_command_reply: tid=0 2024-09-19T07:52:22.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 client.27647 shutdown 2024-09-19T07:52:22.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 msgr2=0x55ad6c595040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c595040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0cda30 tx=0x7ffaec09ccc0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 shutdown_connections 2024-09-19T07:52:22.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55ad6c609a20 0x55ad6c60be40 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad6c58e680 0x55ad6c595040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad6c583dc0 0x55ad6c594b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 --2- 172.21.15.103:0/1202777963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6c5834a0 0x55ad6c5945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 >> 172.21.15.103:0/1202777963 conn(0x55ad6c57fa30 msgr2=0x55ad6c58cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:22.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.704+0000 7ffadcff9640 0 client.27647 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:22.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.705+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 shutdown_connections 2024-09-19T07:52:22.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.705+0000 7ffafd93e740 1 -- 172.21.15.103:0/1202777963 wait complete. 2024-09-19T07:52:22.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.706+0000 7ffafd93e740 20 client.27647 trim_cache size 0 max 16384 2024-09-19T07:52:22.708 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: "208076d4-2ea0-4e1a-8bec-367fb8d8fe3c": { 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "208076d4-2ea0-4e1a-8bec-367fb8d8fe3c", 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:52:22.709 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffae00a3430 msgr2=0x7ffae00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffae00a3430 0x7ffae00a58f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55ad6c580890 tx=0x7ffaf000b390 comp rx=0 tx=0).stop 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf410c160 msgr2=0x7ffaf419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf410c160 0x7ffaf419aa60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009fd0 tx=0x7ffaec00ef10 comp rx=0 tx=0).stop 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 shutdown_connections 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffae00a3430 0x7ffae00a58f0 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55ad6c580890 tx=0x7ffaf000b390 comp rx=0 tx=0).stop 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf410c160 0x7ffaf419aa60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffaf41036a0 0x7ffaf41966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 --2- 172.21.15.103:0/2905732911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf4102d80 0x7ffaf4196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:22.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.709+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 >> 172.21.15.103:0/2905732911 conn(0x7ffaf40fc7a0 msgr2=0x7ffaf40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:22.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.710+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 shutdown_connections 2024-09-19T07:52:22.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:22.710+0000 7ffafb565640 1 -- 172.21.15.103:0/2905732911 wait complete. 2024-09-19T07:52:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:22 smithi134 ceph-mon[35414]: pgmap v450: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:22 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:22 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:52:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:22 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:52:22.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:22 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:52:22.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:22 smithi174 ceph-mon[35370]: pgmap v450: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:22.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:22 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:52:22.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:22 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:52:22.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:22 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:52:22.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:22 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:52:24.690 INFO:tasks.cephfs.filesystem:scrub status for tag:208076d4-2ea0-4e1a-8bec-367fb8d8fe3c - {'path': '/', 'tag': '208076d4-2ea0-4e1a-8bec-367fb8d8fe3c', 'options': 'recursive,force'} 2024-09-19T07:52:24.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:24 smithi103 ceph-mon[31201]: pgmap v451: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:24.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:24 smithi134 ceph-mon[35414]: pgmap v451: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:24.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:24 smithi174 ceph-mon[35370]: pgmap v451: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:25.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 830.0 s, 1930.2 tps, lat 51.908 ms stddev 69.843 2024-09-19T07:52:26.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:26 smithi103 ceph-mon[31201]: pgmap v452: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:26.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:26 smithi134 ceph-mon[35414]: pgmap v452: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:26.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:26 smithi174 ceph-mon[35370]: pgmap v452: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 25 MiB/s wr, 1.62k op/s 2024-09-19T07:52:27.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:27 smithi134 ceph-mon[35414]: pgmap v453: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-09-19T07:52:27.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:27 smithi174 ceph-mon[35370]: pgmap v453: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-09-19T07:52:27.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:27 smithi103 ceph-mon[31201]: pgmap v453: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 24 MiB/s wr, 1.50k op/s 2024-09-19T07:52:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:52:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:52:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:52:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:52:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:29 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:52:29.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:29 smithi174 ceph-mon[35370]: pgmap v454: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.32k op/s 2024-09-19T07:52:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:52:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:52:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:52:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:52:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:29 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:52:29.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:29 smithi103 ceph-mon[31201]: pgmap v454: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.32k op/s 2024-09-19T07:52:30.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:52:30.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:52:30.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:52:30.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:52:30.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:29 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:52:30.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:29 smithi134 ceph-mon[35414]: pgmap v454: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 21 MiB/s wr, 1.32k op/s 2024-09-19T07:52:30.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 835.0 s, 2064.4 tps, lat 47.819 ms stddev 51.492 2024-09-19T07:52:32.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:32 smithi134 ceph-mon[35414]: pgmap v455: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2024-09-19T07:52:32.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:32 smithi174 ceph-mon[35370]: pgmap v455: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2024-09-19T07:52:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:32 smithi103 ceph-mon[31201]: pgmap v455: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 1.59k op/s 2024-09-19T07:52:34.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:34 smithi134 ceph-mon[35414]: pgmap v456: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-19T07:52:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:34 smithi174 ceph-mon[35370]: pgmap v456: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-19T07:52:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:34 smithi103 ceph-mon[31201]: pgmap v456: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-19T07:52:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 840.0 s, 2316.0 tps, lat 43.678 ms stddev 53.489 2024-09-19T07:52:36.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:36 smithi134 ceph-mon[35414]: pgmap v457: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-19T07:52:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:36 smithi174 ceph-mon[35370]: pgmap v457: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-19T07:52:36.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:36 smithi103 ceph-mon[31201]: pgmap v457: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-19T07:52:38.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:38 smithi134 ceph-mon[35414]: pgmap v458: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 908 op/s 2024-09-19T07:52:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:38 smithi174 ceph-mon[35370]: pgmap v458: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 908 op/s 2024-09-19T07:52:38.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:38 smithi103 ceph-mon[31201]: pgmap v458: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 908 op/s 2024-09-19T07:52:40.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:40 smithi134 ceph-mon[35414]: pgmap v459: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 592 op/s 2024-09-19T07:52:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:40 smithi174 ceph-mon[35370]: pgmap v459: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 592 op/s 2024-09-19T07:52:40.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 845.0 s, 2280.2 tps, lat 44.062 ms stddev 54.158 2024-09-19T07:52:40.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:40 smithi103 ceph-mon[31201]: pgmap v459: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 13 MiB/s wr, 592 op/s 2024-09-19T07:52:42.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:42 smithi134 ceph-mon[35414]: pgmap v460: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 20 MiB/s wr, 937 op/s 2024-09-19T07:52:42.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:42 smithi174 ceph-mon[35370]: pgmap v460: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 20 MiB/s wr, 937 op/s 2024-09-19T07:52:42.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:42 smithi103 ceph-mon[31201]: pgmap v460: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 20 MiB/s wr, 937 op/s 2024-09-19T07:52:44.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:44 smithi134 ceph-mon[35414]: pgmap v461: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-19T07:52:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:44 smithi174 ceph-mon[35370]: pgmap v461: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-19T07:52:44.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:44 smithi103 ceph-mon[31201]: pgmap v461: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-19T07:52:45.467 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 850.0 s, 2339.8 tps, lat 40.779 ms stddev 74.882 2024-09-19T07:52:46.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:46 smithi134 ceph-mon[35414]: pgmap v462: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-19T07:52:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:46 smithi174 ceph-mon[35370]: pgmap v462: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-19T07:52:46.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:46 smithi103 ceph-mon[31201]: pgmap v462: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 MiB/s wr, 662 op/s 2024-09-19T07:52:48.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:48 smithi134 ceph-mon[35414]: pgmap v463: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 29 MiB/s wr, 1.68k op/s 2024-09-19T07:52:48.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:48 smithi174 ceph-mon[35370]: pgmap v463: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 29 MiB/s wr, 1.68k op/s 2024-09-19T07:52:48.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:48 smithi103 ceph-mon[31201]: pgmap v463: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 29 MiB/s wr, 1.68k op/s 2024-09-19T07:52:50.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:50 smithi134 ceph-mon[35414]: pgmap v464: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 23 MiB/s wr, 1.36k op/s 2024-09-19T07:52:50.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:50 smithi174 ceph-mon[35370]: pgmap v464: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 23 MiB/s wr, 1.36k op/s 2024-09-19T07:52:50.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 855.0 s, 2214.0 tps, lat 46.736 ms stddev 63.258 2024-09-19T07:52:50.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:50 smithi103 ceph-mon[31201]: pgmap v464: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 23 MiB/s wr, 1.36k op/s 2024-09-19T07:52:52.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:52 smithi134 ceph-mon[35414]: pgmap v465: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 34 MiB/s wr, 2.06k op/s 2024-09-19T07:52:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:52 smithi174 ceph-mon[35370]: pgmap v465: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 34 MiB/s wr, 2.06k op/s 2024-09-19T07:52:52.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:52 smithi103 ceph-mon[31201]: pgmap v465: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 34 MiB/s wr, 2.06k op/s 2024-09-19T07:52:54.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:54 smithi103 ceph-mon[31201]: pgmap v466: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-09-19T07:52:54.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:54 smithi134 ceph-mon[35414]: pgmap v466: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-09-19T07:52:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:54 smithi174 ceph-mon[35370]: pgmap v466: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-09-19T07:52:54.692 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:52:55.067 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:55.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 860.0 s, 1853.8 tps, lat 54.304 ms stddev 63.734 2024-09-19T07:52:56.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.001+0000 7fcf7a986640 1 -- 172.21.15.103:0/3241314044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 msgr2=0x7fcf7410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.001+0000 7fcf7a986640 1 --2- 172.21.15.103:0/3241314044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 0x7fcf7410ec40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009cd0 tx=0x7fcf6c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.001+0000 7fcf7a986640 1 -- 172.21.15.103:0/3241314044 shutdown_connections 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.001+0000 7fcf7a986640 1 --2- 172.21.15.103:0/3241314044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 0x7fcf7410ec40 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.001+0000 7fcf7a986640 1 --2- 172.21.15.103:0/3241314044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf741036e0 0x7fcf74107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.001+0000 7fcf7a986640 1 --2- 172.21.15.103:0/3241314044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf74102d30 0x7fcf74103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.001+0000 7fcf7a986640 1 -- 172.21.15.103:0/3241314044 >> 172.21.15.103:0/3241314044 conn(0x7fcf740fc7d0 msgr2=0x7fcf740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.002+0000 7fcf7a986640 1 -- 172.21.15.103:0/3241314044 shutdown_connections 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.002+0000 7fcf7a986640 1 -- 172.21.15.103:0/3241314044 wait complete. 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.002+0000 7fcf7a986640 1 Processor -- start 2024-09-19T07:52:56.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.002+0000 7fcf7a986640 1 -- start start 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf7a986640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf74102d30 0x7fcf74198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf7a986640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf741036e0 0x7fcf741988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf7a986640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 0x7fcf7419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf7a986640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741120d0 con 0x7fcf741082a0 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf78f23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 0x7fcf7419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf78f23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 0x7fcf7419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34652/0 (socket says 172.21.15.103:34652) 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf78f23640 1 -- 172.21.15.103:0/1430371431 learned_addr learned my addr 172.21.15.103:0/1430371431 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74111f50 con 0x7fcf74102d30 2024-09-19T07:52:56.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.003+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf741036e0 0x7fcf741988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74112250 con 0x7fcf741036e0 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf73fff640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf74102d30 0x7fcf74198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf737fe640 1 -- 172.21.15.103:0/1430371431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf74102d30 msgr2=0x7fcf74198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf74102d30 0x7fcf74198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf737fe640 1 -- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 msgr2=0x7fcf7419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 0x7fcf7419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf737fe640 1 -- 172.21.15.103:0/1430371431 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7419d320 con 0x7fcf741036e0 2024-09-19T07:52:56.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.004+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf741036e0 0x7fcf741988b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf740697d0 tx=0x7fcf6800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.005+0000 7fcf717fa640 1 -- 172.21.15.103:0/1430371431 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800cc40 con 0x7fcf741036e0 2024-09-19T07:52:56.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.005+0000 7fcf717fa640 1 -- 172.21.15.103:0/1430371431 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6800cde0 con 0x7fcf741036e0 2024-09-19T07:52:56.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.005+0000 7fcf717fa640 1 -- 172.21.15.103:0/1430371431 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680188e0 con 0x7fcf741036e0 2024-09-19T07:52:56.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.005+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7419d610 con 0x7fcf741036e0 2024-09-19T07:52:56.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.005+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7419da40 con 0x7fcf741036e0 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.008+0000 7fcf7cd5f740 1 Processor -- start 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.008+0000 7fcf7cd5f740 1 -- start start 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.008+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5127cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc51277030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc511d32e0 con 0x55dc5127c6e0 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc5117ace0 con 0x55dc5127d000 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc511cf2a0 con 0x55dc512776a0 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5127cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf717fa640 1 -- 172.21.15.103:0/1430371431 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcf68018a80 con 0x7fcf741036e0 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5127cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34666/0 (socket says 172.21.15.103:34666) 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 -- 172.21.15.103:0/1719372401 learned_addr learned my addr 172.21.15.103:0/1719372401 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf73fff640 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc51277030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 -- 172.21.15.103:0/1719372401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 msgr2=0x55dc51277b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 -- 172.21.15.103:0/1719372401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 msgr2=0x55dc51277030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc51277030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.009+0000 7fcf78f23640 1 -- 172.21.15.103:0/1719372401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5115f440 con 0x55dc5127c6e0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.010+0000 7fcf78f23640 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5127cac0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005f90 tx=0x7fcf6c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d657465e9e4374a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.010+0000 7fcf5a7fc640 1 -- 172.21.15.103:0/1719372401 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c031640 con 0x55dc5127c6e0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.010+0000 7fcf5a7fc640 1 -- 172.21.15.103:0/1719372401 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c036040 con 0x55dc5127c6e0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.010+0000 7fcf5a7fc640 1 -- 172.21.15.103:0/1719372401 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c004280 con 0x55dc5127c6e0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.010+0000 7fcf717fa640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf5c0a3500 0x7fcf5c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.010+0000 7fcf73fff640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf5c0a3500 0x7fcf5c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf717fa640 1 -- 172.21.15.103:0/1430371431 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fcf680cff40 con 0x7fcf741036e0 2024-09-19T07:52:56.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1719372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 msgr2=0x55dc5127cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5127cac0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005f90 tx=0x7fcf6c009540 comp rx=0 tx=0).stop 2024-09-19T07:52:56.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1719372401 shutdown_connections 2024-09-19T07:52:56.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc51277030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1719372401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5127cac0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1719372401 >> 172.21.15.103:0/1719372401 conn(0x55dc512694e0 msgr2=0x55dc512698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf73fff640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf5c0a3500 0x7fcf5c0a59c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55dc5126abf0 tx=0x7fcf64005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.011+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1719372401 shutdown_connections 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.012+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1719372401 wait complete. 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.013+0000 7fcf7cd5f740 1 -- start start 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.013+0000 7fcf7cd5f740 1 Processor -- start 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.013+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc512776a0 0x55dc51282f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.013+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc5127c6e0 0x55dc512801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.013+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51280780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc5127c6e0 0x55dc512801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc51283960 con 0x55dc5127d000 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc5118e050 con 0x55dc512776a0 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc51283ae0 con 0x55dc5127c6e0 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf7cd5f740 10 client.0 _ll_register_callbacks cb 0x55dc511ce160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc5127c6e0 0x55dc512801e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:59802/0 (socket says 172.21.15.103:59802) 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf737fe640 1 -- 172.21.15.103:0/3215257492 learned_addr learned my addr 172.21.15.103:0/3215257492 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf78f23640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51280780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf737fe640 1 -- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc512776a0 msgr2=0x55dc51282f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.014+0000 7fcf73fff640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc512776a0 0x55dc51282f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf737fe640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc512776a0 0x55dc51282f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf737fe640 1 -- 172.21.15.103:0/3215257492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 msgr2=0x55dc51280780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf737fe640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51280780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf737fe640 1 -- 172.21.15.103:0/3215257492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc50f443c0 con 0x55dc5127c6e0 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf78f23640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51280780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf73fff640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc512776a0 0x55dc51282f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf737fe640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc5127c6e0 0x55dc512801e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcf6809beb0 tx=0x7fcf68009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5ef1df7d3c99b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf537fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf537fe640 1 -- 172.21.15.103:0/3215257492 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6809c8b0 con 0x55dc5127c6e0 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf537fe640 1 -- 172.21.15.103:0/3215257492 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fcf680a01d0 con 0x55dc5127c6e0 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf6c036040 con 0x55dc5127c6e0 2024-09-19T07:52:56.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf537fe640 1 -- 172.21.15.103:0/3215257492 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcf6809c1b0 con 0x55dc5127c6e0 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf7cd5f740 10 client.27616 fetch_fsmap learned FSMap version 52 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.015+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55dc5128f410 con 0x55dc5127c6e0 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf537fe640 1 -- 172.21.15.103:0/3215257492 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c036040 con 0x55dc5127c6e0 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf537fe640 1 -- 172.21.15.103:0/3215257492 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7fcf680cd8a0 con 0x55dc5127c6e0 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf7cd5f740 10 client.27616 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf7cd5f740 10 client.27616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf7cd5f740 20 client.27616 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc51292cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf7cd5f740 4 client.27616 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.016+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc511e0850 con 0x55dc512908d0 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.017+0000 7fcf73fff640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc51292cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.017+0000 7fcf73fff640 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc51292cc0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4af5ff058cd4815 server_cookie=bbaa510966b616ef in_seq=0 out_seq=0 2024-09-19T07:52:56.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.017+0000 7fcf537fe640 10 client.27616 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:56.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.018+0000 7fcf537fe640 1 -- 172.21.15.103:0/3215257492 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc511e0850 con 0x55dc512908d0 2024-09-19T07:52:56.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.018+0000 7fcf537fe640 10 client.27616 handle_command_reply: tid=0 2024-09-19T07:52:56.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.018+0000 7fcf7cd5f740 1 client.27616 shutdown 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc5127c6e0 msgr2=0x55dc512801e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc5127c6e0 0x55dc512801e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcf6809beb0 tx=0x7fcf68009700 comp rx=0 tx=0).stop 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 shutdown_connections 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc51292cc0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51280780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc5127c6e0 0x55dc512801e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf537fe640 0 client.27616 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3215257492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc512776a0 0x55dc51282f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.019+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 >> 172.21.15.103:0/3215257492 conn(0x55dc512694e0 msgr2=0x55dc51268bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.020+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 shutdown_connections 2024-09-19T07:52:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.020+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3215257492 wait complete. 2024-09-19T07:52:56.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.020+0000 7fcf7cd5f740 20 client.27616 trim_cache size 0 max 16384 2024-09-19T07:52:56.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.049+0000 7fcf7cd5f740 1 Processor -- start 2024-09-19T07:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.049+0000 7fcf7cd5f740 1 -- start start 2024-09-19T07:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.050+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.050+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5128cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.050+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc5127daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.050+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741ab6f0 con 0x55dc5127c6e0 2024-09-19T07:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.050+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741ad230 con 0x55dc5127d000 2024-09-19T07:52:56.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.050+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741abdc0 con 0x55dc512776a0 2024-09-19T07:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.053+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5128cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.053+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5128cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34690/0 (socket says 172.21.15.103:34690) 2024-09-19T07:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.053+0000 7fcf737fe640 1 -- 172.21.15.103:0/1297466581 learned_addr learned my addr 172.21.15.103:0/1297466581 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.053+0000 7fcf78f23640 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc5127daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.053+0000 7fcf73fff640 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.055+0000 7fcf737fe640 1 -- 172.21.15.103:0/1297466581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 msgr2=0x55dc51277030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.055+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.055+0000 7fcf737fe640 1 -- 172.21.15.103:0/1297466581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 msgr2=0x55dc5127daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.055+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc5127daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.055+0000 7fcf737fe640 1 -- 172.21.15.103:0/1297466581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc51280af0 con 0x55dc5127c6e0 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.057+0000 7fcf737fe640 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5128cc00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcf680a48f0 tx=0x7fcf680d0230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653ce15deaedad9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.057+0000 7fcf59ffb640 1 -- 172.21.15.103:0/1297466581 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf680a4e70 con 0x55dc5127c6e0 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.057+0000 7fcf59ffb640 1 -- 172.21.15.103:0/1297466581 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6809c8b0 con 0x55dc5127c6e0 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.058+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1297466581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 msgr2=0x55dc5128cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.058+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5128cc00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcf680a48f0 tx=0x7fcf680d0230 comp rx=0 tx=0).stop 2024-09-19T07:52:56.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.065+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1297466581 shutdown_connections 2024-09-19T07:52:56.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.065+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127d000 0x55dc5127daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.065+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127c6e0 0x55dc5128cc00 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.065+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/1297466581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51277030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.065+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1297466581 >> 172.21.15.103:0/1297466581 conn(0x55dc512694e0 msgr2=0x55dc5127d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.071+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1297466581 shutdown_connections 2024-09-19T07:52:56.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.075+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/1297466581 wait complete. 2024-09-19T07:52:56.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.075+0000 7fcf7cd5f740 1 -- start start 2024-09-19T07:52:56.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.075+0000 7fcf7cd5f740 1 Processor -- start 2024-09-19T07:52:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.078+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51288d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.078+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127c6e0 0x55dc512892b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.078+0000 7fcf7cd5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51285d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.078+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741acb60 con 0x55dc5127d000 2024-09-19T07:52:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.078+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741ac490 con 0x55dc5127c6e0 2024-09-19T07:52:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.078+0000 7fcf7cd5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741a96f0 con 0x55dc512776a0 2024-09-19T07:52:56.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.078+0000 7fcf7cd5f740 10 client.0 _ll_register_callbacks cb 0x7fcf680d36e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:52:56.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.079+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127c6e0 0x55dc512892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.081+0000 7fcf73fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51288d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.081+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127c6e0 0x55dc512892b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:43714/0 (socket says 172.21.15.103:43714) 2024-09-19T07:52:56.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.081+0000 7fcf737fe640 1 -- 172.21.15.103:0/3137935867 learned_addr learned my addr 172.21.15.103:0/3137935867 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:56.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.081+0000 7fcf78f23640 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51285d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.083+0000 7fcf78f23640 1 -- 172.21.15.103:0/3137935867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 msgr2=0x55dc51288d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.083+0000 7fcf78f23640 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51288d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.083+0000 7fcf78f23640 1 -- 172.21.15.103:0/3137935867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127c6e0 msgr2=0x55dc512892b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:52:56.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.083+0000 7fcf78f23640 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127c6e0 0x55dc512892b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.083+0000 7fcf78f23640 1 -- 172.21.15.103:0/3137935867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc51286280 con 0x55dc5127d000 2024-09-19T07:52:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.089+0000 7fcf78f23640 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51285d40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x55dc51281450 tx=0x7fcf6c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1338fd89b8373d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:56.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.089+0000 7fcf537fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:52:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.090+0000 7fcf537fe640 1 -- 172.21.15.103:0/3137935867 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c053060 con 0x55dc5127d000 2024-09-19T07:52:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.090+0000 7fcf537fe640 1 -- 172.21.15.103:0/3137935867 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fcf6c041040 con 0x55dc5127d000 2024-09-19T07:52:56.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.091+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf6809c8b0 con 0x55dc5127d000 2024-09-19T07:52:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.091+0000 7fcf537fe640 1 -- 172.21.15.103:0/3137935867 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcf6c0318c0 con 0x55dc5127d000 2024-09-19T07:52:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.092+0000 7fcf7cd5f740 10 client.18357 fetch_fsmap learned FSMap version 52 2024-09-19T07:52:56.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.092+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55dc512fbb70 con 0x55dc5127d000 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf537fe640 1 -- 172.21.15.103:0/3137935867 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6809c8b0 con 0x55dc5127d000 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf537fe640 1 -- 172.21.15.103:0/3137935867 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7fcf6c030070 con 0x55dc5127d000 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf7cd5f740 10 client.18357 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf7cd5f740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf7cd5f740 20 client.18357 populate_metadata read hostname 'smithi103' 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc512fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf7cd5f740 4 client.18357 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:52:56.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.093+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc511dad60 con 0x55dc512908d0 2024-09-19T07:52:56.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.094+0000 7fcf73fff640 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc512fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:56.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.095+0000 7fcf73fff640 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc512fdec0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c26b789731c97890 server_cookie=ffc1823df3993866 in_seq=0 out_seq=0 2024-09-19T07:52:56.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.098+0000 7fcf537fe640 10 client.18357 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:56.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.098+0000 7fcf537fe640 1 -- 172.21.15.103:0/3137935867 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc511dad60 con 0x55dc512908d0 2024-09-19T07:52:56.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.098+0000 7fcf537fe640 10 client.18357 handle_command_reply: tid=0 2024-09-19T07:52:56.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.098+0000 7fcf7cd5f740 1 client.18357 shutdown 2024-09-19T07:52:56.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.098+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 msgr2=0x55dc51285d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.098+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51285d40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x55dc51281450 tx=0x7fcf6c005df0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 shutdown_connections 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55dc512908d0 0x55dc512fdec0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc5127d000 0x55dc51285d40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc5127c6e0 0x55dc512892b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 --2- 172.21.15.103:0/3137935867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc512776a0 0x55dc51288d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 >> 172.21.15.103:0/3137935867 conn(0x55dc512694e0 msgr2=0x55dc5127dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf537fe640 0 client.18357 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 shutdown_connections 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 1 -- 172.21.15.103:0/3137935867 wait complete. 2024-09-19T07:52:56.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.099+0000 7fcf7cd5f740 20 client.18357 trim_cache size 0 max 16384 2024-09-19T07:52:56.101 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:52:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:52:56.101 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:52:56.101 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.100+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf5c0a3500 msgr2=0x7fcf5c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.100+0000 7fcf7a986640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf5c0a3500 0x7fcf5c0a59c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55dc5126abf0 tx=0x7fcf64005f90 comp rx=0 tx=0).stop 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.100+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf741036e0 msgr2=0x7fcf741988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.100+0000 7fcf7a986640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf741036e0 0x7fcf741988b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf740697d0 tx=0x7fcf6800ee50 comp rx=0 tx=0).stop 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 shutdown_connections 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcf5c0a3500 0x7fcf5c0a59c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf741082a0 0x7fcf7419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf741036e0 0x7fcf741988b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 --2- 172.21.15.103:0/1430371431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf74102d30 0x7fcf74198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 >> 172.21.15.103:0/1430371431 conn(0x7fcf740fc7d0 msgr2=0x7fcf740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:56.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 shutdown_connections 2024-09-19T07:52:56.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:56.101+0000 7fcf7a986640 1 -- 172.21.15.103:0/1430371431 wait complete. 2024-09-19T07:52:56.136 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:56 smithi103 ceph-mon[31201]: pgmap v467: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-09-19T07:52:56.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:56 smithi134 ceph-mon[35414]: pgmap v467: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-09-19T07:52:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:56 smithi174 ceph-mon[35370]: pgmap v467: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 28 MiB/s wr, 1.72k op/s 2024-09-19T07:52:57.210 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:52:57.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:52:57.588 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:52:58.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.109+0000 7f7888589640 1 -- 172.21.15.103:0/622119570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109830 msgr2=0x7f7880109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:58.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.109+0000 7f7888589640 1 --2- 172.21.15.103:0/622119570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109830 0x7f7880109c90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f787402a410 comp rx=0 tx=0).stop 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.109+0000 7f7888589640 1 -- 172.21.15.103:0/622119570 shutdown_connections 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.109+0000 7f7888589640 1 --2- 172.21.15.103:0/622119570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f788011b140 0x7f788011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.109+0000 7f7888589640 1 --2- 172.21.15.103:0/622119570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109830 0x7f7880109c90 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.109+0000 7f7888589640 1 --2- 172.21.15.103:0/622119570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7880109450 0x7f7880110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.109+0000 7f7888589640 1 -- 172.21.15.103:0/622119570 >> 172.21.15.103:0/622119570 conn(0x7f788006e140 msgr2=0x7f788006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.110+0000 7f7888589640 1 -- 172.21.15.103:0/622119570 shutdown_connections 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.110+0000 7f7888589640 1 -- 172.21.15.103:0/622119570 wait complete. 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.110+0000 7f7888589640 1 Processor -- start 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7888589640 1 -- start start 2024-09-19T07:52:58.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7888589640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109450 0x7f78801a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7888589640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7880109830 0x7f78801a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7888589640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f788011b140 0x7f78801abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7888589640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7880120df0 con 0x7f7880109450 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7888589640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7880120c70 con 0x7f788011b140 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7888589640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7880120f70 con 0x7f7880109830 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109450 0x7f78801a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109450 0x7f78801a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34708/0 (socket says 172.21.15.103:34708) 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 -- 172.21.15.103:0/2959404708 learned_addr learned my addr 172.21.15.103:0/2959404708 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7885b24640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7880109830 0x7f78801a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886b26640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f788011b140 0x7f78801abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 -- 172.21.15.103:0/2959404708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7880109830 msgr2=0x7f78801a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:58.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7880109830 0x7f78801a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 -- 172.21.15.103:0/2959404708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f788011b140 msgr2=0x7f78801abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f788011b140 0x7f78801abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 -- 172.21.15.103:0/2959404708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78801ac220 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.111+0000 7f7886325640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109450 0x7f78801a7270 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.112+0000 7f78737fe640 1 -- 172.21.15.103:0/2959404708 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c00cd80 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.112+0000 7f78737fe640 1 -- 172.21.15.103:0/2959404708 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787c010910 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.112+0000 7f78737fe640 1 -- 172.21.15.103:0/2959404708 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c018990 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.113+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78801ac510 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.113+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78801b3df0 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.114+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7880073b00 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.117+0000 7f78737fe640 1 -- 172.21.15.103:0/2959404708 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f787c010ac0 con 0x7f7880109450 2024-09-19T07:52:58.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.118+0000 7f78737fe640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f786c0a3500 0x7f786c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:52:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.118+0000 7f78737fe640 1 -- 172.21.15.103:0/2959404708 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7f787c0d04e0 con 0x7f7880109450 2024-09-19T07:52:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.118+0000 7f7885b24640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f786c0a3500 0x7f786c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:52:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.118+0000 7f7885b24640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f786c0a3500 0x7f786c0a59c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f7874005b50 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:52:58.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.122+0000 7f78737fe640 1 -- 172.21.15.103:0/2959404708 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f787c09cf30 con 0x7f7880109450 2024-09-19T07:52:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:58 smithi134 ceph-mon[35414]: pgmap v468: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2024-09-19T07:52:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:58 smithi174 ceph-mon[35370]: pgmap v468: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2024-09-19T07:52:58.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.439+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78801a85d0 con 0x7f7880109450 2024-09-19T07:52:58.439 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:58 smithi103 ceph-mon[31201]: pgmap v468: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 32 MiB/s wr, 2.01k op/s 2024-09-19T07:52:58.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.439+0000 7f78737fe640 1 -- 172.21.15.103:0/2959404708 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11991 (secure 0 0 0) 0x7f787c09c830 con 0x7f7880109450 2024-09-19T07:52:58.440 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:52:58.440 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":52,"btime":"2024-09-19T07:52:19:439277+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:52:18.534822+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:52:58.440 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 52 2024-09-19T07:52:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.442+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f786c0a3500 msgr2=0x7f786c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.442+0000 7f7888589640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f786c0a3500 0x7f786c0a59c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f7874005b50 comp rx=0 tx=0).stop 2024-09-19T07:52:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.442+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109450 msgr2=0x7f78801a7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:52:58.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.442+0000 7f7888589640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109450 0x7f78801a7270 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 shutdown_connections 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f786c0a3500 0x7f786c0a59c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f788011b140 0x7f78801abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7880109830 0x7f78801a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 --2- 172.21.15.103:0/2959404708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7880109450 0x7f78801a7270 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 >> 172.21.15.103:0/2959404708 conn(0x7f788006e140 msgr2=0x7f788011c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 shutdown_connections 2024-09-19T07:52:58.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:52:58.443+0000 7f7888589640 1 -- 172.21.15.103:0/2959404708 wait complete. 2024-09-19T07:52:59.213 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:52:59 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2959404708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:52:59.264 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:52:59.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:52:59 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2959404708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:52:59.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:52:59 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2959404708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:52:59.608 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:00.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 -- 172.21.15.103:0/562374083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e810c160 msgr2=0x7fa7e810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/562374083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e810c160 0x7fa7e810e600 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009ca0 tx=0x7fa7e002a950 comp rx=0 tx=0).stop 2024-09-19T07:53:00.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 -- 172.21.15.103:0/562374083 shutdown_connections 2024-09-19T07:53:00.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/562374083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e810c160 0x7fa7e810e600 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/562374083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e8103650 0x7fa7e810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/562374083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7e8102d30 0x7fa7e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 -- 172.21.15.103:0/562374083 >> 172.21.15.103:0/562374083 conn(0x7fa7e80fc7d0 msgr2=0x7fa7e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:00.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 -- 172.21.15.103:0/562374083 shutdown_connections 2024-09-19T07:53:00.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.184+0000 7fa7ec81c640 1 -- 172.21.15.103:0/562374083 wait complete. 2024-09-19T07:53:00.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 Processor -- start 2024-09-19T07:53:00.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 -- start start 2024-09-19T07:53:00.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e8102d30 0x7fa7e81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e8103650 0x7fa7e81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7e810c160 0x7fa7e819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e8103650 0x7fa7e81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8111fb0 con 0x7fa7e8103650 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e8103650 0x7fa7e81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43776/0 (socket says 172.21.15.103:43776) 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7dffff640 1 -- 172.21.15.103:0/3486006206 learned_addr learned my addr 172.21.15.103:0/3486006206 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7e659c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e8102d30 0x7fa7e81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8111e30 con 0x7fa7e810c160 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.185+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8112130 con 0x7fa7e8102d30 2024-09-19T07:53:00.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7e6d9d640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7e810c160 0x7fa7e819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7dffff640 1 -- 172.21.15.103:0/3486006206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e8102d30 msgr2=0x7fa7e81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7dffff640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e8102d30 0x7fa7e81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7dffff640 1 -- 172.21.15.103:0/3486006206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7e810c160 msgr2=0x7fa7e819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7dffff640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7e810c160 0x7fa7e819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7dffff640 1 -- 172.21.15.103:0/3486006206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e819d160 con 0x7fa7e8103650 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7dffff640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e8103650 0x7fa7e81986f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000be90 tx=0x7fa7d000dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7df7fe640 1 -- 172.21.15.103:0/3486006206 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0018070 con 0x7fa7e8103650 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7df7fe640 1 -- 172.21.15.103:0/3486006206 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0016070 con 0x7fa7e8103650 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.186+0000 7fa7df7fe640 1 -- 172.21.15.103:0/3486006206 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00047a0 con 0x7fa7e8103650 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.187+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e819d450 con 0x7fa7e8103650 2024-09-19T07:53:00.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.187+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81008c0 con 0x7fa7e8103650 2024-09-19T07:53:00.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.190+0000 7fa7df7fe640 1 -- 172.21.15.103:0/3486006206 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa7d00099b0 con 0x7fa7e8103650 2024-09-19T07:53:00.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.191+0000 7fa7eebf5740 1 Processor -- start 2024-09-19T07:53:00.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.191+0000 7fa7eebf5740 1 -- start start 2024-09-19T07:53:00.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.191+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d1a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.191+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 0x561780d2ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561780c802f0 con 0x561780d1bb40 2024-09-19T07:53:00.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561780c27cf0 con 0x561780d1a5d0 2024-09-19T07:53:00.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561780c7c2b0 con 0x561780d2db80 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d1a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d1a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43790/0 (socket says 172.21.15.103:43790) 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 -- 172.21.15.103:0/2761894208 learned_addr learned my addr 172.21.15.103:0/2761894208 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7dffff640 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 0x561780d2ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e659c640 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 -- 172.21.15.103:0/2761894208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 msgr2=0x561780d2ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 0x561780d2ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 -- 172.21.15.103:0/2761894208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 msgr2=0x561780d1aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d1aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e6d9d640 1 -- 172.21.15.103:0/2761894208 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561780c0c3a0 con 0x561780d1bb40 2024-09-19T07:53:00.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e659c640 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:00.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7df7fe640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa7d40a34b0 0x7fa7d40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7e659c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa7d40a34b0 0x7fa7d40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.192+0000 7fa7df7fe640 1 -- 172.21.15.103:0/3486006206 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fa7d00d35f0 con 0x7fa7e8103650 2024-09-19T07:53:00.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.193+0000 7fa7e6d9d640 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d1a090 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009ca0 tx=0x7fa7e0001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea49bd76f5486bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.193+0000 7fa7e659c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa7d40a34b0 0x7fa7d40a5970 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x561780d17ac0 tx=0x7fa7d8005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.193+0000 7fa7dcff9640 1 -- 172.21.15.103:0/2761894208 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0048070 con 0x561780d1bb40 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.194+0000 7fa7dcff9640 1 -- 172.21.15.103:0/2761894208 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0003e50 con 0x561780d1bb40 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.194+0000 7fa7dcff9640 1 -- 172.21.15.103:0/2761894208 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0034040 con 0x561780d1bb40 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.194+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2761894208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 msgr2=0x561780d1a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.194+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d1a090 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009ca0 tx=0x7fa7e0001060 comp rx=0 tx=0).stop 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2761894208 shutdown_connections 2024-09-19T07:53:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 0x561780d2ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2761894208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d1a090 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2761894208 >> 172.21.15.103:0/2761894208 conn(0x561780ce3310 msgr2=0x561780d16c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2761894208 shutdown_connections 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2761894208 wait complete. 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.195+0000 7fa7eebf5740 1 -- start start 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 1 Processor -- start 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d2d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d2a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 0x561780d2b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561780d30870 con 0x561780d1bb40 2024-09-19T07:53:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561780c3b060 con 0x561780d1a5d0 2024-09-19T07:53:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561780d309f0 con 0x561780d2db80 2024-09-19T07:53:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7eebf5740 10 client.0 _ll_register_callbacks cb 0x56178098e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d2d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d2d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45782/0 (socket says 172.21.15.103:45782) 2024-09-19T07:53:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7e659c640 1 -- 172.21.15.103:0/844445282 learned_addr learned my addr 172.21.15.103:0/844445282 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.196+0000 7fa7dffff640 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d2a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.197+0000 7fa7e659c640 1 -- 172.21.15.103:0/844445282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 msgr2=0x561780d2b010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:53:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.197+0000 7fa7e659c640 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 0x561780d2b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.197+0000 7fa7e659c640 1 -- 172.21.15.103:0/844445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 msgr2=0x561780d2a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.197+0000 7fa7e659c640 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d2a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.197+0000 7fa7e659c640 1 -- 172.21.15.103:0/844445282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617809f13c0 con 0x561780d1a5d0 2024-09-19T07:53:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7e659c640 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d2d5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7d801e9d0 tx=0x7fa7d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8db63411339382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7cdffb640 1 -- 172.21.15.103:0/844445282 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d802e060 con 0x561780d1a5d0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7cdffb640 1 -- 172.21.15.103:0/844445282 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fa7d8018030 con 0x561780d1a5d0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e0003e50 con 0x561780d1a5d0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7cdffb640 1 -- 172.21.15.103:0/844445282 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa7d8009db0 con 0x561780d1a5d0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7eebf5740 10 client.27677 fetch_fsmap learned FSMap version 52 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561780d3c3e0 con 0x561780d1a5d0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7cdffb640 1 -- 172.21.15.103:0/844445282 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0003e50 con 0x561780d1a5d0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7cdffb640 1 -- 172.21.15.103:0/844445282 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7fa7d8029720 con 0x561780d1a5d0 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7eebf5740 10 client.27677 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7eebf5740 10 client.27677 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7eebf5740 20 client.27677 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.198+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780d3d8a0 0x561780d3fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.199+0000 7fa7eebf5740 4 client.27677 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.199+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561780c8d860 con 0x561780d3d8a0 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.199+0000 7fa7dffff640 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780d3d8a0 0x561780d3fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.199+0000 7fa7cdffb640 10 client.27677 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.199+0000 7fa7dffff640 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780d3d8a0 0x561780d3fd10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8886a0c35b3098f3 server_cookie=3d413b5f163dca3d in_seq=0 out_seq=0 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.200+0000 7fa7cdffb640 1 -- 172.21.15.103:0/844445282 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561780c8d860 con 0x561780d3d8a0 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.200+0000 7fa7cdffb640 10 client.27677 handle_command_reply: tid=0 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.200+0000 7fa7eebf5740 1 client.27677 shutdown 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 msgr2=0x561780d2d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d2d5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7d801e9d0 tx=0x7fa7d8004730 comp rx=0 tx=0).stop 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 shutdown_connections 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780d3d8a0 0x561780d3fd10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d2db80 0x561780d2b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1bb40 0x561780d2a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/844445282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d2d5d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 >> 172.21.15.103:0/844445282 conn(0x561780ce3310 msgr2=0x561780d1c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:00.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.201+0000 7fa7cdffb640 0 client.27677 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:00.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.202+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 shutdown_connections 2024-09-19T07:53:00.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.202+0000 7fa7eebf5740 1 -- 172.21.15.103:0/844445282 wait complete. 2024-09-19T07:53:00.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.202+0000 7fa7eebf5740 20 client.27677 trim_cache size 0 max 16384 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.218+0000 7fa7eebf5740 1 Processor -- start 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.218+0000 7fa7eebf5740 1 -- start start 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1a5d0 0x561780d16e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d2db80 0x561780d45b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ab710 con 0x561780d1a5d0 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ad250 con 0x561780d2db80 2024-09-19T07:53:00.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81abde0 con 0x561780d1bb40 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1a5d0 0x561780d16e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1a5d0 0x561780d16e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43816/0 (socket says 172.21.15.103:43816) 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7e659c640 1 -- 172.21.15.103:0/2297865818 learned_addr learned my addr 172.21.15.103:0/2297865818 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7dffff640 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7e6d9d640 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d2db80 0x561780d45b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7dffff640 1 -- 172.21.15.103:0/2297865818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d2db80 msgr2=0x561780d45b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7dffff640 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d2db80 0x561780d45b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7dffff640 1 -- 172.21.15.103:0/2297865818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1a5d0 msgr2=0x561780d16e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7dffff640 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1a5d0 0x561780d16e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.219+0000 7fa7dffff640 1 -- 172.21.15.103:0/2297865818 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561780d40390 con 0x561780d1bb40 2024-09-19T07:53:00.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.220+0000 7fa7dffff640 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d455e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00d0de0 tx=0x7fa7d00d7040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a266e4c59b63aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.220+0000 7fa7cffff640 1 -- 172.21.15.103:0/2297865818 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0021040 con 0x561780d1bb40 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.220+0000 7fa7cffff640 1 -- 172.21.15.103:0/2297865818 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0005680 con 0x561780d1bb40 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.220+0000 7fa7cffff640 1 -- 172.21.15.103:0/2297865818 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0004340 con 0x561780d1bb40 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2297865818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 msgr2=0x561780d455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d455e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00d0de0 tx=0x7fa7d00d7040 comp rx=0 tx=0).stop 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2297865818 shutdown_connections 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d2db80 0x561780d45b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d455e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/2297865818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d1a5d0 0x561780d16e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2297865818 >> 172.21.15.103:0/2297865818 conn(0x561780ce3310 msgr2=0x561780d16700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2297865818 shutdown_connections 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.221+0000 7fa7eebf5740 1 -- 172.21.15.103:0/2297865818 wait complete. 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 -- start start 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 Processor -- start 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d35c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d2db80 0x561780d32c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81acb80 con 0x561780d2db80 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ac4b0 con 0x561780d1a5d0 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81a9730 con 0x561780d1bb40 2024-09-19T07:53:00.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7eebf5740 10 client.0 _ll_register_callbacks cb 0x7fa7d8009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7dffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46166/0 (socket says 172.21.15.103:46166) 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.222+0000 7fa7dffff640 1 -- 172.21.15.103:0/3085426668 learned_addr learned my addr 172.21.15.103:0/3085426668 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7e659c640 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d35c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7dffff640 1 -- 172.21.15.103:0/3085426668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 msgr2=0x561780d35c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7dffff640 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d35c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7dffff640 1 -- 172.21.15.103:0/3085426668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d2db80 msgr2=0x561780d32c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:00.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7dffff640 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d2db80 0x561780d32c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7dffff640 1 -- 172.21.15.103:0/3085426668 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561780d368b0 con 0x561780d1bb40 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7dffff640 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d361c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000c00 tx=0x7fa7d009da20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3671d5937d32f248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7cdffb640 1 -- 172.21.15.103:0/3085426668 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0021070 con 0x561780d1bb40 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7cdffb640 1 -- 172.21.15.103:0/3085426668 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fa7d008a0b0 con 0x561780d1bb40 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.223+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d0005680 con 0x561780d1bb40 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.224+0000 7fa7cdffb640 1 -- 172.21.15.103:0/3085426668 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa7d009f980 con 0x561780d1bb40 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.224+0000 7fa7eebf5740 10 client.27640 fetch_fsmap learned FSMap version 52 2024-09-19T07:53:00.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.224+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561780da8bc0 con 0x561780d1bb40 2024-09-19T07:53:00.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.224+0000 7fa7cdffb640 1 -- 172.21.15.103:0/3085426668 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0005680 con 0x561780d1bb40 2024-09-19T07:53:00.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7cdffb640 1 -- 172.21.15.103:0/3085426668 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7fa7d009bff0 con 0x561780d1bb40 2024-09-19T07:53:00.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7eebf5740 10 client.27640 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:53:00.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7eebf5740 10 client.27640 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:53:00.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7eebf5740 20 client.27640 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780da9290 0x561780dab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7eebf5740 4 client.27640 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561780c87d70 con 0x561780da9290 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7e659c640 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780da9290 0x561780dab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7e659c640 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780da9290 0x561780dab6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31699e8c54aa7019 server_cookie=90747d4addcbf866 in_seq=0 out_seq=0 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.225+0000 7fa7cdffb640 10 client.27640 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7cdffb640 1 -- 172.21.15.103:0/3085426668 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561780c87d70 con 0x561780da9290 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7cdffb640 10 client.27640 handle_command_reply: tid=0 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 client.27640 shutdown 2024-09-19T07:53:00.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 msgr2=0x561780d361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d361c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000c00 tx=0x7fa7d009da20 comp rx=0 tx=0).stop 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 shutdown_connections 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x561780da9290 0x561780dab6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561780d2db80 0x561780d32c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7cdffb640 0 client.27640 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561780d1bb40 0x561780d361c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 --2- 172.21.15.103:0/3085426668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561780d1a5d0 0x561780d35c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 >> 172.21.15.103:0/3085426668 conn(0x561780ce3310 msgr2=0x561780d154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 shutdown_connections 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 1 -- 172.21.15.103:0/3085426668 wait complete. 2024-09-19T07:53:00.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.226+0000 7fa7eebf5740 20 client.27640 trim_cache size 0 max 16384 2024-09-19T07:53:00.231 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa7d40a34b0 msgr2=0x7fa7d40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa7d40a34b0 0x7fa7d40a5970 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x561780d17ac0 tx=0x7fa7d8005f90 comp rx=0 tx=0).stop 2024-09-19T07:53:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e8103650 msgr2=0x7fa7e81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e8103650 0x7fa7e81986f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000be90 tx=0x7fa7d000dca0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 shutdown_connections 2024-09-19T07:53:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa7d40a34b0 0x7fa7d40a5970 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7e810c160 0x7fa7e819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7e8103650 0x7fa7e81986f0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 --2- 172.21.15.103:0/3486006206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e8102d30 0x7fa7e81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 >> 172.21.15.103:0/3486006206 conn(0x7fa7e80fc7d0 msgr2=0x7fa7e810dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 shutdown_connections 2024-09-19T07:53:00.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:00.227+0000 7fa7ec81c640 1 -- 172.21.15.103:0/3486006206 wait complete. 2024-09-19T07:53:00.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:00 smithi134 ceph-mon[35414]: pgmap v469: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 997 op/s 2024-09-19T07:53:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:00 smithi174 ceph-mon[35370]: pgmap v469: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 997 op/s 2024-09-19T07:53:00.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 865.0 s, 2227.2 tps, lat 44.972 ms stddev 49.853 2024-09-19T07:53:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:00 smithi103 ceph-mon[31201]: pgmap v469: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 16 MiB/s wr, 997 op/s 2024-09-19T07:53:01.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:53:01.397 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:01.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.958+0000 7f12b54c9640 1 -- 172.21.15.103:0/1499834562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b010d360 msgr2=0x7f12b0114ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:01.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.958+0000 7f12b54c9640 1 --2- 172.21.15.103:0/1499834562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b010d360 0x7f12b0114ee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f12a801acb0 tx=0x7f12a803b920 comp rx=0 tx=0).stop 2024-09-19T07:53:01.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.961+0000 7f12b54c9640 1 -- 172.21.15.103:0/1499834562 shutdown_connections 2024-09-19T07:53:01.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.961+0000 7f12b54c9640 1 --2- 172.21.15.103:0/1499834562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b010d360 0x7f12b0114ee0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.961+0000 7f12b54c9640 1 --2- 172.21.15.103:0/1499834562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b0108c30 0x7f12b010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.961+0000 7f12b54c9640 1 --2- 172.21.15.103:0/1499834562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b0108280 0x7f12b0108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.961+0000 7f12b54c9640 1 -- 172.21.15.103:0/1499834562 >> 172.21.15.103:0/1499834562 conn(0x7f12b00779f0 msgr2=0x7f12b0079e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:01.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.965+0000 7f12b54c9640 1 -- 172.21.15.103:0/1499834562 shutdown_connections 2024-09-19T07:53:01.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.965+0000 7f12b54c9640 1 -- 172.21.15.103:0/1499834562 wait complete. 2024-09-19T07:53:01.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.966+0000 7f12b54c9640 1 Processor -- start 2024-09-19T07:53:01.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.966+0000 7f12b54c9640 1 -- start start 2024-09-19T07:53:01.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.967+0000 7f12b54c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b0108c30 0x7f12b0114c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.967+0000 7f12b54c9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b010d360 0x7f12b0110cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.967+0000 7f12b54c9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b0111210 0x7f12b01116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.967+0000 7f12b54c9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12b006e630 con 0x7f12b0111210 2024-09-19T07:53:01.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.967+0000 7f12b54c9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12b006e4b0 con 0x7f12b010d360 2024-09-19T07:53:01.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.967+0000 7f12b54c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12b006e7b0 con 0x7f12b0108c30 2024-09-19T07:53:01.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.970+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b010d360 0x7f12b0110cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.973+0000 7f12a7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b010d360 0x7f12b0110cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45820/0 (socket says 172.21.15.103:45820) 2024-09-19T07:53:01.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.973+0000 7f12a7fff640 1 -- 172.21.15.103:0/2560746645 learned_addr learned my addr 172.21.15.103:0/2560746645 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:01.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.973+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b0111210 0x7f12b01116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.974+0000 7f12b4cc8640 1 -- 172.21.15.103:0/2560746645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b0108c30 msgr2=0x7f12b0114c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:53:01.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.974+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b0108c30 0x7f12b0114c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.974+0000 7f12b4cc8640 1 -- 172.21.15.103:0/2560746645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b010d360 msgr2=0x7f12b0110cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:01.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.974+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b010d360 0x7f12b0110cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.974+0000 7f12b4cc8640 1 -- 172.21.15.103:0/2560746645 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01c4870 con 0x7f12b0111210 2024-09-19T07:53:01.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.974+0000 7f12a7fff640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b010d360 0x7f12b0110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:01.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.977+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b0111210 0x7f12b01116c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f12a803bf10 tx=0x7f12a80405a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:01.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.977+0000 7f12adffb640 1 -- 172.21.15.103:0/2560746645 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011020 con 0x7f12b0111210 2024-09-19T07:53:01.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.977+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01c4b00 con 0x7f12b0111210 2024-09-19T07:53:01.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.978+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01c4fe0 con 0x7f12b0111210 2024-09-19T07:53:01.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.978+0000 7f12adffb640 1 -- 172.21.15.103:0/2560746645 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a8040710 con 0x7f12b0111210 2024-09-19T07:53:01.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.978+0000 7f12adffb640 1 -- 172.21.15.103:0/2560746645 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8045030 con 0x7f12b0111210 2024-09-19T07:53:01.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.981+0000 7f12b8b04740 1 Processor -- start 2024-09-19T07:53:01.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.983+0000 7f12adffb640 1 -- 172.21.15.103:0/2560746645 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f12a8014020 con 0x7f12b0111210 2024-09-19T07:53:01.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.985+0000 7f12adffb640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f129c0a3540 0x7f129c0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.986+0000 7f12a7fff640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f129c0a3540 0x7f129c0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.987+0000 7f12adffb640 1 -- 172.21.15.103:0/2560746645 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7f12a8003d80 con 0x7f12b0111210 2024-09-19T07:53:01.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.989+0000 7f12b8b04740 1 -- start start 2024-09-19T07:53:01.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.989+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692268cf0 0x55f692267240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.989+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692267780 0x55f692267bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.989+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692274910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:01.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.989+0000 7f12b8b04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6921cd2f0 con 0x55f692267780 2024-09-19T07:53:01.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.989+0000 7f12b8b04740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f692174cf0 con 0x55f692272520 2024-09-19T07:53:01.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.989+0000 7f12b8b04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6921c92b0 con 0x55f692268cf0 2024-09-19T07:53:01.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.991+0000 7f12b4cc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692274910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:01.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.991+0000 7f12b4cc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692274910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45834/0 (socket says 172.21.15.103:45834) 2024-09-19T07:53:01.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.991+0000 7f12b4cc8640 1 -- 172.21.15.103:0/1328790970 learned_addr learned my addr 172.21.15.103:0/1328790970 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:01.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.993+0000 7f12b4cc8640 1 -- 172.21.15.103:0/1328790970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692268cf0 msgr2=0x55f692267240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:53:01.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.993+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/1328790970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692268cf0 0x55f692267240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.993+0000 7f12b4cc8640 1 -- 172.21.15.103:0/1328790970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692267780 msgr2=0x55f692267bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:01.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.993+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/1328790970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692267780 0x55f692267bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:01.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.993+0000 7f12b4cc8640 1 -- 172.21.15.103:0/1328790970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6921593a0 con 0x55f692272520 2024-09-19T07:53:01.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:01.995+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/1328790970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692274910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12a80b90c0 tx=0x7f12a8042c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4456dbaa3384c9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.000+0000 7f12a67fc640 1 -- 172.21.15.103:0/1328790970 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a80d4030 con 0x55f692272520 2024-09-19T07:53:02.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.000+0000 7f12a67fc640 1 -- 172.21.15.103:0/1328790970 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a80bc6f0 con 0x55f692272520 2024-09-19T07:53:02.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.000+0000 7f12a67fc640 1 -- 172.21.15.103:0/1328790970 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a80b8d60 con 0x55f692272520 2024-09-19T07:53:02.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.002+0000 7f12b8b04740 1 -- 172.21.15.103:0/1328790970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 msgr2=0x55f692274910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.002+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1328790970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692274910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12a80b90c0 tx=0x7f12a8042c50 comp rx=0 tx=0).stop 2024-09-19T07:53:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.004+0000 7f12b8b04740 1 -- 172.21.15.103:0/1328790970 shutdown_connections 2024-09-19T07:53:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.004+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1328790970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692274910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.004+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1328790970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692267780 0x55f692267bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.004+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1328790970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692268cf0 0x55f692267240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.004+0000 7f12b8b04740 1 -- 172.21.15.103:0/1328790970 >> 172.21.15.103:0/1328790970 conn(0x55f692230310 msgr2=0x55f692263e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:02.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.007+0000 7f12a7fff640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f129c0a3540 0x7f129c0a5a00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f12b0112490 tx=0x7f12a0010990 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.010+0000 7f12b8b04740 1 -- 172.21.15.103:0/1328790970 shutdown_connections 2024-09-19T07:53:02.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.014+0000 7f12b8b04740 1 -- 172.21.15.103:0/1328790970 wait complete. 2024-09-19T07:53:02.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.015+0000 7f12b8b04740 1 -- start start 2024-09-19T07:53:02.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.015+0000 7f12b8b04740 1 Processor -- start 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.019+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f692271f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.019+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69226f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.019+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f69226f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.019+0000 7f12b8b04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f692174cf0 con 0x55f692272520 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.019+0000 7f12b8b04740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f692188060 con 0x55f692268cf0 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.019+0000 7f12b8b04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f692274fe0 con 0x55f692267780 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.019+0000 7f12b8b04740 10 client.0 _ll_register_callbacks cb 0x55f691edb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.020+0000 7f12b4cc8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f69226f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.020+0000 7f12b4cc8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f69226f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43854/0 (socket says 172.21.15.103:43854) 2024-09-19T07:53:02.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.020+0000 7f12b4cc8640 1 -- 172.21.15.103:0/3613400878 learned_addr learned my addr 172.21.15.103:0/3613400878 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:02.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.021+0000 7f12affff640 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f692271f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.023+0000 7f12a7fff640 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69226f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.023+0000 7f12b4cc8640 1 -- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 msgr2=0x55f692271f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:02.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.023+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f692271f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.023+0000 7f12b4cc8640 1 -- 172.21.15.103:0/3613400878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 msgr2=0x55f69226f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:02.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.023+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69226f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.023+0000 7f12b4cc8640 1 -- 172.21.15.103:0/3613400878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f691f3e3c0 con 0x55f692272520 2024-09-19T07:53:02.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.029+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f69226f7b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f12a801acb0 tx=0x7f12a80c0b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfeabf47d9c098fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.030+0000 7f128f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:02.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.034+0000 7f128f7fe640 1 -- 172.21.15.103:0/3613400878 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a80d4070 con 0x55f692272520 2024-09-19T07:53:02.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.034+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a80bc6f0 con 0x55f692272520 2024-09-19T07:53:02.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.034+0000 7f128f7fe640 1 -- 172.21.15.103:0/3613400878 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7f12a80edde0 con 0x55f692272520 2024-09-19T07:53:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.040+0000 7f128f7fe640 1 -- 172.21.15.103:0/3613400878 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f12a80edfe0 con 0x55f692272520 2024-09-19T07:53:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.041+0000 7f12b8b04740 10 client.18384 fetch_fsmap learned FSMap version 52 2024-09-19T07:53:02.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.041+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f6922808c0 con 0x55f692272520 2024-09-19T07:53:02.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f128f7fe640 1 -- 172.21.15.103:0/3613400878 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a80bc6f0 con 0x55f692272520 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f128f7fe640 1 -- 172.21.15.103:0/3613400878 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7f12a80bb5b0 con 0x55f692272520 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f12b8b04740 10 client.18384 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f12b8b04740 10 client.18384 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f12b8b04740 20 client.18384 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f12b8b04740 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f692281c10 0x55f692284020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f12b8b04740 4 client.18384 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:02.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6921da860 con 0x55f692281c10 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.045+0000 7f12affff640 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f692281c10 0x55f692284020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.047+0000 7f12affff640 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f692281c10 0x55f692284020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c809028e6f10284 server_cookie=7246a139acecc495 in_seq=0 out_seq=0 2024-09-19T07:53:02.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.047+0000 7f128f7fe640 10 client.18384 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.048+0000 7f128f7fe640 1 -- 172.21.15.103:0/3613400878 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6921da860 con 0x55f692281c10 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.048+0000 7f128f7fe640 10 client.18384 handle_command_reply: tid=0 2024-09-19T07:53:02.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.050+0000 7f12b8b04740 1 client.18384 shutdown 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.052+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 msgr2=0x55f69226f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.052+0000 7f12b8b04740 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f69226f7b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f12a801acb0 tx=0x7f12a80c0b70 comp rx=0 tx=0).stop 2024-09-19T07:53:02.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.055+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 shutdown_connections 2024-09-19T07:53:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.055+0000 7f12b8b04740 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f692281c10 0x55f692284020 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.055+0000 7f12b8b04740 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f69226f7b0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.055+0000 7f12b8b04740 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69226f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.055+0000 7f12b8b04740 1 --2- 172.21.15.103:0/3613400878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f692271f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.055+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 >> 172.21.15.103:0/3613400878 conn(0x55f692230310 msgr2=0x55f692269b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.055+0000 7f128f7fe640 0 client.18384 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:02.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.056+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 shutdown_connections 2024-09-19T07:53:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.056+0000 7f12b8b04740 1 -- 172.21.15.103:0/3613400878 wait complete. 2024-09-19T07:53:02.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.056+0000 7f12b8b04740 20 client.18384 trim_cache size 0 max 16384 2024-09-19T07:53:02.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.085+0000 7f12b8b04740 1 Processor -- start 2024-09-19T07:53:02.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.087+0000 7f12b8b04740 1 -- start start 2024-09-19T07:53:02.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.087+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692268cf0 0x55f6922898f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692289e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12b8b04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f129000e8a0 con 0x55f692268cf0 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12affff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12b8b04740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12900103e0 con 0x55f692272520 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12b8b04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f129000ef70 con 0x55f692267780 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12affff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46204/0 (socket says 172.21.15.103:46204) 2024-09-19T07:53:02.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.088+0000 7f12affff640 1 -- 172.21.15.103:0/1780109247 learned_addr learned my addr 172.21.15.103:0/1780109247 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:02.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.090+0000 7f12a7fff640 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692268cf0 0x55f6922898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.090+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692289e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.093+0000 7f12affff640 1 -- 172.21.15.103:0/1780109247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 msgr2=0x55f692289e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.093+0000 7f12affff640 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692289e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.093+0000 7f12affff640 1 -- 172.21.15.103:0/1780109247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692268cf0 msgr2=0x55f6922898f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.093+0000 7f12affff640 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692268cf0 0x55f6922898f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.093+0000 7f12affff640 1 -- 172.21.15.103:0/1780109247 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6922648e0 con 0x55f692267780 2024-09-19T07:53:02.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.096+0000 7f12affff640 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227e0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1298006ed0 tx=0x7f129800b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14419edd6f439af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.100+0000 7f12a5ffb640 1 -- 172.21.15.103:0/1780109247 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129801a040 con 0x55f692267780 2024-09-19T07:53:02.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.100+0000 7f12a5ffb640 1 -- 172.21.15.103:0/1780109247 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1298014d80 con 0x55f692267780 2024-09-19T07:53:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.100+0000 7f12a5ffb640 1 -- 172.21.15.103:0/1780109247 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298016d50 con 0x55f692267780 2024-09-19T07:53:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.102+0000 7f12b8b04740 1 -- 172.21.15.103:0/1780109247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 msgr2=0x55f69227e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.102+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227e0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1298006ed0 tx=0x7f129800b470 comp rx=0 tx=0).stop 2024-09-19T07:53:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.102+0000 7f12b8b04740 1 -- 172.21.15.103:0/1780109247 shutdown_connections 2024-09-19T07:53:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.102+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692272520 0x55f692289e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.102+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692268cf0 0x55f6922898f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.102+0000 7f12b8b04740 1 --2- 172.21.15.103:0/1780109247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227e0c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.102+0000 7f12b8b04740 1 -- 172.21.15.103:0/1780109247 >> 172.21.15.103:0/1780109247 conn(0x55f692230310 msgr2=0x55f69226e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:02.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.105+0000 7f12b8b04740 1 -- 172.21.15.103:0/1780109247 shutdown_connections 2024-09-19T07:53:02.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.105+0000 7f12b8b04740 1 -- 172.21.15.103:0/1780109247 wait complete. 2024-09-19T07:53:02.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.105+0000 7f12b8b04740 1 -- start start 2024-09-19T07:53:02.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.106+0000 7f12b8b04740 1 Processor -- start 2024-09-19T07:53:02.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.106+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12b8b04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69227ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12affff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12affff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46216/0 (socket says 172.21.15.103:46216) 2024-09-19T07:53:02.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12affff640 1 -- 172.21.15.103:0/2333619199 learned_addr learned my addr 172.21.15.103:0/2333619199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:02.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12b8b04740 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f692275b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f129000fd10 con 0x55f692272520 2024-09-19T07:53:02.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12a7fff640 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69227ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.107+0000 7f12b4cc8640 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f692275b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f129000f640 con 0x55f692268cf0 2024-09-19T07:53:02.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f129000c8a0 con 0x55f692267780 2024-09-19T07:53:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12b8b04740 10 client.0 _ll_register_callbacks cb 0x7f12a8042b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12a7fff640 1 -- 172.21.15.103:0/2333619199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 msgr2=0x55f69227a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12a7fff640 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12a7fff640 1 -- 172.21.15.103:0/2333619199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 msgr2=0x55f692275b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12a7fff640 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f692275b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.108+0000 7f12a7fff640 1 -- 172.21.15.103:0/2333619199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f692276370 con 0x55f692268cf0 2024-09-19T07:53:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.112+0000 7f12a7fff640 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69227ab30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f6922724f0 tx=0x7f12a0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34892a06e0287436 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:02.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.112+0000 7f128f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:53:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.114+0000 7f128f7fe640 1 -- 172.21.15.103:0/2333619199 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0011ca0 con 0x55f692268cf0 2024-09-19T07:53:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.114+0000 7f128f7fe640 1 -- 172.21.15.103:0/2333619199 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7f12a0004020 con 0x55f692268cf0 2024-09-19T07:53:02.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.116+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1298014d80 con 0x55f692268cf0 2024-09-19T07:53:02.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.117+0000 7f128f7fe640 1 -- 172.21.15.103:0/2333619199 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f12a0011e40 con 0x55f692268cf0 2024-09-19T07:53:02.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.119+0000 7f12b8b04740 10 client.27698 fetch_fsmap learned FSMap version 52 2024-09-19T07:53:02.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.119+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f6922ec560 con 0x55f692268cf0 2024-09-19T07:53:02.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.120+0000 7f128f7fe640 1 -- 172.21.15.103:0/2333619199 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1298014d80 con 0x55f692268cf0 2024-09-19T07:53:02.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.120+0000 7f128f7fe640 1 -- 172.21.15.103:0/2333619199 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7f12a000e430 con 0x55f692268cf0 2024-09-19T07:53:02.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.120+0000 7f12b8b04740 10 client.27698 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:53:02.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.120+0000 7f12b8b04740 10 client.27698 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:53:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.120+0000 7f12b8b04740 20 client.27698 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.121+0000 7f12b8b04740 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f6922ecce0 0x55f6922ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.121+0000 7f12b8b04740 4 client.27698 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:02.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.121+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6921d4d70 con 0x55f6922ecce0 2024-09-19T07:53:02.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.122+0000 7f12affff640 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f6922ecce0 0x55f6922ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:02.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.123+0000 7f12affff640 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f6922ecce0 0x55f6922ef100 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba3099ddb0cbe059 server_cookie=ea6ff00d8c360f03 in_seq=0 out_seq=0 2024-09-19T07:53:02.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.123+0000 7f128f7fe640 10 client.27698 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:02.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.124+0000 7f128f7fe640 1 -- 172.21.15.103:0/2333619199 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6921d4d70 con 0x55f6922ecce0 2024-09-19T07:53:02.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.124+0000 7f128f7fe640 10 client.27698 handle_command_reply: tid=0 2024-09-19T07:53:02.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.124+0000 7f12b8b04740 1 client.27698 shutdown 2024-09-19T07:53:02.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 msgr2=0x55f69227ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69227ab30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f6922724f0 tx=0x7f12a0004730 comp rx=0 tx=0).stop 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 shutdown_connections 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x55f6922ecce0 0x55f6922ef100 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f692272520 0x55f692275b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f692268cf0 0x55f69227ab30 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f6922724f0 tx=0x7f12a0004730 comp rx=0 tx=0).stop 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 --2- 172.21.15.103:0/2333619199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f692267780 0x55f69227a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 >> 172.21.15.103:0/2333619199 conn(0x55f692230310 msgr2=0x55f692270a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.126+0000 7f128f7fe640 0 client.27698 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.127+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 shutdown_connections 2024-09-19T07:53:02.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.127+0000 7f12b8b04740 1 -- 172.21.15.103:0/2333619199 wait complete. 2024-09-19T07:53:02.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.128+0000 7f12b8b04740 20 client.27698 trim_cache size 0 max 16384 2024-09-19T07:53:02.128 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.132+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f129c0a3540 msgr2=0x7f129c0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.132+0000 7f12b54c9640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f129c0a3540 0x7f129c0a5a00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f12b0112490 tx=0x7f12a0010990 comp rx=0 tx=0).stop 2024-09-19T07:53:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.132+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b0111210 msgr2=0x7f12b01116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:02.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.132+0000 7f12b54c9640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b0111210 0x7f12b01116c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f12a803bf10 tx=0x7f12a80405a0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.133+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 shutdown_connections 2024-09-19T07:53:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.133+0000 7f12b54c9640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f129c0a3540 0x7f129c0a5a00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.133+0000 7f12b54c9640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b0111210 0x7f12b01116c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.133+0000 7f12b54c9640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b010d360 0x7f12b0110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.133+0000 7f12b54c9640 1 --2- 172.21.15.103:0/2560746645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b0108c30 0x7f12b0114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.133+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 >> 172.21.15.103:0/2560746645 conn(0x7f12b00779f0 msgr2=0x7f12b0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:02.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.133+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 shutdown_connections 2024-09-19T07:53:02.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:02.134+0000 7f12b54c9640 1 -- 172.21.15.103:0/2560746645 wait complete. 2024-09-19T07:53:02.401 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:02 smithi103 ceph-mon[31201]: pgmap v470: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T07:53:02.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:02 smithi174 ceph-mon[35370]: pgmap v470: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T07:53:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:02 smithi134 ceph-mon[35414]: pgmap v470: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 20 MiB/s wr, 1.27k op/s 2024-09-19T07:53:02.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:53:03.304 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 -- 172.21.15.103:0/723417683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510102790 msgr2=0x7fd5101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 --2- 172.21.15.103:0/723417683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510102790 0x7fd5101067e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd508009be0 tx=0x7fd50802a6d0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 -- 172.21.15.103:0/723417683 shutdown_connections 2024-09-19T07:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 --2- 172.21.15.103:0/723417683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd510106f10 0x7fd51010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 --2- 172.21.15.103:0/723417683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510102790 0x7fd5101067e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 --2- 172.21.15.103:0/723417683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510101de0 0x7fd5101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 -- 172.21.15.103:0/723417683 >> 172.21.15.103:0/723417683 conn(0x7fd5100fd8e0 msgr2=0x7fd5100ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 -- 172.21.15.103:0/723417683 shutdown_connections 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.913+0000 7fd516769640 1 -- 172.21.15.103:0/723417683 wait complete. 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.914+0000 7fd516769640 1 Processor -- start 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.914+0000 7fd516769640 1 -- start start 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd516769640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510101de0 0x7fd5100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd516769640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510102790 0x7fd510074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd516769640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd510106f10 0x7fd510074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd516769640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd510111f20 con 0x7fd510106f10 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd516769640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd510111da0 con 0x7fd510102790 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd516769640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101120a0 con 0x7fd510101de0 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510102790 0x7fd510074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd514d06640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd510106f10 0x7fd510074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510101de0 0x7fd5100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510102790 0x7fd510074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45878/0 (socket says 172.21.15.103:45878) 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd514d06640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd510106f10 0x7fd510074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43890/0 (socket says 172.21.15.103:43890) 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd514d06640 1 -- 172.21.15.103:0/4007902207 learned_addr learned my addr 172.21.15.103:0/4007902207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50ffff640 1 -- 172.21.15.103:0/4007902207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510102790 msgr2=0x7fd510074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50ffff640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510102790 0x7fd510074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50ffff640 1 -- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd510106f10 msgr2=0x7fd510074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50ffff640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd510106f10 0x7fd510074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.915+0000 7fd50ffff640 1 -- 172.21.15.103:0/4007902207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5101a68e0 con 0x7fd510101de0 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.916+0000 7fd50ffff640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510101de0 0x7fd5100744f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd50000d6d0 tx=0x7fd50000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.916+0000 7fd50d7fa640 1 -- 172.21.15.103:0/4007902207 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500017070 con 0x7fd510101de0 2024-09-19T07:53:03.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.916+0000 7fd50d7fa640 1 -- 172.21.15.103:0/4007902207 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd500010460 con 0x7fd510101de0 2024-09-19T07:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.916+0000 7fd50d7fa640 1 -- 172.21.15.103:0/4007902207 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50000f620 con 0x7fd510101de0 2024-09-19T07:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.917+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5101981e0 con 0x7fd510101de0 2024-09-19T07:53:03.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.917+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd510198610 con 0x7fd510101de0 2024-09-19T07:53:03.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.920+0000 7fd518b42740 1 Processor -- start 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.922+0000 7fd50d7fa640 1 -- 172.21.15.103:0/4007902207 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd50001a070 con 0x7fd510101de0 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.923+0000 7fd50d7fa640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd4fc0abd90 0x7fd4fc0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.923+0000 7fd50d7fa640 1 -- 172.21.15.103:0/4007902207 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fd5000cf8b0 con 0x7fd510101de0 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.923+0000 7fd518b42740 1 -- start start 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.924+0000 7fd518b42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.924+0000 7fd518b42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.924+0000 7fd518b42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b78380 0x5624e5b7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.924+0000 7fd518b42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5624e5ac22f0 con 0x5624e5b5db40 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.924+0000 7fd518b42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5624e5a69cf0 con 0x5624e5b5c5d0 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.924+0000 7fd518b42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5624e5abe2b0 con 0x5624e5b78380 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.927+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.927+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b5c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43896/0 (socket says 172.21.15.103:43896) 2024-09-19T07:53:03.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.927+0000 7fd50ffff640 1 -- 172.21.15.103:0/2649597780 learned_addr learned my addr 172.21.15.103:0/2649597780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.929+0000 7fd514d06640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd4fc0abd90 0x7fd4fc0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.929+0000 7fd514d06640 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b78380 0x5624e5b7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.930+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.931+0000 7fd50ffff640 1 -- 172.21.15.103:0/2649597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b78380 msgr2=0x5624e5b7a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.931+0000 7fd50ffff640 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b78380 0x5624e5b7a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.931+0000 7fd50ffff640 1 -- 172.21.15.103:0/2649597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 msgr2=0x5624e5b5ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.931+0000 7fd50ffff640 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b5ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.931+0000 7fd50ffff640 1 -- 172.21.15.103:0/2649597780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e5a4e3a0 con 0x5624e5b5db40 2024-09-19T07:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.931+0000 7fd50ffff640 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b5c090 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd5000984f0 tx=0x7fd50009b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8936fa6acf7841 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.932+0000 7fd5067fc640 1 -- 172.21.15.103:0/2649597780 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500021040 con 0x5624e5b5db40 2024-09-19T07:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.932+0000 7fd5067fc640 1 -- 172.21.15.103:0/2649597780 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd50009ce50 con 0x5624e5b5db40 2024-09-19T07:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.932+0000 7fd5067fc640 1 -- 172.21.15.103:0/2649597780 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000a4560 con 0x5624e5b5db40 2024-09-19T07:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.933+0000 7fd518b42740 1 -- 172.21.15.103:0/2649597780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 msgr2=0x5624e5b5c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.934+0000 7fd518b42740 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b5c090 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd5000984f0 tx=0x7fd50009b5b0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.943+0000 7fd518b42740 1 -- 172.21.15.103:0/2649597780 shutdown_connections 2024-09-19T07:53:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.943+0000 7fd518b42740 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b78380 0x5624e5b7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.943+0000 7fd518b42740 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b5ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.943+0000 7fd518b42740 1 --2- 172.21.15.103:0/2649597780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b5c090 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.943+0000 7fd518b42740 1 -- 172.21.15.103:0/2649597780 >> 172.21.15.103:0/2649597780 conn(0x5624e5b25310 msgr2=0x5624e5b58c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.946+0000 7fd514d06640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd4fc0abd90 0x7fd4fc0ae250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5624e5b5d070 tx=0x7fd4f80098c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.948+0000 7fd518b42740 1 -- 172.21.15.103:0/2649597780 shutdown_connections 2024-09-19T07:53:03.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.950+0000 7fd518b42740 1 -- 172.21.15.103:0/2649597780 wait complete. 2024-09-19T07:53:03.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.950+0000 7fd518b42740 1 -- start start 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.950+0000 7fd518b42740 1 Processor -- start 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd518b42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5c5d0 0x5624e5b69420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd518b42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd518b42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b64740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd518b42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5624e5b5f550 con 0x5624e5b5c5d0 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd518b42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5624e5a7d060 con 0x5624e5b78380 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd518b42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5624e5b5f6d0 con 0x5624e5b5db40 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd518b42740 10 client.0 _ll_register_callbacks cb 0x5624e57d08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd50f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd50f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b69960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46252/0 (socket says 172.21.15.103:46252) 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd50f7fe640 1 -- 172.21.15.103:0/1429515997 learned_addr learned my addr 172.21.15.103:0/1429515997 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.951+0000 7fd514d06640 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b64740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.952+0000 7fd50f7fe640 1 -- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 msgr2=0x5624e5b64740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.952+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b64740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.952+0000 7fd50f7fe640 1 -- 172.21.15.103:0/1429515997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5c5d0 msgr2=0x5624e5b69420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.952+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5c5d0 0x5624e5b69420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.952+0000 7fd50f7fe640 1 -- 172.21.15.103:0/1429515997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624e58333c0 con 0x5624e5b5db40 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.955+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b69960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd508009be0 tx=0x7fd5080092c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5596da20ef75438 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.955+0000 7fd4ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.955+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/1429515997 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508036040 con 0x5624e5b5db40 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.956+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd50009ce50 con 0x5624e5b5db40 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.956+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/1429515997 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fd5080307d0 con 0x5624e5b5db40 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.956+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/1429515997 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd508003970 con 0x5624e5b5db40 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.956+0000 7fd518b42740 10 client.27664 fetch_fsmap learned FSMap version 52 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.956+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5624e5b730f0 con 0x5624e5b5db40 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.956+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/1429515997 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508037040 con 0x5624e5b5db40 2024-09-19T07:53:03.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.958+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/1429515997 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7fd50803c040 con 0x5624e5b5db40 2024-09-19T07:53:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.958+0000 7fd518b42740 10 client.27664 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:53:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.958+0000 7fd518b42740 10 client.27664 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:53:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.958+0000 7fd518b42740 20 client.27664 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.958+0000 7fd518b42740 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5b74a20 0x5624e5b76e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.958+0000 7fd518b42740 4 client.27664 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.958+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd50009ce50 con 0x5624e5b74a20 2024-09-19T07:53:03.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.959+0000 7fd50ffff640 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5b74a20 0x5624e5b76e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.962+0000 7fd50ffff640 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5b74a20 0x5624e5b76e60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ff710e3eb115969 server_cookie=bd9c8b47ea88ea7f in_seq=0 out_seq=0 2024-09-19T07:53:03.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.962+0000 7fd4ef7fe640 10 client.27664 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.963+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/1429515997 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd50009ce50 con 0x5624e5b74a20 2024-09-19T07:53:03.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.963+0000 7fd4ef7fe640 10 client.27664 handle_command_reply: tid=0 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.963+0000 7fd518b42740 1 client.27664 shutdown 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 msgr2=0x5624e5b69960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b69960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd508009be0 tx=0x7fd5080092c0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 shutdown_connections 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5b74a20 0x5624e5b76e60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b64740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b69960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 --2- 172.21.15.103:0/1429515997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5c5d0 0x5624e5b69420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 >> 172.21.15.103:0/1429515997 conn(0x5624e5b25310 msgr2=0x5624e5b5e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd4ef7fe640 0 client.27664 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 shutdown_connections 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.964+0000 7fd518b42740 1 -- 172.21.15.103:0/1429515997 wait complete. 2024-09-19T07:53:03.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.965+0000 7fd518b42740 20 client.27664 trim_cache size 0 max 16384 2024-09-19T07:53:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 Processor -- start 2024-09-19T07:53:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 -- start start 2024-09-19T07:53:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b58700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b58c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b78380 0x5624e5b64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5100709a0 con 0x5624e5b78380 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5100724e0 con 0x5624e5b5c5d0 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd518b42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd510071070 con 0x5624e5b5db40 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd514d06640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b78380 0x5624e5b64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b58700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd50f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b58c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b58700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45900/0 (socket says 172.21.15.103:45900) 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.986+0000 7fd50ffff640 1 -- 172.21.15.103:0/118277427 learned_addr learned my addr 172.21.15.103:0/118277427 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd514d06640 1 -- 172.21.15.103:0/118277427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 msgr2=0x5624e5b58c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd514d06640 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b58c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd514d06640 1 -- 172.21.15.103:0/118277427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 msgr2=0x5624e5b58700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd514d06640 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b58700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd514d06640 1 -- 172.21.15.103:0/118277427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624e5b59180 con 0x5624e5b78380 2024-09-19T07:53:03.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd50ffff640 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b58700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:03.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd514d06640 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b78380 0x5624e5b64000 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009db0 tx=0x7fd4f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb76c7d938c5428 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd505ffb640 1 -- 172.21.15.103:0/118277427 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8013330 con 0x5624e5b78380 2024-09-19T07:53:03.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd505ffb640 1 -- 172.21.15.103:0/118277427 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f8005500 con 0x5624e5b78380 2024-09-19T07:53:03.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.987+0000 7fd505ffb640 1 -- 172.21.15.103:0/118277427 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80216b0 con 0x5624e5b78380 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.988+0000 7fd518b42740 1 -- 172.21.15.103:0/118277427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b78380 msgr2=0x5624e5b64000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.988+0000 7fd518b42740 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b78380 0x5624e5b64000 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009db0 tx=0x7fd4f8004730 comp rx=0 tx=0).stop 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 -- 172.21.15.103:0/118277427 shutdown_connections 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b78380 0x5624e5b64000 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5db40 0x5624e5b58c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 --2- 172.21.15.103:0/118277427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b5c5d0 0x5624e5b58700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 -- 172.21.15.103:0/118277427 >> 172.21.15.103:0/118277427 conn(0x5624e5b25310 msgr2=0x5624e5b77740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:03.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 -- 172.21.15.103:0/118277427 shutdown_connections 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 -- 172.21.15.103:0/118277427 wait complete. 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.989+0000 7fd518b42740 1 -- start start 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 1 Processor -- start 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5c5d0 0x5624e5b61910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b62390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5c5d0 0x5624e5b61910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd510071e10 con 0x5624e5b5db40 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5c5d0 0x5624e5b61910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46266/0 (socket says 172.21.15.103:46266) 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd510071740 con 0x5624e5b78380 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 -- 172.21.15.103:0/3823983977 learned_addr learned my addr 172.21.15.103:0/3823983977 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd51006e9a0 con 0x5624e5b5c5d0 2024-09-19T07:53:03.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd514d06640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b62390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd518b42740 10 client.0 _ll_register_callbacks cb 0x7fd4f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 -- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 msgr2=0x5624e5b62390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b62390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 -- 172.21.15.103:0/3823983977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 msgr2=0x5624e5b61e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b61e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50ffff640 1 -- 172.21.15.103:0/3823983977 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624e5c02e40 con 0x5624e5b5c5d0 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.990+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b61e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd50ffff640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5c5d0 0x5624e5b61910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd50000d6a0 tx=0x7fd500010910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a7c5b776ac145ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd4ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/3823983977 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500021040 con 0x5624e5b5c5d0 2024-09-19T07:53:03.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/3823983977 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8542+0+0 (secure 0 0 0) 0x7fd5000ba260 con 0x5624e5b5c5d0 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4f8005500 con 0x5624e5b5c5d0 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/3823983977 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd50009c220 con 0x5624e5b5c5d0 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd518b42740 10 client.27670 fetch_fsmap learned FSMap version 52 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5624e5c04060 con 0x5624e5b5c5d0 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/3823983977 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd50009ce50 con 0x5624e5b5c5d0 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/3823983977 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8362+0+0 (secure 0 0 0) 0x7fd50009b090 con 0x5624e5b5c5d0 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd518b42740 10 client.27670 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.991+0000 7fd518b42740 10 client.27670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd518b42740 20 client.27670 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd518b42740 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5c042f0 0x5624e5c066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd518b42740 4 client.27670 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd4f8005500 con 0x5624e5c042f0 2024-09-19T07:53:03.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5c042f0 0x5624e5c066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd50f7fe640 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5c042f0 0x5624e5c066e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e853c57206442966 server_cookie=9e2a1c0d4488eef in_seq=0 out_seq=0 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd4ef7fe640 10 client.27670 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd4ef7fe640 1 -- 172.21.15.103:0/3823983977 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd4f8005500 con 0x5624e5c042f0 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd4ef7fe640 10 client.27670 handle_command_reply: tid=0 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.992+0000 7fd518b42740 1 client.27670 shutdown 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5c5d0 msgr2=0x5624e5b61910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5c5d0 0x5624e5b61910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd50000d6a0 tx=0x7fd500010910 comp rx=0 tx=0).stop 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 shutdown_connections 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x5624e5c042f0 0x5624e5c066e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd4ef7fe640 0 client.27670 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624e5b78380 0x5624e5b62390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5624e5b5db40 0x5624e5b61e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:03.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 --2- 172.21.15.103:0/3823983977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624e5b5c5d0 0x5624e5b61910 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 >> 172.21.15.103:0/3823983977 conn(0x5624e5b25310 msgr2=0x5624e5b64000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:04.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 shutdown_connections 2024-09-19T07:53:04.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 1 -- 172.21.15.103:0/3823983977 wait complete. 2024-09-19T07:53:04.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.993+0000 7fd518b42740 20 client.27670 trim_cache size 0 max 16384 2024-09-19T07:53:04.000 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd4fc0abd90 msgr2=0x7fd4fc0ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd4fc0abd90 0x7fd4fc0ae250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5624e5b5d070 tx=0x7fd4f80098c0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510101de0 msgr2=0x7fd5100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510101de0 0x7fd5100744f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd50000d6d0 tx=0x7fd50000db90 comp rx=0 tx=0).stop 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 shutdown_connections 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd4fc0abd90 0x7fd4fc0ae250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd510106f10 0x7fd510074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510102790 0x7fd510074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 --2- 172.21.15.103:0/4007902207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd510101de0 0x7fd5100744f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:04.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 >> 172.21.15.103:0/4007902207 conn(0x7fd5100fd8e0 msgr2=0x7fd5100fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:04.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 shutdown_connections 2024-09-19T07:53:04.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:03.994+0000 7fd516769640 1 -- 172.21.15.103:0/4007902207 wait complete. 2024-09-19T07:53:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:04 smithi174 ceph-mon[35370]: pgmap v471: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.6 MiB/s wr, 570 op/s 2024-09-19T07:53:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:04 smithi103 ceph-mon[31201]: pgmap v471: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.6 MiB/s wr, 570 op/s 2024-09-19T07:53:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:04 smithi134 ceph-mon[35414]: pgmap v471: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 8.6 MiB/s wr, 570 op/s 2024-09-19T07:53:04.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:53:05.250 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:05.378 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:05 smithi103 ceph-mon[31201]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:53:05.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:05 smithi174 ceph-mon[35370]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:53:05.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 870.0 s, 626.4 tps, lat 158.407 ms stddev 278.893 2024-09-19T07:53:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:05 smithi134 ceph-mon[35414]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T07:53:06.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.104+0000 7fedcffff640 1 -- 172.21.15.103:0/2755366966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 msgr2=0x7fedd0108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.104+0000 7fedcffff640 1 --2- 172.21.15.103:0/2755366966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd0108440 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009860 tx=0x7fedc402a340 comp rx=0 tx=0).stop 2024-09-19T07:53:06.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.105+0000 7fedcffff640 1 -- 172.21.15.103:0/2755366966 shutdown_connections 2024-09-19T07:53:06.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.105+0000 7fedcffff640 1 --2- 172.21.15.103:0/2755366966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd0108980 0x7fedd0114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.105+0000 7fedcffff640 1 --2- 172.21.15.103:0/2755366966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd0108440 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.105+0000 7fedcffff640 1 --2- 172.21.15.103:0/2755366966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd010e060 0x7fedd010e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.105+0000 7fedcffff640 1 -- 172.21.15.103:0/2755366966 >> 172.21.15.103:0/2755366966 conn(0x7fedd0077980 msgr2=0x7fedd0079da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.107+0000 7fedcffff640 1 -- 172.21.15.103:0/2755366966 shutdown_connections 2024-09-19T07:53:06.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.107+0000 7fedcffff640 1 -- 172.21.15.103:0/2755366966 wait complete. 2024-09-19T07:53:06.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.108+0000 7fedcffff640 1 Processor -- start 2024-09-19T07:53:06.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.108+0000 7fedcffff640 1 -- start start 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedcffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd00827c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedcffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd0108980 0x7fedd0082d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedcffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd010e060 0x7fedd007bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedcffff640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedd0118360 con 0x7fedd010e060 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedcffff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01181e0 con 0x7fedd0108020 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedcffff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01184e0 con 0x7fedd0108980 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd00827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedceffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd00827c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45920/0 (socket says 172.21.15.103:45920) 2024-09-19T07:53:06.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.109+0000 7fedceffd640 1 -- 172.21.15.103:0/1810683473 learned_addr learned my addr 172.21.15.103:0/1810683473 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedceffd640 1 -- 172.21.15.103:0/1810683473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd0108980 msgr2=0x7fedd0082d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedceffd640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd0108980 0x7fedd0082d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedceffd640 1 -- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd010e060 msgr2=0x7fedd007bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedceffd640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd010e060 0x7fedd007bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedceffd640 1 -- 172.21.15.103:0/1810683473 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd007c390 con 0x7fedd0108020 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedceffd640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd00827c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fedc000e9f0 tx=0x7fedc000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedd48d2640 1 -- 172.21.15.103:0/1810683473 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc000cd80 con 0x7fedd0108020 2024-09-19T07:53:06.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.110+0000 7fedd48d2640 1 -- 172.21.15.103:0/1810683473 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc0010910 con 0x7fedd0108020 2024-09-19T07:53:06.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.111+0000 7fedd48d2640 1 -- 172.21.15.103:0/1810683473 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0018990 con 0x7fedd0108020 2024-09-19T07:53:06.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.111+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd007c680 con 0x7fedd0108020 2024-09-19T07:53:06.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.112+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd00804c0 con 0x7fedd0108020 2024-09-19T07:53:06.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.116+0000 7fedd48d2640 1 -- 172.21.15.103:0/1810683473 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fedc0010ac0 con 0x7fedd0108020 2024-09-19T07:53:06.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.117+0000 7fedd48d2640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fedb80a3500 0x7fedb80a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.117+0000 7fedd48d2640 1 -- 172.21.15.103:0/1810683473 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7fedc00d02b0 con 0x7fedd0108020 2024-09-19T07:53:06.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.119+0000 7fedce7fc640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fedb80a3500 0x7fedb80a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 Processor -- start 2024-09-19T07:53:06.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 -- start start 2024-09-19T07:53:06.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedce7fc640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fedb80a3500 0x7fedb80a59c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fedc40057c0 tx=0x7fedc4030480 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c67b0 0x5600961c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961d1550 0x5600961d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56009612c2f0 con 0x5600961c7d20 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5600960d3cf0 con 0x5600961c67b0 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedd7cad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600961282b0 con 0x5600961d1550 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.120+0000 7fedce7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c67b0 0x5600961c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c67b0 0x5600961c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:45934/0 (socket says 172.21.15.103:45934) 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 -- 172.21.15.103:0/1019646051 learned_addr learned my addr 172.21.15.103:0/1019646051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 -- 172.21.15.103:0/1019646051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961d1550 msgr2=0x5600961d3940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 --2- 172.21.15.103:0/1019646051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961d1550 0x5600961d3940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 -- 172.21.15.103:0/1019646051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 msgr2=0x5600961c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 --2- 172.21.15.103:0/1019646051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 -- 172.21.15.103:0/1019646051 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600960b83a0 con 0x5600961c67b0 2024-09-19T07:53:06.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.121+0000 7fedce7fc640 1 --2- 172.21.15.103:0/1019646051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c67b0 0x5600961c6bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fedc402a930 tx=0x7fedc403a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21d89aa59ff72cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.123+0000 7fedb5ffb640 1 -- 172.21.15.103:0/1019646051 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4043050 con 0x5600961c67b0 2024-09-19T07:53:06.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.123+0000 7fedb5ffb640 1 -- 172.21.15.103:0/1019646051 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc404d070 con 0x5600961c67b0 2024-09-19T07:53:06.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.123+0000 7fedb5ffb640 1 -- 172.21.15.103:0/1019646051 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4037070 con 0x5600961c67b0 2024-09-19T07:53:06.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 -- 172.21.15.103:0/1019646051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c67b0 msgr2=0x5600961c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 --2- 172.21.15.103:0/1019646051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c67b0 0x5600961c6bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fedc402a930 tx=0x7fedc403a000 comp rx=0 tx=0).stop 2024-09-19T07:53:06.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 -- 172.21.15.103:0/1019646051 shutdown_connections 2024-09-19T07:53:06.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 --2- 172.21.15.103:0/1019646051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961d1550 0x5600961d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 --2- 172.21.15.103:0/1019646051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c67b0 0x5600961c6bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 --2- 172.21.15.103:0/1019646051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 -- 172.21.15.103:0/1019646051 >> 172.21.15.103:0/1019646051 conn(0x56009618f310 msgr2=0x5600961c2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 -- 172.21.15.103:0/1019646051 shutdown_connections 2024-09-19T07:53:06.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.124+0000 7fedd7cad740 1 -- 172.21.15.103:0/1019646051 wait complete. 2024-09-19T07:53:06.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.125+0000 7fedd7cad740 1 -- start start 2024-09-19T07:53:06.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.125+0000 7fedd7cad740 1 Processor -- start 2024-09-19T07:53:06.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.126+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.126+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 0x5600961cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedd7cad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600961d41f0 con 0x5600961c7d20 2024-09-19T07:53:06.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedd7cad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5600960e7060 con 0x5600961d1550 2024-09-19T07:53:06.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedd7cad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600961d4370 con 0x5600961c67b0 2024-09-19T07:53:06.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46298/0 (socket says 172.21.15.103:46298) 2024-09-19T07:53:06.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 -- 172.21.15.103:0/2341402252 learned_addr learned my addr 172.21.15.103:0/2341402252 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:06.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedce7fc640 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 -- 172.21.15.103:0/2341402252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 msgr2=0x5600961cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 0x5600961cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 -- 172.21.15.103:0/2341402252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 msgr2=0x5600961ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedceffd640 1 -- 172.21.15.103:0/2341402252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560095e9d3c0 con 0x5600961c67b0 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedce7fc640 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedd7cad740 10 client.0 _ll_register_callbacks cb 0x560095e3a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.127+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc404d070 con 0x5600961c67b0 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.128+0000 7fedceffd640 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961ce330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fedc0098950 tx=0x7fedc009ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2a6244148b3ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.128+0000 7fedaeffd640 10 client.27676 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.128+0000 7fedaeffd640 1 -- 172.21.15.103:0/2341402252 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00a08a0 con 0x5600961c67b0 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.128+0000 7fedaeffd640 1 -- 172.21.15.103:0/2341402252 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7fedc009c750 con 0x5600961c67b0 2024-09-19T07:53:06.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedaeffd640 1 -- 172.21.15.103:0/2341402252 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fedc009c9d0 con 0x5600961c67b0 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 10 client.27676 fetch_fsmap learned FSMap version 53 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=53}) -- 0x5600961dfce0 con 0x5600961c67b0 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedaeffd640 1 -- 172.21.15.103:0/2341402252 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 53) ==== 8366+0+0 (secure 0 0 0) 0x7fedc00a4cd0 con 0x5600961c67b0 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 10 client.27676 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 10 client.27676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 20 client.27676 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5600961e12d0 0x5600961e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 4 client.27676 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.129+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560096139860 con 0x5600961e12d0 2024-09-19T07:53:06.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.130+0000 7fedce7fc640 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5600961e12d0 0x5600961e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.130+0000 7fedce7fc640 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5600961e12d0 0x5600961e3740 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cec3c8f87472a5e9 server_cookie=ef813fa25ef05448 in_seq=0 out_seq=0 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.130+0000 7fedaeffd640 10 client.27676 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.131+0000 7fedaeffd640 1 -- 172.21.15.103:0/2341402252 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560096139860 con 0x5600961e12d0 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.131+0000 7fedaeffd640 10 client.27676 handle_command_reply: tid=0 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.131+0000 7fedd7cad740 1 client.27676 shutdown 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 msgr2=0x5600961ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961ce330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fedc0098950 tx=0x7fedc009ccb0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 shutdown_connections 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x5600961e12d0 0x5600961e3740 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 0x5600961cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 --2- 172.21.15.103:0/2341402252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961ce330 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 >> 172.21.15.103:0/2341402252 conn(0x56009618f310 msgr2=0x5600961c8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedaeffd640 0 client.27676 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:06.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.132+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 shutdown_connections 2024-09-19T07:53:06.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.133+0000 7fedd7cad740 1 -- 172.21.15.103:0/2341402252 wait complete. 2024-09-19T07:53:06.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.133+0000 7fedd7cad740 20 client.27676 trim_cache size 0 max 16384 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 Processor -- start 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 -- start start 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961c2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c7d20 0x5600961c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961d1550 0x5600961c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01bc680 con 0x5600961d1550 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01be1c0 con 0x5600961c7d20 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedd7cad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01bcd50 con 0x5600961c67b0 2024-09-19T07:53:06.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:46300/0 (socket says 172.21.15.103:46300) 2024-09-19T07:53:06.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedceffd640 1 -- 172.21.15.103:0/3648527132 learned_addr learned my addr 172.21.15.103:0/3648527132 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:06.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedcf7fe640 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961d1550 0x5600961c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.153+0000 7fedce7fc640 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c7d20 0x5600961c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.154+0000 7fedceffd640 1 -- 172.21.15.103:0/3648527132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c7d20 msgr2=0x5600961c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.154+0000 7fedceffd640 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c7d20 0x5600961c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.154+0000 7fedceffd640 1 -- 172.21.15.103:0/3648527132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961d1550 msgr2=0x5600961c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.154+0000 7fedceffd640 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961d1550 0x5600961c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.154+0000 7fedceffd640 1 -- 172.21.15.103:0/3648527132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600961c7940 con 0x5600961c67b0 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.154+0000 7fedceffd640 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961c2250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fedc009c1f0 tx=0x7fedc001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7292e2c64b8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.154+0000 7fedb57fa640 1 -- 172.21.15.103:0/3648527132 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00a51b0 con 0x5600961c67b0 2024-09-19T07:53:06.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.155+0000 7fedb57fa640 1 -- 172.21.15.103:0/3648527132 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc404d070 con 0x5600961c67b0 2024-09-19T07:53:06.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.155+0000 7fedb57fa640 1 -- 172.21.15.103:0/3648527132 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc009be50 con 0x5600961c67b0 2024-09-19T07:53:06.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 -- 172.21.15.103:0/3648527132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 msgr2=0x5600961c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961c2250 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fedc009c1f0 tx=0x7fedc001a000 comp rx=0 tx=0).stop 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 -- 172.21.15.103:0/3648527132 shutdown_connections 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961d1550 0x5600961c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961c7d20 0x5600961c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3648527132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961c2250 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 -- 172.21.15.103:0/3648527132 >> 172.21.15.103:0/3648527132 conn(0x56009618f310 msgr2=0x5600961dcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 -- 172.21.15.103:0/3648527132 shutdown_connections 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.156+0000 7fedd7cad740 1 -- 172.21.15.103:0/3648527132 wait complete. 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 -- start start 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 Processor -- start 2024-09-19T07:53:06.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 0x5600961d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01bdaf0 con 0x5600961c7d20 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01bd420 con 0x5600961d1550 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01ba680 con 0x5600961c67b0 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.157+0000 7fedd7cad740 10 client.0 _ll_register_callbacks cb 0x7fedc0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedcf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 0x5600961d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961d9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43970/0 (socket says 172.21.15.103:43970) 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 -- 172.21.15.103:0/3341757952 learned_addr learned my addr 172.21.15.103:0/3341757952 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 -- 172.21.15.103:0/3341757952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 msgr2=0x5600961cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 -- 172.21.15.103:0/3341757952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 msgr2=0x5600961d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 0x5600961d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 -- 172.21.15.103:0/3341757952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600961da1b0 con 0x5600961c7d20 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedce7fc640 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961d9600 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005b00 tx=0x7fedc4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1bb8432351a4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.158+0000 7fedaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.159+0000 7fedaeffd640 1 -- 172.21.15.103:0/3341757952 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4043070 con 0x5600961c7d20 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.159+0000 7fedaeffd640 1 -- 172.21.15.103:0/3341757952 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7fedc4030a80 con 0x5600961c7d20 2024-09-19T07:53:06.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.159+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc404d070 con 0x5600961c7d20 2024-09-19T07:53:06.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.159+0000 7fedaeffd640 1 -- 172.21.15.103:0/3341757952 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fedc4037040 con 0x5600961c7d20 2024-09-19T07:53:06.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.159+0000 7fedd7cad740 10 client.18414 fetch_fsmap learned FSMap version 53 2024-09-19T07:53:06.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.159+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x56009624bdc0 con 0x5600961c7d20 2024-09-19T07:53:06.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.160+0000 7fedaeffd640 1 -- 172.21.15.103:0/3341757952 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc404d070 con 0x5600961c7d20 2024-09-19T07:53:06.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedaeffd640 1 -- 172.21.15.103:0/3341757952 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 53) ==== 8366+0+0 (secure 0 0 0) 0x7fedc403a070 con 0x5600961c7d20 2024-09-19T07:53:06.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedd7cad740 10 client.18414 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T07:53:06.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedd7cad740 10 client.18414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:53:06.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedd7cad740 20 client.18414 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:06.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56009624c0a0 0x56009624e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:06.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedd7cad740 4 client.18414 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560096133d70 con 0x56009624c0a0 2024-09-19T07:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.161+0000 7fedceffd640 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56009624c0a0 0x56009624e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.162+0000 7fedceffd640 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56009624c0a0 0x56009624e490 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91fda3d0fe27aeb5 server_cookie=516473596e117e75 in_seq=0 out_seq=0 2024-09-19T07:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.162+0000 7fedaeffd640 10 client.18414 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedaeffd640 1 -- 172.21.15.103:0/3341757952 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560096133d70 con 0x56009624c0a0 2024-09-19T07:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedaeffd640 10 client.18414 handle_command_reply: tid=0 2024-09-19T07:53:06.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 client.18414 shutdown 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 msgr2=0x5600961d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961d9600 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005b00 tx=0x7fedc4005a90 comp rx=0 tx=0).stop 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 shutdown_connections 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56009624c0a0 0x56009624e490 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600961d1550 0x5600961d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600961c7d20 0x5600961d9600 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 --2- 172.21.15.103:0/3341757952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600961c67b0 0x5600961cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 >> 172.21.15.103:0/3341757952 conn(0x56009618f310 msgr2=0x5600961cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedaeffd640 0 client.18414 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:06.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 shutdown_connections 2024-09-19T07:53:06.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.163+0000 7fedd7cad740 1 -- 172.21.15.103:0/3341757952 wait complete. 2024-09-19T07:53:06.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.164+0000 7fedd7cad740 20 client.18414 trim_cache size 0 max 16384 2024-09-19T07:53:06.165 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:06.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.165+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fedb80a3500 msgr2=0x7fedb80a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.165+0000 7fedcffff640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fedb80a3500 0x7fedb80a59c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fedc40057c0 tx=0x7fedc4030480 comp rx=0 tx=0).stop 2024-09-19T07:53:06.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.165+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 msgr2=0x7fedd00827c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:06.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.165+0000 7fedcffff640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd00827c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fedc000e9f0 tx=0x7fedc000eeb0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 shutdown_connections 2024-09-19T07:53:06.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fedb80a3500 0x7fedb80a59c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedd010e060 0x7fedd007bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedd0108980 0x7fedd0082d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 --2- 172.21.15.103:0/1810683473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedd0108020 0x7fedd00827c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:06.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 >> 172.21.15.103:0/1810683473 conn(0x7fedd0077980 msgr2=0x7fedd0079460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:06.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 shutdown_connections 2024-09-19T07:53:06.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:06.166+0000 7fedcffff640 1 -- 172.21.15.103:0/1810683473 wait complete. 2024-09-19T07:53:06.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:06 smithi174 ceph-mon[35370]: pgmap v473: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-09-19T07:53:06.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:06 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:53:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:06 smithi103 ceph-mon[31201]: pgmap v473: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-09-19T07:53:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:06 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:53:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:06 smithi134 ceph-mon[35414]: pgmap v473: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 10 MiB/s wr, 684 op/s 2024-09-19T07:53:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:06 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:53:07.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:53:07.455 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:08.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:08 smithi103 ceph-mon[31201]: pgmap v474: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 35 MiB/s wr, 1.83k op/s 2024-09-19T07:53:08.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.235+0000 7fa3b1865640 1 -- 172.21.15.103:0/314569332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 msgr2=0x7fa3ac104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.235+0000 7fa3b1865640 1 --2- 172.21.15.103:0/314569332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 0x7fa3ac104e30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009860 tx=0x7fa3a002a340 comp rx=0 tx=0).stop 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.236+0000 7fa3b1865640 1 -- 172.21.15.103:0/314569332 shutdown_connections 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.236+0000 7fa3b1865640 1 --2- 172.21.15.103:0/314569332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac105560 0x7fa3ac10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.236+0000 7fa3b1865640 1 --2- 172.21.15.103:0/314569332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 0x7fa3ac104e30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.236+0000 7fa3b1865640 1 --2- 172.21.15.103:0/314569332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac100430 0x7fa3ac100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.236+0000 7fa3b1865640 1 -- 172.21.15.103:0/314569332 >> 172.21.15.103:0/314569332 conn(0x7fa3ac077230 msgr2=0x7fa3ac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.236+0000 7fa3b1865640 1 -- 172.21.15.103:0/314569332 shutdown_connections 2024-09-19T07:53:08.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.236+0000 7fa3b1865640 1 -- 172.21.15.103:0/314569332 wait complete. 2024-09-19T07:53:08.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.237+0000 7fa3b1865640 1 Processor -- start 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.237+0000 7fa3b1865640 1 -- start start 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.237+0000 7fa3b1865640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac100430 0x7fa3ac10ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.237+0000 7fa3b1865640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 0x7fa3ac108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3b1865640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac105560 0x7fa3ac1a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac100430 0x7fa3ac10ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac100430 0x7fa3ac10ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49620/0 (socket says 172.21.15.103:49620) 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 0x7fa3ac108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aaffd640 1 -- 172.21.15.103:0/4085551469 learned_addr learned my addr 172.21.15.103:0/4085551469 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac110e40 con 0x7fa3ac105560 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac110cc0 con 0x7fa3ac100de0 2024-09-19T07:53:08.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac110fc0 con 0x7fa3ac100430 2024-09-19T07:53:08.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/4085551469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac100430 msgr2=0x7fa3ac10ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac100430 0x7fa3ac10ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac105560 msgr2=0x7fa3ac1a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac105560 0x7fa3ac1a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.238+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/4085551469 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac109530 con 0x7fa3ac100de0 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.239+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 0x7fa3ac108f70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa3a002a860 tx=0x7fa3a0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.239+0000 7fa3b0863640 1 -- 172.21.15.103:0/4085551469 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0037040 con 0x7fa3ac100de0 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.239+0000 7fa3b0863640 1 -- 172.21.15.103:0/4085551469 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a0030550 con 0x7fa3ac100de0 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.239+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac1097c0 con 0x7fa3ac100de0 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.239+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac109ca0 con 0x7fa3ac100de0 2024-09-19T07:53:08.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.239+0000 7fa3b0863640 1 -- 172.21.15.103:0/4085551469 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0003af0 con 0x7fa3ac100de0 2024-09-19T07:53:08.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.243+0000 7fa3b0863640 1 -- 172.21.15.103:0/4085551469 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa3a0042020 con 0x7fa3ac100de0 2024-09-19T07:53:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.245+0000 7fa3b0863640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3940a3580 0x7fa3940a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.245+0000 7fa3b0863640 1 -- 172.21.15.103:0/4085551469 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7fa3a00ef050 con 0x7fa3ac100de0 2024-09-19T07:53:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.245+0000 7fa3b3c3e740 1 Processor -- start 2024-09-19T07:53:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.246+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3940a3580 0x7fa3940a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.248+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3940a3580 0x7fa3940a5a40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005fd0 tx=0x7fa39c02d490 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:08.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3b3c3e740 1 -- start start 2024-09-19T07:53:08.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c42616b40 0x561c42615090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c426155d0 0x561c42615a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42620370 0x561c42622760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3aa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c426155d0 0x561c42615a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3aa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c426155d0 0x561c42615a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40924/0 (socket says 172.21.15.103:40924) 2024-09-19T07:53:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/1172238898 learned_addr learned my addr 172.21.15.103:0/1172238898 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c42616b40 0x561c42615090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3ab7fe640 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42620370 0x561c42622760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.250+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c4257b2f0 con 0x561c42616b40 2024-09-19T07:53:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c42522cf0 con 0x561c426155d0 2024-09-19T07:53:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c425772b0 con 0x561c42620370 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3aaffd640 1 -- 172.21.15.103:0/1172238898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42620370 msgr2=0x561c42622760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42620370 0x561c42622760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3aaffd640 1 -- 172.21.15.103:0/1172238898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c426155d0 msgr2=0x561c42615a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c426155d0 0x561c42615a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3aaffd640 1 -- 172.21.15.103:0/1172238898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c425073a0 con 0x561c42616b40 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.251+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c426155d0 0x561c42615a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.252+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c42616b40 0x561c42615090 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0058c0 tx=0x7fa39c035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd54fadd371cfbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.252+0000 7fa391ffb640 1 -- 172.21.15.103:0/1172238898 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003e90 con 0x561c42616b40 2024-09-19T07:53:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.254+0000 7fa391ffb640 1 -- 172.21.15.103:0/1172238898 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa39c033e30 con 0x561c42616b40 2024-09-19T07:53:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.254+0000 7fa391ffb640 1 -- 172.21.15.103:0/1172238898 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c03e890 con 0x561c42616b40 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c42616b40 msgr2=0x561c42615090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c42616b40 0x561c42615090 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0058c0 tx=0x7fa39c035000 comp rx=0 tx=0).stop 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 shutdown_connections 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42620370 0x561c42622760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c426155d0 0x561c42615a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/1172238898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c42616b40 0x561c42615090 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 >> 172.21.15.103:0/1172238898 conn(0x561c425de310 msgr2=0x561c42611c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 shutdown_connections 2024-09-19T07:53:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.255+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/1172238898 wait complete. 2024-09-19T07:53:08.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.256+0000 7fa3b3c3e740 1 -- start start 2024-09-19T07:53:08.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.256+0000 7fa3b3c3e740 1 Processor -- start 2024-09-19T07:53:08.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.257+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4261fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.257+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.257+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561c42623070 con 0x561c426155d0 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c42536060 con 0x561c42620370 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3aa7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3aa7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49638/0 (socket says 172.21.15.103:49638) 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/3129389132 learned_addr learned my addr 172.21.15.103:0/3129389132 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3ab7fe640 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c426231f0 con 0x561c42616b40 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3b3c3e740 10 client.0 _ll_register_callbacks cb 0x561c422898c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:08.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.258+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4261fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.259+0000 7fa3aaffd640 1 -- 172.21.15.103:0/3129389132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 msgr2=0x561c4261cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.259+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.259+0000 7fa3aaffd640 1 -- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 msgr2=0x561c4261d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.259+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.259+0000 7fa3aaffd640 1 -- 172.21.15.103:0/3129389132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c422ec3c0 con 0x561c426155d0 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.259+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4261fcf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa39c004650 tx=0x7fa39c040a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04fb42da0400f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.259+0000 7fa38affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.260+0000 7fa38affd640 1 -- 172.21.15.103:0/3129389132 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c005250 con 0x561c426155d0 2024-09-19T07:53:08.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.260+0000 7fa38affd640 1 -- 172.21.15.103:0/3129389132 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7fa39c0053f0 con 0x561c426155d0 2024-09-19T07:53:08.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.260+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39c033e30 con 0x561c426155d0 2024-09-19T07:53:08.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa38affd640 1 -- 172.21.15.103:0/3129389132 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa39c005630 con 0x561c426155d0 2024-09-19T07:53:08.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 10 client.18423 fetch_fsmap learned FSMap version 53 2024-09-19T07:53:08.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x561c4262eae0 con 0x561c426155d0 2024-09-19T07:53:08.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa38affd640 1 -- 172.21.15.103:0/3129389132 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa39c033e30 con 0x561c426155d0 2024-09-19T07:53:08.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa38affd640 1 -- 172.21.15.103:0/3129389132 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 53) ==== 8366+0+0 (secure 0 0 0) 0x7fa39c05b030 con 0x561c426155d0 2024-09-19T07:53:08.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 10 client.18423 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T07:53:08.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:08.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 20 client.18423 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:08.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c42632280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 4 client.18423 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:08.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.261+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c42588860 con 0x561c4262fe90 2024-09-19T07:53:08.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.263+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c42632280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.264+0000 7fa38affd640 10 client.18423 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.264+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c42632280 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525ce22fd6994cf5 server_cookie=be0d66ebcb8d545b in_seq=0 out_seq=0 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa38affd640 1 -- 172.21.15.103:0/3129389132 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c42588860 con 0x561c4262fe90 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa38affd640 10 client.18423 handle_command_reply: tid=0 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 client.18423 shutdown 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 msgr2=0x561c4261fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4261fcf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa39c004650 tx=0x7fa39c040a40 comp rx=0 tx=0).stop 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 shutdown_connections 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c42632280 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa38affd640 0 client.18423 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:08.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.265+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3129389132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4261fcf0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.266+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 >> 172.21.15.103:0/3129389132 conn(0x561c425de310 msgr2=0x561c42617930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:08.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.266+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 shutdown_connections 2024-09-19T07:53:08.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.266+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3129389132 wait complete. 2024-09-19T07:53:08.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.266+0000 7fa3b3c3e740 20 client.18423 trim_cache size 0 max 16384 2024-09-19T07:53:08.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 Processor -- start 2024-09-19T07:53:08.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 -- start start 2024-09-19T07:53:08.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4262c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa39800e930 con 0x561c426155d0 2024-09-19T07:53:08.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa398010470 con 0x561c42620370 2024-09-19T07:53:08.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.285+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa39800f000 con 0x561c42616b40 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49654/0 (socket says 172.21.15.103:49654) 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/148730664 learned_addr learned my addr 172.21.15.103:0/148730664 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3ab7fe640 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/148730664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 msgr2=0x561c4261bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/148730664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 msgr2=0x561c4262c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4262c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 -- 172.21.15.103:0/148730664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4258ee70 con 0x561c42616b40 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3ab7fe640 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:08.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.286+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261b870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00bb330 tx=0x7fa3a003d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96bff82bcf7ea6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:08.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.287+0000 7fa3917fa640 1 -- 172.21.15.103:0/148730664 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00bf750 con 0x561c42616b40 2024-09-19T07:53:08.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.287+0000 7fa3917fa640 1 -- 172.21.15.103:0/148730664 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a00ece50 con 0x561c42616b40 2024-09-19T07:53:08.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.287+0000 7fa3917fa640 1 -- 172.21.15.103:0/148730664 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00ec980 con 0x561c42616b40 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/148730664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 msgr2=0x561c4261b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261b870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00bb330 tx=0x7fa3a003d040 comp rx=0 tx=0).stop 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/148730664 shutdown_connections 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c4261bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c4261b870 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/148730664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c4262c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/148730664 >> 172.21.15.103:0/148730664 conn(0x561c425de310 msgr2=0x561c4261c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/148730664 shutdown_connections 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.288+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/148730664 wait complete. 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 -- start start 2024-09-19T07:53:08.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 Processor -- start 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c426234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c42623a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c42623f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa39800fda0 con 0x561c426155d0 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa39800f6d0 con 0x561c42620370 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa39800c930 con 0x561c42616b40 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3b3c3e740 10 client.0 _ll_register_callbacks cb 0x7fa3a0005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c426234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c426234e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50744/0 (socket says 172.21.15.103:50744) 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.289+0000 7fa3aaffd640 1 -- 172.21.15.103:0/3251235739 learned_addr learned my addr 172.21.15.103:0/3251235739 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:08.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3ab7fe640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c42623f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c42623a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3aaffd640 1 -- 172.21.15.103:0/3251235739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 msgr2=0x561c42623a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c42623a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3aaffd640 1 -- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 msgr2=0x561c42623f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c42623f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3aaffd640 1 -- 172.21.15.103:0/3251235739 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c42698ca0 con 0x561c426155d0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3ab7fe640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c42623f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3aaffd640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c426234e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa39c02dea0 tx=0x7fa39c03ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f9f2d0cd839728 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa38affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa38affd640 1 -- 172.21.15.103:0/3251235739 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0488b0 con 0x561c426155d0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa38affd640 1 -- 172.21.15.103:0/3251235739 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7fa39c048a50 con 0x561c426155d0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a00ece50 con 0x561c426155d0 2024-09-19T07:53:08.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa38affd640 1 -- 172.21.15.103:0/3251235739 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa39c069050 con 0x561c426155d0 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3b3c3e740 10 client.18426 fetch_fsmap learned FSMap version 53 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.290+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x561c42699f00 con 0x561c426155d0 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.291+0000 7fa38affd640 1 -- 172.21.15.103:0/3251235739 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a00ece50 con 0x561c426155d0 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.291+0000 7fa38affd640 1 -- 172.21.15.103:0/3251235739 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 53) ==== 8366+0+0 (secure 0 0 0) 0x7fa39c030040 con 0x561c426155d0 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.291+0000 7fa3b3c3e740 10 client.18426 fetch_fsmap finished waiting for FSMap version 53 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.291+0000 7fa3b3c3e740 10 client.18426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.291+0000 7fa3b3c3e740 20 client.18426 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.292+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c4269c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.292+0000 7fa3b3c3e740 4 client.18426 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.292+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c42582d70 con 0x561c4262fe90 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.292+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c4269c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:08.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.292+0000 7fa38affd640 10 client.18426 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.292+0000 7fa3aa7fc640 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c4269c620 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f006249106808d41 server_cookie=7fb0466e9d00f44f in_seq=0 out_seq=0 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa38affd640 1 -- 172.21.15.103:0/3251235739 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c42582d70 con 0x561c4262fe90 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa38affd640 10 client.18426 handle_command_reply: tid=0 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 client.18426 shutdown 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 msgr2=0x561c426234e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c426234e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa39c02dea0 tx=0x7fa39c03ec40 comp rx=0 tx=0).stop 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 shutdown_connections 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561c4262fe90 0x561c4269c620 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c42620370 0x561c42623f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c42616b40 0x561c42623a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 --2- 172.21.15.103:0/3251235739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561c426155d0 0x561c426234e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa38affd640 0 client.18426 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:08.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 >> 172.21.15.103:0/3251235739 conn(0x561c425de310 msgr2=0x561c42617110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 shutdown_connections 2024-09-19T07:53:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 1 -- 172.21.15.103:0/3251235739 wait complete. 2024-09-19T07:53:08.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.293+0000 7fa3b3c3e740 20 client.18426 trim_cache size 0 max 16384 2024-09-19T07:53:08.299 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3940a3580 msgr2=0x7fa3940a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3940a3580 0x7fa3940a5a40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005fd0 tx=0x7fa39c02d490 comp rx=0 tx=0).stop 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 msgr2=0x7fa3ac108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 0x7fa3ac108f70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa3a002a860 tx=0x7fa3a0033d90 comp rx=0 tx=0).stop 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 shutdown_connections 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3940a3580 0x7fa3940a5a40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac105560 0x7fa3ac1a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ac100de0 0x7fa3ac108f70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 --2- 172.21.15.103:0/4085551469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac100430 0x7fa3ac10ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 >> 172.21.15.103:0/4085551469 conn(0x7fa3ac077230 msgr2=0x7fa3ac102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 shutdown_connections 2024-09-19T07:53:08.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:08.294+0000 7fa3b1865640 1 -- 172.21.15.103:0/4085551469 wait complete. 2024-09-19T07:53:08.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:08 smithi174 ceph-mon[35370]: pgmap v474: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 35 MiB/s wr, 1.83k op/s 2024-09-19T07:53:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:08 smithi134 ceph-mon[35414]: pgmap v474: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 35 MiB/s wr, 1.83k op/s 2024-09-19T07:53:09.252 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T07:53:09.252 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T07:53:09.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:53:09.609 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 -- 172.21.15.103:0/2980435357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 msgr2=0x7ff68c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 --2- 172.21.15.103:0/2980435357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 0x7ff68c103170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff688009900 tx=0x7ff68802a410 comp rx=0 tx=0).stop 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 -- 172.21.15.103:0/2980435357 shutdown_connections 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 --2- 172.21.15.103:0/2980435357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c108290 0x7ff68c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 --2- 172.21.15.103:0/2980435357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c103740 0x7ff68c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 --2- 172.21.15.103:0/2980435357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 0x7ff68c103170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 -- 172.21.15.103:0/2980435357 >> 172.21.15.103:0/2980435357 conn(0x7ff68c0fc7d0 msgr2=0x7ff68c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 -- 172.21.15.103:0/2980435357 shutdown_connections 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.189+0000 7ff693d0d640 1 -- 172.21.15.103:0/2980435357 wait complete. 2024-09-19T07:53:10.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.190+0000 7ff693d0d640 1 Processor -- start 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.190+0000 7ff693d0d640 1 -- start start 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.190+0000 7ff693d0d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 0x7ff68c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.190+0000 7ff693d0d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c103740 0x7ff68c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.190+0000 7ff693d0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c108290 0x7ff68c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff693d0d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1120d0 con 0x7ff68c103740 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff693d0d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c111f50 con 0x7ff68c108290 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff693d0d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c112250 con 0x7ff68c102d90 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c103740 0x7ff68c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff691aa9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 0x7ff68c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c103740 0x7ff68c198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50768/0 (socket says 172.21.15.103:50768) 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 -- 172.21.15.103:0/3498206569 learned_addr learned my addr 172.21.15.103:0/3498206569 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6922aa640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c108290 0x7ff68c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 -- 172.21.15.103:0/3498206569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 msgr2=0x7ff68c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 0x7ff68c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 -- 172.21.15.103:0/3498206569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c108290 msgr2=0x7ff68c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c108290 0x7ff68c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 -- 172.21.15.103:0/3498206569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c19d580 con 0x7ff68c103740 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff6912a8640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c103740 0x7ff68c198b10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c970 tx=0x7ff67c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff682ffd640 1 -- 172.21.15.103:0/3498206569 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c007c70 con 0x7ff68c103740 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff682ffd640 1 -- 172.21.15.103:0/3498206569 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff67c007e10 con 0x7ff68c103740 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff682ffd640 1 -- 172.21.15.103:0/3498206569 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00f450 con 0x7ff68c103740 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.191+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c19d870 con 0x7ff68c103740 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.192+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1a5150 con 0x7ff68c103740 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.193+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c06a7f0 con 0x7ff68c103740 2024-09-19T07:53:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.195+0000 7ff682ffd640 1 -- 172.21.15.103:0/3498206569 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff67c004000 con 0x7ff68c103740 2024-09-19T07:53:10.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.196+0000 7ff682ffd640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff6780a34b0 0x7ff6780a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:10.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.196+0000 7ff682ffd640 1 -- 172.21.15.103:0/3498206569 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7ff67c0cfa50 con 0x7ff68c103740 2024-09-19T07:53:10.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.197+0000 7ff691aa9640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff6780a34b0 0x7ff6780a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:10.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.197+0000 7ff691aa9640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff6780a34b0 0x7ff6780a5970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff688000c00 tx=0x7ff6880058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:10.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.201+0000 7ff682ffd640 1 -- 172.21.15.103:0/3498206569 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff67c014030 con 0x7ff68c103740 2024-09-19T07:53:10.251 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:53:10.252 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:53:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:10 smithi174 ceph-mon[35370]: pgmap v475: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 35 MiB/s wr, 1.83k op/s 2024-09-19T07:53:10.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 875.0 s, 2267.6 tps, lat 44.288 ms stddev 52.978 2024-09-19T07:53:10.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:10 smithi103 ceph-mon[31201]: pgmap v475: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 35 MiB/s wr, 1.83k op/s 2024-09-19T07:53:10.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.510+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff68c10a2a0 con 0x7ff68c103740 2024-09-19T07:53:10.516 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:53:10.516 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":54,"btime":"2024-09-19T07:53:10:110704+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:53:09.872640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:53:10.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.514+0000 7ff682ffd640 1 -- 172.21.15.103:0/3498206569 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7ff67c0b3ef0 con 0x7ff68c103740 2024-09-19T07:53:10.517 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 54 2024-09-19T07:53:10.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff6780a34b0 msgr2=0x7ff6780a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff6780a34b0 0x7ff6780a5970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff688000c00 tx=0x7ff6880058d0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c103740 msgr2=0x7ff68c198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:10.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c103740 0x7ff68c198b10 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c970 tx=0x7ff67c00ce30 comp rx=0 tx=0).stop 2024-09-19T07:53:10.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 shutdown_connections 2024-09-19T07:53:10.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff6780a34b0 0x7ff6780a5970 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff688000c00 tx=0x7ff6880058d0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff68c108290 0x7ff68c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff68c103740 0x7ff68c198b10 secure :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c970 tx=0x7ff67c00ce30 comp rx=0 tx=0).stop 2024-09-19T07:53:10.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 --2- 172.21.15.103:0/3498206569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff68c102d90 0x7ff68c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:10.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.518+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 >> 172.21.15.103:0/3498206569 conn(0x7ff68c0fc7d0 msgr2=0x7ff68c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:10.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.524+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 shutdown_connections 2024-09-19T07:53:10.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:10.530+0000 7ff693d0d640 1 -- 172.21.15.103:0/3498206569 wait complete. 2024-09-19T07:53:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:10 smithi134 ceph-mon[35414]: pgmap v475: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 35 MiB/s wr, 1.83k op/s 2024-09-19T07:53:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:11 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:53:11.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:11 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3498206569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:11 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:53:11.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:11 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3498206569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:11 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:53:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:11 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3498206569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:12.431 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:12 smithi103 ceph-mon[31201]: pgmap v476: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 35 MiB/s wr, 2.01k op/s 2024-09-19T07:53:12.487 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T07:53:12.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T07:53:12.493 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:12 smithi134 ceph-mon[35414]: pgmap v476: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 35 MiB/s wr, 2.01k op/s 2024-09-19T07:53:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:12 smithi174 ceph-mon[35370]: pgmap v476: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 35 MiB/s wr, 2.01k op/s 2024-09-19T07:53:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:14 smithi103 ceph-mon[31201]: pgmap v477: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 35 MiB/s wr, 2.01k op/s 2024-09-19T07:53:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:14 smithi134 ceph-mon[35414]: pgmap v477: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 35 MiB/s wr, 2.01k op/s 2024-09-19T07:53:14.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:14 smithi174 ceph-mon[35370]: pgmap v477: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 35 MiB/s wr, 2.01k op/s 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 -- 172.21.15.103:0/3383411326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 msgr2=0x7f48081095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 --2- 172.21.15.103:0/3383411326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 0x7f48081095d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4804009900 tx=0x7f480402a410 comp rx=0 tx=0).stop 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 -- 172.21.15.103:0/3383411326 shutdown_connections 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 --2- 172.21.15.103:0/3383411326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808114ab0 0x7f4808116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 --2- 172.21.15.103:0/3383411326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808109ba0 0x7f4808114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 --2- 172.21.15.103:0/3383411326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 0x7f48081095d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 -- 172.21.15.103:0/3383411326 >> 172.21.15.103:0/3383411326 conn(0x7f480806e2d0 msgr2=0x7f480806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 -- 172.21.15.103:0/3383411326 shutdown_connections 2024-09-19T07:53:14.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.849+0000 7f480fcb7640 1 -- 172.21.15.103:0/3383411326 wait complete. 2024-09-19T07:53:14.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.850+0000 7f480fcb7640 1 Processor -- start 2024-09-19T07:53:14.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480fcb7640 1 -- start start 2024-09-19T07:53:14.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480fcb7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 0x7f48081a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480fcb7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808109ba0 0x7f48081a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480fcb7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808114ab0 0x7f48081a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480fcb7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f480811a750 con 0x7f48081091f0 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480fcb7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f480811a5d0 con 0x7f4808114ab0 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480fcb7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f480811a8d0 con 0x7f4808109ba0 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480d252640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808109ba0 0x7f48081a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480e254640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808114ab0 0x7f48081a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480e254640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808114ab0 0x7f48081a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40988/0 (socket says 172.21.15.103:40988) 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480da53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 0x7f48081a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480e254640 1 -- 172.21.15.103:0/1790440096 learned_addr learned my addr 172.21.15.103:0/1790440096 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.851+0000 7f480d252640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808109ba0 0x7f48081a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49700/0 (socket says 172.21.15.103:49700) 2024-09-19T07:53:14.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480d252640 1 -- 172.21.15.103:0/1790440096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808114ab0 msgr2=0x7f48081a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480d252640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808114ab0 0x7f48081a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480d252640 1 -- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 msgr2=0x7f48081a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480d252640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 0x7f48081a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480d252640 1 -- 172.21.15.103:0/1790440096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48081a0fd0 con 0x7f4808109ba0 2024-09-19T07:53:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480da53640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 0x7f48081a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480e254640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808114ab0 0x7f48081a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480d252640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808109ba0 0x7f48081a6b60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e9f0 tx=0x7f47f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f47feffd640 1 -- 172.21.15.103:0/1790440096 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f800ccc0 con 0x7f4808109ba0 2024-09-19T07:53:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48081a12c0 con 0x7f4808109ba0 2024-09-19T07:53:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f47feffd640 1 -- 172.21.15.103:0/1790440096 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f8010910 con 0x7f4808109ba0 2024-09-19T07:53:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f47feffd640 1 -- 172.21.15.103:0/1790440096 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8018960 con 0x7f4808109ba0 2024-09-19T07:53:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.852+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480810a4d0 con 0x7f4808109ba0 2024-09-19T07:53:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.854+0000 7f47feffd640 1 -- 172.21.15.103:0/1790440096 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f47f8010ac0 con 0x7f4808109ba0 2024-09-19T07:53:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.855+0000 7f47feffd640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f47f40a3430 0x7f47f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.855+0000 7f47feffd640 1 -- 172.21.15.103:0/1790440096 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7f47f80d0050 con 0x7f4808109ba0 2024-09-19T07:53:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.855+0000 7f480da53640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f47f40a3430 0x7f47f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 Processor -- start 2024-09-19T07:53:14.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 -- start start 2024-09-19T07:53:14.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019aaccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aacd480 0x56019aad7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aad7d40 0x56019aada130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56019aa32280 con 0x56019aaccb60 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56019a9d9b10 con 0x56019aacd480 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f4812090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56019aa2e240 con 0x56019aad7d40 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f480da53640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f47f40a3430 0x7f47f40a58f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4804005ab0 tx=0x7f480400b040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f480da53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019aaccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f480da53640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019aaccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50808/0 (socket says 172.21.15.103:50808) 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f480da53640 1 -- 172.21.15.103:0/2926629832 learned_addr learned my addr 172.21.15.103:0/2926629832 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f480e254640 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aad7d40 0x56019aada130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.856+0000 7f480d252640 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aacd480 0x56019aad7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.857+0000 7f480e254640 1 -- 172.21.15.103:0/2926629832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aacd480 msgr2=0x56019aad7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.857+0000 7f480e254640 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aacd480 0x56019aad7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.857+0000 7f480e254640 1 -- 172.21.15.103:0/2926629832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 msgr2=0x56019aaccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.857+0000 7f480e254640 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019aaccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.857+0000 7f480e254640 1 -- 172.21.15.103:0/2926629832 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56019a9e61f0 con 0x56019aad7d40 2024-09-19T07:53:14.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.857+0000 7f480e254640 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aad7d40 0x56019aada130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48000117b0 tx=0x7f4800011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c372943e2101840 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:14.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.859+0000 7f47f3fff640 1 -- 172.21.15.103:0/2926629832 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480001e070 con 0x56019aad7d40 2024-09-19T07:53:14.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.859+0000 7f47f3fff640 1 -- 172.21.15.103:0/2926629832 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480000fc60 con 0x56019aad7d40 2024-09-19T07:53:14.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.859+0000 7f47f3fff640 1 -- 172.21.15.103:0/2926629832 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480000e670 con 0x56019aad7d40 2024-09-19T07:53:14.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 -- 172.21.15.103:0/2926629832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aad7d40 msgr2=0x56019aada130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aad7d40 0x56019aada130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48000117b0 tx=0x7f4800011c70 comp rx=0 tx=0).stop 2024-09-19T07:53:14.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 -- 172.21.15.103:0/2926629832 shutdown_connections 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aad7d40 0x56019aada130 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aacd480 0x56019aad7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 --2- 172.21.15.103:0/2926629832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019aaccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 -- 172.21.15.103:0/2926629832 >> 172.21.15.103:0/2926629832 conn(0x56019aac90f0 msgr2=0x56019aac94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 -- 172.21.15.103:0/2926629832 shutdown_connections 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 -- 172.21.15.103:0/2926629832 wait complete. 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 -- start start 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.860+0000 7f4812090740 1 Processor -- start 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f4812090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aad77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f4812090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f4812090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f4812090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56019aada960 con 0x56019aacd480 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f4812090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56019a9ecf10 con 0x56019aad7d40 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f4812090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56019aadaae0 con 0x56019aaccb60 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f4812090740 10 client.0 _ll_register_callbacks cb 0x56019a9b5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f480d252640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f480d252640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50824/0 (socket says 172.21.15.103:50824) 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f480d252640 1 -- 172.21.15.103:0/1332135723 learned_addr learned my addr 172.21.15.103:0/1332135723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.861+0000 7f480e254640 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f480e254640 1 -- 172.21.15.103:0/1332135723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 msgr2=0x56019aad77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f480e254640 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aad77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f480e254640 1 -- 172.21.15.103:0/1332135723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 msgr2=0x56019aad4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f480e254640 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f480e254640 1 -- 172.21.15.103:0/1332135723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56019a7a36a0 con 0x56019aad7d40 2024-09-19T07:53:14.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f480e254640 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad5250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4800011780 tx=0x7f48000114f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0052654a41da761 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:14.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f47f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:53:14.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f47f17fa640 1 -- 172.21.15.103:0/1332135723 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480001e050 con 0x56019aad7d40 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480000fc60 con 0x56019aad7d40 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f47f17fa640 1 -- 172.21.15.103:0/1332135723 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7f480000fa60 con 0x56019aad7d40 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f47f17fa640 1 -- 172.21.15.103:0/1332135723 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4800032050 con 0x56019aad7d40 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f4812090740 10 client.27764 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56019aaeec90 con 0x56019aad7d40 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.862+0000 7f47f17fa640 1 -- 172.21.15.103:0/1332135723 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480000fc60 con 0x56019aad7d40 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f47f17fa640 1 -- 172.21.15.103:0/1332135723 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f4800019760 con 0x56019aad7d40 2024-09-19T07:53:14.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f4812090740 10 client.27764 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f4812090740 10 client.27764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f4812090740 20 client.27764 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f4812090740 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aaf2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f4812090740 4 client.27764 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56019aa3f7f0 con 0x56019aaf0040 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.863+0000 7f480da53640 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aaf2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.864+0000 7f480da53640 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aaf2430 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f565a6bcf3b8b77c server_cookie=f75de7760e9a19fb in_seq=0 out_seq=0 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.864+0000 7f47f17fa640 10 client.27764 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.864+0000 7f47f17fa640 1 -- 172.21.15.103:0/1332135723 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56019aa3f7f0 con 0x56019aaf0040 2024-09-19T07:53:14.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.864+0000 7f47f17fa640 10 client.27764 handle_command_reply: tid=0 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 client.27764 shutdown 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 msgr2=0x56019aad5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad5250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4800011780 tx=0x7f48000114f0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 shutdown_connections 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aaf2430 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f47f17fa640 0 client.27764 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad5250 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 --2- 172.21.15.103:0/1332135723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aad77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 >> 172.21.15.103:0/1332135723 conn(0x56019aac90f0 msgr2=0x56019aacee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.865+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 shutdown_connections 2024-09-19T07:53:14.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.866+0000 7f4812090740 1 -- 172.21.15.103:0/1332135723 wait complete. 2024-09-19T07:53:14.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.866+0000 7f4812090740 20 client.27764 trim_cache size 0 max 16384 2024-09-19T07:53:14.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.886+0000 7f4812090740 1 Processor -- start 2024-09-19T07:53:14.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f4812090740 1 -- start start 2024-09-19T07:53:14.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f4812090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aaec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f4812090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f4812090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aaf7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f4812090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b3fb0 con 0x56019aacd480 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f4812090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b5af0 con 0x56019aad7d40 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f4812090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b4680 con 0x56019aaccb60 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f480da53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aaec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f480da53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aaec480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49738/0 (socket says 172.21.15.103:49738) 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f480da53640 1 -- 172.21.15.103:0/2973149530 learned_addr learned my addr 172.21.15.103:0/2973149530 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f480d252640 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.887+0000 7f480e254640 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aaf7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f480da53640 1 -- 172.21.15.103:0/2973149530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 msgr2=0x56019aaf7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f480da53640 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aaf7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f480da53640 1 -- 172.21.15.103:0/2973149530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 msgr2=0x56019aad6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f480da53640 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f480da53640 1 -- 172.21.15.103:0/2973149530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56019aad7b30 con 0x56019aaccb60 2024-09-19T07:53:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f480d252640 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f480da53640 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aaec480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4804009850 tx=0x7f480403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374c3289f9e3f5e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f47f37fe640 1 -- 172.21.15.103:0/2973149530 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804059070 con 0x56019aaccb60 2024-09-19T07:53:14.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.888+0000 7f47f37fe640 1 -- 172.21.15.103:0/2973149530 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4804053a90 con 0x56019aaccb60 2024-09-19T07:53:14.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.889+0000 7f4812090740 1 -- 172.21.15.103:0/2973149530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 msgr2=0x56019aaec480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.889+0000 7f4812090740 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aaec480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4804009850 tx=0x7f480403d000 comp rx=0 tx=0).stop 2024-09-19T07:53:14.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.893+0000 7f4812090740 1 -- 172.21.15.103:0/2973149530 shutdown_connections 2024-09-19T07:53:14.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.893+0000 7f4812090740 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aaf7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.893+0000 7f4812090740 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aacd480 0x56019aad6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.893+0000 7f4812090740 1 --2- 172.21.15.103:0/2973149530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aaccb60 0x56019aaec480 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.893+0000 7f4812090740 1 -- 172.21.15.103:0/2973149530 >> 172.21.15.103:0/2973149530 conn(0x56019aac90f0 msgr2=0x56019aad6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:14.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.897+0000 7f4812090740 1 -- 172.21.15.103:0/2973149530 shutdown_connections 2024-09-19T07:53:14.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.899+0000 7f4812090740 1 -- 172.21.15.103:0/2973149530 wait complete. 2024-09-19T07:53:14.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.899+0000 7f4812090740 1 -- start start 2024-09-19T07:53:14.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.899+0000 7f4812090740 1 Processor -- start 2024-09-19T07:53:14.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.901+0000 7f4812090740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019ab58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.901+0000 7f4812090740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aacd480 0x56019aad18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.901+0000 7f4812090740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.901+0000 7f4812090740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b5420 con 0x56019aaccb60 2024-09-19T07:53:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.901+0000 7f4812090740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b4d50 con 0x56019aad7d40 2024-09-19T07:53:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.901+0000 7f4812090740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48081b1fb0 con 0x56019aacd480 2024-09-19T07:53:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.901+0000 7f4812090740 10 client.0 _ll_register_callbacks cb 0x7f47f8097f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.904+0000 7f480d252640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aacd480 0x56019aad18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.904+0000 7f480d252640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aacd480 0x56019aad18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49746/0 (socket says 172.21.15.103:49746) 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.904+0000 7f480d252640 1 -- 172.21.15.103:0/2823481244 learned_addr learned my addr 172.21.15.103:0/2823481244 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.904+0000 7f480da53640 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019ab58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.908+0000 7f480e254640 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.908+0000 7f480e254640 1 -- 172.21.15.103:0/2823481244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aacd480 msgr2=0x56019aad18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.908+0000 7f480e254640 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aacd480 0x56019aad18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.908+0000 7f480e254640 1 -- 172.21.15.103:0/2823481244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 msgr2=0x56019ab58350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.908+0000 7f480e254640 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019ab58350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.908+0000 7f480e254640 1 -- 172.21.15.103:0/2823481244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56019aad24f0 con 0x56019aad7d40 2024-09-19T07:53:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.908+0000 7f480e254640 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad1e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4800011750 tx=0x7f48000042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14dfc04588117ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:14.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.909+0000 7f47f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:53:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.909+0000 7f47f17fa640 1 -- 172.21.15.103:0/2823481244 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480003c060 con 0x56019aad7d40 2024-09-19T07:53:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.910+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56019aa39d00 con 0x56019aad7d40 2024-09-19T07:53:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.910+0000 7f47f17fa640 1 -- 172.21.15.103:0/2823481244 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8586+0+0 (secure 0 0 0) 0x7f4800019d80 con 0x56019aad7d40 2024-09-19T07:53:14.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.910+0000 7f47f17fa640 1 -- 172.21.15.103:0/2823481244 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f480001e040 con 0x56019aad7d40 2024-09-19T07:53:14.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.911+0000 7f4812090740 10 client.27767 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:14.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.911+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56019aae2d00 con 0x56019aad7d40 2024-09-19T07:53:14.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.914+0000 7f47f17fa640 1 -- 172.21.15.103:0/2823481244 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56019aa39d00 con 0x56019aad7d40 2024-09-19T07:53:14.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.914+0000 7f47f17fa640 1 -- 172.21.15.103:0/2823481244 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f48000195a0 con 0x56019aad7d40 2024-09-19T07:53:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.916+0000 7f4812090740 10 client.27767 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.916+0000 7f4812090740 10 client.27767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.916+0000 7f4812090740 20 client.27767 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.916+0000 7f4812090740 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aae53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.916+0000 7f4812090740 4 client.27767 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:53:14.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.916+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56019a9b6110 con 0x56019aaf0040 2024-09-19T07:53:14.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.919+0000 7f480da53640 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aae53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:14.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.932+0000 7f480da53640 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aae53e0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb1bb567e3e2c0f server_cookie=5e964cc4c0bc6791 in_seq=0 out_seq=0 2024-09-19T07:53:14.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.932+0000 7f47f17fa640 10 client.27767 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:14.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.933+0000 7f47f17fa640 1 -- 172.21.15.103:0/2823481244 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56019a9b6110 con 0x56019aaf0040 2024-09-19T07:53:14.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.933+0000 7f47f17fa640 10 client.27767 handle_command_reply: tid=0 2024-09-19T07:53:14.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.936+0000 7f4812090740 1 client.27767 shutdown 2024-09-19T07:53:14.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.944+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 msgr2=0x56019aad1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.944+0000 7f4812090740 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad1e30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4800011750 tx=0x7f48000042b0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 shutdown_connections 2024-09-19T07:53:14.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56019aaf0040 0x56019aae53e0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56019aad7d40 0x56019aad1e30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.947 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:53:14.948 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:53:14.948 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "e7e7dd4f-0fc6-472b-80ba-5690d0bdcc62", 2024-09-19T07:53:14.948 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:53:14.948 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:53:14.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56019aacd480 0x56019aad18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 --2- 172.21.15.103:0/2823481244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56019aaccb60 0x56019ab58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 >> 172.21.15.103:0/2823481244 conn(0x56019aac90f0 msgr2=0x56019aac8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f47f17fa640 0 client.27767 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 shutdown_connections 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 1 -- 172.21.15.103:0/2823481244 wait complete. 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.946+0000 7f4812090740 20 client.27767 trim_cache size 0 max 16384 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f47f40a3430 msgr2=0x7f47f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f47f40a3430 0x7f47f40a58f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4804005ab0 tx=0x7f480400b040 comp rx=0 tx=0).stop 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808109ba0 msgr2=0x7f48081a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808109ba0 0x7f48081a6b60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e9f0 tx=0x7f47f800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 shutdown_connections 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f47f40a3430 0x7f47f40a58f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4808114ab0 0x7f48081a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808109ba0 0x7f48081a6b60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 --2- 172.21.15.103:0/1790440096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f48081091f0 0x7f48081a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:14.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 >> 172.21.15.103:0/1790440096 conn(0x7f480806e2d0 msgr2=0x7f48080719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:14.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 shutdown_connections 2024-09-19T07:53:14.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:14.947+0000 7f480fcb7640 1 -- 172.21.15.103:0/1790440096 wait complete. 2024-09-19T07:53:15.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 880.0 s, 1597.8 tps, lat 62.263 ms stddev 71.713 2024-09-19T07:53:15.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:15 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:15 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:15 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:15 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:15.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:15 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:15 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:15 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:15 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:15 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:15 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:15 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:15 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:15 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:15 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:15 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:15.934 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:53:15.952 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:16.442 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: pgmap v478: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 34 MiB/s wr, 1.92k op/s 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.443 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.444 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.444 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:16 smithi103 ceph-mon[31201]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:53:16.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 -- 172.21.15.103:0/1573226175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410cf30 msgr2=0x7fcdf4119410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 --2- 172.21.15.103:0/1573226175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410cf30 0x7fcdf4119410 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009900 tx=0x7fcdec02a410 comp rx=0 tx=0).stop 2024-09-19T07:53:16.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 -- 172.21.15.103:0/1573226175 shutdown_connections 2024-09-19T07:53:16.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 --2- 172.21.15.103:0/1573226175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410cf30 0x7fcdf4119410 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 --2- 172.21.15.103:0/1573226175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdf410c590 0x7fcdf410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 --2- 172.21.15.103:0/1573226175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdf4112640 0x7fcdf4112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 -- 172.21.15.103:0/1573226175 >> 172.21.15.103:0/1573226175 conn(0x7fcdf406cdc0 msgr2=0x7fcdf406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:16.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 -- 172.21.15.103:0/1573226175 shutdown_connections 2024-09-19T07:53:16.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.566+0000 7fcdf912a640 1 -- 172.21.15.103:0/1573226175 wait complete. 2024-09-19T07:53:16.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.567+0000 7fcdf912a640 1 Processor -- start 2024-09-19T07:53:16.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.567+0000 7fcdf912a640 1 -- start start 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.567+0000 7fcdf912a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410c590 0x7fcdf41a8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf912a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdf410cf30 0x7fcdf41a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf912a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdf4112640 0x7fcdf41a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdf410cf30 0x7fcdf41a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdf410cf30 0x7fcdf41a8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:41036/0 (socket says 172.21.15.103:41036) 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf259c640 1 -- 172.21.15.103:0/2771287938 learned_addr learned my addr 172.21.15.103:0/2771287938 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf359e640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdf4112640 0x7fcdf41a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf411c9f0 con 0x7fcdf410c590 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf411c870 con 0x7fcdf410cf30 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf411cb70 con 0x7fcdf4112640 2024-09-19T07:53:16.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410c590 0x7fcdf41a8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf359e640 1 -- 172.21.15.103:0/2771287938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdf410cf30 msgr2=0x7fcdf41a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf359e640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdf410cf30 0x7fcdf41a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf359e640 1 -- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410c590 msgr2=0x7fcdf41a8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf359e640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410c590 0x7fcdf41a8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.568+0000 7fcdf359e640 1 -- 172.21.15.103:0/2771287938 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf41a3070 con 0x7fcdf4112640 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.569+0000 7fcdf359e640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdf4112640 0x7fcdf41a2940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcdec005ab0 tx=0x7fcdec030940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.569+0000 7fcddbfff640 1 -- 172.21.15.103:0/2771287938 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec03f070 con 0x7fcdf4112640 2024-09-19T07:53:16.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.569+0000 7fcddbfff640 1 -- 172.21.15.103:0/2771287938 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdec032e40 con 0x7fcdf4112640 2024-09-19T07:53:16.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.569+0000 7fcddbfff640 1 -- 172.21.15.103:0/2771287938 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec035030 con 0x7fcdf4112640 2024-09-19T07:53:16.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.569+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf41a3300 con 0x7fcdf4112640 2024-09-19T07:53:16.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.570+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf41a36d0 con 0x7fcdf4112640 2024-09-19T07:53:16.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.572+0000 7fcddbfff640 1 -- 172.21.15.103:0/2771287938 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcdec042070 con 0x7fcdf4112640 2024-09-19T07:53:16.572 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:16.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.572+0000 7fcddbfff640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcddc0a3430 0x7fcddc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.573+0000 7fcddbfff640 1 -- 172.21.15.103:0/2771287938 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fcdec0f09a0 con 0x7fcdf4112640 2024-09-19T07:53:16.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.573+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcddc0a3430 0x7fcddc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.573+0000 7fcdfb503740 1 Processor -- start 2024-09-19T07:53:16.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.573+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcddc0a3430 0x7fcddc0a58f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcdf41a3a80 tx=0x7fcde4009680 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:16.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.574+0000 7fcdfb503740 1 -- start start 2024-09-19T07:53:16.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.574+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45d770 0x55699d45db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.574+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45e090 0x55699d470d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.574+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d473630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.574+0000 7fcdfb503740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55699d3c2160 con 0x55699d471240 2024-09-19T07:53:16.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.574+0000 7fcdfb503740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55699d369720 con 0x55699d45e090 2024-09-19T07:53:16.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.574+0000 7fcdfb503740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55699d3be120 con 0x55699d45d770 2024-09-19T07:53:16.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45d770 0x55699d45db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45d770 0x55699d45db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49772/0 (socket says 172.21.15.103:49772) 2024-09-19T07:53:16.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/2097795805 learned_addr learned my addr 172.21.15.103:0/2097795805 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:16.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf259c640 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45e090 0x55699d470d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/2097795805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45e090 msgr2=0x55699d470d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45e090 0x55699d470d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/2097795805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 msgr2=0x55699d473630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d473630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.575+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/2097795805 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699d133ba0 con 0x55699d45d770 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.576+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45d770 0x55699d45db50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009cc0 tx=0x7fcde4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b46ade42d6ba630 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.576+0000 7fcdd97fa640 1 -- 172.21.15.103:0/2097795805 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4023070 con 0x55699d45d770 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.576+0000 7fcdd97fa640 1 -- 172.21.15.103:0/2097795805 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde401fb30 con 0x55699d45d770 2024-09-19T07:53:16.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.576+0000 7fcdd97fa640 1 -- 172.21.15.103:0/2097795805 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4011b30 con 0x55699d45d770 2024-09-19T07:53:16.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.577+0000 7fcdfb503740 1 -- 172.21.15.103:0/2097795805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45d770 msgr2=0x55699d45db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.577+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45d770 0x55699d45db50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009cc0 tx=0x7fcde4013000 comp rx=0 tx=0).stop 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.577+0000 7fcdfb503740 1 -- 172.21.15.103:0/2097795805 shutdown_connections 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.577+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d473630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.577+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45e090 0x55699d470d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.577+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2097795805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45d770 0x55699d45db50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.577+0000 7fcdfb503740 1 -- 172.21.15.103:0/2097795805 >> 172.21.15.103:0/2097795805 conn(0x55699d459de0 msgr2=0x55699d45a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.578+0000 7fcdfb503740 1 -- 172.21.15.103:0/2097795805 shutdown_connections 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.578+0000 7fcdfb503740 1 -- 172.21.15.103:0/2097795805 wait complete. 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.578+0000 7fcdfb503740 1 -- start start 2024-09-19T07:53:16.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.578+0000 7fcdfb503740 1 Processor -- start 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: pgmap v478: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 34 MiB/s wr, 1.92k op/s 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:16.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:16.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:53:16.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:16 smithi134 ceph-mon[35414]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:53:16.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.578+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45d770 0x55699d46e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d46e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45d770 0x55699d46e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45d770 0x55699d46e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:41056/0 (socket says 172.21.15.103:41056) 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/1342614199 learned_addr learned my addr 172.21.15.103:0/1342614199 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdfb503740 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d46ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55699d473db0 con 0x55699d471240 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55699d34a900 con 0x55699d45d770 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55699d473f30 con 0x55699d45e090 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf359e640 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d46ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf259c640 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d46e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/1342614199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 msgr2=0x55699d46e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d46e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/1342614199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 msgr2=0x55699d46ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d46ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.579+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/1342614199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699d38ac50 con 0x55699d45d770 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdfb503740 10 client.0 _ll_register_callbacks cb 0x55699d348780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45d770 0x55699d46e360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcde4011940 tx=0x7fcde4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334196dbe128e778 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde401fb30 con 0x55699d45d770 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/1342614199 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4037060 con 0x55699d45d770 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/1342614199 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fcde4011cb0 con 0x55699d45d770 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/1342614199 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fcde4023050 con 0x55699d45d770 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdfb503740 10 client.27776 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=54}) -- 0x55699d47fb10 con 0x55699d45d770 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.580+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/1342614199 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fcde401fce0 con 0x55699d45d770 2024-09-19T07:53:16.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.581+0000 7fcdfb503740 10 client.27776 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.581+0000 7fcdfb503740 10 client.27776 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.581+0000 7fcdfb503740 20 client.27776 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.581+0000 7fcdfb503740 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d480e60 0x55699d483250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.581+0000 7fcdfb503740 4 client.27776 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.581+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcde401fb30 con 0x55699d480e60 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.581+0000 7fcdf259c640 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d480e60 0x55699d483250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.582+0000 7fcdf259c640 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d480e60 0x55699d483250 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98b7cf0e3418ade3 server_cookie=32b63cdf69d4ae6c in_seq=0 out_seq=0 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.582+0000 7fcdd6ff5640 10 client.27776 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/1342614199 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcde401fb30 con 0x55699d480e60 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdd6ff5640 10 client.27776 handle_command_reply: tid=0 2024-09-19T07:53:16.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 client.27776 shutdown 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45d770 msgr2=0x55699d46e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45d770 0x55699d46e360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcde4011940 tx=0x7fcde4005fd0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 shutdown_connections 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d480e60 0x55699d483250 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d46ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d46e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 --2- 172.21.15.103:0/1342614199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d45d770 0x55699d46e360 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcde4011940 tx=0x7fcde4005fd0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 >> 172.21.15.103:0/1342614199 conn(0x55699d459de0 msgr2=0x55699d45fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:16.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.583+0000 7fcdd6ff5640 0 client.27776 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:16.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.587+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 shutdown_connections 2024-09-19T07:53:16.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.588+0000 7fcdfb503740 1 -- 172.21.15.103:0/1342614199 wait complete. 2024-09-19T07:53:16.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.589+0000 7fcdfb503740 20 client.27776 trim_cache size 0 max 16384 2024-09-19T07:53:16.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.625+0000 7fcdfb503740 1 Processor -- start 2024-09-19T07:53:16.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.626+0000 7fcdfb503740 1 -- start start 2024-09-19T07:53:16.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.626+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d458dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d471240 0x55699d4887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d488d30 0x55699d46b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d458dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d458dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49794/0 (socket says 172.21.15.103:49794) 2024-09-19T07:53:16.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/230560104 learned_addr learned my addr 172.21.15.103:0/230560104 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:16.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b3e80 con 0x55699d488d30 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b59c0 con 0x55699d471240 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b4550 con 0x55699d45e090 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/230560104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d471240 msgr2=0x55699d4887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/230560104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d471240 0x55699d4887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/230560104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d488d30 msgr2=0x55699d46b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/230560104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d488d30 0x55699d46b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.627+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/230560104 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699d45ec80 con 0x55699d45e090 2024-09-19T07:53:16.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.628+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/230560104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d458dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcde4004480 tx=0x7fcde402d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a61d2651ef0bc149 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:16.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.630+0000 7fcdd8ff9640 1 -- 172.21.15.103:0/230560104 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4016040 con 0x55699d45e090 2024-09-19T07:53:16.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.631+0000 7fcdd8ff9640 1 -- 172.21.15.103:0/230560104 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde4011cb0 con 0x55699d45e090 2024-09-19T07:53:16.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.631+0000 7fcdd8ff9640 1 -- 172.21.15.103:0/230560104 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4032b00 con 0x55699d45e090 2024-09-19T07:53:16.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.632+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 msgr2=0x55699d458dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.632+0000 7fcdfb503740 1 --2- 172.21.15.103:0/230560104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d458dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcde4004480 tx=0x7fcde402d000 comp rx=0 tx=0).stop 2024-09-19T07:53:16.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.632+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 shutdown_connections 2024-09-19T07:53:16.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.632+0000 7fcdfb503740 1 --2- 172.21.15.103:0/230560104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d488d30 0x55699d46b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.632+0000 7fcdfb503740 1 --2- 172.21.15.103:0/230560104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d471240 0x55699d4887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.632+0000 7fcdfb503740 1 --2- 172.21.15.103:0/230560104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d45e090 0x55699d458dd0 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcde4004480 tx=0x7fcde402d000 comp rx=0 tx=0).stop 2024-09-19T07:53:16.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.632+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 >> 172.21.15.103:0/230560104 conn(0x55699d459de0 msgr2=0x55699d4586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:16.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.633+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 shutdown_connections 2024-09-19T07:53:16.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.633+0000 7fcdfb503740 1 -- 172.21.15.103:0/230560104 wait complete. 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.633+0000 7fcdfb503740 1 -- start start 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.633+0000 7fcdfb503740 1 Processor -- start 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.634+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d474e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.634+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d488d30 0x55699d475360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.634+0000 7fcdfb503740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d4758a0 0x55699d4ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.634+0000 7fcdfb503740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b52f0 con 0x55699d471240 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.634+0000 7fcdfb503740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b4c20 con 0x55699d488d30 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.634+0000 7fcdfb503740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf41b1e80 con 0x55699d4758a0 2024-09-19T07:53:16.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.634+0000 7fcdfb503740 10 client.0 _ll_register_callbacks cb 0x7fcde40048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:16.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.635+0000 7fcdf359e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d4758a0 0x55699d4ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.635+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d488d30 0x55699d475360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.635+0000 7fcdf2d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d474e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.635+0000 7fcdf359e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d4758a0 0x55699d4ea7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:49804/0 (socket says 172.21.15.103:49804) 2024-09-19T07:53:16.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.635+0000 7fcdf359e640 1 -- 172.21.15.103:0/2484702428 learned_addr learned my addr 172.21.15.103:0/2484702428 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:16.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/2484702428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d4758a0 msgr2=0x55699d4ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d4758a0 0x55699d4ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d488d30 msgr2=0x55699d475360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:16.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d488d30 0x55699d475360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf2d9d640 1 -- 172.21.15.103:0/2484702428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699d4ead00 con 0x55699d471240 2024-09-19T07:53:16.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf259c640 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d488d30 0x55699d475360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:16.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf359e640 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d4758a0 0x55699d4ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:16.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdf2d9d640 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d474e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x55699d46db50 tx=0x7fcde4011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f7e22f3c68791a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:16.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:16.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.636+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/2484702428 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4049070 con 0x55699d471240 2024-09-19T07:53:16.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.637+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/2484702428 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fcde4005b70 con 0x55699d471240 2024-09-19T07:53:16.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.637+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde4011cb0 con 0x55699d471240 2024-09-19T07:53:16.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.637+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/2484702428 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fcde4055050 con 0x55699d471240 2024-09-19T07:53:16.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.637+0000 7fcdfb503740 10 client.18459 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:16.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.637+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55699d4ebdf0 con 0x55699d471240 2024-09-19T07:53:16.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.638+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/2484702428 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde4004e40 con 0x55699d471240 2024-09-19T07:53:16.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/2484702428 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fcde400f040 con 0x55699d471240 2024-09-19T07:53:16.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdfb503740 10 client.18459 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:16.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdfb503740 10 client.18459 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:53:16.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdfb503740 20 client.18459 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:16.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d4ec510 0x55699d4ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:16.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdfb503740 4 client.18459 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T07:53:16.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x7fcde4011cb0 con 0x55699d4ec510 2024-09-19T07:53:16.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.639+0000 7fcdf259c640 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d4ec510 0x55699d4ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:16.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.640+0000 7fcdf259c640 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d4ec510 0x55699d4ee900 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6dd9be442ee15c server_cookie=aa0c684a80f700a7 in_seq=0 out_seq=0 2024-09-19T07:53:16.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.640+0000 7fcdd6ff5640 10 client.18459 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: pgmap v478: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 34 MiB/s wr, 1.92k op/s 2024-09-19T07:53:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:53:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:53:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:16 smithi174 ceph-mon[35370]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T07:53:16.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdd6ff5640 1 -- 172.21.15.103:0/2484702428 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fcde4011cb0 con 0x55699d4ec510 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdd6ff5640 10 client.18459 handle_command_reply: tid=0 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 client.18459 shutdown 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 msgr2=0x55699d474e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d474e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x55699d46db50 tx=0x7fcde4011690 comp rx=0 tx=0).stop 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 shutdown_connections 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55699d4ec510 0x55699d4ee900 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55699d4758a0 0x55699d4ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55699d488d30 0x55699d475360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 --2- 172.21.15.103:0/2484702428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55699d471240 0x55699d474e20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdd6ff5640 0 client.18459 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.849+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 >> 172.21.15.103:0/2484702428 conn(0x55699d459de0 msgr2=0x55699d46d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:16.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 shutdown_connections 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdfb503740 1 -- 172.21.15.103:0/2484702428 wait complete. 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdfb503740 20 client.18459 trim_cache size 0 max 16384 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 54, 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 50, 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stdout: "4910a3a6": { 2024-09-19T07:53:16.852 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 50, 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:53:16.853 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:53:16.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:16.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:16.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:16.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:16.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:16.854 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:53:16.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcddc0a3430 msgr2=0x7fcddc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcddc0a3430 0x7fcddc0a58f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcdf41a3a80 tx=0x7fcde4009680 comp rx=0 tx=0).stop 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdf4112640 msgr2=0x7fcdf41a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdf4112640 0x7fcdf41a2940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcdec005ab0 tx=0x7fcdec030940 comp rx=0 tx=0).stop 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 shutdown_connections 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcddc0a3430 0x7fcddc0a58f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdf4112640 0x7fcdf41a2940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdf410cf30 0x7fcdf41a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 --2- 172.21.15.103:0/2771287938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdf410c590 0x7fcdf41a8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.850+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 >> 172.21.15.103:0/2771287938 conn(0x7fcdf406cdc0 msgr2=0x7fcdf410aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.851+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 shutdown_connections 2024-09-19T07:53:16.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:16.851+0000 7fcdf912a640 1 -- 172.21.15.103:0/2771287938 wait complete. 2024-09-19T07:53:18.383 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:18 smithi103 ceph-mon[31201]: pgmap v480: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 831 op/s 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 -- 172.21.15.103:0/1644019983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 msgr2=0x7f94e8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 --2- 172.21.15.103:0/1644019983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e8106e60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009930 tx=0x7f94d802a420 comp rx=0 tx=0).stop 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 -- 172.21.15.103:0/1644019983 shutdown_connections 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 --2- 172.21.15.103:0/1644019983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94e8101420 0x7f94e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 --2- 172.21.15.103:0/1644019983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e8100a80 0x7f94e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 --2- 172.21.15.103:0/1644019983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e8106e60 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 -- 172.21.15.103:0/1644019983 >> 172.21.15.103:0/1644019983 conn(0x7f94e80fc7a0 msgr2=0x7f94e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:18.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 -- 172.21.15.103:0/1644019983 shutdown_connections 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 -- 172.21.15.103:0/1644019983 wait complete. 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.466+0000 7f94ed204640 1 Processor -- start 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94ed204640 1 -- start start 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94ed204640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e8100a80 0x7f94e80fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94ed204640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94e8101420 0x7f94e80fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94ed204640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e80fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94ed204640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8110e40 con 0x7f94e8106a80 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94ed204640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8110cc0 con 0x7f94e8100a80 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94ed204640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8110fc0 con 0x7f94e8101420 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e80fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e80fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37556/0 (socket says 172.21.15.103:37556) 2024-09-19T07:53:18.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 -- 172.21.15.103:0/1861682154 learned_addr learned my addr 172.21.15.103:0/1861682154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e659c640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94e8101420 0x7f94e80fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e8100a80 0x7f94e80fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 -- 172.21.15.103:0/1861682154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94e8101420 msgr2=0x7f94e80fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94e8101420 0x7f94e80fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 -- 172.21.15.103:0/1861682154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e8100a80 msgr2=0x7f94e80fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e8100a80 0x7f94e80fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.467+0000 7f94e759e640 1 -- 172.21.15.103:0/1861682154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e81a6880 con 0x7f94e8106a80 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.468+0000 7f94e759e640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e80fd7c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f94e000eaa0 tx=0x7f94e000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.468+0000 7f94cffff640 1 -- 172.21.15.103:0/1861682154 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0004d20 con 0x7f94e8106a80 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.468+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e81a6b70 con 0x7f94e8106a80 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.468+0000 7f94cffff640 1 -- 172.21.15.103:0/1861682154 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0005020 con 0x7f94e8106a80 2024-09-19T07:53:18.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.468+0000 7f94cffff640 1 -- 172.21.15.103:0/1861682154 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00116d0 con 0x7f94e8106a80 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.468+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e81a7060 con 0x7f94e8106a80 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.470+0000 7f94cffff640 1 -- 172.21.15.103:0/1861682154 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f94e00051d0 con 0x7f94e8106a80 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.470+0000 7f94ef5dd740 1 Processor -- start 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.470+0000 7f94ef5dd740 1 -- start start 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.470+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888913b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.470+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913bdc0 0x558889146140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x558889148a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5588890a12e0 con 0x55888913bdc0 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558889048ce0 con 0x558889146680 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55888909d2a0 con 0x55888913b4a0 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94cffff640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a33b0 0x7f94d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913bdc0 0x558889146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913bdc0 0x558889146140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37560/0 (socket says 172.21.15.103:37560) 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 -- 172.21.15.103:0/2915478713 learned_addr learned my addr 172.21.15.103:0/2915478713 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e759e640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a33b0 0x7f94d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e659c640 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888913b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e659c640 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x558889148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94cffff640 1 -- 172.21.15.103:0/1861682154 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f94e00cfe40 con 0x7f94e8106a80 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 -- 172.21.15.103:0/2915478713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 msgr2=0x55888913b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888913b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 -- 172.21.15.103:0/2915478713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 msgr2=0x558889148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x558889148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 -- 172.21.15.103:0/2915478713 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55888902d440 con 0x55888913bdc0 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e659c640 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888913b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e759e640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a33b0 0x7f94d00a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94e0098570 tx=0x7f94e0066fa0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:18.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913bdc0 0x558889146140 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009930 tx=0x7f94d8031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e560dc9cdeebb29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94cd7fa640 1 -- 172.21.15.103:0/2915478713 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8040070 con 0x55888913bdc0 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.471+0000 7f94cd7fa640 1 -- 172.21.15.103:0/2915478713 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d8005020 con 0x55888913bdc0 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94cd7fa640 1 -- 172.21.15.103:0/2915478713 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8035040 con 0x55888913bdc0 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2915478713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913bdc0 msgr2=0x558889146140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913bdc0 0x558889146140 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009930 tx=0x7f94d8031630 comp rx=0 tx=0).stop 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2915478713 shutdown_connections 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x558889148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913bdc0 0x558889146140 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2915478713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888913b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2915478713 >> 172.21.15.103:0/2915478713 conn(0x558889137a30 msgr2=0x558889137e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2915478713 shutdown_connections 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.472+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2915478713 wait complete. 2024-09-19T07:53:18.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 -- start start 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 Processor -- start 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x558889145df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x5588891430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x5588891435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558889149250 con 0x558889146680 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55888905c050 con 0x55888913bdc0 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5588891493d0 con 0x55888913b4a0 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94ef5dd740 10 client.0 _ll_register_callbacks cb 0x55888909c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x5588891435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e6d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x5588891430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x5588891435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37564/0 (socket says 172.21.15.103:37564) 2024-09-19T07:53:18.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 -- 172.21.15.103:0/2195393681 learned_addr learned my addr 172.21.15.103:0/2195393681 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x558889145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 -- 172.21.15.103:0/2195393681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 msgr2=0x558889145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x558889145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 -- 172.21.15.103:0/2195393681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 msgr2=0x5588891430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x5588891430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 -- 172.21.15.103:0/2195393681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558888e123c0 con 0x558889146680 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94e659c640 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x5588891435e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00f430 tx=0x7f94dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adbb5de11947f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.473+0000 7f94caff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94caff5640 1 -- 172.21.15.103:0/2195393681 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc00d550 con 0x558889146680 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94caff5640 1 -- 172.21.15.103:0/2195393681 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f94dc00d6f0 con 0x558889146680 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d8005020 con 0x558889146680 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94caff5640 1 -- 172.21.15.103:0/2195393681 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f94dc00d8f0 con 0x558889146680 2024-09-19T07:53:18.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 10 client.18471 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558889154a90 con 0x558889146680 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94caff5640 1 -- 172.21.15.103:0/2195393681 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d8005020 con 0x558889146680 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94caff5640 1 -- 172.21.15.103:0/2195393681 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f94dc020a40 con 0x558889146680 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 10 client.18471 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 10 client.18471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 20 client.18471 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558889155f50 0x558889158360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 4 client.18471 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588890ae850 con 0x558889155f50 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.474+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558889155f50 0x558889158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.475+0000 7f94caff5640 10 client.18471 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:18.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.475+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558889155f50 0x558889158360 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5caa1349d4f715 server_cookie=7b33fa20b606f729 in_seq=0 out_seq=0 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.475+0000 7f94caff5640 1 -- 172.21.15.103:0/2195393681 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588890ae850 con 0x558889155f50 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.475+0000 7f94caff5640 10 client.18471 handle_command_reply: tid=0 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 client.18471 shutdown 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 msgr2=0x5588891435e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x5588891435e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00f430 tx=0x7f94dc0046c0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 shutdown_connections 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x558889155f50 0x558889158360 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x5588891435e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x5588891430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94caff5640 0 client.18471 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2195393681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x558889145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 >> 172.21.15.103:0/2195393681 conn(0x558889137a30 msgr2=0x55888913d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:18.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 shutdown_connections 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2195393681 wait complete. 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.476+0000 7f94ef5dd740 20 client.18471 trim_cache size 0 max 16384 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 Processor -- start 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 -- start start 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913b4a0 0x558889141970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913bdc0 0x55888915dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x55888915e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81afad0 con 0x55888913b4a0 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81b1610 con 0x558889146680 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81b01a0 con 0x55888913bdc0 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913b4a0 0x558889141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913b4a0 0x558889141970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37568/0 (socket says 172.21.15.103:37568) 2024-09-19T07:53:18.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94e659c640 1 -- 172.21.15.103:0/211463771 learned_addr learned my addr 172.21.15.103:0/211463771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913bdc0 0x55888915dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.493+0000 7f94e659c640 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x55888915e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94e659c640 1 -- 172.21.15.103:0/211463771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913bdc0 msgr2=0x55888915dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94e659c640 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913bdc0 0x55888915dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94e659c640 1 -- 172.21.15.103:0/211463771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 msgr2=0x55888915e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94e659c640 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x55888915e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94e659c640 1 -- 172.21.15.103:0/211463771 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558889142050 con 0x55888913b4a0 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94e659c640 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913b4a0 0x558889141970 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f94dc011dc0 tx=0x7f94dc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282ae90791a6df2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94ccff9640 1 -- 172.21.15.103:0/211463771 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc00ae80 con 0x55888913b4a0 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94ccff9640 1 -- 172.21.15.103:0/211463771 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc00fe70 con 0x55888913b4a0 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.494+0000 7f94ccff9640 1 -- 172.21.15.103:0/211463771 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc011330 con 0x55888913b4a0 2024-09-19T07:53:18.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- 172.21.15.103:0/211463771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913b4a0 msgr2=0x558889141970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913b4a0 0x558889141970 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f94dc011dc0 tx=0x7f94dc001e50 comp rx=0 tx=0).stop 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- 172.21.15.103:0/211463771 shutdown_connections 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558889146680 0x55888915e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913bdc0 0x55888915dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/211463771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55888913b4a0 0x558889141970 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- 172.21.15.103:0/211463771 >> 172.21.15.103:0/211463771 conn(0x558889137a30 msgr2=0x558889145120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- 172.21.15.103:0/211463771 shutdown_connections 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- 172.21.15.103:0/211463771 wait complete. 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- start start 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 Processor -- start 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888914e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x55888914ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x558889149640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81b0f40 con 0x558889146680 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81b0870 con 0x55888913bdc0 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81adad0 con 0x55888913b4a0 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94ef5dd740 10 client.0 _ll_register_callbacks cb 0x7f94dc011cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.495+0000 7f94e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x558889149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x558889149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37582/0 (socket says 172.21.15.103:37582) 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 -- 172.21.15.103:0/2139242761 learned_addr learned my addr 172.21.15.103:0/2139242761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x55888914ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888914e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 -- 172.21.15.103:0/2139242761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 msgr2=0x55888914e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888914e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 -- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 msgr2=0x55888914ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x55888914ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 -- 172.21.15.103:0/2139242761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558889149b80 con 0x558889146680 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94e659c640 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x558889149640 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f94dc011c50 tx=0x7f94dc011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f5d5480333da11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94caff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94caff5640 1 -- 172.21.15.103:0/2139242761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0112e0 con 0x558889146680 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94caff5640 1 -- 172.21.15.103:0/2139242761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f94dc00dca0 con 0x558889146680 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc00fe70 con 0x558889146680 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94caff5640 1 -- 172.21.15.103:0/2139242761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f94dc045050 con 0x558889146680 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94ef5dd740 10 client.18477 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5588891c13b0 con 0x558889146680 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.496+0000 7f94caff5640 1 -- 172.21.15.103:0/2139242761 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc00fe70 con 0x558889146680 2024-09-19T07:53:18.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94caff5640 1 -- 172.21.15.103:0/2139242761 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f94dc01fc00 con 0x558889146680 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 10 client.18477 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 20 client.18477 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5588891c1a10 0x5588891c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 4 client.18477 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588890a8d60 con 0x5588891c1a10 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5588891c1a10 0x5588891c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94e6d9d640 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5588891c1a10 0x5588891c3e00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f48062c45fcece1 server_cookie=9797285c2b5b6747 in_seq=0 out_seq=0 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94caff5640 10 client.18477 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94caff5640 1 -- 172.21.15.103:0/2139242761 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5588890a8d60 con 0x5588891c1a10 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94caff5640 10 client.18477 handle_command_reply: tid=0 2024-09-19T07:53:18.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 1 client.18477 shutdown 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 msgr2=0x558889149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.497+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x558889149640 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f94dc011c50 tx=0x7f94dc011be0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 shutdown_connections 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5588891c1a10 0x5588891c3e00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558889146680 0x558889149640 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55888913bdc0 0x55888914ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 --2- 172.21.15.103:0/2139242761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55888913b4a0 0x55888914e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 >> 172.21.15.103:0/2139242761 conn(0x558889137a30 msgr2=0x558889136370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94caff5640 0 client.18477 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 shutdown_connections 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 1 -- 172.21.15.103:0/2139242761 wait complete. 2024-09-19T07:53:18.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ef5dd740 20 client.18477 trim_cache size 0 max 16384 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout: "e7e7dd4f-0fc6-472b-80ba-5690d0bdcc62": { 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "e7e7dd4f-0fc6-472b-80ba-5690d0bdcc62", 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:53:18.517 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:18.518 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:18.518 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:53:18.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a33b0 msgr2=0x7f94d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a33b0 0x7f94d00a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f94e0098570 tx=0x7f94e0066fa0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 msgr2=0x7f94e80fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e80fd7c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f94e000eaa0 tx=0x7f94e000eea0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 shutdown_connections 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a33b0 0x7f94d00a5870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94e8106a80 0x7f94e80fd7c0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94e8101420 0x7f94e80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 --2- 172.21.15.103:0/1861682154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94e8100a80 0x7f94e80fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.498+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 >> 172.21.15.103:0/1861682154 conn(0x7f94e80fc7a0 msgr2=0x7f94e806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.499+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 shutdown_connections 2024-09-19T07:53:18.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:18.499+0000 7f94ed204640 1 -- 172.21.15.103:0/1861682154 wait complete. 2024-09-19T07:53:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:18 smithi134 ceph-mon[35414]: pgmap v480: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 831 op/s 2024-09-19T07:53:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:18 smithi174 ceph-mon[35370]: pgmap v480: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 831 op/s 2024-09-19T07:53:19.185 INFO:tasks.cephfs.filesystem:scrub status for tag:e7e7dd4f-0fc6-472b-80ba-5690d0bdcc62 - {'path': '/', 'tag': 'e7e7dd4f-0fc6-472b-80ba-5690d0bdcc62', 'options': 'recursive,force'} 2024-09-19T07:53:19.767 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4910a3a6', quiesce took 10.5 seconds. Will release after: 18.95 2024-09-19T07:53:20.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 885.0 s, 328.6 tps, lat 73.552 ms stddev 84.112 2024-09-19T07:53:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:20 smithi103 ceph-mon[31201]: pgmap v481: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 831 op/s 2024-09-19T07:53:20.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:20 smithi134 ceph-mon[35414]: pgmap v481: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 831 op/s 2024-09-19T07:53:20.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:20 smithi174 ceph-mon[35370]: pgmap v481: 97 pgs: 97 active+clean; 3.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.3 MiB/s wr, 831 op/s 2024-09-19T07:53:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:22 smithi103 ceph-mon[31201]: pgmap v482: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-09-19T07:53:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:22 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:53:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:22 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:53:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:22 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:53:22.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:22 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:53:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:22 smithi134 ceph-mon[35414]: pgmap v482: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-09-19T07:53:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:22 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:53:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:22 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:53:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:22 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:53:22.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:22 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:53:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:22 smithi174 ceph-mon[35370]: pgmap v482: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 1.14k op/s 2024-09-19T07:53:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:22 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:53:22.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:22 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:53:22.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:22 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:53:22.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:22 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:53:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:24 smithi103 ceph-mon[31201]: pgmap v483: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-19T07:53:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:24 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:53:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:24 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:53:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:24 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:53:24.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:24 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:53:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:24 smithi134 ceph-mon[35414]: pgmap v483: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-19T07:53:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:24 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:53:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:24 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:53:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:24 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:53:24.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:24 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:53:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:24 smithi174 ceph-mon[35370]: pgmap v483: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-19T07:53:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:24 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:53:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:24 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:53:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:24 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:53:24.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:24 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:53:25.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:25 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:53:25.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 890.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:53:25.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:25 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:53:25.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:25 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:53:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:26 smithi103 ceph-mon[31201]: pgmap v484: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-19T07:53:26.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:26 smithi134 ceph-mon[35414]: pgmap v484: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-19T07:53:26.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:26 smithi174 ceph-mon[35370]: pgmap v484: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.32k op/s 2024-09-19T07:53:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:28 smithi103 ceph-mon[31201]: pgmap v485: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-09-19T07:53:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:28 smithi134 ceph-mon[35414]: pgmap v485: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-09-19T07:53:28.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:28 smithi174 ceph-mon[35370]: pgmap v485: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-09-19T07:53:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:30 smithi103 ceph-mon[31201]: pgmap v486: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.9 MiB/s wr, 908 op/s 2024-09-19T07:53:30.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 895.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:53:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:30 smithi134 ceph-mon[35414]: pgmap v486: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.9 MiB/s wr, 908 op/s 2024-09-19T07:53:30.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:30 smithi174 ceph-mon[35370]: pgmap v486: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.9 MiB/s wr, 908 op/s 2024-09-19T07:53:32.467 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:32 smithi174 ceph-mon[35370]: pgmap v487: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.2 MiB/s wr, 914 op/s 2024-09-19T07:53:32.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:32 smithi103 ceph-mon[31201]: pgmap v487: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.2 MiB/s wr, 914 op/s 2024-09-19T07:53:32.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:32 smithi134 ceph-mon[35414]: pgmap v487: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.2 MiB/s wr, 914 op/s 2024-09-19T07:53:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:34 smithi103 ceph-mon[31201]: pgmap v488: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-19T07:53:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:34 smithi134 ceph-mon[35414]: pgmap v488: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-19T07:53:34.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:34 smithi174 ceph-mon[35370]: pgmap v488: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-09-19T07:53:35.466 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 900.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:53:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:36 smithi134 ceph-mon[35414]: pgmap v489: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 837 KiB/s wr, 79 op/s 2024-09-19T07:53:36.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:36 smithi174 ceph-mon[35370]: pgmap v489: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 837 KiB/s wr, 79 op/s 2024-09-19T07:53:36.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:36 smithi103 ceph-mon[31201]: pgmap v489: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 837 KiB/s wr, 79 op/s 2024-09-19T07:53:38.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:38 smithi134 ceph-mon[35414]: pgmap v490: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 901 KiB/s wr, 87 op/s 2024-09-19T07:53:38.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:38 smithi174 ceph-mon[35370]: pgmap v490: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 901 KiB/s wr, 87 op/s 2024-09-19T07:53:38.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:53:38.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:38 smithi103 ceph-mon[31201]: pgmap v490: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 901 KiB/s wr, 87 op/s 2024-09-19T07:53:38.981 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.438+0000 7fc453a54640 1 -- 172.21.15.103:0/1736405958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c100e30 msgr2=0x7fc44c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.438+0000 7fc453a54640 1 --2- 172.21.15.103:0/1736405958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c100e30 0x7fc44c104e80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc44800aa10 tx=0x7fc44802b720 comp rx=0 tx=0).stop 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.438+0000 7fc453a54640 1 -- 172.21.15.103:0/1736405958 shutdown_connections 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.438+0000 7fc453a54640 1 --2- 172.21.15.103:0/1736405958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c1055b0 0x7fc44c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.438+0000 7fc453a54640 1 --2- 172.21.15.103:0/1736405958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c100e30 0x7fc44c104e80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.438+0000 7fc453a54640 1 --2- 172.21.15.103:0/1736405958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44c100480 0x7fc44c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.438+0000 7fc453a54640 1 -- 172.21.15.103:0/1736405958 >> 172.21.15.103:0/1736405958 conn(0x7fc44c077210 msgr2=0x7fc44c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:39.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 -- 172.21.15.103:0/1736405958 shutdown_connections 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 -- 172.21.15.103:0/1736405958 wait complete. 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 Processor -- start 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 -- start start 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44c100480 0x7fc44c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c100e30 0x7fc44c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c1055b0 0x7fc44c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c10fd90 con 0x7fc44c1055b0 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c10fc10 con 0x7fc44c100e30 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.439+0000 7fc453a54640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c10ff10 con 0x7fc44c100480 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c1055b0 0x7fc44c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c1055b0 0x7fc44c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41242/0 (socket says 172.21.15.103:41242) 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 -- 172.21.15.103:0/4170453537 learned_addr learned my addr 172.21.15.103:0/4170453537 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc450fef640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c100e30 0x7fc44c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc4517f0640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44c100480 0x7fc44c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 -- 172.21.15.103:0/4170453537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44c100480 msgr2=0x7fc44c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44c100480 0x7fc44c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 -- 172.21.15.103:0/4170453537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c100e30 msgr2=0x7fc44c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c100e30 0x7fc44c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 -- 172.21.15.103:0/4170453537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44c19d410 con 0x7fc44c1055b0 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc451ff1640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c1055b0 0x7fc44c19cd30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc44400eaa0 tx=0x7fc44400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc4427fc640 1 -- 172.21.15.103:0/4170453537 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44400cdb0 con 0x7fc44c1055b0 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc4427fc640 1 -- 172.21.15.103:0/4170453537 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc444005020 con 0x7fc44c1055b0 2024-09-19T07:53:39.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44c19d700 con 0x7fc44c1055b0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc4427fc640 1 -- 172.21.15.103:0/4170453537 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4440116b0 con 0x7fc44c1055b0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.440+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44c1a4fe0 con 0x7fc44c1055b0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.441+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44c100860 con 0x7fc44c1055b0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.442+0000 7fc4427fc640 1 -- 172.21.15.103:0/4170453537 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc444005560 con 0x7fc44c1055b0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.443+0000 7fc4427fc640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4380a33b0 0x7fc4380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.443+0000 7fc4517f0640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4380a33b0 0x7fc4380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.443+0000 7fc4427fc640 1 -- 172.21.15.103:0/4170453537 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fc4440d0240 con 0x7fc44c1055b0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.444+0000 7fc4517f0640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4380a33b0 0x7fc4380a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005fd0 tx=0x7fc43c00b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:39.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.446+0000 7fc4427fc640 1 -- 172.21.15.103:0/4170453537 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc444016030 con 0x7fc44c1055b0 2024-09-19T07:53:39.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.647+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc44c108260 con 0x7fc44c1055b0 2024-09-19T07:53:39.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.648+0000 7fc4427fc640 1 -- 172.21.15.103:0/4170453537 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7fc44409c5c0 con 0x7fc44c1055b0 2024-09-19T07:53:39.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:53:39.649 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":54,"btime":"2024-09-19T07:53:10:110704+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:53:09.872640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:53:39.650 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 54 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4380a33b0 msgr2=0x7fc4380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4380a33b0 0x7fc4380a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005fd0 tx=0x7fc43c00b390 comp rx=0 tx=0).stop 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c1055b0 msgr2=0x7fc44c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c1055b0 0x7fc44c19cd30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc44400eaa0 tx=0x7fc44400ef60 comp rx=0 tx=0).stop 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 shutdown_connections 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc4380a33b0 0x7fc4380a5870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc44c1055b0 0x7fc44c19cd30 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c100e30 0x7fc44c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 --2- 172.21.15.103:0/4170453537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44c100480 0x7fc44c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:39.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 >> 172.21.15.103:0/4170453537 conn(0x7fc44c077210 msgr2=0x7fc44c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:39.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 shutdown_connections 2024-09-19T07:53:39.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:39.650+0000 7fc453a54640 1 -- 172.21.15.103:0/4170453537 wait complete. 2024-09-19T07:53:40.405 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:40 smithi103 ceph-mon[31201]: pgmap v491: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 303 KiB/s wr, 14 op/s 2024-09-19T07:53:40.405 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:40 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/4170453537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:40.434 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 4910a3a6 --release --await' 2024-09-19T07:53:40.435 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 4910a3a6 --release --await 2024-09-19T07:53:40.470 INFO:teuthology.orchestra.run.smithi103.stderr:progress: 905.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-19T07:53:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:40 smithi134 ceph-mon[35414]: pgmap v491: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 303 KiB/s wr, 14 op/s 2024-09-19T07:53:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:40 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/4170453537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:40 smithi174 ceph-mon[35370]: pgmap v491: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 303 KiB/s wr, 14 op/s 2024-09-19T07:53:40.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:40 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/4170453537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:40.685 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:41.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.171+0000 7f161461f640 1 -- 172.21.15.103:0/2496254307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 msgr2=0x7f160c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.171+0000 7f161461f640 1 --2- 172.21.15.103:0/2496254307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c10ea90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1604009ca0 tx=0x7f160402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.171+0000 7f161461f640 1 -- 172.21.15.103:0/2496254307 shutdown_connections 2024-09-19T07:53:41.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.171+0000 7f161461f640 1 --2- 172.21.15.103:0/2496254307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c10ea90 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.171+0000 7f161461f640 1 --2- 172.21.15.103:0/2496254307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160c102790 0x7f160c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.171+0000 7f161461f640 1 --2- 172.21.15.103:0/2496254307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f160c101de0 0x7f160c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.171+0000 7f161461f640 1 -- 172.21.15.103:0/2496254307 >> 172.21.15.103:0/2496254307 conn(0x7f160c0fd8c0 msgr2=0x7f160c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 -- 172.21.15.103:0/2496254307 shutdown_connections 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 -- 172.21.15.103:0/2496254307 wait complete. 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 Processor -- start 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 -- start start 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160c101de0 0x7f160c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f160c102790 0x7f160c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f160c111f70 con 0x7f160c106f10 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f160c111df0 con 0x7f160c101de0 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f161461f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1120f0 con 0x7f160c102790 2024-09-19T07:53:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f1612bbc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.172+0000 7f1612bbc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41262/0 (socket says 172.21.15.103:41262) 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1612bbc640 1 -- 172.21.15.103:0/3408903862 learned_addr learned my addr 172.21.15.103:0/3408903862 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f16123bb640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160c101de0 0x7f160c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1611bba640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f160c102790 0x7f160c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1612bbc640 1 -- 172.21.15.103:0/3408903862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f160c102790 msgr2=0x7f160c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1612bbc640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f160c102790 0x7f160c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1612bbc640 1 -- 172.21.15.103:0/3408903862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160c101de0 msgr2=0x7f160c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1612bbc640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160c101de0 0x7f160c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1612bbc640 1 -- 172.21.15.103:0/3408903862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c19d3a0 con 0x7f160c106f10 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f16123bb640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160c101de0 0x7f160c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f1612bbc640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c19ccc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1604031a20 tx=0x7f1604031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f16037fe640 1 -- 172.21.15.103:0/3408903862 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604003c30 con 0x7f160c106f10 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f16037fe640 1 -- 172.21.15.103:0/3408903862 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1604003dd0 con 0x7f160c106f10 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f16037fe640 1 -- 172.21.15.103:0/3408903862 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160402fcf0 con 0x7f160c106f10 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c19d630 con 0x7f160c106f10 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.173+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c1a4f70 con 0x7f160c106f10 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.175+0000 7f16037fe640 1 -- 172.21.15.103:0/3408903862 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f1604004050 con 0x7f160c106f10 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 Processor -- start 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 -- start start 2024-09-19T07:53:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f48c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f496d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f497240 0x559c2f499630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c2f3f1620 con 0x559c2f48c060 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559c2f398e50 con 0x559c2f48c980 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16169f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c2f3ed5e0 con 0x559c2f497240 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16037fe640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f15f80a33b0 0x7f15f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f48c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f48c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41264/0 (socket says 172.21.15.103:41264) 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f1611bba640 1 -- 172.21.15.103:0/370635586 learned_addr learned my addr 172.21.15.103:0/370635586 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f1612bbc640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f15f80a33b0 0x7f15f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f16123bb640 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f496d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.176+0000 7f1611bba640 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f497240 0x559c2f499630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1611bba640 1 -- 172.21.15.103:0/370635586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f497240 msgr2=0x559c2f499630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1611bba640 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f497240 0x559c2f499630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1611bba640 1 -- 172.21.15.103:0/370635586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 msgr2=0x559c2f496d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1611bba640 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f496d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f16037fe640 1 -- 172.21.15.103:0/3408903862 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f16040f1030 con 0x7f160c106f10 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1611bba640 1 -- 172.21.15.103:0/370635586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2f389d40 con 0x559c2f48c060 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1612bbc640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f15f80a33b0 0x7f15f80a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f16040b9e70 tx=0x7f16040b90f0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1611bba640 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f48c440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00cf90 tx=0x7f15fc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4e2cdc638e8009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1600ff9640 1 -- 172.21.15.103:0/370635586 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc018030 con 0x559c2f48c060 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1600ff9640 1 -- 172.21.15.103:0/370635586 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc012b40 con 0x559c2f48c060 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.177+0000 7f1600ff9640 1 -- 172.21.15.103:0/370635586 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00e940 con 0x559c2f48c060 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 -- 172.21.15.103:0/370635586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 msgr2=0x559c2f48c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f48c440 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00cf90 tx=0x7f15fc00c520 comp rx=0 tx=0).stop 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 -- 172.21.15.103:0/370635586 shutdown_connections 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f497240 0x559c2f499630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f496d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 --2- 172.21.15.103:0/370635586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f48c440 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 -- 172.21.15.103:0/370635586 >> 172.21.15.103:0/370635586 conn(0x559c2f4885f0 msgr2=0x559c2f4889d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 -- 172.21.15.103:0/370635586 shutdown_connections 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 -- 172.21.15.103:0/370635586 wait complete. 2024-09-19T07:53:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 -- start start 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.178+0000 7f16169f8740 1 Processor -- start 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c060 0x559c2f494180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f4946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f494c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c2f499e10 con 0x559c2f497240 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559c2f379f10 con 0x559c2f48c980 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c2f499f90 con 0x559c2f48c060 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 10 client.0 _ll_register_callbacks cb 0x559c2f39ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f494c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f494c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41274/0 (socket says 172.21.15.103:41274) 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 -- 172.21.15.103:0/1761842243 learned_addr learned my addr 172.21.15.103:0/1761842243 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 -- 172.21.15.103:0/1761842243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c060 msgr2=0x559c2f494180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16123bb640 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f4946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c060 0x559c2f494180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 -- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 msgr2=0x559c2f4946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f4946c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 -- 172.21.15.103:0/1761842243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2f100580 con 0x559c2f497240 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f1611bba640 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f494c00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005d80 tx=0x7f15fc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09c40a54242c6d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f15f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f15f5ffb640 1 -- 172.21.15.103:0/1761842243 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc018040 con 0x559c2f497240 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f15f5ffb640 1 -- 172.21.15.103:0/1761842243 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f15fc005250 con 0x559c2f497240 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.179+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc012b40 con 0x559c2f497240 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f15f5ffb640 1 -- 172.21.15.103:0/1761842243 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f15fc02f050 con 0x559c2f497240 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 10 client.18495 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559c2f1ca960 con 0x559c2f497240 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f15f5ffb640 1 -- 172.21.15.103:0/1761842243 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc012b40 con 0x559c2f497240 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f15f5ffb640 1 -- 172.21.15.103:0/1761842243 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f15fc004660 con 0x559c2f497240 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 10 client.18495 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 10 client.18495 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 20 client.18495 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f4a6d50 0x559c2f4a9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 4 client.18495 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c2f3feb90 con 0x559c2f4a6d50 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.180+0000 7f16123bb640 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f4a6d50 0x559c2f4a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.181+0000 7f16123bb640 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f4a6d50 0x559c2f4a9140 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da801d1f37f2fa39 server_cookie=1d6c9e77967d9686 in_seq=0 out_seq=0 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.181+0000 7f15f5ffb640 10 client.18495 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.181+0000 7f15f5ffb640 1 -- 172.21.15.103:0/1761842243 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c2f3feb90 con 0x559c2f4a6d50 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.181+0000 7f15f5ffb640 10 client.18495 handle_command_reply: tid=0 2024-09-19T07:53:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.181+0000 7f16169f8740 1 client.18495 shutdown 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 msgr2=0x559c2f494c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f494c00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f15fc005d80 tx=0x7f15fc00cc50 comp rx=0 tx=0).stop 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 shutdown_connections 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f4a6d50 0x559c2f4a9140 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f494c00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f15f5ffb640 0 client.18495 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c980 0x559c2f4946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 --2- 172.21.15.103:0/1761842243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c060 0x559c2f494180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 >> 172.21.15.103:0/1761842243 conn(0x559c2f4885f0 msgr2=0x559c2f48e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 shutdown_connections 2024-09-19T07:53:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 1 -- 172.21.15.103:0/1761842243 wait complete. 2024-09-19T07:53:41.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.182+0000 7f16169f8740 20 client.18495 trim_cache size 0 max 16384 2024-09-19T07:53:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 Processor -- start 2024-09-19T07:53:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 -- start start 2024-09-19T07:53:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c060 0x559c2f48eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f494850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f4ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1ad920 con 0x559c2f497240 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1af460 con 0x559c2f48c060 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f16169f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1adff0 con 0x559c2f48c980 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f4ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.199+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f4ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41276/0 (socket says 172.21.15.103:41276) 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 -- 172.21.15.103:0/311821277 learned_addr learned my addr 172.21.15.103:0/311821277 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f16123bb640 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f494850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c060 0x559c2f48eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 -- 172.21.15.103:0/311821277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 msgr2=0x559c2f494850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f494850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 -- 172.21.15.103:0/311821277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c060 msgr2=0x559c2f48eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c060 0x559c2f48eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 -- 172.21.15.103:0/311821277 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2f4896a0 con 0x559c2f497240 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f1611bba640 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f4ae7a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c440 tx=0x7f15fc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fa79a2d0a7f887 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f15f7fff640 1 -- 172.21.15.103:0/311821277 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc018070 con 0x559c2f497240 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f15f7fff640 1 -- 172.21.15.103:0/311821277 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc0054c0 con 0x559c2f497240 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.200+0000 7f15f7fff640 1 -- 172.21.15.103:0/311821277 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00e920 con 0x559c2f497240 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 -- 172.21.15.103:0/311821277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 msgr2=0x559c2f4ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f4ae7a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c440 tx=0x7f15fc005a50 comp rx=0 tx=0).stop 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 -- 172.21.15.103:0/311821277 shutdown_connections 2024-09-19T07:53:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f497240 0x559c2f4ae7a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f494850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 --2- 172.21.15.103:0/311821277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f48c060 0x559c2f48eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 -- 172.21.15.103:0/311821277 >> 172.21.15.103:0/311821277 conn(0x559c2f4885f0 msgr2=0x559c2f494140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 -- 172.21.15.103:0/311821277 shutdown_connections 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 -- 172.21.15.103:0/311821277 wait complete. 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 -- start start 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 Processor -- start 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.201+0000 7f16169f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f491200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f491740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f497240 0x559c2f491c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1aed90 con 0x559c2f48c060 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1ae6c0 con 0x559c2f497240 2024-09-19T07:53:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1ab920 con 0x559c2f48c980 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 10 client.0 _ll_register_callbacks cb 0x7f15fc010f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f491200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f491200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41284/0 (socket says 172.21.15.103:41284) 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 -- 172.21.15.103:0/3547605443 learned_addr learned my addr 172.21.15.103:0/3547605443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f497240 0x559c2f491c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16123bb640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f491740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 -- 172.21.15.103:0/3547605443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 msgr2=0x559c2f491740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f491740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 -- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f497240 msgr2=0x559c2f491c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f497240 0x559c2f491c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 -- 172.21.15.103:0/3547605443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2f5107a0 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16123bb640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f491740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f1611bba640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f491200 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f15fc012960 tx=0x7f15fc0049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303cb37b8164c8fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f15f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f15f5ffb640 1 -- 172.21.15.103:0/3547605443 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc043060 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f15f5ffb640 1 -- 172.21.15.103:0/3547605443 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f15fc004660 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc0054c0 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f15f5ffb640 1 -- 172.21.15.103:0/3547605443 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f15fc018040 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 10 client.18501 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.202+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559c2f511aa0 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f15f5ffb640 1 -- 172.21.15.103:0/3547605443 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc0054c0 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f15f5ffb640 1 -- 172.21.15.103:0/3547605443 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f15fc028410 con 0x559c2f48c060 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16169f8740 10 client.18501 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16169f8740 10 client.18501 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:53:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16169f8740 20 client.18501 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16169f8740 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f5120a0 0x559c2f514490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16169f8740 4 client.18501 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4910a3a6", "release": true, "await": true}] 2024-09-19T07:53:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4910a3a6", "release": true, "await": true}) -- 0x559c2f3f90a0 con 0x559c2f5120a0 2024-09-19T07:53:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16123bb640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f5120a0 0x559c2f514490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f15f5ffb640 10 client.18501 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.203+0000 7f16123bb640 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f5120a0 0x559c2f514490 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51ec08a176d0cdd server_cookie=79ea7a01ba380fe1 in_seq=0 out_seq=0 2024-09-19T07:53:41.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.232+0000 7f15f5ffb640 1 -- 172.21.15.103:0/3547605443 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559c2f3f90a0 con 0x559c2f5120a0 2024-09-19T07:53:41.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.232+0000 7f15f5ffb640 10 client.18501 handle_command_reply: tid=0 2024-09-19T07:53:41.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.235+0000 7f16169f8740 1 client.18501 shutdown 2024-09-19T07:53:41.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.243+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 msgr2=0x559c2f491200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.243+0000 7f16169f8740 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f491200 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f15fc012960 tx=0x7f15fc0049c0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 shutdown_connections 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x559c2f5120a0 0x559c2f514490 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c2f497240 0x559c2f491c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c2f48c980 0x559c2f491740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f15f5ffb640 0 client.18501 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 --2- 172.21.15.103:0/3547605443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c2f48c060 0x559c2f491200 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 >> 172.21.15.103:0/3547605443 conn(0x559c2f4885f0 msgr2=0x559c2f494960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 shutdown_connections 2024-09-19T07:53:41.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 1 -- 172.21.15.103:0/3547605443 wait complete. 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.245+0000 7f16169f8740 20 client.18501 trim_cache size 0 max 16384 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 54, 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 52, 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout: "4910a3a6": { 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 52, 2024-09-19T07:53:41.247 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:53:41.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:53:41.249 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:53:41.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:41.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:41.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:41.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:41.249 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:41.249 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:53:41.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.249+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f15f80a33b0 msgr2=0x7f15f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.249+0000 7f161461f640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f15f80a33b0 0x7f15f80a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f16040b9e70 tx=0x7f16040b90f0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.249+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 msgr2=0x7f160c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:41.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.249+0000 7f161461f640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c19ccc0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1604031a20 tx=0x7f1604031a50 comp rx=0 tx=0).stop 2024-09-19T07:53:41.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.250+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 shutdown_connections 2024-09-19T07:53:41.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.250+0000 7f161461f640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f15f80a33b0 0x7f15f80a5870 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.250+0000 7f161461f640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f160c106f10 0x7f160c19ccc0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.250+0000 7f161461f640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f160c102790 0x7f160c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.250+0000 7f161461f640 1 --2- 172.21.15.103:0/3408903862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f160c101de0 0x7f160c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:41.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.250+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 >> 172.21.15.103:0/3408903862 conn(0x7f160c0fd8c0 msgr2=0x7f160c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.254+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 shutdown_connections 2024-09-19T07:53:41.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:41.254+0000 7f161461f640 1 -- 172.21.15.103:0/3408903862 wait complete. 2024-09-19T07:53:41.454 INFO:teuthology.orchestra.run.smithi103.stdout:transaction type: 2024-09-19T07:53:41.454 INFO:teuthology.orchestra.run.smithi103.stdout:scaling factor: 32 2024-09-19T07:53:41.454 INFO:teuthology.orchestra.run.smithi103.stdout:query mode: simple 2024-09-19T07:53:41.454 INFO:teuthology.orchestra.run.smithi103.stdout:number of clients: 100 2024-09-19T07:53:41.454 INFO:teuthology.orchestra.run.smithi103.stdout:number of threads: 4 2024-09-19T07:53:41.454 INFO:teuthology.orchestra.run.smithi103.stdout:duration: 900 s 2024-09-19T07:53:41.454 INFO:teuthology.orchestra.run.smithi103.stdout:number of transactions actually processed: 1568180 2024-09-19T07:53:41.455 INFO:teuthology.orchestra.run.smithi103.stdout:latency average = 57.758 ms 2024-09-19T07:53:41.455 INFO:teuthology.orchestra.run.smithi103.stdout:latency stddev = 699.051 ms 2024-09-19T07:53:41.455 INFO:teuthology.orchestra.run.smithi103.stdout:tps = 1730.906728 (including connections establishing) 2024-09-19T07:53:41.455 INFO:teuthology.orchestra.run.smithi103.stdout:tps = 1730.911849 (excluding connections establishing) 2024-09-19T07:53:41.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-19T07:53:41.959 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4910a3a6', total seconds elapsed: 32.7 2024-09-19T07:53:41.959 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-09-19T07:53:41.959 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:53:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:42 smithi134 ceph-mon[35414]: pgmap v492: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 309 KiB/s wr, 19 op/s 2024-09-19T07:53:42.616 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:42.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:42 smithi174 ceph-mon[35370]: pgmap v492: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 309 KiB/s wr, 19 op/s 2024-09-19T07:53:42.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:42 smithi103 ceph-mon[31201]: pgmap v492: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 309 KiB/s wr, 19 op/s 2024-09-19T07:53:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.265+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/4272430310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0074990 msgr2=0x7fc8a007b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.265+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/4272430310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0074990 0x7fc8a007b3c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc89800afd0 tx=0x7fc89802e720 comp rx=0 tx=0).stop 2024-09-19T07:53:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.265+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/4272430310 shutdown_connections 2024-09-19T07:53:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.265+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/4272430310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0074990 0x7fc8a007b3c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.265+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/4272430310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a0109e00 0x7fc8a00742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.265+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/4272430310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a0109450 0x7fc8a0109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.265+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/4272430310 >> 172.21.15.103:0/4272430310 conn(0x7fc8a006e140 msgr2=0x7fc8a006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:43.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.271+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/4272430310 shutdown_connections 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.271+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/4272430310 wait complete. 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 Processor -- start 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 -- start start 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0109450 0x7fc8a007af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a0109e00 0x7fc8a0075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a0076500 0x7fc8a00769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a007e820 con 0x7fc8a0109450 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a007e6a0 con 0x7fc8a0076500 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.272+0000 7fc8a4c3e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a007e9a0 con 0x7fc8a0109e00 2024-09-19T07:53:43.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.273+0000 7fc89ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a0076500 0x7fc8a00769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.273+0000 7fc89ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a0076500 0x7fc8a00769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40684/0 (socket says 172.21.15.103:40684) 2024-09-19T07:53:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.273+0000 7fc89ed9d640 1 -- 172.21.15.103:0/3552235248 learned_addr learned my addr 172.21.15.103:0/3552235248 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:43.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.274+0000 7fc89dd9b640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a0109e00 0x7fc8a0075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.274+0000 7fc89e59c640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0109450 0x7fc8a007af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.274+0000 7fc89dd9b640 1 -- 172.21.15.103:0/3552235248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a0076500 msgr2=0x7fc8a00769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.274+0000 7fc89dd9b640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a0076500 0x7fc8a00769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.274+0000 7fc89dd9b640 1 -- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0109450 msgr2=0x7fc8a007af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.274+0000 7fc89dd9b640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0109450 0x7fc8a007af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.274+0000 7fc89dd9b640 1 -- 172.21.15.103:0/3552235248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a01bbfa0 con 0x7fc8a0109e00 2024-09-19T07:53:43.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.275+0000 7fc89dd9b640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a0109e00 0x7fc8a0075fc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc8a006f670 tx=0x7fc89400acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:43.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.277+0000 7fc88f7fe640 1 -- 172.21.15.103:0/3552235248 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8940079b0 con 0x7fc8a0109e00 2024-09-19T07:53:43.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.277+0000 7fc88f7fe640 1 -- 172.21.15.103:0/3552235248 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc894005500 con 0x7fc8a0109e00 2024-09-19T07:53:43.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.277+0000 7fc88f7fe640 1 -- 172.21.15.103:0/3552235248 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894004ce0 con 0x7fc8a0109e00 2024-09-19T07:53:43.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.279+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a01bc230 con 0x7fc8a0109e00 2024-09-19T07:53:43.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.279+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a01bc6f0 con 0x7fc8a0109e00 2024-09-19T07:53:43.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.281+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a0073b00 con 0x7fc8a0109e00 2024-09-19T07:53:43.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.285+0000 7fc88f7fe640 1 -- 172.21.15.103:0/3552235248 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc894004e80 con 0x7fc8a0109e00 2024-09-19T07:53:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.286+0000 7fc88f7fe640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc8880a35c0 0x7fc8880a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:43.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.286+0000 7fc88f7fe640 1 -- 172.21.15.103:0/3552235248 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fc8940cf7c0 con 0x7fc8a0109e00 2024-09-19T07:53:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.292+0000 7fc89e59c640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc8880a35c0 0x7fc8880a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:43.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.293+0000 7fc88f7fe640 1 -- 172.21.15.103:0/3552235248 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc89409bb60 con 0x7fc8a0109e00 2024-09-19T07:53:43.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.314+0000 7fc89e59c640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc8880a35c0 0x7fc8880a5a80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc8900090e0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:43.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.712+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8a0074990 con 0x7fc8a0109e00 2024-09-19T07:53:43.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.718+0000 7fc88f7fe640 1 -- 172.21.15.103:0/3552235248 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7fc89409fe60 con 0x7fc8a0109e00 2024-09-19T07:53:43.719 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:53:43.719 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":54,"btime":"2024-09-19T07:53:10:110704+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:53:09.872640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:53:43.719 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 54 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc8880a35c0 msgr2=0x7fc8880a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc8880a35c0 0x7fc8880a5a80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc890006fd0 tx=0x7fc8900090e0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a0109e00 msgr2=0x7fc8a0075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a0109e00 0x7fc8a0075fc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc8a006f670 tx=0x7fc89400acd0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 shutdown_connections 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc8880a35c0 0x7fc8880a5a80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a0076500 0x7fc8a00769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a0109e00 0x7fc8a0075fc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 --2- 172.21.15.103:0/3552235248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a0109450 0x7fc8a007af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:43.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.721+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 >> 172.21.15.103:0/3552235248 conn(0x7fc8a006e140 msgr2=0x7fc8a00720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:43.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.724+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 shutdown_connections 2024-09-19T07:53:43.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:43.725+0000 7fc8a4c3e640 1 -- 172.21.15.103:0/3552235248 wait complete. 2024-09-19T07:53:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:44 smithi134 ceph-mon[35414]: pgmap v493: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 13 op/s 2024-09-19T07:53:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:44 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3552235248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:44 smithi174 ceph-mon[35370]: pgmap v493: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 13 op/s 2024-09-19T07:53:44.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:44 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3552235248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:44.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:44 smithi103 ceph-mon[31201]: pgmap v493: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 13 op/s 2024-09-19T07:53:44.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:44 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3552235248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:46.371 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-09-19T07:53:46.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-09-19T07:53:46.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:46 smithi134 ceph-mon[35414]: pgmap v494: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 12 op/s 2024-09-19T07:53:46.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:46 smithi174 ceph-mon[35370]: pgmap v494: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 12 op/s 2024-09-19T07:53:46.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:46 smithi103 ceph-mon[31201]: pgmap v494: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 71 KiB/s wr, 12 op/s 2024-09-19T07:53:46.900 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:47.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.532+0000 7f94f2986640 1 -- 172.21.15.103:0/3029046857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec109120 msgr2=0x7f94ec110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.532+0000 7f94f2986640 1 --2- 172.21.15.103:0/3029046857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec109120 0x7f94ec110ca0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009ca0 tx=0x7f94e402a8c0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.533+0000 7f94f2986640 1 -- 172.21.15.103:0/3029046857 shutdown_connections 2024-09-19T07:53:47.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.533+0000 7f94f2986640 1 --2- 172.21.15.103:0/3029046857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec109120 0x7f94ec110ca0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.533+0000 7f94f2986640 1 --2- 172.21.15.103:0/3029046857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec1049a0 0x7f94ec1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.533+0000 7f94f2986640 1 --2- 172.21.15.103:0/3029046857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94ec103ff0 0x7f94ec1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.533+0000 7f94f2986640 1 -- 172.21.15.103:0/3029046857 >> 172.21.15.103:0/3029046857 conn(0x7f94ec0ffa70 msgr2=0x7f94ec101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.533+0000 7f94f2986640 1 -- 172.21.15.103:0/3029046857 shutdown_connections 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.533+0000 7f94f2986640 1 -- 172.21.15.103:0/3029046857 wait complete. 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.534+0000 7f94f2986640 1 Processor -- start 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.534+0000 7f94f2986640 1 -- start start 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.534+0000 7f94f2986640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94ec103ff0 0x7f94ec10f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.534+0000 7f94f2986640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec1049a0 0x7f94ec073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.534+0000 7f94f2986640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec109120 0x7f94ec10fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f2986640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec077810 con 0x7f94ec103ff0 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f2986640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec077690 con 0x7f94ec1049a0 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f2986640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec077990 con 0x7f94ec109120 2024-09-19T07:53:47.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec109120 0x7f94ec10fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec109120 0x7f94ec10fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:59696/0 (socket says 172.21.15.103:59696) 2024-09-19T07:53:47.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 -- 172.21.15.103:0/33670821 learned_addr learned my addr 172.21.15.103:0/33670821 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:47.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec1049a0 0x7f94ec073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 -- 172.21.15.103:0/33670821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec1049a0 msgr2=0x7f94ec073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec1049a0 0x7f94ec073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 -- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94ec103ff0 msgr2=0x7f94ec10f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94ec103ff0 0x7f94ec10f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.535+0000 7f94f0f23640 1 -- 172.21.15.103:0/33670821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec112fa0 con 0x7f94ec109120 2024-09-19T07:53:47.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.536+0000 7f94f0f23640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec109120 0x7f94ec10fc10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f94e4005e30 tx=0x7f94e4031680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.538+0000 7f94e97fa640 1 -- 172.21.15.103:0/33670821 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4031bb0 con 0x7f94ec109120 2024-09-19T07:53:47.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.538+0000 7f94e97fa640 1 -- 172.21.15.103:0/33670821 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e4004520 con 0x7f94ec109120 2024-09-19T07:53:47.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.538+0000 7f94e97fa640 1 -- 172.21.15.103:0/33670821 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4030cb0 con 0x7f94ec109120 2024-09-19T07:53:47.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.539+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec113230 con 0x7f94ec109120 2024-09-19T07:53:47.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.539+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec113720 con 0x7f94ec109120 2024-09-19T07:53:47.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.543+0000 7f94f4d5f740 1 Processor -- start 2024-09-19T07:53:47.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.545+0000 7f94e97fa640 1 -- 172.21.15.103:0/33670821 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f94e4004040 con 0x7f94ec109120 2024-09-19T07:53:47.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.545+0000 7f94e97fa640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a35c0 0x7f94d00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.546+0000 7f94f4d5f740 1 -- start start 2024-09-19T07:53:47.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.546+0000 7f94e97fa640 1 -- 172.21.15.103:0/33670821 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f94e40f1210 con 0x7f94ec109120 2024-09-19T07:53:47.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.546+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee98850 0x56296ee98c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.546+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee99170 0x56296eea34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.546+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee98850 0x56296ee98c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.547+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee98850 0x56296ee98c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40718/0 (socket says 172.21.15.103:40718) 2024-09-19T07:53:47.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.547+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.547+0000 7f94f4d5f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56296edfd160 con 0x56296ee99170 2024-09-19T07:53:47.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.547+0000 7f94f4d5f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56296eda4720 con 0x56296ee98850 2024-09-19T07:53:47.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.547+0000 7f94f4d5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56296edf9120 con 0x56296eea3a30 2024-09-19T07:53:47.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.548+0000 7f94ebfff640 1 -- 172.21.15.103:0/2853466849 learned_addr learned my addr 172.21.15.103:0/2853466849 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:47.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.549+0000 7f94f0f23640 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.550+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a35c0 0x7f94d00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.550+0000 7f94ebfff640 1 -- 172.21.15.103:0/2853466849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 msgr2=0x56296eea5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.550+0000 7f94ebfff640 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.550+0000 7f94ebfff640 1 -- 172.21.15.103:0/2853466849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee99170 msgr2=0x56296eea34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.550+0000 7f94ebfff640 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee99170 0x56296eea34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.550+0000 7f94ebfff640 1 -- 172.21.15.103:0/2853466849 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296eb6eba0 con 0x56296ee98850 2024-09-19T07:53:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.554+0000 7f94ebfff640 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee98850 0x56296ee98c30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00bb40 tx=0x7f94dc00d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ddbc06414fa716d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.555+0000 7f94e27fc640 1 -- 172.21.15.103:0/2853466849 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc014070 con 0x56296ee98850 2024-09-19T07:53:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.555+0000 7f94e27fc640 1 -- 172.21.15.103:0/2853466849 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc00dd10 con 0x56296ee98850 2024-09-19T07:53:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2853466849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee98850 msgr2=0x56296ee98c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94e27fc640 1 -- 172.21.15.103:0/2853466849 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc014070 con 0x56296ee98850 2024-09-19T07:53:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee98850 0x56296ee98c30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00bb40 tx=0x7f94dc00d2e0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2853466849 shutdown_connections 2024-09-19T07:53:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee99170 0x56296eea34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2853466849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee98850 0x56296ee98c30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.556+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2853466849 >> 172.21.15.103:0/2853466849 conn(0x56296ee94de0 msgr2=0x56296ee951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.557+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2853466849 shutdown_connections 2024-09-19T07:53:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.557+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a35c0 0x7f94d00a5a80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x56296ee95c40 tx=0x7f94d800a780 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.557+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2853466849 wait complete. 2024-09-19T07:53:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.557+0000 7f94f4d5f740 1 -- start start 2024-09-19T07:53:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.558+0000 7f94f4d5f740 1 Processor -- start 2024-09-19T07:53:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.561+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eea0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.561+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296eea0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.561+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.561+0000 7f94f4d5f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56296eea6650 con 0x56296ee98850 2024-09-19T07:53:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.561+0000 7f94f4d5f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56296ed85900 con 0x56296ee99170 2024-09-19T07:53:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.561+0000 7f94f4d5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56296eea67d0 con 0x56296eea3a30 2024-09-19T07:53:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.561+0000 7f94f4d5f740 10 client.0 _ll_register_callbacks cb 0x56296ed83780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.563+0000 7f94eb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296eea0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.563+0000 7f94eb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296eea0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.103:40730/0 (socket says 172.21.15.103:40730) 2024-09-19T07:53:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.563+0000 7f94eb7fe640 1 -- 172.21.15.103:0/1568084191 learned_addr learned my addr 172.21.15.103:0/1568084191 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.563+0000 7f94f0f23640 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.564+0000 7f94eb7fe640 1 -- 172.21.15.103:0/1568084191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 msgr2=0x56296eea1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.564+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.564+0000 7f94eb7fe640 1 -- 172.21.15.103:0/1568084191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 msgr2=0x56296eea0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.564+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eea0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.564+0000 7f94eb7fe640 1 -- 172.21.15.103:0/1568084191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296edc5c50 con 0x56296ee99170 2024-09-19T07:53:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.565+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296eea0d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94d8010af0 tx=0x7f94d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaccc69a28d4f2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.565+0000 7f94cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:53:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.565+0000 7f94cf7fe640 1 -- 172.21.15.103:0/1568084191 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8016070 con 0x56296ee99170 2024-09-19T07:53:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.566+0000 7f94cf7fe640 1 -- 172.21.15.103:0/1568084191 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f94d801be00 con 0x56296ee99170 2024-09-19T07:53:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.566+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc00dd10 con 0x56296ee99170 2024-09-19T07:53:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.566+0000 7f94cf7fe640 1 -- 172.21.15.103:0/1568084191 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f94d8027480 con 0x56296ee99170 2024-09-19T07:53:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.567+0000 7f94f4d5f740 10 client.27827 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.567+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56296eeb22b0 con 0x56296ee99170 2024-09-19T07:53:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.568+0000 7f94cf7fe640 1 -- 172.21.15.103:0/1568084191 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc00dd10 con 0x56296ee99170 2024-09-19T07:53:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.568+0000 7f94cf7fe640 1 -- 172.21.15.103:0/1568084191 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f94d801c670 con 0x56296ee99170 2024-09-19T07:53:47.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.570+0000 7f94f4d5f740 10 client.27827 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.570+0000 7f94f4d5f740 10 client.27827 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:53:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.570+0000 7f94f4d5f740 20 client.27827 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.570+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296eeb37c0 0x56296eeb5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.570+0000 7f94f4d5f740 4 client.27827 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.570+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56296ee0a890 con 0x56296eeb37c0 2024-09-19T07:53:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.571+0000 7f94ebfff640 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296eeb37c0 0x56296eeb5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.579+0000 7f94ebfff640 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296eeb37c0 0x56296eeb5c80 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4bbf413ec3e96f4 server_cookie=141b9926ef59908d in_seq=0 out_seq=0 2024-09-19T07:53:47.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.579+0000 7f94cf7fe640 10 client.27827 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.580+0000 7f94cf7fe640 1 -- 172.21.15.103:0/1568084191 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56296ee0a890 con 0x56296eeb37c0 2024-09-19T07:53:47.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.580+0000 7f94cf7fe640 10 client.27827 handle_command_reply: tid=0 2024-09-19T07:53:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.580+0000 7f94f4d5f740 1 client.27827 shutdown 2024-09-19T07:53:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 msgr2=0x56296eea0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296eea0d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94d8010af0 tx=0x7f94d8004730 comp rx=0 tx=0).stop 2024-09-19T07:53:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 shutdown_connections 2024-09-19T07:53:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296eeb37c0 0x56296eeb5c80 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296eea1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296eea0d50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/1568084191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eea0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.582+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 >> 172.21.15.103:0/1568084191 conn(0x56296ee94de0 msgr2=0x56296ee9aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.583+0000 7f94cf7fe640 0 client.27827 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.583+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 shutdown_connections 2024-09-19T07:53:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.583+0000 7f94f4d5f740 1 -- 172.21.15.103:0/1568084191 wait complete. 2024-09-19T07:53:47.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.583+0000 7f94f4d5f740 20 client.27827 trim_cache size 0 max 16384 2024-09-19T07:53:47.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.614+0000 7f94f4d5f740 1 Processor -- start 2024-09-19T07:53:47.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.614+0000 7f94f4d5f740 1 -- start start 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.615+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296ee9bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.615+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296ee99170 0x56296ee9f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.615+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296eea3a30 0x56296eebb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.615+0000 7f94eb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296ee99170 0x56296ee9f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.615+0000 7f94eb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296ee99170 0x56296ee9f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:59718/0 (socket says 172.21.15.103:59718) 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.616+0000 7f94eb7fe640 1 -- 172.21.15.103:0/76049251 learned_addr learned my addr 172.21.15.103:0/76049251 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.616+0000 7f94f0f23640 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296eea3a30 0x56296eebb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.616+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec1c4f50 con 0x56296ee98850 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.616+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec1c6a90 con 0x56296eea3a30 2024-09-19T07:53:47.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.616+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec1c5620 con 0x56296ee99170 2024-09-19T07:53:47.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.617+0000 7f94ebfff640 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296ee9bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.617+0000 7f94ebfff640 1 -- 172.21.15.103:0/76049251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296ee99170 msgr2=0x56296ee9f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.617+0000 7f94ebfff640 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296ee99170 0x56296ee9f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.617+0000 7f94ebfff640 1 -- 172.21.15.103:0/76049251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296eea3a30 msgr2=0x56296eebb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.618+0000 7f94ebfff640 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296eea3a30 0x56296eebb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.618+0000 7f94ebfff640 1 -- 172.21.15.103:0/76049251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296ee94c10 con 0x56296ee98850 2024-09-19T07:53:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.618+0000 7f94f0f23640 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296eea3a30 0x56296eebb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:53:47.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.619+0000 7f94ebfff640 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296ee9bd40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006010 tx=0x7f94dc003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c004ca26f4d77814 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.620+0000 7f94e1ffb640 1 -- 172.21.15.103:0/76049251 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc014040 con 0x56296ee98850 2024-09-19T07:53:47.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.620+0000 7f94e1ffb640 1 -- 172.21.15.103:0/76049251 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc003c60 con 0x56296ee98850 2024-09-19T07:53:47.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.620+0000 7f94e1ffb640 1 -- 172.21.15.103:0/76049251 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0106c0 con 0x56296ee98850 2024-09-19T07:53:47.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 msgr2=0x56296ee9bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296ee9bd40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006010 tx=0x7f94dc003a90 comp rx=0 tx=0).stop 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 shutdown_connections 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296eea3a30 0x56296eebb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296ee99170 0x56296ee9f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/76049251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296ee9bd40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 >> 172.21.15.103:0/76049251 conn(0x56296ee94de0 msgr2=0x56296ee9efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 shutdown_connections 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.621+0000 7f94f4d5f740 1 -- 172.21.15.103:0/76049251 wait complete. 2024-09-19T07:53:47.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.622+0000 7f94f4d5f740 1 -- start start 2024-09-19T07:53:47.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.622+0000 7f94f4d5f740 1 Processor -- start 2024-09-19T07:53:47.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.623+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eeac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.623+0000 7f94f4d5f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296ee9d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.623+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eeac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.623+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eeac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41344/0 (socket says 172.21.15.103:41344) 2024-09-19T07:53:47.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.624+0000 7f94ebfff640 1 -- 172.21.15.103:0/2729267085 learned_addr learned my addr 172.21.15.103:0/2729267085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:47.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.624+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296ee9d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.624+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296ee9da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.624+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec1c63c0 con 0x56296ee98850 2024-09-19T07:53:47.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.624+0000 7f94f0f23640 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296ee9da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.624+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec1c5cf0 con 0x56296ee99170 2024-09-19T07:53:47.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.625+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec104410 con 0x56296eea3a30 2024-09-19T07:53:47.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.625+0000 7f94eb7fe640 1 -- 172.21.15.103:0/2729267085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 msgr2=0x56296ee9da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.625+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296ee9da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.625+0000 7f94eb7fe640 1 -- 172.21.15.103:0/2729267085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 msgr2=0x56296eeac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.625+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eeac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.625+0000 7f94eb7fe640 1 -- 172.21.15.103:0/2729267085 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296ee9e190 con 0x56296ee99170 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.626+0000 7f94eb7fe640 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296ee9d4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f94d801cae0 tx=0x7f94d801b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=909848a4b9338ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.626+0000 7f94cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.626+0000 7f94f4d5f740 10 client.0 _ll_register_callbacks cb 0x7f94d8005d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.626+0000 7f94cf7fe640 1 -- 172.21.15.103:0/2729267085 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d800a2d0 con 0x56296ee99170 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.626+0000 7f94cf7fe640 1 -- 172.21.15.103:0/2729267085 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f94d801c630 con 0x56296ee99170 2024-09-19T07:53:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.627+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc003c60 con 0x56296ee99170 2024-09-19T07:53:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.627+0000 7f94cf7fe640 1 -- 172.21.15.103:0/2729267085 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f94d8047050 con 0x56296ee99170 2024-09-19T07:53:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.627+0000 7f94f4d5f740 10 client.27833 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.627+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=54}) -- 0x56296ef1e520 con 0x56296ee99170 2024-09-19T07:53:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94cf7fe640 1 -- 172.21.15.103:0/2729267085 <== mon.2 v2:172.21.15.134:3300/0 4 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f94d8038070 con 0x56296ee99170 2024-09-19T07:53:47.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94f4d5f740 10 client.27833 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:47.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94f4d5f740 10 client.27833 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:53:47.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94f4d5f740 20 client.27833 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:47.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296ef1eae0 0x56296ef20f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:47.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94f4d5f740 4 client.27833 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-09-19T07:53:47.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x7f94dc003c60 con 0x56296ef1eae0 2024-09-19T07:53:47.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.628+0000 7f94ebfff640 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296ef1eae0 0x56296ef20f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:47.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.629+0000 7f94ebfff640 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296ef1eae0 0x56296ef20f00 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2143a9f33dc08ff server_cookie=f0b969edf6d131db in_seq=0 out_seq=0 2024-09-19T07:53:47.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.630+0000 7f94cf7fe640 10 client.27833 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:47.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94cf7fe640 1 -- 172.21.15.103:0/2729267085 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f94dc003c60 con 0x56296ef1eae0 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94cf7fe640 10 client.27833 handle_command_reply: tid=0 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 client.27833 shutdown 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 msgr2=0x56296ee9d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296ee9d4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f94d801cae0 tx=0x7f94d801b8b0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 shutdown_connections 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56296ef1eae0 0x56296ef20f00 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56296eea3a30 0x56296ee9da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296ee99170 0x56296ee9d4e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 --2- 172.21.15.103:0/2729267085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56296ee98850 0x56296eeac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 >> 172.21.15.103:0/2729267085 conn(0x56296ee94de0 msgr2=0x56296ee93ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94cf7fe640 0 client.27833 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:47.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 shutdown_connections 2024-09-19T07:53:47.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.893+0000 7f94f4d5f740 1 -- 172.21.15.103:0/2729267085 wait complete. 2024-09-19T07:53:47.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.894+0000 7f94f4d5f740 20 client.27833 trim_cache size 0 max 16384 2024-09-19T07:53:47.896 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:53:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 54, 2024-09-19T07:53:47.896 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 54, 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "2cb8dfe1": { 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 54, 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:53:47.897 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:53:47.898 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:53:47.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.894+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a35c0 msgr2=0x7f94d00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a35c0 0x7f94d00a5a80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x56296ee95c40 tx=0x7f94d800a780 comp rx=0 tx=0).stop 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec109120 msgr2=0x7f94ec10fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec109120 0x7f94ec10fc10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f94e4005e30 tx=0x7f94e4031680 comp rx=0 tx=0).stop 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 shutdown_connections 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f94d00a35c0 0x7f94d00a5a80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94ec109120 0x7f94ec10fc10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94ec1049a0 0x7f94ec073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 --2- 172.21.15.103:0/33670821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94ec103ff0 0x7f94ec10f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 >> 172.21.15.103:0/33670821 conn(0x7f94ec0ffa70 msgr2=0x7f94ec101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:47.901 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 shutdown_connections 2024-09-19T07:53:47.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:47.895+0000 7f94f2986640 1 -- 172.21.15.103:0/33670821 wait complete. 2024-09-19T07:53:48.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:48 smithi134 ceph-mon[35414]: pgmap v495: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.8 MiB/s wr, 522 op/s 2024-09-19T07:53:48.625 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:48 smithi103 ceph-mon[31201]: pgmap v495: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.8 MiB/s wr, 522 op/s 2024-09-19T07:53:48.654 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2cb8dfe1', quiesce took 6.7 seconds. Will release after: 22.75 2024-09-19T07:53:48.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:48 smithi174 ceph-mon[35370]: pgmap v495: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.8 MiB/s wr, 522 op/s 2024-09-19T07:53:49.186 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:53:49.448 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:50.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 -- 172.21.15.103:0/910659873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100d00 msgr2=0x7f61f8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 --2- 172.21.15.103:0/910659873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100d00 0x7f61f8106840 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009930 tx=0x7f61e802a420 comp rx=0 tx=0).stop 2024-09-19T07:53:50.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 -- 172.21.15.103:0/910659873 shutdown_connections 2024-09-19T07:53:50.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 --2- 172.21.15.103:0/910659873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f8106f50 0x7f61f810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 --2- 172.21.15.103:0/910659873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100d00 0x7f61f8106840 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 --2- 172.21.15.103:0/910659873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61f8100350 0x7f61f8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 -- 172.21.15.103:0/910659873 >> 172.21.15.103:0/910659873 conn(0x7f61f8077210 msgr2=0x7f61f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 -- 172.21.15.103:0/910659873 shutdown_connections 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.128+0000 7f61ff668640 1 -- 172.21.15.103:0/910659873 wait complete. 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 Processor -- start 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 -- start start 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100350 0x7f61f81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f8100d00 0x7f61f8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61f8106f50 0x7f61f819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8111fa0 con 0x7f61f8100350 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8111e20 con 0x7f61f8100d00 2024-09-19T07:53:50.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61ff668640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8112120 con 0x7f61f8106f50 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61fd404640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100350 0x7f61f81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61fd404640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100350 0x7f61f81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45250/0 (socket says 172.21.15.103:45250) 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61fd404640 1 -- 172.21.15.103:0/2560229620 learned_addr learned my addr 172.21.15.103:0/2560229620 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.129+0000 7f61fcc03640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f8100d00 0x7f61f8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fdc05640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61f8106f50 0x7f61f819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fd404640 1 -- 172.21.15.103:0/2560229620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61f8106f50 msgr2=0x7f61f819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fd404640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61f8106f50 0x7f61f819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fd404640 1 -- 172.21.15.103:0/2560229620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f8100d00 msgr2=0x7f61f8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fd404640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f8100d00 0x7f61f8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fd404640 1 -- 172.21.15.103:0/2560229620 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f819d490 con 0x7f61f8100350 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fdc05640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61f8106f50 0x7f61f819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:50.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61fd404640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100350 0x7f61f81984e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f61f400e9f0 tx=0x7f61f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61ee7fc640 1 -- 172.21.15.103:0/2560229620 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f400cd80 con 0x7f61f8100350 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f819d780 con 0x7f61f8100350 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61ee7fc640 1 -- 172.21.15.103:0/2560229620 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f4005500 con 0x7f61f8100350 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61ee7fc640 1 -- 172.21.15.103:0/2560229620 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4010690 con 0x7f61f8100350 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.130+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f81a5060 con 0x7f61f8100350 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 Processor -- start 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 -- start start 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61ee7fc640 1 -- 172.21.15.103:0/2560229620 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f61f40046c0 con 0x7f61f8100350 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dddc0 0x557d620e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620e8680 0x557d620eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d620432e0 con 0x557d620dddc0 2024-09-19T07:53:50.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d61feace0 con 0x557d620dd4a0 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f6201a41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d6203f2a0 con 0x557d620e8680 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dddc0 0x557d620e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dddc0 0x557d620e8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45256/0 (socket says 172.21.15.103:45256) 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 -- 172.21.15.103:0/2668801775 learned_addr learned my addr 172.21.15.103:0/2668801775 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fdc05640 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fcc03640 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620e8680 0x557d620eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 -- 172.21.15.103:0/2668801775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620e8680 msgr2=0x557d620eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620e8680 0x557d620eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 -- 172.21.15.103:0/2668801775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 msgr2=0x557d620dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fd404640 1 -- 172.21.15.103:0/2668801775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d61fcf440 con 0x557d620dddc0 2024-09-19T07:53:50.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.133+0000 7f61fcc03640 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620e8680 0x557d620eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.134+0000 7f61fd404640 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dddc0 0x557d620e8140 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cd870 tx=0x7f61f40d83b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759cdad4c3540175 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.134+0000 7f61ee7fc640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f61e40a33b0 0x7f61e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.134+0000 7f61fdc05640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f61e40a33b0 0x7f61e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.134+0000 7f61e37fe640 1 -- 172.21.15.103:0/2668801775 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f40e3070 con 0x557d620dddc0 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.134+0000 7f61ee7fc640 1 -- 172.21.15.103:0/2560229620 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f61f40d01c0 con 0x7f61f8100350 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.134+0000 7f61e37fe640 1 -- 172.21.15.103:0/2668801775 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f409cb90 con 0x557d620dddc0 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.134+0000 7f61e37fe640 1 -- 172.21.15.103:0/2668801775 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f409c260 con 0x557d620dddc0 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 -- 172.21.15.103:0/2668801775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dddc0 msgr2=0x557d620e8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dddc0 0x557d620e8140 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cd870 tx=0x7f61f40d83b0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f61fdc05640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f61e40a33b0 0x7f61e40a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005fd0 tx=0x7f61f0009500 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 -- 172.21.15.103:0/2668801775 shutdown_connections 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620e8680 0x557d620eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dddc0 0x557d620e8140 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 --2- 172.21.15.103:0/2668801775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 -- 172.21.15.103:0/2668801775 >> 172.21.15.103:0/2668801775 conn(0x557d620d9a30 msgr2=0x557d620d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 -- 172.21.15.103:0/2668801775 shutdown_connections 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 -- 172.21.15.103:0/2668801775 wait complete. 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 -- start start 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 Processor -- start 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.135+0000 7f6201a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f6201a41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620e5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f6201a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f6201a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d620eb250 con 0x557d620e8680 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f6201a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d61ffe050 con 0x557d620dd4a0 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f6201a41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d620eb3d0 con 0x557d620dddc0 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f6201a41740 10 client.0 _ll_register_callbacks cb 0x557d6203e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620e57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45262/0 (socket says 172.21.15.103:45262) 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 -- 172.21.15.103:0/1684302184 learned_addr learned my addr 172.21.15.103:0/1684302184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fcc03640 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fd404640 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 -- 172.21.15.103:0/1684302184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 msgr2=0x557d620e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 -- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 msgr2=0x557d620e7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620e7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 -- 172.21.15.103:0/1684302184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d61db43c0 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61fdc05640 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620e57e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f61f0012010 tx=0x7f61f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41155418ba6830a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1684302184 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f002c060 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f409cb90 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.136+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1684302184 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f61f0017030 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1684302184 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f61f001fc80 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 10 client.18528 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557d620f6cd0 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1684302184 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f409cb90 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1684302184 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f61f001c3f0 con 0x557d620e8680 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 10 client.18528 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:50.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 10 client.18528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 20 client.18528 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d620f80e0 0x557d620fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 4 client.18528 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d62050850 con 0x557d620f80e0 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.137+0000 7f61fd404640 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d620f80e0 0x557d620fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.138+0000 7f61fd404640 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d620f80e0 0x557d620fa4d0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ffb0d39b696bdb server_cookie=234f670556abfdf6 in_seq=0 out_seq=0 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.138+0000 7f61e0ff9640 10 client.18528 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.138+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1684302184 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d62050850 con 0x557d620f80e0 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.138+0000 7f61e0ff9640 10 client.18528 handle_command_reply: tid=0 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.138+0000 7f6201a41740 1 client.18528 shutdown 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.138+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 msgr2=0x557d620e57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.138+0000 7f6201a41740 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620e57e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f61f0012010 tx=0x7f61f0004ab0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 shutdown_connections 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d620f80e0 0x557d620fa4d0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620e57e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620e5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 --2- 172.21.15.103:0/1684302184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dd4a0 0x557d620e7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 >> 172.21.15.103:0/1684302184 conn(0x557d620d9a30 msgr2=0x557d620df760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f61e0ff9640 0 client.18528 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 shutdown_connections 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 1 -- 172.21.15.103:0/1684302184 wait complete. 2024-09-19T07:53:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.139+0000 7f6201a41740 20 client.18528 trim_cache size 0 max 16384 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 Processor -- start 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 -- start start 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dd4a0 0x557d620f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620e8680 0x557d620d8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61f81ada00 con 0x557d620dd4a0 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61f81af540 con 0x557d620e8680 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f6201a41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61f81ae0d0 con 0x557d620dddc0 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dd4a0 0x557d620f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dd4a0 0x557d620f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45272/0 (socket says 172.21.15.103:45272) 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 -- 172.21.15.103:0/1271844949 learned_addr learned my addr 172.21.15.103:0/1271844949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fcc03640 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 -- 172.21.15.103:0/1271844949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 msgr2=0x557d620fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 -- 172.21.15.103:0/1271844949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620e8680 msgr2=0x557d620d8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620e8680 0x557d620d8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.156+0000 7f61fd404640 1 -- 172.21.15.103:0/1271844949 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d620e4230 con 0x557d620dd4a0 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f61fd404640 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dd4a0 0x557d620f44b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cd9b0 tx=0x7f61f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b231b2e0f0e99c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f61e2ffd640 1 -- 172.21.15.103:0/1271844949 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f409bdf0 con 0x557d620dd4a0 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f61e2ffd640 1 -- 172.21.15.103:0/1271844949 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f40d3ae0 con 0x557d620dd4a0 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f61e2ffd640 1 -- 172.21.15.103:0/1271844949 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f409bfd0 con 0x557d620dd4a0 2024-09-19T07:53:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 -- 172.21.15.103:0/1271844949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dd4a0 msgr2=0x557d620f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dd4a0 0x557d620f44b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f61f40cd9b0 tx=0x7f61f4004a40 comp rx=0 tx=0).stop 2024-09-19T07:53:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 -- 172.21.15.103:0/1271844949 shutdown_connections 2024-09-19T07:53:50.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620e8680 0x557d620d8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dddc0 0x557d620fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 --2- 172.21.15.103:0/1271844949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620dd4a0 0x557d620f44b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 -- 172.21.15.103:0/1271844949 >> 172.21.15.103:0/1271844949 conn(0x557d620d9a30 msgr2=0x557d620d8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:50.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 -- 172.21.15.103:0/1271844949 shutdown_connections 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.157+0000 7f6201a41740 1 -- 172.21.15.103:0/1271844949 wait complete. 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 -- start start 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 Processor -- start 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dd4a0 0x557d620f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dddc0 0x557d620ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61f81aee70 con 0x557d620e8680 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61f81ae7a0 con 0x557d620dddc0 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61f81aba00 con 0x557d620dd4a0 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f6201a41740 10 client.0 _ll_register_callbacks cb 0x7f61f0009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f61fdc05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f61fdc05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620ec640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45280/0 (socket says 172.21.15.103:45280) 2024-09-19T07:53:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f61fdc05640 1 -- 172.21.15.103:0/1917265019 learned_addr learned my addr 172.21.15.103:0/1917265019 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.158+0000 7f61fd404640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dd4a0 0x557d620f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fcc03640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dddc0 0x557d620ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fdc05640 1 -- 172.21.15.103:0/1917265019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dd4a0 msgr2=0x557d620f0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fdc05640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dd4a0 0x557d620f0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fdc05640 1 -- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dddc0 msgr2=0x557d620ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fdc05640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dddc0 0x557d620ec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fdc05640 1 -- 172.21.15.103:0/1917265019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d620e8340 con 0x557d620e8680 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fd404640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dd4a0 0x557d620f0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61fdc05640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620ec640 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f61f0011e90 tx=0x7f61f001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bdf2a50a15f94d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1917265019 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0004e10 con 0x557d620e8680 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1917265019 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f61f0005040 con 0x557d620e8680 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f40d3ae0 con 0x557d620e8680 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1917265019 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f61f002c030 con 0x557d620e8680 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f6201a41740 10 client.18534 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557d62162bf0 con 0x557d620e8680 2024-09-19T07:53:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1917265019 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f40d3ae0 con 0x557d620e8680 2024-09-19T07:53:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.159+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1917265019 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f61f0030440 con 0x557d620e8680 2024-09-19T07:53:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 10 client.18534 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:53:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 20 client.18534 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d62163300 0x557d621656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 4 client.18534 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:53:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d6204ad60 con 0x557d62163300 2024-09-19T07:53:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f61fd404640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d62163300 0x557d621656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f61e0ff9640 10 client.18534 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f61fd404640 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d62163300 0x557d621656f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e328d1878dba9b server_cookie=13ed80cb4b8fb36b in_seq=0 out_seq=0 2024-09-19T07:53:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f61e0ff9640 1 -- 172.21.15.103:0/1917265019 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557d6204ad60 con 0x557d62163300 2024-09-19T07:53:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f61e0ff9640 10 client.18534 handle_command_reply: tid=0 2024-09-19T07:53:50.384 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:53:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:53:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:53:50.384 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:53:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 client.18534 shutdown 2024-09-19T07:53:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 msgr2=0x557d620ec640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620ec640 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f61f0011e90 tx=0x7f61f001c6a0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 shutdown_connections 2024-09-19T07:53:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x557d62163300 0x557d621656f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d620e8680 0x557d620ec640 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f61e0ff9640 0 client.18534 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:53:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d620dddc0 0x557d620ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.160+0000 7f6201a41740 1 --2- 172.21.15.103:0/1917265019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d620dd4a0 0x557d620f0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 >> 172.21.15.103:0/1917265019 conn(0x557d620d9a30 msgr2=0x557d620e7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 shutdown_connections 2024-09-19T07:53:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f6201a41740 1 -- 172.21.15.103:0/1917265019 wait complete. 2024-09-19T07:53:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f6201a41740 20 client.18534 trim_cache size 0 max 16384 2024-09-19T07:53:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f61e40a33b0 msgr2=0x7f61e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f61e40a33b0 0x7f61e40a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f61f0005fd0 tx=0x7f61f0009500 comp rx=0 tx=0).stop 2024-09-19T07:53:50.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100350 msgr2=0x7f61f81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:50.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100350 0x7f61f81984e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f61f400e9f0 tx=0x7f61f400edf0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 shutdown_connections 2024-09-19T07:53:50.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f61e40a33b0 0x7f61e40a5870 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61f8106f50 0x7f61f819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f8100d00 0x7f61f8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 --2- 172.21.15.103:0/2560229620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f8100350 0x7f61f81984e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 >> 172.21.15.103:0/2560229620 conn(0x7f61f8077210 msgr2=0x7f61f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 shutdown_connections 2024-09-19T07:53:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:50.161+0000 7f61ff668640 1 -- 172.21.15.103:0/2560229620 wait complete. 2024-09-19T07:53:50.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:50 smithi103 ceph-mon[31201]: pgmap v496: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 514 op/s 2024-09-19T07:53:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:50 smithi134 ceph-mon[35414]: pgmap v496: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 514 op/s 2024-09-19T07:53:50.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:50 smithi174 ceph-mon[35370]: pgmap v496: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 514 op/s 2024-09-19T07:53:51.369 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:53:51.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:53:51.621 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:52.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 -- 172.21.15.103:0/570427041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102ef0 msgr2=0x7f894c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 --2- 172.21.15.103:0/570427041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102ef0 0x7f894c10da40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8944009ca0 tx=0x7f894402a950 comp rx=0 tx=0).stop 2024-09-19T07:53:52.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 -- 172.21.15.103:0/570427041 shutdown_connections 2024-09-19T07:53:52.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 --2- 172.21.15.103:0/570427041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102ef0 0x7f894c10da40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 --2- 172.21.15.103:0/570427041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c102570 0x7f894c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 --2- 172.21.15.103:0/570427041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c100e20 0x7f894c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 -- 172.21.15.103:0/570427041 >> 172.21.15.103:0/570427041 conn(0x7f894c0fc7d0 msgr2=0x7f894c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 -- 172.21.15.103:0/570427041 shutdown_connections 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.208+0000 7f8954066640 1 -- 172.21.15.103:0/570427041 wait complete. 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 Processor -- start 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 -- start start 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c100e20 0x7f894c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102570 0x7f894c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c102ef0 0x7f894c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f894c110f80 con 0x7f894c102570 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f894c110e00 con 0x7f894c100e20 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8954066640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f894c111100 con 0x7f894c102ef0 2024-09-19T07:53:52.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8951601640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102570 0x7f894c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8951601640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102570 0x7f894c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45298/0 (socket says 172.21.15.103:45298) 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.209+0000 7f8951601640 1 -- 172.21.15.103:0/2846570798 learned_addr learned my addr 172.21.15.103:0/2846570798 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951e02640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c100e20 0x7f894c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8952603640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c102ef0 0x7f894c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951601640 1 -- 172.21.15.103:0/2846570798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c102ef0 msgr2=0x7f894c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951601640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c102ef0 0x7f894c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951601640 1 -- 172.21.15.103:0/2846570798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c100e20 msgr2=0x7f894c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951601640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c100e20 0x7f894c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951601640 1 -- 172.21.15.103:0/2846570798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c19d480 con 0x7f894c102570 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951e02640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c100e20 0x7f894c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8952603640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c102ef0 0x7f894c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:52.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8951601640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102570 0x7f894c198a10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f893c00e9f0 tx=0x7f893c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8942ffd640 1 -- 172.21.15.103:0/2846570798 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c00cd80 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c19d770 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8942ffd640 1 -- 172.21.15.103:0/2846570798 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f893c010910 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8942ffd640 1 -- 172.21.15.103:0/2846570798 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c0188f0 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.210+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c1a5050 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.211+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c101200 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.212+0000 7f8942ffd640 1 -- 172.21.15.103:0/2846570798 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f893c010ac0 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.213+0000 7f8942ffd640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f89380a33b0 0x7f89380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.213+0000 7f8951e02640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f89380a33b0 0x7f89380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.213+0000 7f8942ffd640 1 -- 172.21.15.103:0/2846570798 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f893c0d01d0 con 0x7f894c102570 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.213+0000 7f8951e02640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f89380a33b0 0x7f89380a5870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89480058c0 tx=0x7f8948005850 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:52.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.217+0000 7f8942ffd640 1 -- 172.21.15.103:0/2846570798 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f893c09cba0 con 0x7f894c102570 2024-09-19T07:53:52.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.427+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f894c10a0a0 con 0x7f894c102570 2024-09-19T07:53:52.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.428+0000 7f8942ffd640 1 -- 172.21.15.103:0/2846570798 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7f893c09c4a0 con 0x7f894c102570 2024-09-19T07:53:52.427 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:53:52.428 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":54,"btime":"2024-09-19T07:53:10:110704+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:53:09.872640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:53:52.428 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 54 2024-09-19T07:53:52.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f89380a33b0 msgr2=0x7f89380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f89380a33b0 0x7f89380a5870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89480058c0 tx=0x7f8948005850 comp rx=0 tx=0).stop 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102570 msgr2=0x7f894c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102570 0x7f894c198a10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f893c00e9f0 tx=0x7f893c00eeb0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 shutdown_connections 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f89380a33b0 0x7f89380a5870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c102ef0 0x7f894c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f894c102570 0x7f894c198a10 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 --2- 172.21.15.103:0/2846570798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c100e20 0x7f894c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 >> 172.21.15.103:0/2846570798 conn(0x7f894c0fc7d0 msgr2=0x7f894c06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 shutdown_connections 2024-09-19T07:53:52.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:52.430+0000 7f8954066640 1 -- 172.21.15.103:0/2846570798 wait complete. 2024-09-19T07:53:52.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:52 smithi134 ceph-mon[35414]: pgmap v497: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.5 MiB/s wr, 701 op/s 2024-09-19T07:53:52.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:52 smithi174 ceph-mon[35370]: pgmap v497: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.5 MiB/s wr, 701 op/s 2024-09-19T07:53:52.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:52 smithi103 ceph-mon[31201]: pgmap v497: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.5 MiB/s wr, 701 op/s 2024-09-19T07:53:53.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:53:53.466 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:53.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:53 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2846570798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:53.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:53 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2846570798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:53.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:53 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2846570798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:53:53.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 -- 172.21.15.103:0/1221427085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8101420 msgr2=0x7f16e810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 --2- 172.21.15.103:0/1221427085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8101420 0x7f16e810d950 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009cd0 tx=0x7f16e002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 -- 172.21.15.103:0/1221427085 shutdown_connections 2024-09-19T07:53:53.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 --2- 172.21.15.103:0/1221427085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8101420 0x7f16e810d950 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 --2- 172.21.15.103:0/1221427085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e8100a80 0x7f16e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 --2- 172.21.15.103:0/1221427085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e8106a80 0x7f16e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 -- 172.21.15.103:0/1221427085 >> 172.21.15.103:0/1221427085 conn(0x7f16e80fc7a0 msgr2=0x7f16e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 -- 172.21.15.103:0/1221427085 shutdown_connections 2024-09-19T07:53:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 -- 172.21.15.103:0/1221427085 wait complete. 2024-09-19T07:53:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.961+0000 7f16eeb06640 1 Processor -- start 2024-09-19T07:53:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16eeb06640 1 -- start start 2024-09-19T07:53:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16eeb06640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8100a80 0x7f16e80fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16eeb06640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e8101420 0x7f16e80fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16eeb06640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e8106a80 0x7f16e80fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16eeb06640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8110f70 con 0x7f16e8100a80 2024-09-19T07:53:53.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16eeb06640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16e8110df0 con 0x7f16e8106a80 2024-09-19T07:53:53.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16eeb06640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81110f0 con 0x7f16e8101420 2024-09-19T07:53:53.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8100a80 0x7f16e80fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8100a80 0x7f16e80fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45310/0 (socket says 172.21.15.103:45310) 2024-09-19T07:53:53.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 -- 172.21.15.103:0/3886790745 learned_addr learned my addr 172.21.15.103:0/3886790745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:53.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e8106a80 0x7f16e80fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 -- 172.21.15.103:0/3886790745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e8101420 msgr2=0x7f16e80fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16e7fff640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e8101420 0x7f16e80fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e8101420 0x7f16e80fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 -- 172.21.15.103:0/3886790745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e8106a80 msgr2=0x7f16e80fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e8106a80 0x7f16e80fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.962+0000 7f16ec8a2640 1 -- 172.21.15.103:0/3886790745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e81a6a90 con 0x7f16e8100a80 2024-09-19T07:53:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.963+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8100a80 0x7f16e80fcd90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00b4a0 tx=0x7f16dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.963+0000 7f16e5ffb640 1 -- 172.21.15.103:0/3886790745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc014070 con 0x7f16e8100a80 2024-09-19T07:53:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.963+0000 7f16e5ffb640 1 -- 172.21.15.103:0/3886790745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16dc010af0 con 0x7f16e8100a80 2024-09-19T07:53:53.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.963+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e81a6d80 con 0x7f16e8100a80 2024-09-19T07:53:53.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.963+0000 7f16e5ffb640 1 -- 172.21.15.103:0/3886790745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc017d10 con 0x7f16e8100a80 2024-09-19T07:53:53.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.963+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e81a7210 con 0x7f16e8100a80 2024-09-19T07:53:53.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.965+0000 7f16e5ffb640 1 -- 172.21.15.103:0/3886790745 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f16dc017070 con 0x7f16e8100a80 2024-09-19T07:53:53.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 Processor -- start 2024-09-19T07:53:53.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e5ffb640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16d80a33b0 0x7f16d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 -- start start 2024-09-19T07:53:53.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16d80a33b0 0x7f16d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e5ffb640 1 -- 172.21.15.103:0/3886790745 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f16dc0ceff0 con 0x7f16e8100a80 2024-09-19T07:53:53.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b03b40 0x55a153b02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b025d0 0x55a153b02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b0d370 0x55a153b0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a153a682f0 con 0x55a153b025d0 2024-09-19T07:53:53.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a153a0fcf0 con 0x55a153b03b40 2024-09-19T07:53:53.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16f0edf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a153a642b0 con 0x55a153b0d370 2024-09-19T07:53:53.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b025d0 0x55a153b02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b025d0 0x55a153b02a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45314/0 (socket says 172.21.15.103:45314) 2024-09-19T07:53:53.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 -- 172.21.15.103:0/2406422998 learned_addr learned my addr 172.21.15.103:0/2406422998 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:53.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b03b40 0x55a153b02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b0d370 0x55a153b0f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 -- 172.21.15.103:0/2406422998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b0d370 msgr2=0x55a153b0f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16d80a33b0 0x7f16d80a5870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006010 tx=0x7f16e0032fd0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b0d370 0x55a153b0f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 -- 172.21.15.103:0/2406422998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b03b40 msgr2=0x55a153b02090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b03b40 0x55a153b02090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16e7fff640 1 -- 172.21.15.103:0/2406422998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1539f43a0 con 0x55a153b025d0 2024-09-19T07:53:53.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.966+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b03b40 0x55a153b02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:53.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.967+0000 7f16e7fff640 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b025d0 0x55a153b02a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f16d400a8e0 tx=0x7f16d400ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247006d25be01760 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.967+0000 7f16d2ffd640 1 -- 172.21.15.103:0/2406422998 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4015070 con 0x55a153b025d0 2024-09-19T07:53:53.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.967+0000 7f16d2ffd640 1 -- 172.21.15.103:0/2406422998 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d400cb40 con 0x55a153b025d0 2024-09-19T07:53:53.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.967+0000 7f16d2ffd640 1 -- 172.21.15.103:0/2406422998 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d40185c0 con 0x55a153b025d0 2024-09-19T07:53:53.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.967+0000 7f16f0edf740 1 -- 172.21.15.103:0/2406422998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b025d0 msgr2=0x55a153b02a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:53.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.967+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b025d0 0x55a153b02a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f16d400a8e0 tx=0x7f16d400ada0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- 172.21.15.103:0/2406422998 shutdown_connections 2024-09-19T07:53:53.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b0d370 0x55a153b0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b025d0 0x55a153b02a10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2406422998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b03b40 0x55a153b02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- 172.21.15.103:0/2406422998 >> 172.21.15.103:0/2406422998 conn(0x55a153acb310 msgr2=0x55a153afec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:53.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- 172.21.15.103:0/2406422998 shutdown_connections 2024-09-19T07:53:53.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- 172.21.15.103:0/2406422998 wait complete. 2024-09-19T07:53:53.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- start start 2024-09-19T07:53:53.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 Processor -- start 2024-09-19T07:53:53.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b0ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b09f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b0a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a153b10070 con 0x55a153b0d370 2024-09-19T07:53:53.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a153a23060 con 0x55a153b025d0 2024-09-19T07:53:53.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a153b101f0 con 0x55a153b03b40 2024-09-19T07:53:53.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16f0edf740 10 client.0 _ll_register_callbacks cb 0x55a1537768c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:53.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16ed0a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b0a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.968+0000 7f16ed0a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b0a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45330/0 (socket says 172.21.15.103:45330) 2024-09-19T07:53:53.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ed0a3640 1 -- 172.21.15.103:0/922707705 learned_addr learned my addr 172.21.15.103:0/922707705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:53.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b0ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16e7fff640 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b09f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:53.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ed0a3640 1 -- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 msgr2=0x55a153b09f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b09f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ed0a3640 1 -- 172.21.15.103:0/922707705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 msgr2=0x55a153b0ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:53.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b0ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:53.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ed0a3640 1 -- 172.21.15.103:0/922707705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1537d93c0 con 0x55a153b0d370 2024-09-19T07:53:53.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b0a4d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009cd0 tx=0x7f16e002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec81883b262b56c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:53.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:53.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16c7fff640 1 -- 172.21.15.103:0/922707705 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0004450 con 0x55a153b0d370 2024-09-19T07:53:53.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16c7fff640 1 -- 172.21.15.103:0/922707705 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f16e00045f0 con 0x55a153b0d370 2024-09-19T07:53:53.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d400cb40 con 0x55a153b0d370 2024-09-19T07:53:53.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16c7fff640 1 -- 172.21.15.103:0/922707705 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f16e00047f0 con 0x55a153b0d370 2024-09-19T07:53:53.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16f0edf740 10 client.18552 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:53.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a153b1bae0 con 0x55a153b0d370 2024-09-19T07:53:53.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.969+0000 7f16c7fff640 1 -- 172.21.15.103:0/922707705 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d400cb40 con 0x55a153b0d370 2024-09-19T07:53:53.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16c7fff640 1 -- 172.21.15.103:0/922707705 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f16e004f070 con 0x55a153b0d370 2024-09-19T07:53:53.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16f0edf740 10 client.18552 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:53.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16f0edf740 10 client.18552 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:53:53.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16f0edf740 20 client.18552 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:53.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16f0edf740 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b1cef0 0x55a153b1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:53.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16f0edf740 4 client.18552 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:54.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a153a75860 con 0x55a153b1cef0 2024-09-19T07:53:54.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b1cef0 0x55a153b1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16c7fff640 10 client.18552 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:54.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.970+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b1cef0 0x55a153b1f340 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=198e4e44b5304ad0 server_cookie=a7331d1d7caf3bb9 in_seq=0 out_seq=0 2024-09-19T07:53:54.000 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16c7fff640 1 -- 172.21.15.103:0/922707705 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a153a75860 con 0x55a153b1cef0 2024-09-19T07:53:54.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16c7fff640 10 client.18552 handle_command_reply: tid=0 2024-09-19T07:53:54.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 client.18552 shutdown 2024-09-19T07:53:54.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 msgr2=0x55a153b0a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:54.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b0a4d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009cd0 tx=0x7f16e002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 shutdown_connections 2024-09-19T07:53:54.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b1cef0 0x55a153b1f340 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b0a4d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b09f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 --2- 172.21.15.103:0/922707705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b0ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16c7fff640 0 client.18552 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:54.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 >> 172.21.15.103:0/922707705 conn(0x55a153acb310 msgr2=0x55a153b04930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:54.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.971+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 shutdown_connections 2024-09-19T07:53:54.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.972+0000 7f16f0edf740 1 -- 172.21.15.103:0/922707705 wait complete. 2024-09-19T07:53:54.004 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.972+0000 7f16f0edf740 20 client.18552 trim_cache size 0 max 16384 2024-09-19T07:53:54.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.984+0000 7f16f0edf740 1 Processor -- start 2024-09-19T07:53:54.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.984+0000 7f16f0edf740 1 -- start start 2024-09-19T07:53:54.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b09830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:54.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b0bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b05120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16f0edf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81afcb0 con 0x55a153b0d370 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16f0edf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81b17f0 con 0x55a153b025d0 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16f0edf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81b0380 con 0x55a153b03b40 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b05120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b05120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45336/0 (socket says 172.21.15.103:45336) 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 -- 172.21.15.103:0/418185382 learned_addr learned my addr 172.21.15.103:0/418185382 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b09830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16e7fff640 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b0bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 -- 172.21.15.103:0/418185382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 msgr2=0x55a153b0bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:54.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b0bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 -- 172.21.15.103:0/418185382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 msgr2=0x55a153b09830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b09830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 -- 172.21.15.103:0/418185382 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a153b08f40 con 0x55a153b0d370 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b05120 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009ca0 tx=0x7f16e0031c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6855f7ca26dab9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16d27fc640 1 -- 172.21.15.103:0/418185382 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0051a10 con 0x55a153b0d370 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16d27fc640 1 -- 172.21.15.103:0/418185382 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0036040 con 0x55a153b0d370 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.985+0000 7f16d27fc640 1 -- 172.21.15.103:0/418185382 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0030920 con 0x55a153b0d370 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 -- 172.21.15.103:0/418185382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 msgr2=0x55a153b05120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b05120 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009ca0 tx=0x7f16e0031c60 comp rx=0 tx=0).stop 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 -- 172.21.15.103:0/418185382 shutdown_connections 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b05120 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b0bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 --2- 172.21.15.103:0/418185382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b09830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 -- 172.21.15.103:0/418185382 >> 172.21.15.103:0/418185382 conn(0x55a153acb310 msgr2=0x55a153b0b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 -- 172.21.15.103:0/418185382 shutdown_connections 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 -- 172.21.15.103:0/418185382 wait complete. 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 -- start start 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.986+0000 7f16f0edf740 1 Processor -- start 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b15bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b06c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16f0edf740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16f0edf740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81b1120 con 0x55a153b0d370 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16f0edf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81b0a50 con 0x55a153b025d0 2024-09-19T07:53:54.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16f0edf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16e81adcb0 con 0x55a153b03b40 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16f0edf740 10 client.0 _ll_register_callbacks cb 0x7f16e0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ec8a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b15bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45340/0 (socket says 172.21.15.103:45340) 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 -- 172.21.15.103:0/2129669209 learned_addr learned my addr 172.21.15.103:0/2129669209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16e7fff640 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b06c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 -- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 msgr2=0x55a153b06c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b06c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 -- 172.21.15.103:0/2129669209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 msgr2=0x55a153b15bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b15bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.987+0000 7f16ed0a3640 1 -- 172.21.15.103:0/2129669209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a153b079a0 con 0x55a153b0d370 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16ed0a3640 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b071a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000f80 tx=0x7f16e0051990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4ca63e453d7f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16c7fff640 1 -- 172.21.15.103:0/2129669209 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0004450 con 0x55a153b0d370 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16c7fff640 1 -- 172.21.15.103:0/2129669209 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f16e00045f0 con 0x55a153b0d370 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e0036040 con 0x55a153b0d370 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16c7fff640 1 -- 172.21.15.103:0/2129669209 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f16e0073050 con 0x55a153b0d370 2024-09-19T07:53:54.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 10 client.18558 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a153b87b80 con 0x55a153b0d370 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16c7fff640 1 -- 172.21.15.103:0/2129669209 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0036040 con 0x55a153b0d370 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16c7fff640 1 -- 172.21.15.103:0/2129669209 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f16e004f050 con 0x55a153b0d370 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 10 client.18558 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 10 client.18558 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 20 client.18558 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b88190 0x55a153b8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 4 client.18558 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.988+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a153a6fd70 con 0x55a153b88190 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b88190 0x55a153b8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16c7fff640 10 client.18558 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:54.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16ec8a2640 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b88190 0x55a153b8a580 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f1e8df4c17b63c server_cookie=332ef619d3832951 in_seq=0 out_seq=0 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16c7fff640 1 -- 172.21.15.103:0/2129669209 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a153a6fd70 con 0x55a153b88190 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16c7fff640 10 client.18558 handle_command_reply: tid=0 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 client.18558 shutdown 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 msgr2=0x55a153b071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b071a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f16e0000f80 tx=0x7f16e0051990 comp rx=0 tx=0).stop 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 shutdown_connections 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55a153b88190 0x55a153b8a580 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a153b0d370 0x55a153b071a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a153b03b40 0x55a153b06c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16f0edf740 1 --2- 172.21.15.103:0/2129669209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a153b025d0 0x55a153b15bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 >> 172.21.15.103:0/2129669209 conn(0x55a153acb310 msgr2=0x55a153b0b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.989+0000 7f16c7fff640 0 client.18558 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:53:54.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 shutdown_connections 2024-09-19T07:53:54.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16f0edf740 1 -- 172.21.15.103:0/2129669209 wait complete. 2024-09-19T07:53:54.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16f0edf740 20 client.18558 trim_cache size 0 max 16384 2024-09-19T07:53:54.022 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:54.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16d80a33b0 msgr2=0x7f16d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:54.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16d80a33b0 0x7f16d80a5870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006010 tx=0x7f16e0032fd0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8100a80 msgr2=0x7f16e80fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8100a80 0x7f16e80fcd90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00b4a0 tx=0x7f16dc00b960 comp rx=0 tx=0).stop 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 shutdown_connections 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f16d80a33b0 0x7f16d80a5870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e8106a80 0x7f16e80fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e8101420 0x7f16e80fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 --2- 172.21.15.103:0/3886790745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16e8100a80 0x7f16e80fcd90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 >> 172.21.15.103:0/3886790745 conn(0x7f16e80fc7a0 msgr2=0x7f16e806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:54.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 shutdown_connections 2024-09-19T07:53:54.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:53.990+0000 7f16eeb06640 1 -- 172.21.15.103:0/3886790745 wait complete. 2024-09-19T07:53:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:54 smithi134 ceph-mon[35414]: pgmap v498: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.3 MiB/s wr, 779 op/s 2024-09-19T07:53:54.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:54 smithi174 ceph-mon[35370]: pgmap v498: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.3 MiB/s wr, 779 op/s 2024-09-19T07:53:54.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:54 smithi103 ceph-mon[31201]: pgmap v498: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.3 MiB/s wr, 779 op/s 2024-09-19T07:53:54.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:53:54.994 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:55.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.514+0000 7fcaae20d640 1 -- 172.21.15.103:0/2904568135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102570 msgr2=0x7fcaa81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.514+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2904568135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102570 0x7fcaa81029b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fca90009900 tx=0x7fca9002a480 comp rx=0 tx=0).stop 2024-09-19T07:53:55.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.514+0000 7fcaae20d640 1 -- 172.21.15.103:0/2904568135 shutdown_connections 2024-09-19T07:53:55.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.514+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2904568135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8102ef0 0x7fcaa810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.514+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2904568135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102570 0x7fcaa81029b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.514+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2904568135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa8100e20 0x7fcaa8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.514+0000 7fcaae20d640 1 -- 172.21.15.103:0/2904568135 >> 172.21.15.103:0/2904568135 conn(0x7fcaa80fc7d0 msgr2=0x7fcaa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.515+0000 7fcaae20d640 1 -- 172.21.15.103:0/2904568135 shutdown_connections 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.515+0000 7fcaae20d640 1 -- 172.21.15.103:0/2904568135 wait complete. 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.515+0000 7fcaae20d640 1 Processor -- start 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.515+0000 7fcaae20d640 1 -- start start 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaae20d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100e20 0x7fcaa8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaae20d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa8102570 0x7fcaa8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaae20d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102ef0 0x7fcaa819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaae20d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8110f80 con 0x7fcaa8102ef0 2024-09-19T07:53:55.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaae20d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8110e00 con 0x7fcaa8102570 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaae20d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8111100 con 0x7fcaa8100e20 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102ef0 0x7fcaa819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102ef0 0x7fcaa819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45366/0 (socket says 172.21.15.103:45366) 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 -- 172.21.15.103:0/2517802659 learned_addr learned my addr 172.21.15.103:0/2517802659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100e20 0x7fcaa8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa8102570 0x7fcaa8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 -- 172.21.15.103:0/2517802659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100e20 msgr2=0x7fcaa8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100e20 0x7fcaa8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 -- 172.21.15.103:0/2517802659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa8102570 msgr2=0x7fcaa8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa8102570 0x7fcaa8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 -- 172.21.15.103:0/2517802659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa819d510 con 0x7fcaa8102ef0 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100e20 0x7fcaa8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa8102570 0x7fcaa8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa7fff640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102ef0 0x7fcaa819ce30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000eaa0 tx=0x7fcaa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa4ff9640 1 -- 172.21.15.103:0/2517802659 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000cdb0 con 0x7fcaa8102ef0 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa4ff9640 1 -- 172.21.15.103:0/2517802659 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa0005020 con 0x7fcaa8102ef0 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa819d800 con 0x7fcaa8102ef0 2024-09-19T07:53:55.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.516+0000 7fcaa4ff9640 1 -- 172.21.15.103:0/2517802659 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00116b0 con 0x7fcaa8102ef0 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.517+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81a50e0 con 0x7fcaa8102ef0 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.519+0000 7fcaa4ff9640 1 -- 172.21.15.103:0/2517802659 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcaa0005560 con 0x7fcaa8102ef0 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.519+0000 7fcab05e6740 1 Processor -- start 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.519+0000 7fcab05e6740 1 -- start start 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560153755b40 0x560153754090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601537545d0 0x560153754a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa4ff9640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fca940a33b0 0x7fca940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56015375f370 0x560153761760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcab05e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5601536ba2f0 con 0x560153755b40 2024-09-19T07:53:55.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcab05e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560153661cf0 con 0x56015375f370 2024-09-19T07:53:55.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcab05e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601536b62b0 con 0x5601537545d0 2024-09-19T07:53:55.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560153755b40 0x560153754090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa7fff640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fca940a33b0 0x7fca940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560153755b40 0x560153754090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45382/0 (socket says 172.21.15.103:45382) 2024-09-19T07:53:55.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/33431276 learned_addr learned my addr 172.21.15.103:0/33431276 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:55.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa4ff9640 1 -- 172.21.15.103:0/2517802659 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fcaa00cffc0 con 0x7fcaa8102ef0 2024-09-19T07:53:55.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601537545d0 0x560153754a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56015375f370 0x560153761760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/33431276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601537545d0 msgr2=0x560153754a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601537545d0 0x560153754a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/33431276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56015375f370 msgr2=0x560153761760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56015375f370 0x560153761760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/33431276 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601536463a0 con 0x560153755b40 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa7fff640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fca940a33b0 0x7fca940a5870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00cd7b0 tx=0x7fcaa006e010 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560153755b40 0x560153754090 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fca90006010 tx=0x7fca90031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cc95c5dfc95b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.520+0000 7fca9dffb640 1 -- 172.21.15.103:0/33431276 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90035070 con 0x560153755b40 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fca9dffb640 1 -- 172.21.15.103:0/33431276 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca900046c0 con 0x560153755b40 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fca9dffb640 1 -- 172.21.15.103:0/33431276 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90003830 con 0x560153755b40 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 -- 172.21.15.103:0/33431276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560153755b40 msgr2=0x560153754090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560153755b40 0x560153754090 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fca90006010 tx=0x7fca90031480 comp rx=0 tx=0).stop 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 -- 172.21.15.103:0/33431276 shutdown_connections 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56015375f370 0x560153761760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601537545d0 0x560153754a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 --2- 172.21.15.103:0/33431276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560153755b40 0x560153754090 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 -- 172.21.15.103:0/33431276 >> 172.21.15.103:0/33431276 conn(0x56015371d310 msgr2=0x560153750c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 -- 172.21.15.103:0/33431276 shutdown_connections 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 -- 172.21.15.103:0/33431276 wait complete. 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.521+0000 7fcab05e6740 1 -- start start 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 1 Processor -- start 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601537545d0 0x56015375edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560153755b40 0x56015375c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56015375f370 0x56015375c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560153762010 con 0x56015375f370 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560153675060 con 0x5601537545d0 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560153762190 con 0x560153755b40 2024-09-19T07:53:55.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcab05e6740 10 client.0 _ll_register_callbacks cb 0x5601533c88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56015375f370 0x56015375c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56015375f370 0x56015375c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45392/0 (socket says 172.21.15.103:45392) 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/2781731079 learned_addr learned my addr 172.21.15.103:0/2781731079 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601537545d0 0x56015375edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560153755b40 0x56015375c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560153755b40 msgr2=0x56015375c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560153755b40 0x56015375c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/2781731079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601537545d0 msgr2=0x56015375edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601537545d0 0x56015375edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/2781731079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015342b3c0 con 0x56015375f370 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.522+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560153755b40 0x56015375c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56015375f370 0x56015375c7e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fca900096c0 tx=0x7fca90004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1f7c8467d52eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fca86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fca86ffd640 1 -- 172.21.15.103:0/2781731079 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9004f060 con 0x56015375f370 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fca86ffd640 1 -- 172.21.15.103:0/2781731079 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fca90031740 con 0x56015375f370 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca900046c0 con 0x56015375f370 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fca86ffd640 1 -- 172.21.15.103:0/2781731079 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fca90035040 con 0x56015375f370 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 10 client.18570 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56015376db40 con 0x56015375f370 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fca86ffd640 1 -- 172.21.15.103:0/2781731079 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca900046c0 con 0x56015375f370 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fca86ffd640 1 -- 172.21.15.103:0/2781731079 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fca9004a900 con 0x56015375f370 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 10 client.18570 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 10 client.18570 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 20 client.18570 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56015376ee90 0x560153771280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 4 client.18570 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.523+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601536c7860 con 0x56015376ee90 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.524+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56015376ee90 0x560153771280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.524+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56015376ee90 0x560153771280 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b69c6de2d58cc98 server_cookie=5ce8a0c4e584add8 in_seq=0 out_seq=0 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fca86ffd640 10 client.18570 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fca86ffd640 1 -- 172.21.15.103:0/2781731079 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601536c7860 con 0x56015376ee90 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fca86ffd640 10 client.18570 handle_command_reply: tid=0 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 client.18570 shutdown 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56015375f370 msgr2=0x56015375c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56015375f370 0x56015375c7e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fca900096c0 tx=0x7fca90004560 comp rx=0 tx=0).stop 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 shutdown_connections 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x56015376ee90 0x560153771280 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56015375f370 0x56015375c7e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560153755b40 0x56015375c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fca86ffd640 0 client.18570 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 --2- 172.21.15.103:0/2781731079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601537545d0 0x56015375edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 >> 172.21.15.103:0/2781731079 conn(0x56015371d310 msgr2=0x560153756960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 shutdown_connections 2024-09-19T07:53:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 1 -- 172.21.15.103:0/2781731079 wait complete. 2024-09-19T07:53:55.537 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.525+0000 7fcab05e6740 20 client.18570 trim_cache size 0 max 16384 2024-09-19T07:53:55.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.538+0000 7fcab05e6740 1 Processor -- start 2024-09-19T07:53:55.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.538+0000 7fcab05e6740 1 -- start start 2024-09-19T07:53:55.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x56015376b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153757120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153757660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcab05e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ad990 con 0x5601537545d0 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcab05e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81af4d0 con 0x560153755b40 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcab05e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ae060 con 0x56015375f370 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x56015376b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x56015376b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45402/0 (socket says 172.21.15.103:45402) 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4071787670 learned_addr learned my addr 172.21.15.103:0/4071787670 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153757120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153757660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4071787670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 msgr2=0x560153757660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153757660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4071787670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 msgr2=0x560153757120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153757120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.540 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4071787670 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560153757ba0 con 0x5601537545d0 2024-09-19T07:53:55.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x56015376b430 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fca9002ac70 tx=0x7fca900042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bab8599652d113 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fca9d7fa640 1 -- 172.21.15.103:0/4071787670 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9004f040 con 0x5601537545d0 2024-09-19T07:53:55.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fca9d7fa640 1 -- 172.21.15.103:0/4071787670 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca900046c0 con 0x5601537545d0 2024-09-19T07:53:55.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.539+0000 7fca9d7fa640 1 -- 172.21.15.103:0/4071787670 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90039990 con 0x5601537545d0 2024-09-19T07:53:55.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 -- 172.21.15.103:0/4071787670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 msgr2=0x56015376b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x56015376b430 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fca9002ac70 tx=0x7fca900042e0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 -- 172.21.15.103:0/4071787670 shutdown_connections 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153757660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153757120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4071787670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x56015376b430 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 -- 172.21.15.103:0/4071787670 >> 172.21.15.103:0/4071787670 conn(0x56015371d310 msgr2=0x56015375e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 -- 172.21.15.103:0/4071787670 shutdown_connections 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.540+0000 7fcab05e6740 1 -- 172.21.15.103:0/4071787670 wait complete. 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 -- start start 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 Processor -- start 2024-09-19T07:53:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x560153772010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153772550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153772a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81aee00 con 0x5601537545d0 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ae730 con 0x560153755b40 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ab990 con 0x56015375f370 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcab05e6740 10 client.0 _ll_register_callbacks cb 0x7fca90004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x560153772010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcaa6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x560153772010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45408/0 (socket says 172.21.15.103:45408) 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4040762127 learned_addr learned my addr 172.21.15.103:0/4040762127 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153772550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.541+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153772a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 msgr2=0x560153772a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153772a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4040762127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 msgr2=0x560153772550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153772550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcaa6ffd640 1 -- 172.21.15.103:0/4040762127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601537d87d0 con 0x5601537545d0 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153772550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcaa6ffd640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x560153772010 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fca9004acd0 tx=0x7fca9004aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abfecd1d1ac02ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fca86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fca86ffd640 1 -- 172.21.15.103:0/4040762127 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9004f040 con 0x5601537545d0 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fca86ffd640 1 -- 172.21.15.103:0/4040762127 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fca90039b40 con 0x5601537545d0 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca900046c0 con 0x5601537545d0 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fca86ffd640 1 -- 172.21.15.103:0/4040762127 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fca90039520 con 0x5601537545d0 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcab05e6740 10 client.18576 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:55.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5601537d9a50 con 0x5601537545d0 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fca86ffd640 1 -- 172.21.15.103:0/4040762127 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca900046c0 con 0x5601537545d0 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fca86ffd640 1 -- 172.21.15.103:0/4040762127 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fca90053470 con 0x5601537545d0 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.542+0000 7fcab05e6740 10 client.18576 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcab05e6740 10 client.18576 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcab05e6740 20 client.18576 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5601537d9e50 0x5601537dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcab05e6740 4 client.18576 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601536c1d70 con 0x5601537d9e50 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5601537d9e50 0x5601537dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcaa77fe640 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5601537d9e50 0x5601537dc270 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4cc670703ce725b9 server_cookie=61aac74d9f5bf154 in_seq=0 out_seq=0 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fca86ffd640 10 client.18576 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:55.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fca86ffd640 1 -- 172.21.15.103:0/4040762127 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601536c1d70 con 0x5601537d9e50 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fca86ffd640 10 client.18576 handle_command_reply: tid=0 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcab05e6740 1 client.18576 shutdown 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.543+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 msgr2=0x560153772010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x560153772010 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fca9004acd0 tx=0x7fca9004aab0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 shutdown_connections 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5601537d9e50 0x5601537dc270 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56015375f370 0x560153772a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560153755b40 0x560153772550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 --2- 172.21.15.103:0/4040762127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5601537545d0 0x560153772010 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fca86ffd640 0 client.18576 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 >> 172.21.15.103:0/4040762127 conn(0x56015371d310 msgr2=0x56015375a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 shutdown_connections 2024-09-19T07:53:55.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 1 -- 172.21.15.103:0/4040762127 wait complete. 2024-09-19T07:53:55.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcab05e6740 20 client.18576 trim_cache size 0 max 16384 2024-09-19T07:53:55.550 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:55.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fca940a33b0 msgr2=0x7fca940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fca940a33b0 0x7fca940a5870 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00cd7b0 tx=0x7fcaa006e010 comp rx=0 tx=0).stop 2024-09-19T07:53:55.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102ef0 msgr2=0x7fcaa819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:55.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102ef0 0x7fcaa819ce30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000eaa0 tx=0x7fcaa000ef60 comp rx=0 tx=0).stop 2024-09-19T07:53:55.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 shutdown_connections 2024-09-19T07:53:55.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fca940a33b0 0x7fca940a5870 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa8102ef0 0x7fcaa819ce30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa8102570 0x7fcaa8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 --2- 172.21.15.103:0/2517802659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100e20 0x7fcaa8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:55.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 >> 172.21.15.103:0/2517802659 conn(0x7fcaa80fc7d0 msgr2=0x7fcaa806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:55.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 shutdown_connections 2024-09-19T07:53:55.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:55.544+0000 7fcaae20d640 1 -- 172.21.15.103:0/2517802659 wait complete. 2024-09-19T07:53:56.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:53:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:56.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:56 smithi103 ceph-mon[31201]: pgmap v499: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.3 MiB/s wr, 779 op/s 2024-09-19T07:53:56.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:56 smithi134 ceph-mon[35414]: pgmap v499: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.3 MiB/s wr, 779 op/s 2024-09-19T07:53:56.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:56 smithi174 ceph-mon[35370]: pgmap v499: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.3 MiB/s wr, 779 op/s 2024-09-19T07:53:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 -- 172.21.15.103:0/124218992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444106a80 msgr2=0x7f2444106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 --2- 172.21.15.103:0/124218992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444106a80 0x7f2444106e60 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2434009930 tx=0x7f243402a440 comp rx=0 tx=0).stop 2024-09-19T07:53:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 -- 172.21.15.103:0/124218992 shutdown_connections 2024-09-19T07:53:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 --2- 172.21.15.103:0/124218992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2444101420 0x7f244410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 --2- 172.21.15.103:0/124218992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2444100a80 0x7f2444100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 --2- 172.21.15.103:0/124218992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444106a80 0x7f2444106e60 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 -- 172.21.15.103:0/124218992 >> 172.21.15.103:0/124218992 conn(0x7f24440fc7a0 msgr2=0x7f24440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 -- 172.21.15.103:0/124218992 shutdown_connections 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.924+0000 7f24488fb640 1 -- 172.21.15.103:0/124218992 wait complete. 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 Processor -- start 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 -- start start 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444100a80 0x7f2444197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2444101420 0x7f2444197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2444106a80 0x7f244419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2444110e80 con 0x7f2444100a80 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2444110d00 con 0x7f2444106a80 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f24488fb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2444111000 con 0x7f2444101420 2024-09-19T07:53:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444100a80 0x7f2444197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444100a80 0x7f2444197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45426/0 (socket says 172.21.15.103:45426) 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 -- 172.21.15.103:0/3743542947 learned_addr learned my addr 172.21.15.103:0/3743542947 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2444106a80 0x7f244419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f243bfff640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2444101420 0x7f2444197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 -- 172.21.15.103:0/3743542947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2444101420 msgr2=0x7f2444197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2444101420 0x7f2444197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 -- 172.21.15.103:0/3743542947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2444106a80 msgr2=0x7f244419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2444106a80 0x7f244419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.925+0000 7f244259c640 1 -- 172.21.15.103:0/3743542947 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244419c1f0 con 0x7f2444100a80 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.926+0000 7f244259c640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444100a80 0x7f2444197240 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2434009a50 tx=0x7f2434004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.926+0000 7f243b7fe640 1 -- 172.21.15.103:0/3743542947 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434046070 con 0x7f2444100a80 2024-09-19T07:53:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.926+0000 7f243b7fe640 1 -- 172.21.15.103:0/3743542947 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24340044a0 con 0x7f2444100a80 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.926+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244419c480 con 0x7f2444100a80 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.926+0000 7f243b7fe640 1 -- 172.21.15.103:0/3743542947 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434035050 con 0x7f2444100a80 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.926+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244406a790 con 0x7f2444100a80 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 Processor -- start 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 -- start start 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f243b7fe640 1 -- 172.21.15.103:0/3743542947 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2434049070 con 0x7f2444100a80 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a036e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a036ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a037b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1a02d42f0 con 0x55f1a036fb40 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f1a027bcf0 con 0x55f1a0379370 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244acd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1a02d02b0 con 0x55f1a036e5d0 2024-09-19T07:53:56.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f2442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a036e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f2442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a036e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45434/0 (socket says 172.21.15.103:45434) 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f2442d9d640 1 -- 172.21.15.103:0/3378437574 learned_addr learned my addr 172.21.15.103:0/3378437574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f244259c640 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a036ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.929+0000 7f243bfff640 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a037b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2442d9d640 1 -- 172.21.15.103:0/3378437574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 msgr2=0x55f1a036ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a036ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2442d9d640 1 -- 172.21.15.103:0/3378437574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 msgr2=0x55f1a037b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a037b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2442d9d640 1 -- 172.21.15.103:0/3378437574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a02603a0 con 0x55f1a036fb40 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f243bfff640 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a037b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a036e090 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f243c007dd0 tx=0x7f243c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f192e0cafd80a59c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f243b7fe640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24300a33b0 0x7f24300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2438ff9640 1 -- 172.21.15.103:0/3378437574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c014070 con 0x55f1a036fb40 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2438ff9640 1 -- 172.21.15.103:0/3378437574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243c004d20 con 0x55f1a036fb40 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f244259c640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24300a33b0 0x7f24300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f2438ff9640 1 -- 172.21.15.103:0/3378437574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c010850 con 0x55f1a036fb40 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.930+0000 7f243b7fe640 1 -- 172.21.15.103:0/3743542947 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f24340f1a20 con 0x7f2444100a80 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- 172.21.15.103:0/3378437574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 msgr2=0x55f1a036e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244259c640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24300a33b0 0x7f24300a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f24340ef210 tx=0x7f24340f54c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a036e090 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f243c007dd0 tx=0x7f243c00e720 comp rx=0 tx=0).stop 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- 172.21.15.103:0/3378437574 shutdown_connections 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a037b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a036ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 --2- 172.21.15.103:0/3378437574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a036e090 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- 172.21.15.103:0/3378437574 >> 172.21.15.103:0/3378437574 conn(0x55f1a0337310 msgr2=0x55f1a036ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:56.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- 172.21.15.103:0/3378437574 shutdown_connections 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- 172.21.15.103:0/3378437574 wait complete. 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- start start 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 Processor -- start 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a0378d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036fb40 0x55f1a0376060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a0379370 0x55f1a03766c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1a037c070 con 0x55f1a0379370 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f1a028f060 con 0x55f1a036fb40 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.931+0000 7f244acd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1a037c1f0 con 0x55f1a036e5d0 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f244acd4740 10 client.0 _ll_register_callbacks cb 0x55f19ffe28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a0379370 0x55f1a03766c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a0379370 0x55f1a03766c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45440/0 (socket says 172.21.15.103:45440) 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 -- 172.21.15.103:0/3426643284 learned_addr learned my addr 172.21.15.103:0/3426643284 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f243bfff640 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036fb40 0x55f1a0376060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a0378d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 -- 172.21.15.103:0/3426643284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 msgr2=0x55f1a0378d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a0378d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 -- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036fb40 msgr2=0x55f1a0376060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036fb40 0x55f1a0376060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 -- 172.21.15.103:0/3426643284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a00453c0 con 0x55f1a0379370 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a0379370 0x55f1a03766c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f243c011e20 tx=0x7f243c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ba43237834e77a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2429ffb640 1 -- 172.21.15.103:0/3426643284 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c014070 con 0x55f1a0379370 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2429ffb640 1 -- 172.21.15.103:0/3426643284 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f243c010c30 con 0x55f1a0379370 2024-09-19T07:53:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243c004d20 con 0x55f1a0379370 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2429ffb640 1 -- 172.21.15.103:0/3426643284 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f243c02f050 con 0x55f1a0379370 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f244acd4740 10 client.18588 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f1a0387ac0 con 0x55f1a0379370 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.932+0000 7f2429ffb640 1 -- 172.21.15.103:0/3426643284 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243c004d20 con 0x55f1a0379370 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f2429ffb640 1 -- 172.21.15.103:0/3426643284 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f243c004300 con 0x55f1a0379370 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f244acd4740 10 client.18588 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f244acd4740 10 client.18588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f244acd4740 20 client.18588 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f244acd4740 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a038b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f244acd4740 4 client.18588 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1a02e1860 con 0x55f1a0388e10 2024-09-19T07:53:56.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f243bfff640 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a038b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f2429ffb640 10 client.18588 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.933+0000 7f243bfff640 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a038b200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6580f67c559744b server_cookie=ee37f13efca2bfb3 in_seq=0 out_seq=0 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f2429ffb640 1 -- 172.21.15.103:0/3426643284 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1a02e1860 con 0x55f1a0388e10 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f2429ffb640 10 client.18588 handle_command_reply: tid=0 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 client.18588 shutdown 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a0379370 msgr2=0x55f1a03766c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a0379370 0x55f1a03766c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f243c011e20 tx=0x7f243c010430 comp rx=0 tx=0).stop 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 shutdown_connections 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a038b200 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a0379370 0x55f1a03766c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f2429ffb640 0 client.18588 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036fb40 0x55f1a0376060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 --2- 172.21.15.103:0/3426643284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036e5d0 0x55f1a0378d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 >> 172.21.15.103:0/3426643284 conn(0x55f1a0337310 msgr2=0x55f1a0370960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 shutdown_connections 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.934+0000 7f244acd4740 1 -- 172.21.15.103:0/3426643284 wait complete. 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.935+0000 7f244acd4740 20 client.18588 trim_cache size 0 max 16384 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 Processor -- start 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 -- start start 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036e5d0 0x55f1a0385310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a0371120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a0379370 0x55f1a0371660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24441ab5a0 con 0x55f1a036fb40 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24441ad0e0 con 0x55f1a036e5d0 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f244acd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24441abc70 con 0x55f1a0379370 2024-09-19T07:53:56.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.947+0000 7f243bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a0371120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a0371120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45452/0 (socket says 172.21.15.103:45452) 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 -- 172.21.15.103:0/3039422706 learned_addr learned my addr 172.21.15.103:0/3039422706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036e5d0 0x55f1a0385310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 -- 172.21.15.103:0/3039422706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a0379370 msgr2=0x55f1a0371660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a0379370 0x55f1a0371660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 -- 172.21.15.103:0/3039422706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036e5d0 msgr2=0x55f1a0385310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036e5d0 0x55f1a0385310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 -- 172.21.15.103:0/3039422706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a0378960 con 0x55f1a036fb40 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f243bfff640 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a0371120 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x55f1a036f070 tx=0x7f242c00a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46630b890789bf66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f242bfff640 1 -- 172.21.15.103:0/3039422706 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c01a330 con 0x55f1a036fb40 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f242bfff640 1 -- 172.21.15.103:0/3039422706 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242c01a9b0 con 0x55f1a036fb40 2024-09-19T07:53:56.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.948+0000 7f242bfff640 1 -- 172.21.15.103:0/3039422706 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c011ae0 con 0x55f1a036fb40 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- 172.21.15.103:0/3039422706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 msgr2=0x55f1a0371120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a0371120 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x55f1a036f070 tx=0x7f242c00a470 comp rx=0 tx=0).stop 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- 172.21.15.103:0/3039422706 shutdown_connections 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a0379370 0x55f1a0371660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036fb40 0x55f1a0371120 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 --2- 172.21.15.103:0/3039422706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a036e5d0 0x55f1a0385310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- 172.21.15.103:0/3039422706 >> 172.21.15.103:0/3039422706 conn(0x55f1a0337310 msgr2=0x55f1a03780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- 172.21.15.103:0/3039422706 shutdown_connections 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- 172.21.15.103:0/3039422706 wait complete. 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- start start 2024-09-19T07:53:56.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 Processor -- start 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036e5d0 0x55f1a0381710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036fb40 0x55f1a0381c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a03728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24441aca10 con 0x55f1a036e5d0 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24441ac340 con 0x55f1a0379370 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.949+0000 7f244acd4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24441a95a0 con 0x55f1a036fb40 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f244acd4740 10 client.0 _ll_register_callbacks cb 0x7f24340f6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036e5d0 0x55f1a0381710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036e5d0 0x55f1a0381710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45460/0 (socket says 172.21.15.103:45460) 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 -- 172.21.15.103:0/3789863040 learned_addr learned my addr 172.21.15.103:0/3789863040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a03728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f243bfff640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036fb40 0x55f1a0381c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 -- 172.21.15.103:0/3789863040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036fb40 msgr2=0x55f1a0381c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036fb40 0x55f1a0381c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 -- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 msgr2=0x55f1a03728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a03728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 -- 172.21.15.103:0/3789863040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a0373120 con 0x55f1a036e5d0 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f243bfff640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036fb40 0x55f1a0381c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2442d9d640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036e5d0 0x55f1a0381710 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f243c002230 tx=0x7f243c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850d544a47fe8f69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2429ffb640 1 -- 172.21.15.103:0/3789863040 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c03a020 con 0x55f1a036e5d0 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f242c01a9b0 con 0x55f1a036e5d0 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2429ffb640 1 -- 172.21.15.103:0/3789863040 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f243c003970 con 0x55f1a036e5d0 2024-09-19T07:53:56.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f2429ffb640 1 -- 172.21.15.103:0/3789863040 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f243c014070 con 0x55f1a036e5d0 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f244acd4740 10 client.18594 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.950+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f1a03f3480 con 0x55f1a036e5d0 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f2429ffb640 1 -- 172.21.15.103:0/3789863040 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f242c01a9b0 con 0x55f1a036e5d0 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f2429ffb640 1 -- 172.21.15.103:0/3789863040 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f243c003c70 con 0x55f1a036e5d0 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f244acd4740 10 client.18594 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f244acd4740 10 client.18594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f244acd4740 20 client.18594 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f244acd4740 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a03f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f244acd4740 4 client.18594 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1a02dbd70 con 0x55f1a0388e10 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.951+0000 7f243bfff640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a03f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f243bfff640 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a03f5c20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b42762043a9d90b server_cookie=1f7cd3ae7a67dde4 in_seq=0 out_seq=0 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f2429ffb640 10 client.18594 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:56.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f2429ffb640 1 -- 172.21.15.103:0/3789863040 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1a02dbd70 con 0x55f1a0388e10 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f2429ffb640 10 client.18594 handle_command_reply: tid=0 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 client.18594 shutdown 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036e5d0 msgr2=0x55f1a0381710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036e5d0 0x55f1a0381710 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f243c002230 tx=0x7f243c0048a0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 shutdown_connections 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55f1a0388e10 0x55f1a03f5c20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1a0379370 0x55f1a03728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f2429ffb640 0 client.18594 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1a036fb40 0x55f1a0381c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 --2- 172.21.15.103:0/3789863040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a036e5d0 0x55f1a0381710 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 >> 172.21.15.103:0/3789863040 conn(0x55f1a0337310 msgr2=0x55f1a03694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:56.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 shutdown_connections 2024-09-19T07:53:56.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 1 -- 172.21.15.103:0/3789863040 wait complete. 2024-09-19T07:53:56.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.952+0000 7f244acd4740 20 client.18594 trim_cache size 0 max 16384 2024-09-19T07:53:56.969 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24300a33b0 msgr2=0x7f24300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24300a33b0 0x7f24300a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f24340ef210 tx=0x7f24340f54c0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444100a80 msgr2=0x7f2444197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444100a80 0x7f2444197240 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2434009a50 tx=0x7f2434004270 comp rx=0 tx=0).stop 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 shutdown_connections 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f24300a33b0 0x7f24300a5870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2444106a80 0x7f244419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2444101420 0x7f2444197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 --2- 172.21.15.103:0/3743542947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2444100a80 0x7f2444197240 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 >> 172.21.15.103:0/3743542947 conn(0x7f24440fc7a0 msgr2=0x7f24440fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:56.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 shutdown_connections 2024-09-19T07:53:56.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:56.953+0000 7f24488fb640 1 -- 172.21.15.103:0/3743542947 wait complete. 2024-09-19T07:53:57.795 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:53:58.045 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:53:58.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:53:58 smithi103 ceph-mon[31201]: pgmap v500: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-09-19T07:53:58.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:53:58 smithi134 ceph-mon[35414]: pgmap v500: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- 172.21.15.103:0/3562417007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c102ac0 msgr2=0x7f828c1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 --2- 172.21.15.103:0/3562417007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c102ac0 0x7f828c1000b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8284009900 tx=0x7f828402a480 comp rx=0 tx=0).stop 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- 172.21.15.103:0/3562417007 shutdown_connections 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 --2- 172.21.15.103:0/3562417007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f828c100c10 0x7f828c110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 --2- 172.21.15.103:0/3562417007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f828c102ea0 0x7f828c1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 --2- 172.21.15.103:0/3562417007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c102ac0 0x7f828c1000b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- 172.21.15.103:0/3562417007 >> 172.21.15.103:0/3562417007 conn(0x7f828c077210 msgr2=0x7f828c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- 172.21.15.103:0/3562417007 shutdown_connections 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- 172.21.15.103:0/3562417007 wait complete. 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 Processor -- start 2024-09-19T07:53:58.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- start start 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c100c10 0x7f828c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f828c102ac0 0x7f828c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f828c102ea0 0x7f828c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f828c114240 con 0x7f828c100c10 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1140c0 con 0x7f828c102ea0 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.576+0000 7f8292e0b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1143c0 con 0x7f828c102ac0 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c100c10 0x7f828c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c100c10 0x7f828c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58260/0 (socket says 172.21.15.103:58260) 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 -- 172.21.15.103:0/2266974783 learned_addr learned my addr 172.21.15.103:0/2266974783 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f828bfff640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f828c102ac0 0x7f828c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f82913a8640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f828c102ea0 0x7f828c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 -- 172.21.15.103:0/2266974783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f828c102ac0 msgr2=0x7f828c1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f828c102ac0 0x7f828c1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 -- 172.21.15.103:0/2266974783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f828c102ea0 msgr2=0x7f828c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f828c102ea0 0x7f828c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 -- 172.21.15.103:0/2266974783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c19ace0 con 0x7f828c100c10 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f828bfff640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f828c102ac0 0x7f828c1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f82913a8640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f828c102ea0 0x7f828c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8290ba7640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c100c10 0x7f828c1a03f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8284005ab0 tx=0x7f82840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8289ffb640 1 -- 172.21.15.103:0/2266974783 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828403f030 con 0x7f828c100c10 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8289ffb640 1 -- 172.21.15.103:0/2266974783 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8284003ce0 con 0x7f828c100c10 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828c19af70 con 0x7f828c100c10 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8289ffb640 1 -- 172.21.15.103:0/2266974783 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284035070 con 0x7f828c100c10 2024-09-19T07:53:58.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.577+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828c1a72c0 con 0x7f828c100c10 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.579+0000 7f8289ffb640 1 -- 172.21.15.103:0/2266974783 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f8284042030 con 0x7f828c100c10 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 Processor -- start 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 -- start start 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704521090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704521a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a70452e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a7044872f0 con 0x55a704522b40 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a70442ecf0 con 0x55a7045215d0 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82951e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a7044832b0 con 0x55a70452c370 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f8289ffb640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82780a33b0 0x7f82780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82913a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704521090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82913a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704521090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58270/0 (socket says 172.21.15.103:58270) 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82913a8640 1 -- 172.21.15.103:0/2859709994 learned_addr learned my addr 172.21.15.103:0/2859709994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f82913a8640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82780a33b0 0x7f82780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f8289ffb640 1 -- 172.21.15.103:0/2266974783 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f82840f0730 con 0x7f828c100c10 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f8290ba7640 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704521a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.580+0000 7f828bfff640 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a70452e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f82913a8640 1 -- 172.21.15.103:0/2859709994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 msgr2=0x55a70452e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f82913a8640 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a70452e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f82913a8640 1 -- 172.21.15.103:0/2859709994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 msgr2=0x55a704521a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f82913a8640 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704521a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f82913a8640 1 -- 172.21.15.103:0/2859709994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7044133a0 con 0x55a704522b40 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f828bfff640 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a70452e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:53:58.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f82913a8640 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704521090 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8274015760 tx=0x7f8274015c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a511b652476d6036 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f82913a8640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82780a33b0 0x7f82780a5870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8274013650 tx=0x7f827401a000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f8282ffd640 1 -- 172.21.15.103:0/2859709994 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827400f040 con 0x55a704522b40 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f8282ffd640 1 -- 172.21.15.103:0/2859709994 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827400c700 con 0x55a704522b40 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.581+0000 7f8282ffd640 1 -- 172.21.15.103:0/2859709994 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274020720 con 0x55a704522b40 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 -- 172.21.15.103:0/2859709994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 msgr2=0x55a704521090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704521090 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8274015760 tx=0x7f8274015c20 comp rx=0 tx=0).stop 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 -- 172.21.15.103:0/2859709994 shutdown_connections 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a70452e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704521a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 --2- 172.21.15.103:0/2859709994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704521090 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 -- 172.21.15.103:0/2859709994 >> 172.21.15.103:0/2859709994 conn(0x55a7044ea310 msgr2=0x55a70451dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 -- 172.21.15.103:0/2859709994 shutdown_connections 2024-09-19T07:53:58.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 -- 172.21.15.103:0/2859709994 wait complete. 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 -- start start 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.582+0000 7f82951e4740 1 Processor -- start 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704529050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a7045297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a70452f010 con 0x55a704522b40 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a704442060 con 0x55a7045215d0 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a70452f190 con 0x55a70452c370 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 10 client.0 _ll_register_callbacks cb 0x55a7041958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704529050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704529050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58282/0 (socket says 172.21.15.103:58282) 2024-09-19T07:53:58.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 -- 172.21.15.103:0/1813922133 learned_addr learned my addr 172.21.15.103:0/1813922133 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f8290ba7640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82913a8640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a7045297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 -- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 msgr2=0x55a7045297e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a7045297e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 -- 172.21.15.103:0/1813922133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 msgr2=0x55a70452bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 -- 172.21.15.103:0/1813922133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7041f83c0 con 0x55a704522b40 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82913a8640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a7045297e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f828bfff640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704529050 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f827c00aba0 tx=0x7f827c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9308607a49ccf546 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f826bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f826bfff640 1 -- 172.21.15.103:0/1813922133 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c01b060 con 0x55a704522b40 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f826bfff640 1 -- 172.21.15.103:0/1813922133 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f827c0165f0 con 0x55a704522b40 2024-09-19T07:53:58.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.583+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f827400c700 con 0x55a704522b40 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f826bfff640 1 -- 172.21.15.103:0/1813922133 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f827c0167f0 con 0x55a704522b40 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 10 client.18606 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a70453ab50 con 0x55a704522b40 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f826bfff640 1 -- 172.21.15.103:0/1813922133 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827400c700 con 0x55a704522b40 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f826bfff640 1 -- 172.21.15.103:0/1813922133 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f827c015070 con 0x55a704522b40 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 10 client.18606 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 10 client.18606 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 20 client.18606 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a70453c060 0x55a70453e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 4 client.18606 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a704494860 con 0x55a70453c060 2024-09-19T07:53:58.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.584+0000 7f8290ba7640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a70453c060 0x55a70453e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f826bfff640 10 client.18606 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f8290ba7640 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a70453c060 0x55a70453e490 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65aed14ed9a55cbc server_cookie=7f8fe497019323bd in_seq=0 out_seq=0 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f826bfff640 1 -- 172.21.15.103:0/1813922133 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a704494860 con 0x55a70453c060 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f826bfff640 10 client.18606 handle_command_reply: tid=0 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 client.18606 shutdown 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 msgr2=0x55a704529050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704529050 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f827c00aba0 tx=0x7f827c007680 comp rx=0 tx=0).stop 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 shutdown_connections 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a70453c060 0x55a70453e490 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a7045297e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704529050 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 --2- 172.21.15.103:0/1813922133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f826bfff640 0 client.18606 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.585+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 >> 172.21.15.103:0/1813922133 conn(0x55a7044ea310 msgr2=0x55a704523960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.586+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 shutdown_connections 2024-09-19T07:53:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.586+0000 7f82951e4740 1 -- 172.21.15.103:0/1813922133 wait complete. 2024-09-19T07:53:58.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.586+0000 7f82951e4740 20 client.18606 trim_cache size 0 max 16384 2024-09-19T07:53:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.598+0000 7f82951e4740 1 Processor -- start 2024-09-19T07:53:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.598+0000 7f82951e4740 1 -- start start 2024-09-19T07:53:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82951e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82951e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704528910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82951e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a704543da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82951e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1afbd0 con 0x55a704522b40 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82951e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1b1710 con 0x55a7045215d0 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82951e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1b02a0 con 0x55a70452c370 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704528910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704528910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58288/0 (socket says 172.21.15.103:58288) 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 -- 172.21.15.103:0/495974824 learned_addr learned my addr 172.21.15.103:0/495974824 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f8290ba7640 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82913a8640 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a704543da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 -- 172.21.15.103:0/495974824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 msgr2=0x55a704543da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a704543da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 -- 172.21.15.103:0/495974824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 msgr2=0x55a70452bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 -- 172.21.15.103:0/495974824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a704529b50 con 0x55a704522b40 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f828bfff640 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704528910 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f827c000f80 tx=0x7f827c00a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdc633af47e9d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82827fc640 1 -- 172.21.15.103:0/495974824 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0165f0 con 0x55a704522b40 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82827fc640 1 -- 172.21.15.103:0/495974824 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827c0156a0 con 0x55a704522b40 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.599+0000 7f82827fc640 1 -- 172.21.15.103:0/495974824 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c01f680 con 0x55a704522b40 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 -- 172.21.15.103:0/495974824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 msgr2=0x55a704528910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704528910 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f827c000f80 tx=0x7f827c00a840 comp rx=0 tx=0).stop 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 -- 172.21.15.103:0/495974824 shutdown_connections 2024-09-19T07:53:58.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a70452c370 0x55a704543da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a704522b40 0x55a704528910 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 --2- 172.21.15.103:0/495974824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a70452bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 -- 172.21.15.103:0/495974824 >> 172.21.15.103:0/495974824 conn(0x55a7044ea310 msgr2=0x55a704527920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 -- 172.21.15.103:0/495974824 shutdown_connections 2024-09-19T07:53:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.600+0000 7f82951e4740 1 -- 172.21.15.103:0/495974824 wait complete. 2024-09-19T07:53:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 -- start start 2024-09-19T07:53:58.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 Processor -- start 2024-09-19T07:53:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704534920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a704522b40 0x55a70452fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a70452c370 0x55a7045301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1b1040 con 0x55a70452c370 2024-09-19T07:53:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1b0970 con 0x55a7045215d0 2024-09-19T07:53:58.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1adbd0 con 0x55a704522b40 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82951e4740 10 client.0 _ll_register_callbacks cb 0x7f827c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82913a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a70452c370 0x55a7045301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82913a8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a70452c370 0x55a7045301e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58304/0 (socket says 172.21.15.103:58304) 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f82913a8640 1 -- 172.21.15.103:0/4007796575 learned_addr learned my addr 172.21.15.103:0/4007796575 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f8290ba7640 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704534920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.601+0000 7f828bfff640 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a704522b40 0x55a70452fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82913a8640 1 -- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a704522b40 msgr2=0x55a70452fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82913a8640 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a704522b40 0x55a70452fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82913a8640 1 -- 172.21.15.103:0/4007796575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 msgr2=0x55a704534920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82913a8640 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704534920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82913a8640 1 -- 172.21.15.103:0/4007796575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7045307a0 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82913a8640 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a70452c370 0x55a7045301e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f827400cf00 tx=0x7f827400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d446608f5fd7fde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f826bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f826bfff640 1 -- 172.21.15.103:0/4007796575 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274030070 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f826bfff640 1 -- 172.21.15.103:0/4007796575 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7f8274020600 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f827c0156a0 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f826bfff640 1 -- 172.21.15.103:0/4007796575 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f827403c050 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82951e4740 10 client.18612 fetch_fsmap learned FSMap version 54 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a7045af330 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f826bfff640 1 -- 172.21.15.103:0/4007796575 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827c0156a0 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.602+0000 7f826bfff640 1 -- 172.21.15.103:0/4007796575 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7f8274018c00 con 0x55a70452c370 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f82951e4740 10 client.18612 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:53:58.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f82951e4740 10 client.18612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f82951e4740 20 client.18612 populate_metadata read hostname 'smithi103' 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f82951e4740 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a7045af710 0x55a7045b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f82951e4740 4 client.18612 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a70448ed70 con 0x55a7045af710 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f8290ba7640 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a7045af710 0x55a7045b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f8290ba7640 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a7045af710 0x55a7045b1b30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20244b41dd00e9b3 server_cookie=e8ad5f847a022a9b in_seq=0 out_seq=0 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f826bfff640 10 client.18612 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f826bfff640 1 -- 172.21.15.103:0/4007796575 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a70448ed70 con 0x55a7045af710 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f826bfff640 10 client.18612 handle_command_reply: tid=0 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.603+0000 7f82951e4740 1 client.18612 shutdown 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a70452c370 msgr2=0x55a7045301e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a70452c370 0x55a7045301e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f827400cf00 tx=0x7f827400cf30 comp rx=0 tx=0).stop 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 shutdown_connections 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x55a7045af710 0x55a7045b1b30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a70452c370 0x55a7045301e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f826bfff640 0 client.18612 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a704522b40 0x55a70452fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 --2- 172.21.15.103:0/4007796575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7045215d0 0x55a704534920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 >> 172.21.15.103:0/4007796575 conn(0x55a7044ea310 msgr2=0x55a70452b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 shutdown_connections 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 1 -- 172.21.15.103:0/4007796575 wait complete. 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f82951e4740 20 client.18612 trim_cache size 0 max 16384 2024-09-19T07:53:58.610 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:53:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82780a33b0 msgr2=0x7f82780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82780a33b0 0x7f82780a5870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8274013650 tx=0x7f827401a000 comp rx=0 tx=0).stop 2024-09-19T07:53:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c100c10 msgr2=0x7f828c1a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:53:58.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c100c10 0x7f828c1a03f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8284005ab0 tx=0x7f82840042f0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 shutdown_connections 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82780a33b0 0x7f82780a5870 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f828c102ea0 0x7f828c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f828c102ac0 0x7f828c1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 --2- 172.21.15.103:0/2266974783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f828c100c10 0x7f828c1a03f0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.604+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 >> 172.21.15.103:0/2266974783 conn(0x7f828c077210 msgr2=0x7f828c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.605+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 shutdown_connections 2024-09-19T07:53:58.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:53:58.605+0000 7f8292e0b640 1 -- 172.21.15.103:0/2266974783 wait complete. 2024-09-19T07:53:58.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:53:58 smithi174 ceph-mon[35370]: pgmap v500: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.3 MiB/s wr, 785 op/s 2024-09-19T07:53:59.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:53:59.693 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 -- 172.21.15.103:0/1881333761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954101420 msgr2=0x7fa95410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 --2- 172.21.15.103:0/1881333761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954101420 0x7fa95410d950 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009cd0 tx=0x7fa94c02a8e0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 -- 172.21.15.103:0/1881333761 shutdown_connections 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 --2- 172.21.15.103:0/1881333761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954101420 0x7fa95410d950 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 --2- 172.21.15.103:0/1881333761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954100a80 0x7fa954100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 --2- 172.21.15.103:0/1881333761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa954106a80 0x7fa954106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 -- 172.21.15.103:0/1881333761 >> 172.21.15.103:0/1881333761 conn(0x7fa9540fc7a0 msgr2=0x7fa9540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 -- 172.21.15.103:0/1881333761 shutdown_connections 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.164+0000 7fa95afbd640 1 -- 172.21.15.103:0/1881333761 wait complete. 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 Processor -- start 2024-09-19T07:54:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 -- start start 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954100a80 0x7fa954197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa954101420 0x7fa954197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954106a80 0x7fa95419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa954110e80 con 0x7fa954106a80 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa954110d00 con 0x7fa954101420 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95afbd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111000 con 0x7fa954100a80 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95955a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954106a80 0x7fa95419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95955a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954106a80 0x7fa95419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58332/0 (socket says 172.21.15.103:58332) 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa95955a640 1 -- 172.21.15.103:0/3076660648 learned_addr learned my addr 172.21.15.103:0/3076660648 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.165+0000 7fa958d59640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954100a80 0x7fa954197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa953fff640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa954101420 0x7fa954197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95955a640 1 -- 172.21.15.103:0/3076660648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954100a80 msgr2=0x7fa954197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95955a640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954100a80 0x7fa954197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95955a640 1 -- 172.21.15.103:0/3076660648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa954101420 msgr2=0x7fa954197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95955a640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa954101420 0x7fa954197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95955a640 1 -- 172.21.15.103:0/3076660648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95419c2c0 con 0x7fa954106a80 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa958d59640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954100a80 0x7fa954197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95955a640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954106a80 0x7fa95419bbe0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005cc0 tx=0x7fa94c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa951ffb640 1 -- 172.21.15.103:0/3076660648 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004040 con 0x7fa954106a80 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95419c550 con 0x7fa954106a80 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa951ffb640 1 -- 172.21.15.103:0/3076660648 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94c0046c0 con 0x7fa954106a80 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa951ffb640 1 -- 172.21.15.103:0/3076660648 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c046480 con 0x7fa954106a80 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.166+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a3e90 con 0x7fa954106a80 2024-09-19T07:54:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa951ffb640 1 -- 172.21.15.103:0/3076660648 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa94c0305a0 con 0x7fa954106a80 2024-09-19T07:54:00.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 Processor -- start 2024-09-19T07:54:00.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 -- start start 2024-09-19T07:54:00.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5ab62f0 con 0x55c9e5b505d0 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5a5dcf0 con 0x55c9e5b51b40 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa95d396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5ab22b0 con 0x55c9e5b5d580 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa958d59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa958d59640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b50a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58344/0 (socket says 172.21.15.103:58344) 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa958d59640 1 -- 172.21.15.103:0/2878129065 learned_addr learned my addr 172.21.15.103:0/2878129065 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa951ffb640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa9440a33b0 0x7fa9440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa953fff640 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.169+0000 7fa953fff640 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa958d59640 1 -- 172.21.15.103:0/2878129065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b5d580 msgr2=0x55c9e5b5f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa958d59640 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa958d59640 1 -- 172.21.15.103:0/2878129065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 msgr2=0x55c9e5b50090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa95955a640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa9440a33b0 0x7fa9440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa958d59640 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b50090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa958d59640 1 -- 172.21.15.103:0/2878129065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e5a423a0 con 0x55c9e5b505d0 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa953fff640 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa951ffb640 1 -- 172.21.15.103:0/3076660648 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fa94c02e030 con 0x7fa954106a80 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa958d59640 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b50a10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa94800bcc0 tx=0x7fa94800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821eeede712ef72f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa93effd640 1 -- 172.21.15.103:0/2878129065 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948015070 con 0x55c9e5b505d0 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa93effd640 1 -- 172.21.15.103:0/2878129065 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94801a940 con 0x55c9e5b505d0 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.170+0000 7fa93effd640 1 -- 172.21.15.103:0/2878129065 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948011a80 con 0x55c9e5b505d0 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 -- 172.21.15.103:0/2878129065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 msgr2=0x55c9e5b50a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95955a640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa9440a33b0 0x7fa9440a5870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0ef250 tx=0x7fa94c0a8040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b50a10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa94800bcc0 tx=0x7fa94800f730 comp rx=0 tx=0).stop 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 -- 172.21.15.103:0/2878129065 shutdown_connections 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b50a10 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 --2- 172.21.15.103:0/2878129065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b50090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 -- 172.21.15.103:0/2878129065 >> 172.21.15.103:0/2878129065 conn(0x55c9e5b19310 msgr2=0x55c9e5b4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 -- 172.21.15.103:0/2878129065 shutdown_connections 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 -- 172.21.15.103:0/2878129065 wait complete. 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 -- start start 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.171+0000 7fa95d396740 1 Processor -- start 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa95d396740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b505d0 0x55c9e5b5cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa95d396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b5a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa95d396740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa95d396740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5b60280 con 0x55c9e5b5d580 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa95d396740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5a71060 con 0x55c9e5b505d0 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa95d396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5b60400 con 0x55c9e5b51b40 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa95d396740 10 client.0 _ll_register_callbacks cb 0x55c9e57c48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58348/0 (socket says 172.21.15.103:58348) 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 -- 172.21.15.103:0/2028476112 learned_addr learned my addr 172.21.15.103:0/2028476112 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b505d0 0x55c9e5b5cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa958d59640 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b5a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 -- 172.21.15.103:0/2028476112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 msgr2=0x55c9e5b5a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b5a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 -- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b505d0 msgr2=0x55c9e5b5cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b505d0 0x55c9e5b5cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 -- 172.21.15.103:0/2028476112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9e58273c0 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa953fff640 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5a810 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa9400058c0 tx=0x7fa9400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7059e9ec802d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa933fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.172+0000 7fa933fff640 1 -- 172.21.15.103:0/2028476112 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940004020 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa933fff640 1 -- 172.21.15.103:0/2028476112 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fa9400041c0 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa94801a940 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa933fff640 1 -- 172.21.15.103:0/2028476112 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa940025050 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 10 client.18624 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c9e5b6bdd0 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa933fff640 1 -- 172.21.15.103:0/2028476112 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94801a940 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa933fff640 1 -- 172.21.15.103:0/2028476112 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fa9400104c0 con 0x55c9e5b5d580 2024-09-19T07:54:00.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 10 client.18624 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 10 client.18624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 20 client.18624 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5b6d290 0x55c9e5b6f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 4 client.18624 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.173+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9e5ac3860 con 0x55c9e5b6d290 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.174+0000 7fa958d59640 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5b6d290 0x55c9e5b6f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.174+0000 7fa933fff640 10 client.18624 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.174+0000 7fa958d59640 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5b6d290 0x55c9e5b6f6c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3c6a3a5fd4fdc0 server_cookie=dac1d3a6eb591427 in_seq=0 out_seq=0 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa933fff640 1 -- 172.21.15.103:0/2028476112 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9e5ac3860 con 0x55c9e5b6d290 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa933fff640 10 client.18624 handle_command_reply: tid=0 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 client.18624 shutdown 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 msgr2=0x55c9e5b5a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5a810 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa9400058c0 tx=0x7fa9400046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 shutdown_connections 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5b6d290 0x55c9e5b6f6c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b5a810 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa933fff640 0 client.18624 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 --2- 172.21.15.103:0/2028476112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b505d0 0x55c9e5b5cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 >> 172.21.15.103:0/2028476112 conn(0x55c9e5b19310 msgr2=0x55c9e5b52990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 shutdown_connections 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 1 -- 172.21.15.103:0/2028476112 wait complete. 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.175+0000 7fa95d396740 20 client.18624 trim_cache size 0 max 16384 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 Processor -- start 2024-09-19T07:54:00.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 -- start start 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b505d0 0x55c9e5b4ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b69520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541ab630 con 0x55c9e5b5d580 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541ad170 con 0x55c9e5b51b40 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa95d396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541abd00 con 0x55c9e5b505d0 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa953fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa953fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b74f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58364/0 (socket says 172.21.15.103:58364) 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa958d59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b69520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.188+0000 7fa953fff640 1 -- 172.21.15.103:0/1607426420 learned_addr learned my addr 172.21.15.103:0/1607426420 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:00.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa953fff640 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b505d0 0x55c9e5b4ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa958d59640 1 -- 172.21.15.103:0/1607426420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b505d0 msgr2=0x55c9e5b4ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa958d59640 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b505d0 0x55c9e5b4ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa958d59640 1 -- 172.21.15.103:0/1607426420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 msgr2=0x55c9e5b74f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa958d59640 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b74f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa958d59640 1 -- 172.21.15.103:0/1607426420 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e5b53120 con 0x55c9e5b51b40 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa958d59640 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b69520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa948000c00 tx=0x7fa948011680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c23976ab8df374 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa93e7fc640 1 -- 172.21.15.103:0/1607426420 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948015040 con 0x55c9e5b51b40 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa93e7fc640 1 -- 172.21.15.103:0/1607426420 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa948005040 con 0x55c9e5b51b40 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.189+0000 7fa93e7fc640 1 -- 172.21.15.103:0/1607426420 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480048e0 con 0x55c9e5b51b40 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 -- 172.21.15.103:0/1607426420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 msgr2=0x55c9e5b69520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b69520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa948000c00 tx=0x7fa948011680 comp rx=0 tx=0).stop 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 -- 172.21.15.103:0/1607426420 shutdown_connections 2024-09-19T07:54:00.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b5d580 0x55c9e5b74f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b51b40 0x55c9e5b69520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 --2- 172.21.15.103:0/1607426420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b505d0 0x55c9e5b4ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 -- 172.21.15.103:0/1607426420 >> 172.21.15.103:0/1607426420 conn(0x55c9e5b19310 msgr2=0x55c9e5b4c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 -- 172.21.15.103:0/1607426420 shutdown_connections 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 -- 172.21.15.103:0/1607426420 wait complete. 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 -- start start 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.190+0000 7fa95d396740 1 Processor -- start 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa95d396740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b65690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa95d396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b65bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa95d396740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b5d580 0x55c9e5b62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa95d396740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541acaa0 con 0x55c9e5b505d0 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa95d396740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541ac3d0 con 0x55c9e5b5d580 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa95d396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9541a9630 con 0x55c9e5b51b40 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa95d396740 10 client.0 _ll_register_callbacks cb 0x7fa94000eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b65690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b65690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58380/0 (socket says 172.21.15.103:58380) 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 -- 172.21.15.103:0/3581985368 learned_addr learned my addr 172.21.15.103:0/3581985368 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b5d580 0x55c9e5b62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa958d59640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b65bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 -- 172.21.15.103:0/3581985368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 msgr2=0x55c9e5b65bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b65bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 -- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b5d580 msgr2=0x55c9e5b62660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b5d580 0x55c9e5b62660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 -- 172.21.15.103:0/3581985368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9e5b62ba0 con 0x55c9e5b505d0 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa958d59640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b65bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa953fff640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b65690 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa940001eb0 tx=0x7fa94000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51aea0abf8e6134 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa933fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.191+0000 7fa933fff640 1 -- 172.21.15.103:0/3581985368 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940010640 con 0x55c9e5b505d0 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa948005040 con 0x55c9e5b505d0 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa933fff640 1 -- 172.21.15.103:0/3581985368 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fa940010ab0 con 0x55c9e5b505d0 2024-09-19T07:54:00.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa933fff640 1 -- 172.21.15.103:0/3581985368 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa940010cb0 con 0x55c9e5b505d0 2024-09-19T07:54:00.297 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:54:00.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 10 client.18630 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c9e5bd8610 con 0x55c9e5b505d0 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa933fff640 1 -- 172.21.15.103:0/3581985368 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa948005040 con 0x55c9e5b505d0 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa933fff640 1 -- 172.21.15.103:0/3581985368 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fa940003a30 con 0x55c9e5b505d0 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 10 client.18630 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 10 client.18630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 20 client.18630 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5bd8c70 0x55c9e5bdb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 4 client.18630 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9e5abdd70 con 0x55c9e5bd8c70 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.192+0000 7fa958d59640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5bd8c70 0x55c9e5bdb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa958d59640 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5bd8c70 0x55c9e5bdb060 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe701d04eb489bc server_cookie=ddc3440b864701c9 in_seq=0 out_seq=0 2024-09-19T07:54:00.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa933fff640 10 client.18630 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa933fff640 1 -- 172.21.15.103:0/3581985368 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9e5abdd70 con 0x55c9e5bd8c70 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa933fff640 10 client.18630 handle_command_reply: tid=0 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 client.18630 shutdown 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 msgr2=0x55c9e5b65690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b65690 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa940001eb0 tx=0x7fa94000ec70 comp rx=0 tx=0).stop 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 shutdown_connections 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55c9e5bd8c70 0x55c9e5bdb060 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9e5b5d580 0x55c9e5b62660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9e5b51b40 0x55c9e5b65bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa933fff640 0 client.18630 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 --2- 172.21.15.103:0/3581985368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c9e5b505d0 0x55c9e5b65690 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 >> 172.21.15.103:0/3581985368 conn(0x55c9e5b19310 msgr2=0x55c9e5b5ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:00.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 shutdown_connections 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 1 -- 172.21.15.103:0/3581985368 wait complete. 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.193+0000 7fa95d396740 20 client.18630 trim_cache size 0 max 16384 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa9440a33b0 msgr2=0x7fa9440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa9440a33b0 0x7fa9440a5870 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0ef250 tx=0x7fa94c0a8040 comp rx=0 tx=0).stop 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954106a80 msgr2=0x7fa95419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954106a80 0x7fa95419bbe0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005cc0 tx=0x7fa94c003970 comp rx=0 tx=0).stop 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 shutdown_connections 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa9440a33b0 0x7fa9440a5870 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa954106a80 0x7fa95419bbe0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa954101420 0x7fa954197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 --2- 172.21.15.103:0/3076660648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954100a80 0x7fa954197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:00.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 >> 172.21.15.103:0/3076660648 conn(0x7fa9540fc7a0 msgr2=0x7fa9540fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 shutdown_connections 2024-09-19T07:54:00.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:00.194+0000 7fa95afbd640 1 -- 172.21.15.103:0/3076660648 wait complete. 2024-09-19T07:54:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:00 smithi103 ceph-mon[31201]: pgmap v501: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-09-19T07:54:00.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:00 smithi134 ceph-mon[35414]: pgmap v501: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-09-19T07:54:00.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:00 smithi174 ceph-mon[35370]: pgmap v501: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-09-19T07:54:02.280 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T07:54:02.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T07:54:02.547 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:02.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:02 smithi134 ceph-mon[35414]: pgmap v502: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-09-19T07:54:02.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:02 smithi174 ceph-mon[35370]: pgmap v502: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-09-19T07:54:02.668 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:02 smithi103 ceph-mon[31201]: pgmap v502: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-09-19T07:54:03.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 -- 172.21.15.103:0/4054079981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 msgr2=0x7fa84010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 --2- 172.21.15.103:0/4054079981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa84010eaf0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa838009cd0 tx=0x7fa83802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 -- 172.21.15.103:0/4054079981 shutdown_connections 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 --2- 172.21.15.103:0/4054079981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa84010eaf0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 --2- 172.21.15.103:0/4054079981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840100d00 0x7fa840106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 --2- 172.21.15.103:0/4054079981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa840100350 0x7fa840100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 -- 172.21.15.103:0/4054079981 >> 172.21.15.103:0/4054079981 conn(0x7fa840077210 msgr2=0x7fa8400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 -- 172.21.15.103:0/4054079981 shutdown_connections 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.050+0000 7fa8486bf640 1 -- 172.21.15.103:0/4054079981 wait complete. 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 Processor -- start 2024-09-19T07:54:03.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 -- start start 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa840100350 0x7fa8401012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840100d00 0x7fa840101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa840101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa840112020 con 0x7fa840106f50 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa840111ea0 con 0x7fa840100350 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa8486bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401121a0 con 0x7fa840100d00 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa840101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa840101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58402/0 (socket says 172.21.15.103:58402) 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 -- 172.21.15.103:0/1485813803 learned_addr learned my addr 172.21.15.103:0/1485813803 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa845c5a640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840100d00 0x7fa840101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa84645b640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa840100350 0x7fa8401012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 -- 172.21.15.103:0/1485813803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840100d00 msgr2=0x7fa840101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840100d00 0x7fa840101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 -- 172.21.15.103:0/1485813803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa840100350 msgr2=0x7fa8401012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa840100350 0x7fa8401012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.051+0000 7fa846c5c640 1 -- 172.21.15.103:0/1485813803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8401a69c0 con 0x7fa840106f50 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.052+0000 7fa846c5c640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa840101d50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa838005cc0 tx=0x7fa838003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.052+0000 7fa8377fe640 1 -- 172.21.15.103:0/1485813803 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838004040 con 0x7fa840106f50 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.052+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8401a6c50 con 0x7fa840106f50 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.052+0000 7fa8377fe640 1 -- 172.21.15.103:0/1485813803 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8380046c0 con 0x7fa840106f50 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.052+0000 7fa8377fe640 1 -- 172.21.15.103:0/1485813803 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838046480 con 0x7fa840106f50 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.052+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8401a71e0 con 0x7fa840106f50 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.054+0000 7fa8377fe640 1 -- 172.21.15.103:0/1485813803 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa8380305a0 con 0x7fa840106f50 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.054+0000 7fa84aa98740 1 Processor -- start 2024-09-19T07:54:03.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84aa98740 1 -- start start 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d26b60 0x55e257d26f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d31800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d31d40 0x55e257d34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84aa98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e257c8c280 con 0x55e257d31d40 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84aa98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e257c33b10 con 0x55e257d27480 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84aa98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e257c88240 con 0x55e257d26b60 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa8377fe640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa82c0a33b0 0x7fa82c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d31d40 0x55e257d34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d31d40 0x55e257d34130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58418/0 (socket says 172.21.15.103:58418) 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 -- 172.21.15.103:0/3891627808 learned_addr learned my addr 172.21.15.103:0/3891627808 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:03.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa846c5c640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa82c0a33b0 0x7fa82c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa84645b640 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d31800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d26b60 0x55e257d26f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa8377fe640 1 -- 172.21.15.103:0/1485813803 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fa83802e030 con 0x7fa840106f50 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 -- 172.21.15.103:0/3891627808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d26b60 msgr2=0x55e257d26f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d26b60 0x55e257d26f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 -- 172.21.15.103:0/3891627808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 msgr2=0x55e257d31800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d31800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.055+0000 7fa845c5a640 1 -- 172.21.15.103:0/3891627808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e257c401f0 con 0x55e257d31d40 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.056+0000 7fa846c5c640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa82c0a33b0 0x7fa82c0a5870 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8380eed40 tx=0x7fa8380a8040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:03.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.056+0000 7fa845c5a640 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d31d40 0x55e257d34130 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa830005880 tx=0x7fa83000b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a52625ec45e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.056+0000 7fa834ff9640 1 -- 172.21.15.103:0/3891627808 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004030 con 0x55e257d31d40 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.056+0000 7fa834ff9640 1 -- 172.21.15.103:0/3891627808 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa830005500 con 0x55e257d31d40 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.056+0000 7fa834ff9640 1 -- 172.21.15.103:0/3891627808 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004bc0 con 0x55e257d31d40 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.056+0000 7fa84aa98740 1 -- 172.21.15.103:0/3891627808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d31d40 msgr2=0x55e257d34130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.056+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d31d40 0x55e257d34130 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa830005880 tx=0x7fa83000b9f0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- 172.21.15.103:0/3891627808 shutdown_connections 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d31d40 0x55e257d34130 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d31800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3891627808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d26b60 0x55e257d26f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- 172.21.15.103:0/3891627808 >> 172.21.15.103:0/3891627808 conn(0x55e257d230f0 msgr2=0x55e257d234d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- 172.21.15.103:0/3891627808 shutdown_connections 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- 172.21.15.103:0/3891627808 wait complete. 2024-09-19T07:54:03.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- start start 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 Processor -- start 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d2ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d27480 0x55e257d2f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d31d40 0x55e257d2f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e257d34910 con 0x55e257d26b60 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e257c46f10 con 0x55e257d31d40 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e257d34a90 con 0x55e257d27480 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa84aa98740 10 client.0 _ll_register_callbacks cb 0x55e257c0fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d2ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d2ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58422/0 (socket says 172.21.15.103:58422) 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.057+0000 7fa845c5a640 1 -- 172.21.15.103:0/940068816 learned_addr learned my addr 172.21.15.103:0/940068816 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:03.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa84645b640 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d27480 0x55e257d2f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa845c5a640 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d31d40 0x55e257d2f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa845c5a640 1 -- 172.21.15.103:0/940068816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d27480 msgr2=0x55e257d2f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa845c5a640 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d27480 0x55e257d2f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa845c5a640 1 -- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d31d40 msgr2=0x55e257d2f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa845c5a640 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d31d40 0x55e257d2f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa845c5a640 1 -- 172.21.15.103:0/940068816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2579fd6a0 con 0x55e257d26b60 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa845c5a640 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d2ec20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b3f0 tx=0x7fa830004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343037af426fa010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa829ffb640 1 -- 172.21.15.103:0/940068816 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004b40 con 0x55e257d26b60 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa829ffb640 1 -- 172.21.15.103:0/940068816 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fa830004ce0 con 0x55e257d26b60 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa830005500 con 0x55e257d26b60 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa829ffb640 1 -- 172.21.15.103:0/940068816 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa830035050 con 0x55e257d26b60 2024-09-19T07:54:03.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa84aa98740 10 client.18642 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e257d40540 con 0x55e257d26b60 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.058+0000 7fa829ffb640 1 -- 172.21.15.103:0/940068816 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa830005500 con 0x55e257d26b60 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa829ffb640 1 -- 172.21.15.103:0/940068816 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fa83001f580 con 0x55e257d26b60 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84aa98740 10 client.18642 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84aa98740 10 client.18642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84aa98740 20 client.18642 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84aa98740 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257d418f0 0x55e257d43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84aa98740 4 client.18642 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:03.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e257c997f0 con 0x55e257d418f0 2024-09-19T07:54:03.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84645b640 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257d418f0 0x55e257d43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa829ffb640 10 client.18642 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:03.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.059+0000 7fa84645b640 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257d418f0 0x55e257d43ce0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f6b461a3a1831 server_cookie=6b9bf3a868c55ed5 in_seq=0 out_seq=0 2024-09-19T07:54:03.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa829ffb640 1 -- 172.21.15.103:0/940068816 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e257c997f0 con 0x55e257d418f0 2024-09-19T07:54:03.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa829ffb640 10 client.18642 handle_command_reply: tid=0 2024-09-19T07:54:03.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 client.18642 shutdown 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 msgr2=0x55e257d2ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d2ec20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b3f0 tx=0x7fa830004840 comp rx=0 tx=0).stop 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 shutdown_connections 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257d418f0 0x55e257d43ce0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d31d40 0x55e257d2f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa829ffb640 0 client.18642 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d27480 0x55e257d2f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 --2- 172.21.15.103:0/940068816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d2ec20 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 >> 172.21.15.103:0/940068816 conn(0x55e257d230f0 msgr2=0x55e257d28dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:03.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 shutdown_connections 2024-09-19T07:54:03.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.060+0000 7fa84aa98740 1 -- 172.21.15.103:0/940068816 wait complete. 2024-09-19T07:54:03.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.061+0000 7fa84aa98740 20 client.18642 trim_cache size 0 max 16384 2024-09-19T07:54:03.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.077+0000 7fa84aa98740 1 Processor -- start 2024-09-19T07:54:03.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.077+0000 7fa84aa98740 1 -- start start 2024-09-19T07:54:03.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d30e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 0x55e257d49a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa84aa98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ab760 con 0x55e257d26b60 2024-09-19T07:54:03.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa84aa98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ad2a0 con 0x55e257d27480 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa84aa98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401abe30 con 0x55e257d31d40 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d30e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d30e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58426/0 (socket says 172.21.15.103:58426) 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 -- 172.21.15.103:0/3862755101 learned_addr learned my addr 172.21.15.103:0/3862755101 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa84645b640 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 -- 172.21.15.103:0/3862755101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 msgr2=0x55e257d49a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 0x55e257d49a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 -- 172.21.15.103:0/3862755101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 msgr2=0x55e257d494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 -- 172.21.15.103:0/3862755101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e257d44050 con 0x55e257d26b60 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa845c5a640 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d30e40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa830005f90 tx=0x7fa83000b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca8268314533b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa82bfff640 1 -- 172.21.15.103:0/3862755101 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83003e070 con 0x55e257d26b60 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa82bfff640 1 -- 172.21.15.103:0/3862755101 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa830029040 con 0x55e257d26b60 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.078+0000 7fa82bfff640 1 -- 172.21.15.103:0/3862755101 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004cd0 con 0x55e257d26b60 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 -- 172.21.15.103:0/3862755101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 msgr2=0x55e257d30e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d30e40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa830005f90 tx=0x7fa83000b9f0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 -- 172.21.15.103:0/3862755101 shutdown_connections 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 0x55e257d49a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 --2- 172.21.15.103:0/3862755101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d30e40 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 -- 172.21.15.103:0/3862755101 >> 172.21.15.103:0/3862755101 conn(0x55e257d230f0 msgr2=0x55e257d28dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 -- 172.21.15.103:0/3862755101 shutdown_connections 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 -- 172.21.15.103:0/3862755101 wait complete. 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 -- start start 2024-09-19T07:54:03.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.079+0000 7fa84aa98740 1 Processor -- start 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d35b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d36070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 0x55e257d365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401acbd0 con 0x55e257d26b60 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ac500 con 0x55e257d27480 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401a9780 con 0x55e257d31d40 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 10 client.0 _ll_register_callbacks cb 0x7fa83c00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:03.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d35b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d35b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58442/0 (socket says 172.21.15.103:58442) 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 -- 172.21.15.103:0/157063958 learned_addr learned my addr 172.21.15.103:0/157063958 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84645b640 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 0x55e257d365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 -- 172.21.15.103:0/157063958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 msgr2=0x55e257d365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 0x55e257d365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 -- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 msgr2=0x55e257d36070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d36070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 -- 172.21.15.103:0/157063958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e257dab510 con 0x55e257d26b60 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa845c5a640 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d35b30 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa8300057c0 tx=0x7fa830005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4790b5d5bd18bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:03.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa829ffb640 1 -- 172.21.15.103:0/157063958 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83003e040 con 0x55e257d26b60 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa829ffb640 1 -- 172.21.15.103:0/157063958 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8542+0+0 (secure 0 0 0) 0x7fa83001f660 con 0x55e257d26b60 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.080+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa830029040 con 0x55e257d26b60 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa829ffb640 1 -- 172.21.15.103:0/157063958 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa830004a50 con 0x55e257d26b60 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 10 client.18648 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e257dac830 con 0x55e257d26b60 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa829ffb640 1 -- 172.21.15.103:0/157063958 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa830029040 con 0x55e257d26b60 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa829ffb640 1 -- 172.21.15.103:0/157063958 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fa830010740 con 0x55e257d26b60 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 10 client.18648 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 10 client.18648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 20 client.18648 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257dace90 0x55e257daf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:03.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 4 client.18648 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T07:54:03.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e257c93d00 con 0x55e257dace90 2024-09-19T07:54:03.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.081+0000 7fa84645b640 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257dace90 0x55e257daf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:03.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.082+0000 7fa84645b640 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257dace90 0x55e257daf280 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ed96e8c0e8487a server_cookie=5f74d0f75eb4768e in_seq=0 out_seq=0 2024-09-19T07:54:03.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.082+0000 7fa829ffb640 10 client.18648 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:03.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.083+0000 7fa829ffb640 1 -- 172.21.15.103:0/157063958 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e257c93d00 con 0x55e257dace90 2024-09-19T07:54:03.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.083+0000 7fa829ffb640 10 client.18648 handle_command_reply: tid=0 2024-09-19T07:54:03.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.083+0000 7fa84aa98740 1 client.18648 shutdown 2024-09-19T07:54:03.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.083+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 msgr2=0x55e257d35b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.083+0000 7fa84aa98740 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d35b30 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa8300057c0 tx=0x7fa830005c10 comp rx=0 tx=0).stop 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.083+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 shutdown_connections 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55e257dace90 0x55e257daf280 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e257d31d40 0x55e257d365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e257d27480 0x55e257d36070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 1 --2- 172.21.15.103:0/157063958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e257d26b60 0x55e257d35b30 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 >> 172.21.15.103:0/157063958 conn(0x55e257d230f0 msgr2=0x55e257d22270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa829ffb640 0 client.18648 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:03.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 shutdown_connections 2024-09-19T07:54:03.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 1 -- 172.21.15.103:0/157063958 wait complete. 2024-09-19T07:54:03.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa84aa98740 20 client.18648 trim_cache size 0 max 16384 2024-09-19T07:54:03.093 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:03.093 INFO:teuthology.orchestra.run.smithi103.stdout: "return_code": 0, 2024-09-19T07:54:03.093 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_tag": "d42807b4-7243-4b7e-8c6c-6187c298f673", 2024-09-19T07:54:03.094 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": "asynchronous" 2024-09-19T07:54:03.094 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:03.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa82c0a33b0 msgr2=0x7fa82c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa82c0a33b0 0x7fa82c0a5870 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8380eed40 tx=0x7fa8380a8040 comp rx=0 tx=0).stop 2024-09-19T07:54:03.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 msgr2=0x7fa840101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:03.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa840101d50 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa838005cc0 tx=0x7fa838003970 comp rx=0 tx=0).stop 2024-09-19T07:54:03.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 shutdown_connections 2024-09-19T07:54:03.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa82c0a33b0 0x7fa82c0a5870 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa840106f50 0x7fa840101d50 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840100d00 0x7fa840101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 --2- 172.21.15.103:0/1485813803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa840100350 0x7fa8401012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:03.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 >> 172.21.15.103:0/1485813803 conn(0x7fa840077210 msgr2=0x7fa8401051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:03.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 shutdown_connections 2024-09-19T07:54:03.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:03.084+0000 7fa8486bf640 1 -- 172.21.15.103:0/1485813803 wait complete. 2024-09-19T07:54:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:03 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:03.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:03 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:03 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:03.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:03 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:03.694 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:03 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:03.694 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:03 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:03.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:54:04.168 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:04.573 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: pgmap v503: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 861 KiB/s wr, 95 op/s 2024-09-19T07:54:04.573 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub queued for path: / 2024-09-19T07:54:04.573 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.573 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:04.573 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:04.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:04.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:04.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:04.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:04.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: active paths [/] 2024-09-19T07:54:04.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.575 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:04 smithi103 ceph-mon[31201]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: pgmap v503: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 861 KiB/s wr, 95 op/s 2024-09-19T07:54:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub queued for path: / 2024-09-19T07:54:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: active paths [/] 2024-09-19T07:54:04.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:04 smithi134 ceph-mon[35414]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: pgmap v503: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 861 KiB/s wr, 95 op/s 2024-09-19T07:54:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub queued for path: / 2024-09-19T07:54:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:04.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: active paths [/] 2024-09-19T07:54:04.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.667 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:04 smithi174 ceph-mon[35370]: scrub summary: idle+waiting paths [/] 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 -- 172.21.15.103:0/111868699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c1036e0 msgr2=0x7fb17c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 --2- 172.21.15.103:0/111868699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c1036e0 0x7fb17c107b90 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009930 tx=0x7fb16c02a420 comp rx=0 tx=0).stop 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 -- 172.21.15.103:0/111868699 shutdown_connections 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 --2- 172.21.15.103:0/111868699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c1082a0 0x7fb17c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 --2- 172.21.15.103:0/111868699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c1036e0 0x7fb17c107b90 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 --2- 172.21.15.103:0/111868699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c102d30 0x7fb17c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 -- 172.21.15.103:0/111868699 >> 172.21.15.103:0/111868699 conn(0x7fb17c0fc7d0 msgr2=0x7fb17c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:04.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 -- 172.21.15.103:0/111868699 shutdown_connections 2024-09-19T07:54:04.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.687+0000 7fb1815a4640 1 -- 172.21.15.103:0/111868699 wait complete. 2024-09-19T07:54:04.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 Processor -- start 2024-09-19T07:54:04.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 -- start start 2024-09-19T07:54:04.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c102d30 0x7fb17c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c1036e0 0x7fb17c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c1082a0 0x7fb17c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1120c0 con 0x7fb17c102d30 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c111f40 con 0x7fb17c1036e0 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb1815a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c112240 con 0x7fb17c1082a0 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c102d30 0x7fb17c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c102d30 0x7fb17c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58450/0 (socket says 172.21.15.103:58450) 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c1082a0 0x7fb17c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 -- 172.21.15.103:0/1135084706 learned_addr learned my addr 172.21.15.103:0/1135084706 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c1036e0 0x7fb17c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 -- 172.21.15.103:0/1135084706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c1082a0 msgr2=0x7fb17c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c1082a0 0x7fb17c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 -- 172.21.15.103:0/1135084706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c1036e0 msgr2=0x7fb17c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c1036e0 0x7fb17c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17affd640 1 -- 172.21.15.103:0/1135084706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c1a6b00 con 0x7fb17c102d30 2024-09-19T07:54:04.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.688+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c1036e0 0x7fb17c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:04.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.689+0000 7fb17affd640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c102d30 0x7fb17c100990 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b4a0 tx=0x7fb16400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:04.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.689+0000 7fb173fff640 1 -- 172.21.15.103:0/1135084706 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164014070 con 0x7fb17c102d30 2024-09-19T07:54:04.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.689+0000 7fb173fff640 1 -- 172.21.15.103:0/1135084706 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb164010af0 con 0x7fb17c102d30 2024-09-19T07:54:04.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.689+0000 7fb173fff640 1 -- 172.21.15.103:0/1135084706 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164017d10 con 0x7fb17c102d30 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.689+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c1a6df0 con 0x7fb17c102d30 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.689+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c1a7250 con 0x7fb17c102d30 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.691+0000 7fb173fff640 1 -- 172.21.15.103:0/1135084706 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb164017070 con 0x7fb17c102d30 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.691+0000 7fb18397d740 1 Processor -- start 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.691+0000 7fb18397d740 1 -- start start 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb18397d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb18397d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40300140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb18397d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40302a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb18397d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e4025b2e0 con 0x561e40300680 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb18397d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561e40202ce0 con 0x561e402f54a0 2024-09-19T07:54:04.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb18397d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e402572a0 con 0x561e402f5dc0 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb17a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40302a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb173fff640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb1680a3430 0x7fb1680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb17a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40302a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58456/0 (socket says 172.21.15.103:58456) 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb17a7fc640 1 -- 172.21.15.103:0/1943886242 learned_addr learned my addr 172.21.15.103:0/1943886242 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb17b7fe640 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.692+0000 7fb17affd640 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40300140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17b7fe640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb1680a3430 0x7fb1680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb173fff640 1 -- 172.21.15.103:0/1135084706 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8586+0+0 (secure 0 0 0) 0x7fb1640cefa0 con 0x7fb17c102d30 2024-09-19T07:54:04.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17affd640 1 -- 172.21.15.103:0/1943886242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 msgr2=0x561e402f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17affd640 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17affd640 1 -- 172.21.15.103:0/1943886242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 msgr2=0x561e40302a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17affd640 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40302a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17affd640 1 -- 172.21.15.103:0/1943886242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e401e7440 con 0x561e402f5dc0 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17affd640 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40300140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb1640cc760 tx=0x7fb164017730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21eec146fbebd9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb17b7fe640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb1680a3430 0x7fb1680a58f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb174009990 tx=0x7fb174009b20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb1717fa640 1 -- 172.21.15.103:0/1943886242 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16401e070 con 0x561e402f5dc0 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb1717fa640 1 -- 172.21.15.103:0/1943886242 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16409b9f0 con 0x561e402f5dc0 2024-09-19T07:54:04.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.693+0000 7fb1717fa640 1 -- 172.21.15.103:0/1943886242 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164010ca0 con 0x561e402f5dc0 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.694+0000 7fb18397d740 1 -- 172.21.15.103:0/1943886242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 msgr2=0x561e40300140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.694+0000 7fb18397d740 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40300140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb1640cc760 tx=0x7fb164017730 comp rx=0 tx=0).stop 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.694+0000 7fb18397d740 1 -- 172.21.15.103:0/1943886242 shutdown_connections 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.694+0000 7fb18397d740 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40302a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.694+0000 7fb18397d740 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40300140 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.694+0000 7fb18397d740 1 --2- 172.21.15.103:0/1943886242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.694+0000 7fb18397d740 1 -- 172.21.15.103:0/1943886242 >> 172.21.15.103:0/1943886242 conn(0x561e402f1a30 msgr2=0x561e402f1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 -- 172.21.15.103:0/1943886242 shutdown_connections 2024-09-19T07:54:04.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 -- 172.21.15.103:0/1943886242 wait complete. 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 -- start start 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 Processor -- start 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e402fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e402fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e40303250 con 0x561e402f5dc0 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561e40216050 con 0x561e402f54a0 2024-09-19T07:54:04.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb17a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e402fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e403033d0 con 0x561e40300680 2024-09-19T07:54:04.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb17a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e402fd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58462/0 (socket says 172.21.15.103:58462) 2024-09-19T07:54:04.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb18397d740 10 client.0 _ll_register_callbacks cb 0x561e40256160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:04.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb17a7fc640 1 -- 172.21.15.103:0/2996945985 learned_addr learned my addr 172.21.15.103:0/2996945985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:04.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb17affd640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.695+0000 7fb17b7fe640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e402fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb17a7fc640 1 -- 172.21.15.103:0/2996945985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 msgr2=0x561e402fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e402fd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb17a7fc640 1 -- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 msgr2=0x561e402ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb17a7fc640 1 -- 172.21.15.103:0/2996945985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e3ffcc3c0 con 0x561e402f5dc0 2024-09-19T07:54:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb17b7fe640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e402fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e402fd2a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009230 tx=0x7fb16c031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae2c60310e28c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:04.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb1627fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb1627fc640 1 -- 172.21.15.103:0/2996945985 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c035040 con 0x561e402f5dc0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb1627fc640 1 -- 172.21.15.103:0/2996945985 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8586+0+0 (secure 0 0 0) 0x7fb16c004300 con 0x561e402f5dc0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16409b9f0 con 0x561e402f5dc0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb1627fc640 1 -- 172.21.15.103:0/2996945985 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb16c005030 con 0x561e402f5dc0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb18397d740 10 client.18660 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.696+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561e4030ec20 con 0x561e402f5dc0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb1627fc640 1 -- 172.21.15.103:0/2996945985 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16409b9f0 con 0x561e402f5dc0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb1627fc640 1 -- 172.21.15.103:0/2996945985 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fb16c004cd0 con 0x561e402f5dc0 2024-09-19T07:54:04.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb18397d740 10 client.18660 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb18397d740 10 client.18660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb18397d740 20 client.18660 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb18397d740 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e403100e0 0x561e40312520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb18397d740 4 client.18660 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e40268850 con 0x561e403100e0 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb17affd640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e403100e0 0x561e40312520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb1627fc640 10 client.18660 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.697+0000 7fb17affd640 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e403100e0 0x561e40312520 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4423c5b900861ef8 server_cookie=e0e7dc1ba8ab6b48 in_seq=0 out_seq=0 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb1627fc640 1 -- 172.21.15.103:0/2996945985 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e40268850 con 0x561e403100e0 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb1627fc640 10 client.18660 handle_command_reply: tid=0 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 client.18660 shutdown 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 msgr2=0x561e402fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e402fd2a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009230 tx=0x7fb16c031970 comp rx=0 tx=0).stop 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 shutdown_connections 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e403100e0 0x561e40312520 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e402fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e402fd2a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 --2- 172.21.15.103:0/2996945985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e402ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 >> 172.21.15.103:0/2996945985 conn(0x561e402f1a30 msgr2=0x561e402f7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb1627fc640 0 client.18660 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 shutdown_connections 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.698+0000 7fb18397d740 1 -- 172.21.15.103:0/2996945985 wait complete. 2024-09-19T07:54:04.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.699+0000 7fb18397d740 20 client.18660 trim_cache size 0 max 16384 2024-09-19T07:54:04.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 Processor -- start 2024-09-19T07:54:04.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 -- start start 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e4030c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40317df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40318330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1afbd0 con 0x561e40300680 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1b1710 con 0x561e402f54a0 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb18397d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1b02a0 con 0x561e402f5dc0 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40318330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e4030c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40318330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58478/0 (socket says 172.21.15.103:58478) 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17b7fe640 1 -- 172.21.15.103:0/564694537 learned_addr learned my addr 172.21.15.103:0/564694537 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40317df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17affd640 1 -- 172.21.15.103:0/564694537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 msgr2=0x561e40317df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17affd640 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40317df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17affd640 1 -- 172.21.15.103:0/564694537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 msgr2=0x561e40318330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17affd640 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40318330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.716+0000 7fb17affd640 1 -- 172.21.15.103:0/564694537 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e402fdb70 con 0x561e402f54a0 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.717+0000 7fb17affd640 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e4030c4b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb164097cc0 tx=0x7fb1640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a1317ad0eb9f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.717+0000 7fb170ff9640 1 -- 172.21.15.103:0/564694537 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16401e040 con 0x561e402f54a0 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.717+0000 7fb170ff9640 1 -- 172.21.15.103:0/564694537 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb164017360 con 0x561e402f54a0 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.717+0000 7fb170ff9640 1 -- 172.21.15.103:0/564694537 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164095e60 con 0x561e402f54a0 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 -- 172.21.15.103:0/564694537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 msgr2=0x561e4030c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e4030c4b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb164097cc0 tx=0x7fb1640046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 -- 172.21.15.103:0/564694537 shutdown_connections 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e40300680 0x561e40318330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e402f5dc0 0x561e40317df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 --2- 172.21.15.103:0/564694537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e4030c4b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 -- 172.21.15.103:0/564694537 >> 172.21.15.103:0/564694537 conn(0x561e402f1a30 msgr2=0x561e402f8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 -- 172.21.15.103:0/564694537 shutdown_connections 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 -- 172.21.15.103:0/564694537 wait complete. 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 -- start start 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 Processor -- start 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.718+0000 7fb18397d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e40303b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb18397d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e403040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb18397d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e40378db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb18397d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1b1040 con 0x561e402f5dc0 2024-09-19T07:54:04.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb18397d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1b0970 con 0x561e402f54a0 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb18397d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c1adbd0 con 0x561e40300680 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb18397d740 10 client.0 _ll_register_callbacks cb 0x7fb16c031620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e403040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e40303b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e403040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58486/0 (socket says 172.21.15.103:58486) 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 -- 172.21.15.103:0/845463255 learned_addr learned my addr 172.21.15.103:0/845463255 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17b7fe640 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e40378db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 -- 172.21.15.103:0/845463255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 msgr2=0x561e40378db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e40378db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 -- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 msgr2=0x561e40303b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:04.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e40303b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 -- 172.21.15.103:0/845463255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e40379370 con 0x561e402f5dc0 2024-09-19T07:54:04.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17affd640 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e40303b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:04.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb17a7fc640 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e403040c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009850 tx=0x7fb16c003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5258869d1d18f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:04.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb1627fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:04.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb1627fc640 1 -- 172.21.15.103:0/845463255 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c051040 con 0x561e402f5dc0 2024-09-19T07:54:04.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb1627fc640 1 -- 172.21.15.103:0/845463255 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8586+0+0 (secure 0 0 0) 0x7fb16c004300 con 0x561e402f5dc0 2024-09-19T07:54:04.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.719+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb164017360 con 0x561e402f5dc0 2024-09-19T07:54:04.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb1627fc640 1 -- 172.21.15.103:0/845463255 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb16c035040 con 0x561e402f5dc0 2024-09-19T07:54:04.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 10 client.18666 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:04.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561e4037a650 con 0x561e402f5dc0 2024-09-19T07:54:04.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb1627fc640 1 -- 172.21.15.103:0/845463255 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb164017360 con 0x561e402f5dc0 2024-09-19T07:54:04.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb1627fc640 1 -- 172.21.15.103:0/845463255 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fb16c0046c0 con 0x561e402f5dc0 2024-09-19T07:54:04.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 10 client.18666 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:04.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 10 client.18666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:04.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 20 client.18666 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:04.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e4037ad70 0x561e4037d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:04.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 4 client.18666 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:54:04.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e40262d60 con 0x561e4037ad70 2024-09-19T07:54:04.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.720+0000 7fb17affd640 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e4037ad70 0x561e4037d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:04.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb1627fc640 10 client.18666 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb17affd640 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e4037ad70 0x561e4037d190 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1afa997f88fcfaae server_cookie=123ce7e5f0460ed8 in_seq=0 out_seq=0 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb1627fc640 1 -- 172.21.15.103:0/845463255 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561e40262d60 con 0x561e4037ad70 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb1627fc640 10 client.18666 handle_command_reply: tid=0 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 client.18666 shutdown 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 msgr2=0x561e403040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e403040c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009850 tx=0x7fb16c003b30 comp rx=0 tx=0).stop 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 shutdown_connections 2024-09-19T07:54:04.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x561e4037ad70 0x561e4037d190 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e40300680 0x561e40378db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e402f5dc0 0x561e403040c0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 --2- 172.21.15.103:0/845463255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e402f54a0 0x561e40303b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 >> 172.21.15.103:0/845463255 conn(0x561e402f1a30 msgr2=0x561e402feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:04.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb1627fc640 0 client.18666 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:04.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 shutdown_connections 2024-09-19T07:54:04.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 1 -- 172.21.15.103:0/845463255 wait complete. 2024-09-19T07:54:04.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.721+0000 7fb18397d740 20 client.18666 trim_cache size 0 max 16384 2024-09-19T07:54:04.738 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T07:54:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": { 2024-09-19T07:54:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "d42807b4-7243-4b7e-8c6c-6187c298f673": { 2024-09-19T07:54:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "path": "/", 2024-09-19T07:54:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tag": "d42807b4-7243-4b7e-8c6c-6187c298f673", 2024-09-19T07:54:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "options": "recursive,force" 2024-09-19T07:54:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:04.739 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:04.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb1680a3430 msgr2=0x7fb1680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb1680a3430 0x7fb1680a58f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb174009990 tx=0x7fb174009b20 comp rx=0 tx=0).stop 2024-09-19T07:54:04.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c102d30 msgr2=0x7fb17c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:04.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c102d30 0x7fb17c100990 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b4a0 tx=0x7fb16400b960 comp rx=0 tx=0).stop 2024-09-19T07:54:04.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 shutdown_connections 2024-09-19T07:54:04.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb1680a3430 0x7fb1680a58f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c1082a0 0x7fb17c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c1036e0 0x7fb17c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 --2- 172.21.15.103:0/1135084706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb17c102d30 0x7fb17c100990 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:04.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 >> 172.21.15.103:0/1135084706 conn(0x7fb17c0fc7d0 msgr2=0x7fb17c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:04.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 shutdown_connections 2024-09-19T07:54:04.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:04.722+0000 7fb1815a4640 1 -- 172.21.15.103:0/1135084706 wait complete. 2024-09-19T07:54:05.389 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:05 smithi103 ceph-mon[31201]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:54:05.418 INFO:tasks.cephfs.filesystem:scrub status for tag:d42807b4-7243-4b7e-8c6c-6187c298f673 - {'path': '/', 'tag': 'd42807b4-7243-4b7e-8c6c-6187c298f673', 'options': 'recursive,force'} 2024-09-19T07:54:05.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:05 smithi134 ceph-mon[35414]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:54:05.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:05 smithi174 ceph-mon[35370]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T07:54:06.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:06 smithi134 ceph-mon[35414]: pgmap v505: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-09-19T07:54:06.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:06 smithi174 ceph-mon[35370]: pgmap v505: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-09-19T07:54:06.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:06 smithi103 ceph-mon[31201]: pgmap v505: 97 pgs: 97 active+clean; 3.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-09-19T07:54:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:08 smithi134 ceph-mon[35414]: pgmap v506: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 83 KiB/s wr, 33 op/s 2024-09-19T07:54:08.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:08 smithi174 ceph-mon[35370]: pgmap v506: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 83 KiB/s wr, 33 op/s 2024-09-19T07:54:08.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:08 smithi103 ceph-mon[31201]: pgmap v506: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 83 KiB/s wr, 33 op/s 2024-09-19T07:54:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:09 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:54:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:09 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:54:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:09 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:54:09.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:09 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:54:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:09 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:54:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:09 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:54:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:09 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:54:09.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:09 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:54:09.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:09 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:54:09.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:09 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:54:09.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:09 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:54:09.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:09 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:54:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:10 smithi134 ceph-mon[35414]: scrub summary: idle 2024-09-19T07:54:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:10 smithi134 ceph-mon[35414]: pgmap v507: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 83 KiB/s wr, 33 op/s 2024-09-19T07:54:10.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:10 smithi174 ceph-mon[35370]: scrub summary: idle 2024-09-19T07:54:10.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:10 smithi174 ceph-mon[35370]: pgmap v507: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 83 KiB/s wr, 33 op/s 2024-09-19T07:54:10.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:10 smithi103 ceph-mon[31201]: scrub summary: idle 2024-09-19T07:54:10.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:10 smithi103 ceph-mon[31201]: pgmap v507: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 83 KiB/s wr, 33 op/s 2024-09-19T07:54:11.405 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:54:11.668 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:12.067 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:12 smithi103 ceph-mon[31201]: pgmap v508: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 74 KiB/s wr, 32 op/s 2024-09-19T07:54:12.067 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:12 smithi103 ceph-mon[31201]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T07:54:12.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 -- 172.21.15.103:0/2906063258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c100e20 msgr2=0x7fa39c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 --2- 172.21.15.103:0/2906063258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c100e20 0x7fa39c101200 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa398009900 tx=0x7fa39802a480 comp rx=0 tx=0).stop 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 -- 172.21.15.103:0/2906063258 shutdown_connections 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 --2- 172.21.15.103:0/2906063258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa39c102ef0 0x7fa39c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 --2- 172.21.15.103:0/2906063258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa39c102570 0x7fa39c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 --2- 172.21.15.103:0/2906063258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c100e20 0x7fa39c101200 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 -- 172.21.15.103:0/2906063258 >> 172.21.15.103:0/2906063258 conn(0x7fa39c0fc7d0 msgr2=0x7fa39c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 -- 172.21.15.103:0/2906063258 shutdown_connections 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 -- 172.21.15.103:0/2906063258 wait complete. 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 Processor -- start 2024-09-19T07:54:12.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.148+0000 7fa3a3345640 1 -- start start 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a3345640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa39c100e20 0x7fa39c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a3345640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa39c102570 0x7fa39c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a3345640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c102ef0 0x7fa39c1a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a3345640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c110f80 con 0x7fa39c102ef0 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a3345640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c110e00 con 0x7fa39c102570 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a3345640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c111100 con 0x7fa39c100e20 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c102ef0 0x7fa39c1a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c102ef0 0x7fa39c1a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51120/0 (socket says 172.21.15.103:51120) 2024-09-19T07:54:12.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 -- 172.21.15.103:0/1542235601 learned_addr learned my addr 172.21.15.103:0/1542235601 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a10e1640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa39c100e20 0x7fa39c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a08e0640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa39c102570 0x7fa39c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 -- 172.21.15.103:0/1542235601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa39c100e20 msgr2=0x7fa39c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa39c100e20 0x7fa39c1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 -- 172.21.15.103:0/1542235601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa39c102570 msgr2=0x7fa39c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa39c102570 0x7fa39c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 -- 172.21.15.103:0/1542235601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c1a4990 con 0x7fa39c102ef0 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a18e2640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c102ef0 0x7fa39c1a4450 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa39400eaa0 tx=0x7fa39400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3927fc640 1 -- 172.21.15.103:0/1542235601 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39400cdb0 con 0x7fa39c102ef0 2024-09-19T07:54:12.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3927fc640 1 -- 172.21.15.103:0/1542235601 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa394005020 con 0x7fa39c102ef0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.149+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c1a4b60 con 0x7fa39c102ef0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.150+0000 7fa3927fc640 1 -- 172.21.15.103:0/1542235601 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940116b0 con 0x7fa39c102ef0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.150+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c1a5070 con 0x7fa39c102ef0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.151+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39c101200 con 0x7fa39c102ef0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.152+0000 7fa3927fc640 1 -- 172.21.15.103:0/1542235601 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fa394005560 con 0x7fa39c102ef0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.152+0000 7fa3927fc640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3880a3430 0x7fa3880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.152+0000 7fa3927fc640 1 -- 172.21.15.103:0/1542235601 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fa3940d0290 con 0x7fa39c102ef0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.153+0000 7fa3a10e1640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3880a3430 0x7fa3880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.153+0000 7fa3a10e1640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3880a3430 0x7fa3880a58f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa398005ab0 tx=0x7fa398005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:12.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.156+0000 7fa3927fc640 1 -- 172.21.15.103:0/1542235601 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa394016030 con 0x7fa39c102ef0 2024-09-19T07:54:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:12 smithi134 ceph-mon[35414]: pgmap v508: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 74 KiB/s wr, 32 op/s 2024-09-19T07:54:12.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:12 smithi134 ceph-mon[35414]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T07:54:12.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.366+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa39c06a560 con 0x7fa39c102ef0 2024-09-19T07:54:12.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.367+0000 7fa3927fc640 1 -- 172.21.15.103:0/1542235601 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7fa39c06a560 con 0x7fa39c102ef0 2024-09-19T07:54:12.368 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:12.368 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":54,"btime":"2024-09-19T07:53:10:110704+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:53:09.872640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:54:12.368 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 54 2024-09-19T07:54:12.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3880a3430 msgr2=0x7fa3880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3880a3430 0x7fa3880a58f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa398005ab0 tx=0x7fa398005a20 comp rx=0 tx=0).stop 2024-09-19T07:54:12.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c102ef0 msgr2=0x7fa39c1a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:12.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c102ef0 0x7fa39c1a4450 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa39400eaa0 tx=0x7fa39400ef60 comp rx=0 tx=0).stop 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 shutdown_connections 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fa3880a3430 0x7fa3880a58f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa39c102ef0 0x7fa39c1a4450 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa39c102570 0x7fa39c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 --2- 172.21.15.103:0/1542235601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa39c100e20 0x7fa39c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 >> 172.21.15.103:0/1542235601 conn(0x7fa39c0fc7d0 msgr2=0x7fa39c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 shutdown_connections 2024-09-19T07:54:12.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:12.369+0000 7fa3a3345640 1 -- 172.21.15.103:0/1542235601 wait complete. 2024-09-19T07:54:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:12 smithi174 ceph-mon[35370]: pgmap v508: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 74 KiB/s wr, 32 op/s 2024-09-19T07:54:12.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:12 smithi174 ceph-mon[35370]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T07:54:13.035 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 2cb8dfe1 --release --await' 2024-09-19T07:54:13.035 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 2cb8dfe1 --release --await 2024-09-19T07:54:13.293 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:13.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:13 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1542235601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:13.334 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:13 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1542235601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:13.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:13 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1542235601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:13.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.765+0000 7fdfa4107640 1 -- 172.21.15.103:0/3575054195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c105ad0 msgr2=0x7fdf9c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 --2- 172.21.15.103:0/3575054195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c105ad0 0x7fdf9c105eb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009900 tx=0x7fdf9802a410 comp rx=0 tx=0).stop 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- 172.21.15.103:0/3575054195 shutdown_connections 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 --2- 172.21.15.103:0/3575054195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf9c101ba0 0x7fdf9c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 --2- 172.21.15.103:0/3575054195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c101200 0x7fdf9c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 --2- 172.21.15.103:0/3575054195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c105ad0 0x7fdf9c105eb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- 172.21.15.103:0/3575054195 >> 172.21.15.103:0/3575054195 conn(0x7fdf9c077210 msgr2=0x7fdf9c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- 172.21.15.103:0/3575054195 shutdown_connections 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- 172.21.15.103:0/3575054195 wait complete. 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 Processor -- start 2024-09-19T07:54:13.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- start start 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf9c101200 0x7fdf9c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c101ba0 0x7fdf9c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c105ad0 0x7fdf9c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c110eb0 con 0x7fdf9c101ba0 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c110d30 con 0x7fdf9c105ad0 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.766+0000 7fdfa4107640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c111030 con 0x7fdf9c101200 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c101ba0 0x7fdf9c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c101ba0 0x7fdf9c0fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51142/0 (socket says 172.21.15.103:51142) 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 -- 172.21.15.103:0/786987270 learned_addr learned my addr 172.21.15.103:0/786987270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf9c101200 0x7fdf9c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c105ad0 0x7fdf9c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 -- 172.21.15.103:0/786987270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf9c101200 msgr2=0x7fdf9c0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf9c101200 0x7fdf9c0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 -- 172.21.15.103:0/786987270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c105ad0 msgr2=0x7fdf9c0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c105ad0 0x7fdf9c0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 -- 172.21.15.103:0/786987270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c1a6800 con 0x7fdf9c101ba0 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf9c101200 0x7fdf9c0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c105ad0 0x7fdf9c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c101ba0 0x7fdf9c0fcbd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c890 tx=0x7fdf8c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdf92ffd640 1 -- 172.21.15.103:0/786987270 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c004220 con 0x7fdf9c101ba0 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdf92ffd640 1 -- 172.21.15.103:0/786987270 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c0043c0 con 0x7fdf9c101ba0 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c1a6af0 con 0x7fdf9c101ba0 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdf92ffd640 1 -- 172.21.15.103:0/786987270 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0049e0 con 0x7fdf9c101ba0 2024-09-19T07:54:13.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.767+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c1a7030 con 0x7fdf9c101ba0 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdf92ffd640 1 -- 172.21.15.103:0/786987270 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fdf8c0079b0 con 0x7fdf9c101ba0 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 Processor -- start 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 -- start start 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611060 0x562cb4611440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611980 0x562cb461bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb461c240 0x562cb461e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cb4576620 con 0x562cb4611060 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562cb451de50 con 0x562cb4611980 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.770+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562cb45725e0 con 0x562cb461c240 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611060 0x562cb4611440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdf92ffd640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdf880a33b0 0x7fdf880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611060 0x562cb4611440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51154/0 (socket says 172.21.15.103:51154) 2024-09-19T07:54:13.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 -- 172.21.15.103:0/3879091934 learned_addr learned my addr 172.21.15.103:0/3879091934 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611980 0x562cb461bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdf880a33b0 0x7fdf880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdf92ffd640 1 -- 172.21.15.103:0/786987270 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fdf8c0cfe40 con 0x7fdf9c101ba0 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb461c240 0x562cb461e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 -- 172.21.15.103:0/3879091934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb461c240 msgr2=0x562cb461e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb461c240 0x562cb461e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 -- 172.21.15.103:0/3879091934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611980 msgr2=0x562cb461bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611980 0x562cb461bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 -- 172.21.15.103:0/3879091934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb450ed40 con 0x562cb4611060 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611060 0x562cb4611440 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdf940138e0 tx=0x7fdf94013da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c91d47e41eb218 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdf87fff640 1 -- 172.21.15.103:0/3879091934 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011c40 con 0x562cb4611060 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdf87fff640 1 -- 172.21.15.103:0/3879091934 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf94011de0 con 0x562cb4611060 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdf87fff640 1 -- 172.21.15.103:0/3879091934 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9401d6a0 con 0x562cb4611060 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.771+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdf880a33b0 0x7fdf880a5870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdf94020000 tx=0x7fdf9401f000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 -- 172.21.15.103:0/3879091934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611060 msgr2=0x562cb4611440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611060 0x562cb4611440 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdf940138e0 tx=0x7fdf94013da0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 -- 172.21.15.103:0/3879091934 shutdown_connections 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb461c240 0x562cb461e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611980 0x562cb461bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/3879091934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611060 0x562cb4611440 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 -- 172.21.15.103:0/3879091934 >> 172.21.15.103:0/3879091934 conn(0x562cb460d5f0 msgr2=0x562cb460d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 -- 172.21.15.103:0/3879091934 shutdown_connections 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.772+0000 7fdfa64e0740 1 -- 172.21.15.103:0/3879091934 wait complete. 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 -- start start 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 Processor -- start 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb46190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb4619610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb4619b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cb461ee10 con 0x562cb4611980 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562cb44fef10 con 0x562cb461c240 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562cb461ef90 con 0x562cb4611060 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa64e0740 10 client.0 _ll_register_callbacks cb 0x562cb4522db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:13.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa16a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb4619610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa16a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb4619610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51162/0 (socket says 172.21.15.103:51162) 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa16a2640 1 -- 172.21.15.103:0/2893537940 learned_addr learned my addr 172.21.15.103:0/2893537940 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.773+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb46190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb4619b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa16a2640 1 -- 172.21.15.103:0/2893537940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 msgr2=0x562cb46190d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb46190d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa16a2640 1 -- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 msgr2=0x562cb4619b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb4619b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa16a2640 1 -- 172.21.15.103:0/2893537940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb4285580 con 0x562cb4611980 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb46190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb4619610 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c000c00 tx=0x7fdf8c09bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b4c02f629bad7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdf857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdf857fa640 1 -- 172.21.15.103:0/2893537940 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c01d070 con 0x562cb4611980 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdf857fa640 1 -- 172.21.15.103:0/2893537940 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fdf8c0a0120 con 0x562cb4611980 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf94011de0 con 0x562cb4611980 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdf857fa640 1 -- 172.21.15.103:0/2893537940 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c0a04c0 con 0x562cb4611980 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa64e0740 10 client.18684 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562cb434f960 con 0x562cb4611980 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdf857fa640 1 -- 172.21.15.103:0/2893537940 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf94011de0 con 0x562cb4611980 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdf857fa640 1 -- 172.21.15.103:0/2893537940 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fdf8c0d1040 con 0x562cb4611980 2024-09-19T07:54:13.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa64e0740 10 client.18684 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.774+0000 7fdfa64e0740 10 client.18684 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.775+0000 7fdfa64e0740 20 client.18684 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.775+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb462bcf0 0x562cb462e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.775+0000 7fdfa64e0740 4 client.18684 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.775+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cb4583b90 con 0x562cb462bcf0 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.775+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb462bcf0 0x562cb462e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.775+0000 7fdf857fa640 10 client.18684 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.775+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb462bcf0 0x562cb462e0e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed2a32752642707 server_cookie=738979323ec39edb in_seq=0 out_seq=0 2024-09-19T07:54:13.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdf857fa640 1 -- 172.21.15.103:0/2893537940 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cb4583b90 con 0x562cb462bcf0 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdf857fa640 10 client.18684 handle_command_reply: tid=0 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 client.18684 shutdown 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 msgr2=0x562cb4619610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb4619610 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c000c00 tx=0x7fdf8c09bb30 comp rx=0 tx=0).stop 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 shutdown_connections 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb462bcf0 0x562cb462e0e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb4619b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb4619610 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdf857fa640 0 client.18684 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2893537940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb46190d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 >> 172.21.15.103:0/2893537940 conn(0x562cb460d5f0 msgr2=0x562cb460c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 shutdown_connections 2024-09-19T07:54:13.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2893537940 wait complete. 2024-09-19T07:54:13.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.776+0000 7fdfa64e0740 20 client.18684 trim_cache size 0 max 16384 2024-09-19T07:54:13.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.793+0000 7fdfa64e0740 1 Processor -- start 2024-09-19T07:54:13.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.793+0000 7fdfa64e0740 1 -- start start 2024-09-19T07:54:13.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb4613ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb461afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb46339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1afa20 con 0x562cb4611980 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b1560 con 0x562cb461c240 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b00f0 con 0x562cb4611060 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb461afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb461afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51172/0 (socket says 172.21.15.103:51172) 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 -- 172.21.15.103:0/1940510874 learned_addr learned my addr 172.21.15.103:0/1940510874 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb46339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb4613ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 -- 172.21.15.103:0/1940510874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 msgr2=0x562cb4613ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb4613ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 -- 172.21.15.103:0/1940510874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 msgr2=0x562cb46339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb46339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 -- 172.21.15.103:0/1940510874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb46117b0 con 0x562cb4611980 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb461afa0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c09a3c0 tx=0x7fdf8c0b5240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9626ffd95f14a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdf877fe640 1 -- 172.21.15.103:0/1940510874 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c01d070 con 0x562cb4611980 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdf877fe640 1 -- 172.21.15.103:0/1940510874 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c09b880 con 0x562cb4611980 2024-09-19T07:54:13.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.794+0000 7fdf877fe640 1 -- 172.21.15.103:0/1940510874 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c007ca0 con 0x562cb4611980 2024-09-19T07:54:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 -- 172.21.15.103:0/1940510874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 msgr2=0x562cb461afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb461afa0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c09a3c0 tx=0x7fdf8c0b5240 comp rx=0 tx=0).stop 2024-09-19T07:54:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 -- 172.21.15.103:0/1940510874 shutdown_connections 2024-09-19T07:54:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb461c240 0x562cb46339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb4611980 0x562cb461afa0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/1940510874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611060 0x562cb4613ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 -- 172.21.15.103:0/1940510874 >> 172.21.15.103:0/1940510874 conn(0x562cb460d5f0 msgr2=0x562cb460c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 -- 172.21.15.103:0/1940510874 shutdown_connections 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.795+0000 7fdfa64e0740 1 -- 172.21.15.103:0/1940510874 wait complete. 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 -- start start 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 Processor -- start 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611060 0x562cb4620720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611980 0x562cb4620c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb461c240 0x562cb4695980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b0e90 con 0x562cb461c240 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1b07c0 con 0x562cb4611060 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1ada20 con 0x562cb4611980 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa64e0740 10 client.0 _ll_register_callbacks cb 0x7fdf8c09bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa26a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb461c240 0x562cb4695980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa26a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb461c240 0x562cb4695980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51188/0 (socket says 172.21.15.103:51188) 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa26a4640 1 -- 172.21.15.103:0/2818895635 learned_addr learned my addr 172.21.15.103:0/2818895635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa16a2640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611980 0x562cb4620c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.796+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611060 0x562cb4620720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa26a4640 1 -- 172.21.15.103:0/2818895635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611980 msgr2=0x562cb4620c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611980 0x562cb4620c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa26a4640 1 -- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611060 msgr2=0x562cb4620720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611060 0x562cb4620720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa26a4640 1 -- 172.21.15.103:0/2818895635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb4695f70 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611060 0x562cb4620720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa26a4640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb461c240 0x562cb4695980 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf94005fd0 tx=0x7fdf94013760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c93c70d01b0dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdf857fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdf857fa640 1 -- 172.21.15.103:0/2818895635 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94036060 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdf857fa640 1 -- 172.21.15.103:0/2818895635 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fdf9401d440 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8c09b880 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdf857fa640 1 -- 172.21.15.103:0/2818895635 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdf9401d640 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa64e0740 10 client.18690 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562cb4697290 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdf857fa640 1 -- 172.21.15.103:0/2818895635 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8c09b880 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdf857fa640 1 -- 172.21.15.103:0/2818895635 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fdf94030070 con 0x562cb461c240 2024-09-19T07:54:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa64e0740 10 client.18690 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa64e0740 10 client.18690 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.797+0000 7fdfa64e0740 20 client.18690 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.798+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb46976f0 0x562cb4699b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.798+0000 7fdfa64e0740 4 client.18690 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2cb8dfe1", "release": true, "await": true}] 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.798+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2cb8dfe1", "release": true, "await": true}) -- 0x562cb457e0a0 con 0x562cb46976f0 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.798+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb46976f0 0x562cb4699b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.798+0000 7fdf857fa640 10 client.18690 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.798+0000 7fdfa1ea3640 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb46976f0 0x562cb4699b10 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afce3b27d9e55316 server_cookie=a9273f5ffac92c73 in_seq=0 out_seq=0 2024-09-19T07:54:13.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdf857fa640 1 -- 172.21.15.103:0/2818895635 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562cb457e0a0 con 0x562cb46976f0 2024-09-19T07:54:13.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdf857fa640 10 client.18690 handle_command_reply: tid=0 2024-09-19T07:54:13.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 client.18690 shutdown 2024-09-19T07:54:13.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb461c240 msgr2=0x562cb4695980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb461c240 0x562cb4695980 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf94005fd0 tx=0x7fdf94013760 comp rx=0 tx=0).stop 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 shutdown_connections 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x562cb46976f0 0x562cb4699b10 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cb461c240 0x562cb4695980 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdf857fa640 0 client.18690 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562cb4611980 0x562cb4620c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 --2- 172.21.15.103:0/2818895635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cb4611060 0x562cb4620720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 >> 172.21.15.103:0/2818895635 conn(0x562cb460d5f0 msgr2=0x562cb4627060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 shutdown_connections 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 1 -- 172.21.15.103:0/2818895635 wait complete. 2024-09-19T07:54:13.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.835+0000 7fdfa64e0740 20 client.18690 trim_cache size 0 max 16384 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 54, 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 56, 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "2cb8dfe1": { 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 56, 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:54:13.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 149.4, 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:13.839 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:13.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdf880a33b0 msgr2=0x7fdf880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdf880a33b0 0x7fdf880a5870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdf94020000 tx=0x7fdf9401f000 comp rx=0 tx=0).stop 2024-09-19T07:54:13.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c101ba0 msgr2=0x7fdf9c0fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:13.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c101ba0 0x7fdf9c0fcbd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c890 tx=0x7fdf8c00cd50 comp rx=0 tx=0).stop 2024-09-19T07:54:13.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 shutdown_connections 2024-09-19T07:54:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fdf880a33b0 0x7fdf880a5870 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c105ad0 0x7fdf9c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf9c101ba0 0x7fdf9c0fcbd0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 --2- 172.21.15.103:0/786987270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf9c101200 0x7fdf9c0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 >> 172.21.15.103:0/786987270 conn(0x7fdf9c077210 msgr2=0x7fdf9c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 shutdown_connections 2024-09-19T07:54:13.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:13.836+0000 7fdfa4107640 1 -- 172.21.15.103:0/786987270 wait complete. 2024-09-19T07:54:14.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:14 smithi134 ceph-mon[35414]: pgmap v510: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 88 KiB/s wr, 39 op/s 2024-09-19T07:54:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:14 smithi174 ceph-mon[35370]: pgmap v510: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 88 KiB/s wr, 39 op/s 2024-09-19T07:54:14.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:14 smithi103 ceph-mon[31201]: pgmap v510: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 88 KiB/s wr, 39 op/s 2024-09-19T07:54:14.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-19T07:54:14.551 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/: 2024-09-19T07:54:14.551 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 .. 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 data 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data: 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:total 36 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 . 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 .. 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 19 07:38 base 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 19 07:38 current_logfiles 2024-09-19T07:54:14.552 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 19 07:38 global 2024-09-19T07:54:14.553 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 log 2024-09-19T07:54:14.553 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_commit_ts 2024-09-19T07:54:14.553 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_dynshmem 2024-09-19T07:54:14.553 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 19 07:38 pg_hba.conf 2024-09-19T07:54:14.554 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 19 07:38 pg_ident.conf 2024-09-19T07:54:14.554 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:54 pg_logical 2024-09-19T07:54:14.554 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 pg_multixact 2024-09-19T07:54:14.554 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_notify 2024-09-19T07:54:14.554 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_replslot 2024-09-19T07:54:14.554 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_serial 2024-09-19T07:54:14.554 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_snapshots 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:54 pg_stat 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:54 pg_stat_tmp 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:52 pg_subtrans 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_tblspc 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pg_twophase 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 19 07:54 pg_wal 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:52 pg_xact 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 19 07:38 postgresql.auto.conf 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 19 07:38 postgresql.conf 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 19 07:38 postmaster.opts 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.555 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base: 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 19 07:38 . 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 1 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 13493 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 19 07:43 13494 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 pgsql_tmp 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:total 7777 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 . 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 19 07:38 .. 2024-09-19T07:54:14.556 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 112 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 113 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 1247 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1247_fsm 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1247_vm 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 07:38 1249 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1249_fsm 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1249_vm 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 07:38 1255 2024-09-19T07:54:14.557 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1255_fsm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1255_vm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 07:38 1259 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1259_fsm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1259_vm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 07:38 13344 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13344_fsm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13344_vm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13346 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13348 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13349_fsm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349_vm 2024-09-19T07:54:14.558 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13351 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13353 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13354_fsm 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354_vm 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13356 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13358 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13359_fsm 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359_vm 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13361 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13363 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1417 2024-09-19T07:54:14.559 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1418 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 174 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 175 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2187 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2224 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2328 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2336 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2337 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2579 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2600 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2600_fsm 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2600_vm 2024-09-19T07:54:14.560 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2601_fsm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601_vm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2602 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2602_fsm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2602_vm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2603 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2603_fsm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2603_vm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2604 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2605 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2605_fsm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2605_vm 2024-09-19T07:54:14.561 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606 2024-09-19T07:54:14.562 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2606_fsm 2024-09-19T07:54:14.562 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606_vm 2024-09-19T07:54:14.562 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2607 2024-09-19T07:54:14.562 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2607_fsm 2024-09-19T07:54:14.562 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2607_vm 2024-09-19T07:54:14.562 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 07:38 2608 2024-09-19T07:54:14.562 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2608_fsm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2608_vm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 07:38 2609 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2609_fsm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2609_vm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2610 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2610_fsm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2610_vm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2611 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2612_fsm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612_vm 2024-09-19T07:54:14.563 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2613 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2615_fsm 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615_vm 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2616 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2616_fsm 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2616_vm 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 07:38 2617 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2617_fsm 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2617_vm 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 07:38 2618 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2618_fsm 2024-09-19T07:54:14.564 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2618_vm 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 07:38 2619 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2619_fsm 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2619_vm 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2620 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2650 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2651 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2652 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2653 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2654 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2655 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2656 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2657 2024-09-19T07:54:14.565 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 07:38 2658 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 2659 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2660 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2661 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2662 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2663 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2664 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2665 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2666 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2667 2024-09-19T07:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2668 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2669 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2670 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 07:38 2673 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 07:38 2674 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 07:38 2675 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2678 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2679 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2680 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2681 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2682 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2683 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2684 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2685 2024-09-19T07:54:14.567 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2686 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2687 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2688 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2689 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 07:38 2690 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 07:38 2691 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2692 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2693 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2696 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2699 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2701 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2702 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2703 2024-09-19T07:54:14.568 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2704 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2753 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2753_fsm 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2753_vm 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2754 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2755 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2756 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2757 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2830 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2831 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2832 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2833 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2834 2024-09-19T07:54:14.569 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2835 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2836 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2837 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 07:38 2838 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2838_fsm 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2838_vm 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2839 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840_fsm 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2840_vm 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2841 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2995 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2996 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079 2024-09-19T07:54:14.570 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3079_fsm 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079_vm 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3080 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3081 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3085 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3118 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3119 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 07:38 3164 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3256 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3257 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3258 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3350 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3351 2024-09-19T07:54:14.571 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3379 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3380 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3381 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3394 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3394_fsm 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3394_vm 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3395 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3429 2024-09-19T07:54:14.572 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3430 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3431 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3433 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3439 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3440 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3455 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 07:38 3456 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3456_fsm 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3456_vm 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3466 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3467 2024-09-19T07:54:14.573 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3468 2024-09-19T07:54:14.574 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3501 2024-09-19T07:54:14.574 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3502 2024-09-19T07:54:14.574 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3503 2024-09-19T07:54:14.574 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3534 2024-09-19T07:54:14.574 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541 2024-09-19T07:54:14.574 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3541_fsm 2024-09-19T07:54:14.574 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541_vm 2024-09-19T07:54:14.575 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3542 2024-09-19T07:54:14.575 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3574 2024-09-19T07:54:14.575 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3575 2024-09-19T07:54:14.575 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3576 2024-09-19T07:54:14.575 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3596 2024-09-19T07:54:14.575 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3597 2024-09-19T07:54:14.576 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3598 2024-09-19T07:54:14.576 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3599 2024-09-19T07:54:14.576 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600 2024-09-19T07:54:14.576 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3600_fsm 2024-09-19T07:54:14.576 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600_vm 2024-09-19T07:54:14.576 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601 2024-09-19T07:54:14.576 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3601_fsm 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601_vm 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3602_fsm 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602_vm 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603_fsm 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3603_vm 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3604 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3605 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3606 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3607 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3608 2024-09-19T07:54:14.577 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 3609 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3712 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3764_fsm 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764_vm 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3766 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3767 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3997 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4143 2024-09-19T07:54:14.578 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4144 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4145 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4146 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4147 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4148 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4149 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4150 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4151 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4152 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4153 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4154 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4155 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4156 2024-09-19T07:54:14.579 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4157 2024-09-19T07:54:14.580 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4158 2024-09-19T07:54:14.580 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4159 2024-09-19T07:54:14.580 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4160 2024-09-19T07:54:14.580 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4161 2024-09-19T07:54:14.580 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4162 2024-09-19T07:54:14.581 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4163 2024-09-19T07:54:14.581 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4164 2024-09-19T07:54:14.581 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4165 2024-09-19T07:54:14.581 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4166 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4167 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4168 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4169 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4170 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4171 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4172 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4173 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4174 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 5002 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 548 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 549 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6102 2024-09-19T07:54:14.582 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6104 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6106 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6110 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6111 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6112 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6113 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6117 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 826 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 827 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 828 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-19T07:54:14.583 INFO:teuthology.orchestra.run.smithi103.stdout:total 7777 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 19 07:38 . 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 19 07:38 .. 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 112 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 113 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 1247 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1247_fsm 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1247_vm 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 07:38 1249 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1249_fsm 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1249_vm 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 07:38 1255 2024-09-19T07:54:14.584 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1255_fsm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1255_vm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 07:38 1259 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1259_fsm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1259_vm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 07:38 13344 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13344_fsm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13344_vm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13346 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13348 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13349_fsm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349_vm 2024-09-19T07:54:14.585 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13351 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13353 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13354_fsm 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354_vm 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13356 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13358 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13359_fsm 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359_vm 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13361 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13363 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1417 2024-09-19T07:54:14.586 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1418 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 174 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 175 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2187 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2224 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2328 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2336 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2337 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2579 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2600 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2600_fsm 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2600_vm 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601 2024-09-19T07:54:14.587 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2601_fsm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601_vm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2602 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2602_fsm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2602_vm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2603 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2603_fsm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2603_vm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2604 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2605 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2605_fsm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2605_vm 2024-09-19T07:54:14.588 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2606_fsm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606_vm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2607 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2607_fsm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2607_vm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 07:38 2608 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2608_fsm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2608_vm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 07:38 2609 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2609_fsm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2609_vm 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2610 2024-09-19T07:54:14.589 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2610_fsm 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2610_vm 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2611 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2612_fsm 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612_vm 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2613 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2615_fsm 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615_vm 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2616 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2616_fsm 2024-09-19T07:54:14.590 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2616_vm 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 07:38 2617 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2617_fsm 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2617_vm 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 07:38 2618 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2618_fsm 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2618_vm 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 19 07:38 2619 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2619_fsm 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2619_vm 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2620 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2650 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2651 2024-09-19T07:54:14.591 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2652 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2653 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2654 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2655 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2656 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2657 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 07:38 2658 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 2659 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2660 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2661 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2662 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2663 2024-09-19T07:54:14.592 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2664 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2665 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2666 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2667 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2668 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2669 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2670 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 07:38 2673 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 07:38 2674 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 07:38 2675 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2678 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2679 2024-09-19T07:54:14.593 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2680 2024-09-19T07:54:14.594 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2681 2024-09-19T07:54:14.594 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2682 2024-09-19T07:54:14.594 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2683 2024-09-19T07:54:14.594 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2684 2024-09-19T07:54:14.594 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2685 2024-09-19T07:54:14.597 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2686 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2687 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2688 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2689 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 07:38 2690 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 07:38 2691 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2692 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2693 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2696 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2699 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2701 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2702 2024-09-19T07:54:14.598 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2703 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2704 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2753 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2753_fsm 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2753_vm 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2754 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2755 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2756 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2757 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2830 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2831 2024-09-19T07:54:14.599 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2832 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2833 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2834 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2835 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2836 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2837 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 07:38 2838 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2838_fsm 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2838_vm 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2839 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840_fsm 2024-09-19T07:54:14.600 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2840_vm 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2841 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2995 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2996 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3079_fsm 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079_vm 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3080 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3081 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3085 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3118 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3119 2024-09-19T07:54:14.601 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 07:38 3164 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3256 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3257 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3258 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3350 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3351 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3379 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3380 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3381 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3394 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3394_fsm 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3394_vm 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3395 2024-09-19T07:54:14.602 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3429 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3430 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3431 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3433 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3439 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3440 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3455 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 07:38 3456 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3456_fsm 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3456_vm 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3466 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3467 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3468 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3501 2024-09-19T07:54:14.603 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3502 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3503 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3534 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3541_fsm 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541_vm 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3542 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3574 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3575 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3576 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3596 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3597 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3598 2024-09-19T07:54:14.604 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3599 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3600_fsm 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600_vm 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3601_fsm 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601_vm 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3602_fsm 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602_vm 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603_fsm 2024-09-19T07:54:14.605 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3603_vm 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3604 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3605 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3606 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3607 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3608 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 3609 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3712 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3764_fsm 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764_vm 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3766 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3767 2024-09-19T07:54:14.606 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3997 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4143 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4144 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4145 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4146 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4147 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4148 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4149 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4150 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4151 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4152 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4153 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4154 2024-09-19T07:54:14.607 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4155 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4156 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4157 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4158 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4159 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4160 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4161 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4162 2024-09-19T07:54:14.608 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4163 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4164 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4165 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4166 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4167 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4168 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4169 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4170 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4171 2024-09-19T07:54:14.609 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4172 2024-09-19T07:54:14.612 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4173 2024-09-19T07:54:14.612 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4174 2024-09-19T07:54:14.612 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 5002 2024-09-19T07:54:14.612 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 548 2024-09-19T07:54:14.612 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 549 2024-09-19T07:54:14.612 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6102 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6104 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6106 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6110 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6111 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6112 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6113 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6117 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 826 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 827 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 828 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:54:14.613 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:total 594766 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 19 07:43 . 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 19 07:38 .. 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 112 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 113 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 1247 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1247_fsm 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1247_vm 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 19 07:38 1249 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1249_fsm 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1249_vm 2024-09-19T07:54:14.614 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 19 07:38 1255 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1255_fsm 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1255_vm 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 19 07:53 1259 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1259_fsm 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1259_vm 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 19 07:38 13344 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13344_fsm 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13344_vm 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13346 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13348 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349 2024-09-19T07:54:14.615 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13349_fsm 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13349_vm 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13351 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13353 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13354_fsm 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13354_vm 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13356 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13358 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 13359_fsm 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13359_vm 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 13361 2024-09-19T07:54:14.616 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 13363 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1417 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 1418 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443400192 Sep 19 07:53 16396 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 19 07:53 16396_fsm 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:43 16396_vm 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073152 Sep 19 07:53 16397 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:53 16397_fsm 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:53 16397_vm 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1613824 Sep 19 07:53 16399 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:53 16399_fsm 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:53 16399_vm 2024-09-19T07:54:14.617 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:53 16400 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 07:53 16402 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 19 07:53 16404 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 82567168 Sep 19 07:53 16406 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:53 16406_fsm 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:52 16406_vm 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 174 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 175 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2187 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2224 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2328 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2336 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2337 2024-09-19T07:54:14.618 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2579 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2600 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2600_fsm 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2600_vm 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2601_fsm 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2601_vm 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2602 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2602_fsm 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2602_vm 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2603 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2603_fsm 2024-09-19T07:54:14.619 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2603_vm 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2604 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2605 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2605_fsm 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2605_vm 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2606_fsm 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2606_vm 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2607 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2607_fsm 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2607_vm 2024-09-19T07:54:14.621 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 19 07:38 2608 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2608_fsm 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2608_vm 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 19 07:38 2609 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2609_fsm 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2609_vm 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2610 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2610_fsm 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2610_vm 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2611 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2612_fsm 2024-09-19T07:54:14.622 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2612_vm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2613 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2615_fsm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2615_vm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2616 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2616_fsm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2616_vm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 19 07:38 2617 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2617_fsm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2617_vm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 19 07:38 2618 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2618_fsm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2618_vm 2024-09-19T07:54:14.623 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 19 07:53 2619 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:40 2619_fsm 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2619_vm 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2620 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2650 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2651 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2652 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2653 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2654 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2655 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2656 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2657 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 19 07:38 2658 2024-09-19T07:54:14.624 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 19 07:38 2659 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2660 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2661 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2662 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2663 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2664 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2665 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2666 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2667 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2668 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2669 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2670 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 19 07:38 2673 2024-09-19T07:54:14.625 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 19 07:38 2674 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 19 07:38 2675 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2678 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2679 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2680 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2681 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2682 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2683 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2684 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2685 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2686 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2687 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2688 2024-09-19T07:54:14.626 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 19 07:38 2689 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 19 07:38 2690 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 19 07:38 2691 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2692 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2693 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:42 2696 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2699 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2701 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2702 2024-09-19T07:54:14.627 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2703 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 2704 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2753 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2753_fsm 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2753_vm 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2754 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2755 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2756 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 2757 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2830 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2831 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2832 2024-09-19T07:54:14.628 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2833 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2834 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2835 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2836 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2837 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 19 07:38 2838 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2838_fsm 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2838_vm 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2839 2024-09-19T07:54:14.629 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2840_fsm 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2840_vm 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2841 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2995 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2996 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3079_fsm 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3079_vm 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3080 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3081 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3085 2024-09-19T07:54:14.630 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3118 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3119 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 19 07:38 3164 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3256 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3257 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3258 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3350 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3351 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3379 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3380 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3381 2024-09-19T07:54:14.631 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3394 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3394_fsm 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3394_vm 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3395 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3429 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3430 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3431 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3433 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3439 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3440 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 19 07:38 3455 2024-09-19T07:54:14.632 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 19 07:38 3456 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3456_fsm 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3456_vm 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3466 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3467 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3468 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3501 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3502 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3503 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3534 2024-09-19T07:54:14.633 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3541_fsm 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3541_vm 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3542 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3574 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3575 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3576 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3596 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3597 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3598 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3599 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600 2024-09-19T07:54:14.634 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3600_fsm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3600_vm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3601_fsm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3601_vm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3602_fsm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3602_vm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3603_fsm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3603_vm 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3604 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3605 2024-09-19T07:54:14.635 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3606 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3607 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3608 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 19 07:38 3609 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3712 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 3764_fsm 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3764_vm 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3766 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 3767 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3997 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4143 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4144 2024-09-19T07:54:14.636 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4145 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4146 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4147 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4148 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4149 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4150 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4151 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4152 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4153 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4154 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4155 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4156 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4157 2024-09-19T07:54:14.637 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4158 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4159 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4160 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4161 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4162 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4163 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4164 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4165 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4166 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4167 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4168 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4169 2024-09-19T07:54:14.638 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4170 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4171 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4172 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4173 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4174 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 5002 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 548 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 549 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6102 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6104 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6106 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6110 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6111 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6112 2024-09-19T07:54:14.639 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6113 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6117 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 826 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 827 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 828 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 19 07:38 pg_internal.init 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:38 PG_VERSION 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.640 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 19 07:38 .. 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/global: 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:total 560 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 19 07:38 . 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.641 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1213 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1213_fsm 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1213_vm 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1214 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1214_fsm 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1214_vm 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 1232 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 1233 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1260 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1260_fsm 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1260_vm 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1261 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1261_fsm 2024-09-19T07:54:14.642 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1261_vm 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1262 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 1262_fsm 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 1262_vm 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2396 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 19 07:38 2396_fsm 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2396_vm 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2397 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2671 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2672 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2676 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2677 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2694 2024-09-19T07:54:14.643 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2695 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2697 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 19 07:38 2698 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2846 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2847 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2964 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2965 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 2966 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 2967 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 3592 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 3593 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4060 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4061 2024-09-19T07:54:14.644 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4175 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4176 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4177 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4178 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4181 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4182 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4183 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4184 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 4185 2024-09-19T07:54:14.645 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 4186 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6000 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6001 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6002 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 6100 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6114 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 6115 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:54 pg_control 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 19 07:38 pg_filenode.map 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 19 07:38 pg_internal.init 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/log: 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:total 2 2024-09-19T07:54:14.646 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1495 Sep 19 07:54 postgresql-Thu.log 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.647 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:total 1 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:54 . 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 mappings 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 19 07:54 replorigin_checkpoint 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 snapshots 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.648 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:54 .. 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:54 .. 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 . 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 members 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 offsets 2024-09-19T07:54:14.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:total 8 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 .. 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:38 0000 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:total 8 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:38 . 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:38 .. 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 19 07:43 0000 2024-09-19T07:54:14.650 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.651 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.652 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2cb8dfe1', total seconds elapsed: 32.7 2024-09-19T07:54:14.653 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.653 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.653 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-19T07:54:14.653 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:total 15 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 19 07:54 . 2024-09-19T07:54:14.654 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 19 07:54 db_0.stat 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 19 07:54 db_13494.stat 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 19 07:54 global.stat 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:54 . 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-19T07:54:14.655 INFO:teuthology.orchestra.run.smithi103.stdout:total 240 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 19 07:52 . 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 245760 Sep 19 07:53 0017 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.656 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:total 1048576 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 19 07:54 . 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:54 000000010000000000000096 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:50 000000010000000000000097 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:50 000000010000000000000098 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:49 000000010000000000000099 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 00000001000000000000009A 2024-09-19T07:54:14.657 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 00000001000000000000009B 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 00000001000000000000009C 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:50 00000001000000000000009D 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 00000001000000000000009E 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 00000001000000000000009F 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000A0 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000A1 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:49 0000000100000000000000A2 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000A3 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000A4 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000A5 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000A6 2024-09-19T07:54:14.658 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000A7 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:49 0000000100000000000000A8 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:49 0000000100000000000000A9 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000AA 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000AB 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000AC 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000AD 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000AE 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000AF 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:49 0000000100000000000000B0 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000B1 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000B2 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:48 0000000100000000000000B3 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000B4 2024-09-19T07:54:14.659 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000B5 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:51 0000000100000000000000B6 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:49 0000000100000000000000B7 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:50 0000000100000000000000B8 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:47 0000000100000000000000B9 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000BA 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000BB 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000BC 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000BD 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000BE 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000BF 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C0 2024-09-19T07:54:14.660 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C1 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C2 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C3 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C4 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C5 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:53 0000000100000000000000C6 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C7 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C8 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000C9 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000CA 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000CB 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000CC 2024-09-19T07:54:14.661 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000CD 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000CE 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000CF 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000D0 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000D1 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000D2 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000D3 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:53 0000000100000000000000D4 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 19 07:52 0000000100000000000000D5 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 archive_status 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:total 0 2024-09-19T07:54:14.662 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 19 07:38 . 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 19 07:54 .. 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout:total 384 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 19 07:52 . 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 19 07:54 .. 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 19 07:52 0000 2024-09-19T07:54:14.663 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 19 07:53 0001 2024-09-19T07:54:14.663 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T07:54:14.674 INFO:tasks.quiescer:joining Quiescers 2024-09-19T07:54:14.674 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T07:54:14.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:54:14.940 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.593+0000 7f37eca9c640 1 -- 172.21.15.103:0/2576140965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 msgr2=0x7f37e81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.593+0000 7f37eca9c640 1 --2- 172.21.15.103:0/2576140965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81067e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009900 tx=0x7f37d002a410 comp rx=0 tx=0).stop 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.593+0000 7f37eca9c640 1 -- 172.21.15.103:0/2576140965 shutdown_connections 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 --2- 172.21.15.103:0/2576140965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e8106f10 0x7f37e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 --2- 172.21.15.103:0/2576140965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81067e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 --2- 172.21.15.103:0/2576140965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e8101de0 0x7f37e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 -- 172.21.15.103:0/2576140965 >> 172.21.15.103:0/2576140965 conn(0x7f37e80fd8c0 msgr2=0x7f37e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 -- 172.21.15.103:0/2576140965 shutdown_connections 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 -- 172.21.15.103:0/2576140965 wait complete. 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 Processor -- start 2024-09-19T07:54:15.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 -- start start 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e8101de0 0x7f37e8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.594+0000 7f37eca9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37eca9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e8106f10 0x7f37e819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37eca9c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8111f30 con 0x7f37e8102790 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37eca9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8111db0 con 0x7f37e8101de0 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37eca9c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81120b0 con 0x7f37e8106f10 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51214/0 (socket says 172.21.15.103:51214) 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 -- 172.21.15.103:0/789067479 learned_addr learned my addr 172.21.15.103:0/789067479 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:15.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e6d9d640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e8106f10 0x7f37e819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e659c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e8101de0 0x7f37e8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 -- 172.21.15.103:0/789067479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e8106f10 msgr2=0x7f37e819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e8106f10 0x7f37e819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 -- 172.21.15.103:0/789067479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e8101de0 msgr2=0x7f37e8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e8101de0 0x7f37e8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 -- 172.21.15.103:0/789067479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e819d330 con 0x7f37e8102790 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37e5d9b640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81988c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f37d0006010 tx=0x7f37d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37df7fe640 1 -- 172.21.15.103:0/789067479 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d003f030 con 0x7f37e8102790 2024-09-19T07:54:15.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37df7fe640 1 -- 172.21.15.103:0/789067479 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d0003ce0 con 0x7f37e8102790 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e819d5c0 con 0x7f37e8102790 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.595+0000 7f37df7fe640 1 -- 172.21.15.103:0/789067479 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0035070 con 0x7f37e8102790 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.596+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a4f00 con 0x7f37e8102790 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.597+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e81032c0 con 0x7f37e8102790 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.598+0000 7f37df7fe640 1 -- 172.21.15.103:0/789067479 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f37d0042030 con 0x7f37e8102790 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.598+0000 7f37df7fe640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37d40a33b0 0x7f37d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.599+0000 7f37e659c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37d40a33b0 0x7f37d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.599+0000 7f37df7fe640 1 -- 172.21.15.103:0/789067479 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f37d00f0430 con 0x7f37e8102790 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.599+0000 7f37e659c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37d40a33b0 0x7f37d40a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f37d8007ec0 tx=0x7f37d80075b0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:15.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.602+0000 7f37df7fe640 1 -- 172.21.15.103:0/789067479 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f37d00bc870 con 0x7f37e8102790 2024-09-19T07:54:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.808+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37e810a0f0 con 0x7f37e8102790 2024-09-19T07:54:15.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.809+0000 7f37df7fe640 1 -- 172.21.15.103:0/789067479 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7f37d00c0b70 con 0x7f37e8102790 2024-09-19T07:54:15.810 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:15.810 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":54,"btime":"2024-09-19T07:53:10:110704+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:53:09.872640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:54:15.810 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 54 2024-09-19T07:54:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37d40a33b0 msgr2=0x7f37d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37d40a33b0 0x7f37d40a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f37d8007ec0 tx=0x7f37d80075b0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 msgr2=0x7f37e81988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81988c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f37d0006010 tx=0x7f37d00042f0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 shutdown_connections 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f37d40a33b0 0x7f37d40a5870 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e8106f10 0x7f37e819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37e8102790 0x7f37e81988c0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 --2- 172.21.15.103:0/789067479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37e8101de0 0x7f37e8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 >> 172.21.15.103:0/789067479 conn(0x7f37e80fd8c0 msgr2=0x7f37e80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 shutdown_connections 2024-09-19T07:54:15.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:15.811+0000 7f37eca9c640 1 -- 172.21.15.103:0/789067479 wait complete. 2024-09-19T07:54:16.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:16 smithi134 ceph-mon[35414]: pgmap v511: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-09-19T07:54:16.387 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:16 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/789067479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:16.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:16 smithi103 ceph-mon[31201]: pgmap v511: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-09-19T07:54:16.388 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:16 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/789067479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:16 smithi174 ceph-mon[35370]: pgmap v511: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 73 KiB/s wr, 32 op/s 2024-09-19T07:54:16.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:16 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/789067479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:16.674 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --cancel --all' 2024-09-19T07:54:16.674 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --cancel --all 2024-09-19T07:54:16.923 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 -- 172.21.15.103:0/3584645564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8106a90 msgr2=0x7fc2e8106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3584645564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8106a90 0x7fc2e8106e70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009cd0 tx=0x7fc2e002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 -- 172.21.15.103:0/3584645564 shutdown_connections 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3584645564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2e8101430 0x7fc2e810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3584645564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2e8100a90 0x7fc2e8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3584645564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8106a90 0x7fc2e8106e70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 -- 172.21.15.103:0/3584645564 >> 172.21.15.103:0/3584645564 conn(0x7fc2e80fc7d0 msgr2=0x7fc2e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 -- 172.21.15.103:0/3584645564 shutdown_connections 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.474+0000 7fc2ed057640 1 -- 172.21.15.103:0/3584645564 wait complete. 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 Processor -- start 2024-09-19T07:54:17.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 -- start start 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8100a90 0x7fc2e8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2e8101430 0x7fc2e8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2e8106a90 0x7fc2e819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8110eb0 con 0x7fc2e8100a90 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8110d30 con 0x7fc2e8106a90 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2ed057640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8111030 con 0x7fc2e8101430 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8100a90 0x7fc2e8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8100a90 0x7fc2e8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51234/0 (socket says 172.21.15.103:51234) 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/3785459819 learned_addr learned my addr 172.21.15.103:0/3785459819 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2e759e640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2e8106a90 0x7fc2e819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.475+0000 7fc2e659c640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2e8101430 0x7fc2e8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/3785459819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2e8101430 msgr2=0x7fc2e8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2e8101430 0x7fc2e8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/3785459819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2e8106a90 msgr2=0x7fc2e819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2e8106a90 0x7fc2e819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/3785459819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e819d4c0 con 0x7fc2e8100a90 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2e659c640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2e8101430 0x7fc2e8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8100a90 0x7fc2e8198760 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc2e002ae00 tx=0x7fc2e0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2dffff640 1 -- 172.21.15.103:0/3785459819 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0004040 con 0x7fc2e8100a90 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e819d750 con 0x7fc2e8100a90 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2dffff640 1 -- 172.21.15.103:0/3785459819 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e00046c0 con 0x7fc2e8100a90 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2dffff640 1 -- 172.21.15.103:0/3785459819 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0046480 con 0x7fc2e8100a90 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.476+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e81a5040 con 0x7fc2e8100a90 2024-09-19T07:54:17.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.478+0000 7fc2dffff640 1 -- 172.21.15.103:0/3785459819 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fc2e00305a0 con 0x7fc2e8100a90 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 Processor -- start 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 -- start start 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ab1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3a16200 con 0x55d0c3ab2220 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0c39bcdc0 con 0x55d0c3ab1900 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2ef430740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3a121c0 con 0x55d0c3ac0ff0 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51242/0 (socket says 172.21.15.103:51242) 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/2397038 learned_addr learned my addr 172.21.15.103:0/2397038 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:17.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2dffff640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc2d40a3430 0x7fc2d40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.479+0000 7fc2e759e640 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ab1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e659c640 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e759e640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc2d40a3430 0x7fc2d40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2dffff640 1 -- 172.21.15.103:0/3785459819 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fc2e002e030 con 0x7fc2e8100a90 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/2397038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 msgr2=0x55d0c3ac33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/2397038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 msgr2=0x55d0c3ab1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ab1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/2397038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c3787620 con 0x55d0c3ab2220 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e759e640 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ab1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e659c640 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac0ab0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00bd500 tx=0x7fc2e00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a505a5a79772cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2e759e640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc2d40a3430 0x7fc2d40a58f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80153e0 tx=0x7fc2d80046c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2dd7fa640 1 -- 172.21.15.103:0/2397038 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00bd5d0 con 0x55d0c3ab2220 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2dd7fa640 1 -- 172.21.15.103:0/2397038 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e0036070 con 0x55d0c3ab2220 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.480+0000 7fc2dd7fa640 1 -- 172.21.15.103:0/2397038 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00bd160 con 0x55d0c3ab2220 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 -- 172.21.15.103:0/2397038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 msgr2=0x55d0c3ac0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac0ab0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00bd500 tx=0x7fc2e00095c0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 -- 172.21.15.103:0/2397038 shutdown_connections 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac0ab0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2397038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ab1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 -- 172.21.15.103:0/2397038 >> 172.21.15.103:0/2397038 conn(0x55d0c3aadf90 msgr2=0x55d0c3aae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 -- 172.21.15.103:0/2397038 shutdown_connections 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 -- 172.21.15.103:0/2397038 wait complete. 2024-09-19T07:54:17.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 -- start start 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.481+0000 7fc2ef430740 1 Processor -- start 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3abdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3abe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3ac39e0 con 0x55d0c3ab2220 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0c39b0c50 con 0x55d0c3ab1900 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0c3ac3b60 con 0x55d0c3ac0ff0 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 10 client.0 _ll_register_callbacks cb 0x55d0c39c2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3abdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3abdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51256/0 (socket says 172.21.15.103:51256) 2024-09-19T07:54:17.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 -- 172.21.15.103:0/4063634202 learned_addr learned my addr 172.21.15.103:0/4063634202 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e759e640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3abe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 -- 172.21.15.103:0/4063634202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 msgr2=0x55d0c3abe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3abe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 -- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 msgr2=0x55d0c3ac0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 -- 172.21.15.103:0/4063634202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c3abea20 con 0x55d0c3ab2220 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e759e640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3abe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2e659c640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3abdc50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fc2e810dcf0 tx=0x7fc2d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7314f5517d995c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/4063634202 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0004220 con 0x55d0c3ab2220 2024-09-19T07:54:17.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/4063634202 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fc2d00043c0 con 0x55d0c3ab2220 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.482+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e0036070 con 0x55d0c3ab2220 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/4063634202 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc2d00035f0 con 0x55d0c3ab2220 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 10 client.18708 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d0c3acf5a0 con 0x55d0c3ab2220 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/4063634202 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e0036070 con 0x55d0c3ab2220 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/4063634202 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fc2d001b9d0 con 0x55d0c3ab2220 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 10 client.18708 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 10 client.18708 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 20 client.18708 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3ad0ab0 0x55d0c3ad2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 4 client.18708 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c3a23770 con 0x55d0c3ad0ab0 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.483+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3ad0ab0 0x55d0c3ad2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.484+0000 7fc2ce7fc640 10 client.18708 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.484+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3ad0ab0 0x55d0c3ad2ed0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a235af405cbe90 server_cookie=808139f9c607bdfa in_seq=0 out_seq=0 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.484+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/4063634202 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0c3a23770 con 0x55d0c3ad0ab0 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.484+0000 7fc2ce7fc640 10 client.18708 handle_command_reply: tid=0 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.484+0000 7fc2ef430740 1 client.18708 shutdown 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 msgr2=0x55d0c3abdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3abdc50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fc2e810dcf0 tx=0x7fc2d00117b0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 shutdown_connections 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3ad0ab0 0x55d0c3ad2ed0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3abe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3abdc50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 --2- 172.21.15.103:0/4063634202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ce7fc640 0 client.18708 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 >> 172.21.15.103:0/4063634202 conn(0x55d0c3aadf90 msgr2=0x55d0c3ab3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 shutdown_connections 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 1 -- 172.21.15.103:0/4063634202 wait complete. 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.485+0000 7fc2ef430740 20 client.18708 trim_cache size 0 max 16384 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.501+0000 7fc2ef430740 1 Processor -- start 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.501+0000 7fc2ef430740 1 -- start start 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab1900 0x55d0c3abcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab2220 0x55d0c3ad8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ad8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2ef430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e81ab730 con 0x55d0c3ab1900 2024-09-19T07:54:17.529 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:17.529 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 54, 2024-09-19T07:54:17.529 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24494, 2024-09-19T07:54:17.529 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 56, 2024-09-19T07:54:17.529 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": {} 2024-09-19T07:54:17.529 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:17.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2ef430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e81ad270 con 0x55d0c3ab2220 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2ef430740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e81abe00 con 0x55d0c3ac0ff0 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab1900 0x55d0c3abcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab1900 0x55d0c3abcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51262/0 (socket says 172.21.15.103:51262) 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/848860734 learned_addr learned my addr 172.21.15.103:0/848860734 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/848860734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 msgr2=0x55d0c3ad8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/848860734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ad8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/848860734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab2220 msgr2=0x55d0c3ad8780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/848860734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab2220 0x55d0c3ad8780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 -- 172.21.15.103:0/848860734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c3aaddc0 con 0x55d0c3ab1900 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/848860734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab1900 0x55d0c3abcb90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005e30 tx=0x7fc2e00b9bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb777a45b41eac41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2dcff9640 1 -- 172.21.15.103:0/848860734 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00d5020 con 0x55d0c3ab1900 2024-09-19T07:54:17.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2dcff9640 1 -- 172.21.15.103:0/848860734 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e0036040 con 0x55d0c3ab1900 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.502+0000 7fc2dcff9640 1 -- 172.21.15.103:0/848860734 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00bd340 con 0x55d0c3ab1900 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 -- 172.21.15.103:0/848860734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab1900 msgr2=0x55d0c3abcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 --2- 172.21.15.103:0/848860734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab1900 0x55d0c3abcb90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005e30 tx=0x7fc2e00b9bd0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 -- 172.21.15.103:0/848860734 shutdown_connections 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 --2- 172.21.15.103:0/848860734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ad8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 --2- 172.21.15.103:0/848860734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab2220 0x55d0c3ad8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 --2- 172.21.15.103:0/848860734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab1900 0x55d0c3abcb90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 -- 172.21.15.103:0/848860734 >> 172.21.15.103:0/848860734 conn(0x55d0c3aadf90 msgr2=0x55d0c3ab9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 -- 172.21.15.103:0/848860734 shutdown_connections 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 -- 172.21.15.103:0/848860734 wait complete. 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 -- start start 2024-09-19T07:54:17.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.503+0000 7fc2ef430740 1 Processor -- start 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e81acba0 con 0x55d0c3ab2220 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e81ac4d0 con 0x55d0c3ab1900 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e81a9750 con 0x55d0c3ac0ff0 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 10 client.0 _ll_register_callbacks cb 0x7fc2d000ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51268/0 (socket says 172.21.15.103:51268) 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 -- 172.21.15.103:0/2438063432 learned_addr learned my addr 172.21.15.103:0/2438063432 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 -- 172.21.15.103:0/2438063432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 msgr2=0x55d0c3ac56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 -- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 msgr2=0x55d0c3ac4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 -- 172.21.15.103:0/2438063432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c3b3aa20 con 0x55d0c3ab2220 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2e659c640 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac5160 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00058c0 tx=0x7fc2d0012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25af5ea6a2213995 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/2438063432 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0028070 con 0x55d0c3ab2220 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e0036040 con 0x55d0c3ab2220 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.504+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/2438063432 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fc2d0005480 con 0x55d0c3ab2220 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/2438063432 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc2d000b2c0 con 0x55d0c3ab2220 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 10 client.18714 fetch_fsmap learned FSMap version 54 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d0c3b3bd60 con 0x55d0c3ab2220 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/2438063432 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e0036040 con 0x55d0c3ab2220 2024-09-19T07:54:17.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/2438063432 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 54) ==== 8378+0+0 (secure 0 0 0) 0x7fc2d001b850 con 0x55d0c3ab2220 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 10 client.18714 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 10 client.18714 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 20 client.18714 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3b3c430 0x55d0c3b3e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 4 client.18714 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55d0c3a1dc80 con 0x55d0c3b3c430 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.505+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3b3c430 0x55d0c3b3e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ce7fc640 10 client.18714 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2e6d9d640 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3b3c430 0x55d0c3b3e850 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf98c1448d159c6 server_cookie=b57475368ba78c92 in_seq=0 out_seq=0 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ce7fc640 1 -- 172.21.15.103:0/2438063432 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55d0c3a1dc80 con 0x55d0c3b3c430 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ce7fc640 10 client.18714 handle_command_reply: tid=0 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 client.18714 shutdown 2024-09-19T07:54:17.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 msgr2=0x55d0c3ac5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac5160 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00058c0 tx=0x7fc2d0012730 comp rx=0 tx=0).stop 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 shutdown_connections 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x55d0c3b3c430 0x55d0c3b3e850 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c3ac0ff0 0x55d0c3ac56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0c3ab2220 0x55d0c3ac5160 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ce7fc640 0 client.18714 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 --2- 172.21.15.103:0/2438063432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0c3ab1900 0x55d0c3ac4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 >> 172.21.15.103:0/2438063432 conn(0x55d0c3aadf90 msgr2=0x55d0c3ab3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 shutdown_connections 2024-09-19T07:54:17.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 1 -- 172.21.15.103:0/2438063432 wait complete. 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.506+0000 7fc2ef430740 20 client.18714 trim_cache size 0 max 16384 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc2d40a3430 msgr2=0x7fc2d40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc2d40a3430 0x7fc2d40a58f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80153e0 tx=0x7fc2d80046c0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8100a90 msgr2=0x7fc2e8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8100a90 0x7fc2e8198760 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc2e002ae00 tx=0x7fc2e0003970 comp rx=0 tx=0).stop 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 shutdown_connections 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fc2d40a3430 0x7fc2d40a58f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2e8106a90 0x7fc2e819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2e8101430 0x7fc2e8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 --2- 172.21.15.103:0/3785459819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8100a90 0x7fc2e8198760 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 >> 172.21.15.103:0/3785459819 conn(0x7fc2e80fc7d0 msgr2=0x7fc2e80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 shutdown_connections 2024-09-19T07:54:17.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:17.507+0000 7fc2ed057640 1 -- 172.21.15.103:0/3785459819 wait complete. 2024-09-19T07:54:18.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:18 smithi134 ceph-mon[35414]: pgmap v512: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 179 KiB/s wr, 38 op/s 2024-09-19T07:54:18.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:18 smithi174 ceph-mon[35370]: pgmap v512: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 179 KiB/s wr, 38 op/s 2024-09-19T07:54:18.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:18 smithi103 ceph-mon[31201]: pgmap v512: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 179 KiB/s wr, 38 op/s 2024-09-19T07:54:18.531 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T07:54:18.531 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T07:54:18.542 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T07:54:18.542 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T07:54:18.542 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:54:18.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T07:54:18.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T07:54:18.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T07:54:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 -- 172.21.15.103:0/4054608265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd81069c0 msgr2=0x7ffbd8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 --2- 172.21.15.103:0/4054608265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd81069c0 0x7ffbd8106da0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009930 tx=0x7ffbd402a420 comp rx=0 tx=0).stop 2024-09-19T07:54:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 -- 172.21.15.103:0/4054608265 shutdown_connections 2024-09-19T07:54:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 --2- 172.21.15.103:0/4054608265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd8101350 0x7ffbd810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 --2- 172.21.15.103:0/4054608265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbd81009b0 0x7ffbd8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 --2- 172.21.15.103:0/4054608265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd81069c0 0x7ffbd8106da0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 -- 172.21.15.103:0/4054608265 >> 172.21.15.103:0/4054608265 conn(0x7ffbd8077990 msgr2=0x7ffbd80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 -- 172.21.15.103:0/4054608265 shutdown_connections 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.784+0000 7ffbe012c640 1 -- 172.21.15.103:0/4054608265 wait complete. 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 Processor -- start 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 -- start start 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd81009b0 0x7ffbd8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd8101350 0x7ffbd8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbd81069c0 0x7ffbd81a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8110e00 con 0x7ffbd8101350 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8110c80 con 0x7ffbd81009b0 2024-09-19T07:54:18.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbe012c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8110f80 con 0x7ffbd81069c0 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd8101350 0x7ffbd8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd8101350 0x7ffbd8199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35512/0 (socket says 172.21.15.103:35512) 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 -- 172.21.15.103:0/2921188752 learned_addr learned my addr 172.21.15.103:0/2921188752 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbde6c9640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbd81069c0 0x7ffbd81a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbddec8640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd81009b0 0x7ffbd8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 -- 172.21.15.103:0/2921188752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbd81069c0 msgr2=0x7ffbd81a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbd81069c0 0x7ffbd81a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 -- 172.21.15.103:0/2921188752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd81009b0 msgr2=0x7ffbd8199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd81009b0 0x7ffbd8199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 -- 172.21.15.103:0/2921188752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd81a47c0 con 0x7ffbd8101350 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbddec8640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd81009b0 0x7ffbd8199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.785+0000 7ffbdd6c7640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd8101350 0x7ffbd8199740 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800d770 tx=0x7ffbc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.786+0000 7ffbceffd640 1 -- 172.21.15.103:0/2921188752 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8017070 con 0x7ffbd8101350 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.786+0000 7ffbceffd640 1 -- 172.21.15.103:0/2921188752 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8005500 con 0x7ffbd8101350 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.786+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd81a4990 con 0x7ffbd8101350 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.786+0000 7ffbceffd640 1 -- 172.21.15.103:0/2921188752 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8004e00 con 0x7ffbd8101350 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.786+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd81a4e60 con 0x7ffbd8101350 2024-09-19T07:54:18.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.786+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd8100e10 con 0x7ffbd8101350 2024-09-19T07:54:18.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.788+0000 7ffbceffd640 1 -- 172.21.15.103:0/2921188752 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ffbc8004fa0 con 0x7ffbd8101350 2024-09-19T07:54:18.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.789+0000 7ffbceffd640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffbc40a3460 0x7ffbc40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:18.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.789+0000 7ffbddec8640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffbc40a3460 0x7ffbc40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:18.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.789+0000 7ffbceffd640 1 -- 172.21.15.103:0/2921188752 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7ffbc80cf860 con 0x7ffbd8101350 2024-09-19T07:54:18.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.789+0000 7ffbddec8640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffbc40a3460 0x7ffbc40a5920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4000c00 tx=0x7ffbd4032c70 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:18.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.793+0000 7ffbceffd640 1 -- 172.21.15.103:0/2921188752 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffbc809c230 con 0x7ffbd8101350 2024-09-19T07:54:18.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.978+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ffbd8062630 con 0x7ffbc40a3460 2024-09-19T07:54:18.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.986+0000 7ffbceffd640 1 -- 172.21.15.103:0/2921188752 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7ffbd8062630 con 0x7ffbc40a3460 2024-09-19T07:54:18.986 INFO:teuthology.orchestra.run.smithi103.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T07:38:06", "first": "2024-09-19T07:39:00", "last": "2024-09-19T07:54:00", "last_pruned": "2024-09-19T07:54:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-09-19T07:54:18.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffbc40a3460 msgr2=0x7ffbc40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffbc40a3460 0x7ffbc40a5920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4000c00 tx=0x7ffbd4032c70 comp rx=0 tx=0).stop 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd8101350 msgr2=0x7ffbd8199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd8101350 0x7ffbd8199740 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800d770 tx=0x7ffbc800dc30 comp rx=0 tx=0).stop 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 shutdown_connections 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ffbc40a3460 0x7ffbc40a5920 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbd81069c0 0x7ffbd81a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbd8101350 0x7ffbd8199740 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 --2- 172.21.15.103:0/2921188752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbd81009b0 0x7ffbd8199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 >> 172.21.15.103:0/2921188752 conn(0x7ffbd8077990 msgr2=0x7ffbd8103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 shutdown_connections 2024-09-19T07:54:18.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:18.988+0000 7ffbe012c640 1 -- 172.21.15.103:0/2921188752 wait complete. 2024-09-19T07:54:18.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T07:54:19.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 -- 172.21.15.103:0/1594425588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81069c0 msgr2=0x7f33d8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 --2- 172.21.15.103:0/1594425588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81069c0 0x7f33d8106da0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009930 tx=0x7f33c802a420 comp rx=0 tx=0).stop 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 -- 172.21.15.103:0/1594425588 shutdown_connections 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 --2- 172.21.15.103:0/1594425588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d8101350 0x7f33d810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 --2- 172.21.15.103:0/1594425588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d81009b0 0x7f33d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 --2- 172.21.15.103:0/1594425588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81069c0 0x7f33d8106da0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 -- 172.21.15.103:0/1594425588 >> 172.21.15.103:0/1594425588 conn(0x7f33d8077990 msgr2=0x7f33d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 -- 172.21.15.103:0/1594425588 shutdown_connections 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.102+0000 7f33dd11a640 1 -- 172.21.15.103:0/1594425588 wait complete. 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 Processor -- start 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 -- start start 2024-09-19T07:54:19.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81009b0 0x7f33d8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d8101350 0x7f33d81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d81069c0 0x7f33d819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8110e00 con 0x7f33d81009b0 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8110c80 con 0x7f33d81069c0 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.103+0000 7f33dd11a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8110f80 con 0x7f33d8101350 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81009b0 0x7f33d8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81009b0 0x7f33d8197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35542/0 (socket says 172.21.15.103:35542) 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 -- 172.21.15.103:0/385971093 learned_addr learned my addr 172.21.15.103:0/385971093 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33ce59c640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d8101350 0x7f33d81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d759e640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d81069c0 0x7f33d819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 -- 172.21.15.103:0/385971093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d8101350 msgr2=0x7f33d81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d8101350 0x7f33d81977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 -- 172.21.15.103:0/385971093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d81069c0 msgr2=0x7f33d819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d81069c0 0x7f33d819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 -- 172.21.15.103:0/385971093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d819c280 con 0x7f33d81009b0 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d6d9d640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81009b0 0x7f33d8197290 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009230 tx=0x7f33c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d4d99640 1 -- 172.21.15.103:0/385971093 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c803f030 con 0x7f33d81009b0 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d819c510 con 0x7f33d81009b0 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d4d99640 1 -- 172.21.15.103:0/385971093 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c8003ad0 con 0x7f33d81009b0 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33d4d99640 1 -- 172.21.15.103:0/385971093 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8035070 con 0x7f33d81009b0 2024-09-19T07:54:19.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.104+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d81a3e00 con 0x7f33d81009b0 2024-09-19T07:54:19.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.105+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d8100e10 con 0x7f33d81009b0 2024-09-19T07:54:19.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.107+0000 7f33d4d99640 1 -- 172.21.15.103:0/385971093 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f33c8042030 con 0x7f33d81009b0 2024-09-19T07:54:19.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.107+0000 7f33d4d99640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33c40a3460 0x7f33c40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:19.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.107+0000 7f33ce59c640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33c40a3460 0x7f33c40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:19.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.107+0000 7f33d4d99640 1 -- 172.21.15.103:0/385971093 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f33c80f0150 con 0x7f33d81009b0 2024-09-19T07:54:19.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.108+0000 7f33ce59c640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33c40a3460 0x7f33c40a5920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f33d80ffe60 tx=0x7f33c000b410 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:19.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.111+0000 7f33d4d99640 1 -- 172.21.15.103:0/385971093 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33c80bc4d0 con 0x7f33d81009b0 2024-09-19T07:54:19.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.302+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 --> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f33d8103e80 con 0x7f33c40a3460 2024-09-19T07:54:19.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.309+0000 7f33d4d99640 1 -- 172.21.15.103:0/385971093 <== mgr.15183 v2:172.21.15.103:6800/3066951043 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f33d8103e80 con 0x7f33c40a3460 2024-09-19T07:54:19.310 INFO:teuthology.orchestra.run.smithi103.stdout:/ 1m 6m3h 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33c40a3460 msgr2=0x7f33c40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33c40a3460 0x7f33c40a5920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f33d80ffe60 tx=0x7f33c000b410 comp rx=0 tx=0).stop 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81009b0 msgr2=0x7f33d8197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81009b0 0x7f33d8197290 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009230 tx=0x7f33c8004270 comp rx=0 tx=0).stop 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 shutdown_connections 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f33c40a3460 0x7f33c40a5920 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33d81069c0 0x7f33d819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d8101350 0x7f33d81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 --2- 172.21.15.103:0/385971093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33d81009b0 0x7f33d8197290 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:19.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 >> 172.21.15.103:0/385971093 conn(0x7f33d8077990 msgr2=0x7f33d8103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:19.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 shutdown_connections 2024-09-19T07:54:19.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:19.311+0000 7f33dd11a640 1 -- 172.21.15.103:0/385971093 wait complete. 2024-09-19T07:54:19.323 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T07:54:19.335 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T07:54:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:20 smithi134 ceph-mon[35414]: pgmap v513: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 179 KiB/s wr, 38 op/s 2024-09-19T07:54:20.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:20 smithi134 ceph-mon[35414]: from='client.18720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:54:20.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:20 smithi134 ceph-mon[35414]: from='client.18726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:54:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:20 smithi174 ceph-mon[35370]: pgmap v513: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 179 KiB/s wr, 38 op/s 2024-09-19T07:54:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:20 smithi174 ceph-mon[35370]: from='client.18720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:54:20.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:20 smithi174 ceph-mon[35370]: from='client.18726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:54:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:20 smithi103 ceph-mon[31201]: pgmap v513: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 179 KiB/s wr, 38 op/s 2024-09-19T07:54:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:20 smithi103 ceph-mon[31201]: from='client.18720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:54:20.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:20 smithi103 ceph-mon[31201]: from='client.18726 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T07:54:21.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:21 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:21.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:21 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:21.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:21 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:22.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:22 smithi134 ceph-mon[35414]: pgmap v514: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 186 KiB/s wr, 45 op/s 2024-09-19T07:54:22.370 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:22 smithi103 ceph-mon[31201]: pgmap v514: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 186 KiB/s wr, 45 op/s 2024-09-19T07:54:22.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:22 smithi174 ceph-mon[35370]: pgmap v514: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 186 KiB/s wr, 45 op/s 2024-09-19T07:54:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:23 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:54:23.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:23 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:54:23.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:23 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:54:23.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:23 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:54:23.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:23 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:54:23.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:23 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:54:23.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:23 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:54:23.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:23 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:54:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:23 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:54:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:23 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:54:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:23 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:54:23.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:23 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:54:24.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:24 smithi134 ceph-mon[35414]: pgmap v515: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 162 KiB/s wr, 39 op/s 2024-09-19T07:54:24.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:24 smithi174 ceph-mon[35370]: pgmap v515: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 162 KiB/s wr, 39 op/s 2024-09-19T07:54:24.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:24 smithi103 ceph-mon[31201]: pgmap v515: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 162 KiB/s wr, 39 op/s 2024-09-19T07:54:26.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:26 smithi134 ceph-mon[35414]: pgmap v516: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 155 KiB/s wr, 38 op/s 2024-09-19T07:54:26.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:26 smithi174 ceph-mon[35370]: pgmap v516: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 155 KiB/s wr, 38 op/s 2024-09-19T07:54:26.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:26 smithi103 ceph-mon[31201]: pgmap v516: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 155 KiB/s wr, 38 op/s 2024-09-19T07:54:28.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:28 smithi134 ceph-mon[35414]: pgmap v517: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 172 KiB/s wr, 44 op/s 2024-09-19T07:54:28.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:28 smithi174 ceph-mon[35370]: pgmap v517: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 172 KiB/s wr, 44 op/s 2024-09-19T07:54:28.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:28 smithi103 ceph-mon[31201]: pgmap v517: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 172 KiB/s wr, 44 op/s 2024-09-19T07:54:30.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:30 smithi174 ceph-mon[35370]: pgmap v518: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-09-19T07:54:30.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:30 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:30 smithi103 ceph-mon[31201]: pgmap v518: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-09-19T07:54:30.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:30 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:30 smithi134 ceph-mon[35414]: pgmap v518: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-09-19T07:54:30.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:30 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:32.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:32 smithi134 ceph-mon[35414]: pgmap v519: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 80 KiB/s wr, 22 op/s 2024-09-19T07:54:32.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:32 smithi174 ceph-mon[35370]: pgmap v519: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 80 KiB/s wr, 22 op/s 2024-09-19T07:54:32.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:32 smithi103 ceph-mon[31201]: pgmap v519: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 80 KiB/s wr, 22 op/s 2024-09-19T07:54:34.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:34 smithi174 ceph-mon[35370]: pgmap v520: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 66 KiB/s wr, 12 op/s 2024-09-19T07:54:34.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:34 smithi103 ceph-mon[31201]: pgmap v520: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 66 KiB/s wr, 12 op/s 2024-09-19T07:54:34.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:34 smithi134 ceph-mon[35414]: pgmap v520: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 66 KiB/s wr, 12 op/s 2024-09-19T07:54:35.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:35 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:35.419 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T07:54:35.448 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:35 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:35.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:35 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:35.682 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:36.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 -- 172.21.15.103:0/1062924532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc101280 msgr2=0x7fcfcc10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 --2- 172.21.15.103:0/1062924532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc101280 0x7fcfcc10df40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009ca0 tx=0x7fcfc802a8c0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 -- 172.21.15.103:0/1062924532 shutdown_connections 2024-09-19T07:54:36.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 --2- 172.21.15.103:0/1062924532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfcc10e480 0x7fcfcc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 --2- 172.21.15.103:0/1062924532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc101280 0x7fcfcc10df40 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 --2- 172.21.15.103:0/1062924532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfcc1008d0 0x7fcfcc100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 -- 172.21.15.103:0/1062924532 >> 172.21.15.103:0/1062924532 conn(0x7fcfcc0fc7a0 msgr2=0x7fcfcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 -- 172.21.15.103:0/1062924532 shutdown_connections 2024-09-19T07:54:36.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.151+0000 7fcfd3948640 1 -- 172.21.15.103:0/1062924532 wait complete. 2024-09-19T07:54:36.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 Processor -- start 2024-09-19T07:54:36.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 -- start start 2024-09-19T07:54:36.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc1008d0 0x7fcfcc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfcc101280 0x7fcfcc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfcc10e480 0x7fcfcc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc114190 con 0x7fcfcc1008d0 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc114010 con 0x7fcfcc10e480 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd3948640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc114310 con 0x7fcfcc101280 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc1008d0 0x7fcfcc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc1008d0 0x7fcfcc19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48464/0 (socket says 172.21.15.103:48464) 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 -- 172.21.15.103:0/4112026782 learned_addr learned my addr 172.21.15.103:0/4112026782 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfcc101280 0x7fcfcc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfcc10e480 0x7fcfcc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 -- 172.21.15.103:0/4112026782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfcc101280 msgr2=0x7fcfcc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfcc101280 0x7fcfcc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 -- 172.21.15.103:0/4112026782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfcc10e480 msgr2=0x7fcfcc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfcc10e480 0x7fcfcc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd16e4640 1 -- 172.21.15.103:0/4112026782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc19f660 con 0x7fcfcc1008d0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfcc101280 0x7fcfcc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.152+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfcc10e480 0x7fcfcc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.153+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc1008d0 0x7fcfcc19a6b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d950 tx=0x7fcfc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.153+0000 7fcfc27fc640 1 -- 172.21.15.103:0/4112026782 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4017070 con 0x7fcfcc1008d0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.153+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfcc19f950 con 0x7fcfcc1008d0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.153+0000 7fcfc27fc640 1 -- 172.21.15.103:0/4112026782 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc4009c30 con 0x7fcfcc1008d0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.153+0000 7fcfc27fc640 1 -- 172.21.15.103:0/4112026782 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400f6b0 con 0x7fcfcc1008d0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.153+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfcc1a7230 con 0x7fcfcc1008d0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.155+0000 7fcfc27fc640 1 -- 172.21.15.103:0/4112026782 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fcfc40109d0 con 0x7fcfcc1008d0 2024-09-19T07:54:36.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 Processor -- start 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 -- start start 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfc27fc640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcfb40a33b0 0x7fcfb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397103c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971047140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971049a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563970fa22e0 con 0x56397103c4a0 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563970f49ce0 con 0x56397103cdc0 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563970f9e2a0 con 0x563971047680 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397103c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfc27fc640 1 -- 172.21.15.103:0/4112026782 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fcfc40cfde0 con 0x7fcfcc1008d0 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcfb40a33b0 0x7fcfb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397103c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48468/0 (socket says 172.21.15.103:48468) 2024-09-19T07:54:36.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2413585489 learned_addr learned my addr 172.21.15.103:0/2413585489 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971049a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971047140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2413585489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 msgr2=0x563971049a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971049a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2413585489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 msgr2=0x563971047140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971047140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2413585489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563970f2e440 con 0x56397103c4a0 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.156+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971047140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.157+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcfb40a33b0 0x7fcfb40a5870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc06a0f0 tx=0x7fcfbc014840 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.157+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397103c880 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcfc409bd50 tx=0x7fcfc4079610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecdf7a40d448d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.157+0000 7fcfb37fe640 1 -- 172.21.15.103:0/2413585489 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4009de0 con 0x56397103c4a0 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.157+0000 7fcfb37fe640 1 -- 172.21.15.103:0/2413585489 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc409c750 con 0x56397103c4a0 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.157+0000 7fcfb37fe640 1 -- 172.21.15.103:0/2413585489 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40bc140 con 0x56397103c4a0 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.157+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2413585489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 msgr2=0x56397103c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.157+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397103c880 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcfc409bd50 tx=0x7fcfc4079610 comp rx=0 tx=0).stop 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2413585489 shutdown_connections 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971049a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971047140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2413585489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397103c880 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2413585489 >> 172.21.15.103:0/2413585489 conn(0x563971038a30 msgr2=0x563971038e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2413585489 shutdown_connections 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2413585489 wait complete. 2024-09-19T07:54:36.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- start start 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 Processor -- start 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x563971046ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971044210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971044750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56397104a250 con 0x56397103c4a0 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563970f5d050 con 0x56397103cdc0 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56397104a3d0 con 0x563971047680 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd5d21740 10 client.0 _ll_register_callbacks cb 0x563970f9d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.158+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x563971046ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x563971046ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48478/0 (socket says 172.21.15.103:48478) 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2224849149 learned_addr learned my addr 172.21.15.103:0/2224849149 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971044210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971044750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2224849149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 msgr2=0x563971044750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971044750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 msgr2=0x563971044210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971044210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 -- 172.21.15.103:0/2224849149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563970d133c0 con 0x56397103c4a0 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971044750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x563971046ec0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcfc407bff0 tx=0x7fcfc409c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37b8901f8d55c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/2224849149 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4020030 con 0x56397103c4a0 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/2224849149 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fcfc40a00b0 con 0x56397103c4a0 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc409c750 con 0x56397103c4a0 2024-09-19T07:54:36.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/2224849149 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcfc40610f0 con 0x56397103c4a0 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd5d21740 10 client.18738 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x563971055b10 con 0x56397103c4a0 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.159+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/2224849149 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc409c750 con 0x56397103c4a0 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/2224849149 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7fcfc40cd740 con 0x56397103c4a0 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfd5d21740 10 client.18738 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfd5d21740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfd5d21740 20 client.18738 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563971056e60 0x563971059250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfd5d21740 4 client.18738 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563970faf850 con 0x563971056e60 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.160+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563971056e60 0x563971059250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563971056e60 0x563971059250 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=448a26268175c5f server_cookie=ffcf322ef896671e in_seq=0 out_seq=0 2024-09-19T07:54:36.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfb0ff9640 10 client.18738 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/2224849149 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563970faf850 con 0x563971056e60 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfb0ff9640 10 client.18738 handle_command_reply: tid=0 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 client.18738 shutdown 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 msgr2=0x563971046ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x563971046ec0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcfc407bff0 tx=0x7fcfc409c170 comp rx=0 tx=0).stop 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 shutdown_connections 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x563971056e60 0x563971059250 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971044750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x563971044210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfb0ff9640 0 client.18738 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:36.198 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "status": "no active scrubs running", 2024-09-19T07:54:36.198 INFO:teuthology.orchestra.run.smithi103.stdout: "scrubs": {} 2024-09-19T07:54:36.198 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:36.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/2224849149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x563971046ec0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.161+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 >> 172.21.15.103:0/2224849149 conn(0x563971038a30 msgr2=0x56397103e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.162+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 shutdown_connections 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.162+0000 7fcfd5d21740 1 -- 172.21.15.103:0/2224849149 wait complete. 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.162+0000 7fcfd5d21740 20 client.18738 trim_cache size 0 max 16384 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.178+0000 7fcfd5d21740 1 Processor -- start 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.178+0000 7fcfd5d21740 1 -- start start 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x56397105c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971041470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1afba0 con 0x56397103c4a0 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1b1770 con 0x56397103cdc0 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1b0270 con 0x563971047680 2024-09-19T07:54:36.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48486/0 (socket says 172.21.15.103:48486) 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1190306910 learned_addr learned my addr 172.21.15.103:0/1190306910 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1190306910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 msgr2=0x563971041470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/1190306910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971041470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1190306910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 msgr2=0x56397105c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/1190306910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x56397105c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1190306910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563971039ae0 con 0x56397103c4a0 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/1190306910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c270 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4098a20 tx=0x7fcfc40b5120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252fa571216f4f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfb2ffd640 1 -- 172.21.15.103:0/1190306910 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4020070 con 0x56397103c4a0 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfb2ffd640 1 -- 172.21.15.103:0/1190306910 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc4009de0 con 0x56397103c4a0 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.179+0000 7fcfb2ffd640 1 -- 172.21.15.103:0/1190306910 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40cd920 con 0x56397103c4a0 2024-09-19T07:54:36.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1190306910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 msgr2=0x56397105c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1190306910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c270 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4098a20 tx=0x7fcfc40b5120 comp rx=0 tx=0).stop 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1190306910 shutdown_connections 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1190306910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563971047680 0x563971041470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1190306910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56397103cdc0 0x56397105c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1190306910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c270 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1190306910 >> 172.21.15.103:0/1190306910 conn(0x563971038a30 msgr2=0x563971043300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1190306910 shutdown_connections 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1190306910 wait complete. 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 -- start start 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.180+0000 7fcfd5d21740 1 Processor -- start 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56397103cdc0 0x563971059bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563971047680 0x56397105a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1b1040 con 0x56397103c4a0 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1b0940 con 0x563971047680 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1adba0 con 0x56397103cdc0 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 10 client.0 _ll_register_callbacks cb 0x7fcfc40a0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48496/0 (socket says 172.21.15.103:48496) 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1996871764 learned_addr learned my addr 172.21.15.103:0/1996871764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563971047680 0x56397105a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1996871764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56397103cdc0 msgr2=0x563971059bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56397103cdc0 0x563971059bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563971047680 msgr2=0x56397105a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563971047680 0x56397105a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 -- 172.21.15.103:0/1996871764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397105a7a0 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd1ee5640 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563971047680 0x56397105a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd16e4640 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c870 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400dfe0 tx=0x7fcfc409be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb47d5fbe577d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/1996871764 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4020070 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/1996871764 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fcfc40a0110 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc4009de0 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/1996871764 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fcfc409c750 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 10 client.18744 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.181+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56397105a7a0 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/1996871764 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc4009de0 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/1996871764 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7fcfc4010460 con 0x56397103c4a0 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfd5d21740 10 client.18744 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:36.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfd5d21740 10 client.18744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfd5d21740 20 client.18744 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639710c1940 0x5639710c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfd5d21740 4 client.18744 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563970fa9d60 con 0x5639710c1940 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.182+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639710c1940 0x5639710c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfb0ff9640 10 client.18744 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd0ee3640 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639710c1940 0x5639710c3d30 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b651f2e422fdaa1c server_cookie=a12ecdce0ada9a12 in_seq=0 out_seq=0 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfb0ff9640 1 -- 172.21.15.103:0/1996871764 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563970fa9d60 con 0x5639710c1940 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfb0ff9640 10 client.18744 handle_command_reply: tid=0 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 client.18744 shutdown 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 msgr2=0x56397105c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c870 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400dfe0 tx=0x7fcfc409be20 comp rx=0 tx=0).stop 2024-09-19T07:54:36.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 shutdown_connections 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x5639710c1940 0x5639710c3d30 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563971047680 0x56397105a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56397103cdc0 0x563971059bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 --2- 172.21.15.103:0/1996871764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56397103c4a0 0x56397105c870 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 >> 172.21.15.103:0/1996871764 conn(0x563971038a30 msgr2=0x56397103def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfb0ff9640 0 client.18744 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 shutdown_connections 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 1 -- 172.21.15.103:0/1996871764 wait complete. 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.183+0000 7fcfd5d21740 20 client.18744 trim_cache size 0 max 16384 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcfb40a33b0 msgr2=0x7fcfb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcfb40a33b0 0x7fcfb40a5870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc06a0f0 tx=0x7fcfbc014840 comp rx=0 tx=0).stop 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc1008d0 msgr2=0x7fcfcc19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:36.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc1008d0 0x7fcfcc19a6b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d950 tx=0x7fcfc400de10 comp rx=0 tx=0).stop 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 shutdown_connections 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fcfb40a33b0 0x7fcfb40a5870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfcc10e480 0x7fcfcc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfcc101280 0x7fcfcc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 --2- 172.21.15.103:0/4112026782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfcc1008d0 0x7fcfcc19a6b0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 >> 172.21.15.103:0/4112026782 conn(0x7fcfcc0fc7a0 msgr2=0x7fcfcc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 shutdown_connections 2024-09-19T07:54:36.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:36.184+0000 7fcfd3948640 1 -- 172.21.15.103:0/4112026782 wait complete. 2024-09-19T07:54:36.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:36 smithi174 ceph-mon[35370]: pgmap v521: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 66 KiB/s wr, 11 op/s 2024-09-19T07:54:36.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:36 smithi103 ceph-mon[31201]: pgmap v521: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 66 KiB/s wr, 11 op/s 2024-09-19T07:54:36.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:36 smithi134 ceph-mon[35414]: pgmap v521: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 66 KiB/s wr, 11 op/s 2024-09-19T07:54:37.055 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T07:54:37.055 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:54:37.345 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:37.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 -- 172.21.15.103:0/1488473762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c101de0 msgr2=0x7f288c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 --2- 172.21.15.103:0/1488473762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c101de0 0x7f288c1021c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2888009900 tx=0x7f288802a410 comp rx=0 tx=0).stop 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 -- 172.21.15.103:0/1488473762 shutdown_connections 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 --2- 172.21.15.103:0/1488473762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f288c106f10 0x7f288c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 --2- 172.21.15.103:0/1488473762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f288c102790 0x7f288c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 --2- 172.21.15.103:0/1488473762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c101de0 0x7f288c1021c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 -- 172.21.15.103:0/1488473762 >> 172.21.15.103:0/1488473762 conn(0x7f288c0fd8c0 msgr2=0x7f288c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 -- 172.21.15.103:0/1488473762 shutdown_connections 2024-09-19T07:54:37.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 -- 172.21.15.103:0/1488473762 wait complete. 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 Processor -- start 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.842+0000 7f2893513640 1 -- start start 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2893513640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f288c101de0 0x7f288c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2893513640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c102790 0x7f288c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2893513640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f288c106f10 0x7f288c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2893513640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f288c111f70 con 0x7f288c102790 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2893513640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f288c111df0 con 0x7f288c106f10 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2893513640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f288c1120f0 con 0x7f288c101de0 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c102790 0x7f288c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c102790 0x7f288c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48518/0 (socket says 172.21.15.103:48518) 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 -- 172.21.15.103:0/2028588745 learned_addr learned my addr 172.21.15.103:0/2028588745 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:37.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f28912af640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f288c101de0 0x7f288c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2891ab0640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f288c106f10 0x7f288c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 -- 172.21.15.103:0/2028588745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f288c101de0 msgr2=0x7f288c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f288c101de0 0x7f288c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 -- 172.21.15.103:0/2028588745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f288c106f10 msgr2=0x7f288c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f288c106f10 0x7f288c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 -- 172.21.15.103:0/2028588745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c19d3e0 con 0x7f288c102790 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2891ab0640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f288c106f10 0x7f288c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2890aae640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c102790 0x7f288c198970 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d950 tx=0x7f287c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f28827fc640 1 -- 172.21.15.103:0/2028588745 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c017030 con 0x7f288c102790 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c19d6d0 con 0x7f288c102790 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f28827fc640 1 -- 172.21.15.103:0/2028588745 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f287c004020 con 0x7f288c102790 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.843+0000 7f28827fc640 1 -- 172.21.15.103:0/2028588745 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c0047a0 con 0x7f288c102790 2024-09-19T07:54:37.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.844+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c1a4fb0 con 0x7f288c102790 2024-09-19T07:54:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.844+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c1021c0 con 0x7f288c102790 2024-09-19T07:54:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.846+0000 7f28827fc640 1 -- 172.21.15.103:0/2028588745 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f287c0041d0 con 0x7f288c102790 2024-09-19T07:54:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.846+0000 7f28827fc640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28780a33b0 0x7f28780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.846+0000 7f28912af640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28780a33b0 0x7f28780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.846+0000 7f28827fc640 1 -- 172.21.15.103:0/2028588745 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f287c0cfe10 con 0x7f288c102790 2024-09-19T07:54:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.847+0000 7f28912af640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28780a33b0 0x7f28780a5870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2888009230 tx=0x7f28880058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:37.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:37.850+0000 7f28827fc640 1 -- 172.21.15.103:0/2028588745 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f287c09c780 con 0x7f288c102790 2024-09-19T07:54:38.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.051+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f288c10a150 con 0x7f288c102790 2024-09-19T07:54:38.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.052+0000 7f28827fc640 1 -- 172.21.15.103:0/2028588745 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+12002 (secure 0 0 0) 0x7f287c09c080 con 0x7f288c102790 2024-09-19T07:54:38.053 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:38.053 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":57,"btime":"2024-09-19T07:54:34:088745+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:54:33.537610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24448,24454,24455,24494,24466]},"id":1}]} 2024-09-19T07:54:38.053 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 57 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28780a33b0 msgr2=0x7f28780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28780a33b0 0x7f28780a5870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2888009230 tx=0x7f28880058d0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c102790 msgr2=0x7f288c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c102790 0x7f288c198970 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d950 tx=0x7f287c00de10 comp rx=0 tx=0).stop 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 shutdown_connections 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28780a33b0 0x7f28780a5870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f288c106f10 0x7f288c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f288c102790 0x7f288c198970 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 --2- 172.21.15.103:0/2028588745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f288c101de0 0x7f288c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:38.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 >> 172.21.15.103:0/2028588745 conn(0x7f288c0fd8c0 msgr2=0x7f288c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:38.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 shutdown_connections 2024-09-19T07:54:38.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:38.054+0000 7f2893513640 1 -- 172.21.15.103:0/2028588745 wait complete. 2024-09-19T07:54:38.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:38 smithi103 ceph-mon[31201]: pgmap v522: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-19T07:54:38.359 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:38 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2028588745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:38 smithi174 ceph-mon[35370]: pgmap v522: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-19T07:54:38.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:38 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2028588745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:38.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:38 smithi134 ceph-mon[35414]: pgmap v522: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-19T07:54:38.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:38 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2028588745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:54:38.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-19T07:54:39.064 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 -- 172.21.15.103:0/2303559905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 msgr2=0x7ff79010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 --2- 172.21.15.103:0/2303559905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff79010d950 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff788009cd0 tx=0x7ff78802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 -- 172.21.15.103:0/2303559905 shutdown_connections 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 --2- 172.21.15.103:0/2303559905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff79010d950 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 --2- 172.21.15.103:0/2303559905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff790100a80 0x7ff790100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 --2- 172.21.15.103:0/2303559905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff790106a80 0x7ff790106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 -- 172.21.15.103:0/2303559905 >> 172.21.15.103:0/2303559905 conn(0x7ff7900fc7a0 msgr2=0x7ff7900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 -- 172.21.15.103:0/2303559905 shutdown_connections 2024-09-19T07:54:39.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.557+0000 7ff79562b640 1 -- 172.21.15.103:0/2303559905 wait complete. 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 Processor -- start 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 -- start start 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff790100a80 0x7ff7901983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff790198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff790106a80 0x7ff79019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff790111fa0 con 0x7ff790101420 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff790111e20 con 0x7ff790100a80 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff79562b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff790112120 con 0x7ff790106a80 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff790198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff790198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44196/0 (socket says 172.21.15.103:44196) 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 -- 172.21.15.103:0/1935798864 learned_addr learned my addr 172.21.15.103:0/1935798864 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:39.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff790106a80 0x7ff79019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff78effd640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff790100a80 0x7ff7901983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 -- 172.21.15.103:0/1935798864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff790106a80 msgr2=0x7ff79019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff790106a80 0x7ff79019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 -- 172.21.15.103:0/1935798864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff790100a80 msgr2=0x7ff7901983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff790100a80 0x7ff7901983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff787fff640 1 -- 172.21.15.103:0/1935798864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79019d390 con 0x7ff790101420 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.558+0000 7ff78effd640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff790100a80 0x7ff7901983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.559+0000 7ff787fff640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff790198920 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff77800c920 tx=0x7ff77800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.559+0000 7ff78cff9640 1 -- 172.21.15.103:0/1935798864 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778007c70 con 0x7ff790101420 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.559+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79019d680 con 0x7ff790101420 2024-09-19T07:54:39.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.559+0000 7ff78cff9640 1 -- 172.21.15.103:0/1935798864 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff778007e10 con 0x7ff790101420 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.559+0000 7ff78cff9640 1 -- 172.21.15.103:0/1935798864 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778018890 con 0x7ff790101420 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.559+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7901a4f60 con 0x7ff790101420 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.561+0000 7ff78cff9640 1 -- 172.21.15.103:0/1935798864 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff778018a70 con 0x7ff790101420 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.561+0000 7ff797a04740 1 Processor -- start 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff797a04740 1 -- start start 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff797a04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f708b40 0x55b49f707090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff797a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f707a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff797a04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f712370 0x55b49f714760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff797a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49f66d2f0 con 0x55b49f7075d0 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff797a04740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b49f614cf0 con 0x55b49f712370 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff797a04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b49f6692b0 con 0x55b49f708b40 2024-09-19T07:54:39.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f707a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78cff9640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff77c0a33b0 0x7ff77c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f707a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44198/0 (socket says 172.21.15.103:44198) 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 -- 172.21.15.103:0/81704427 learned_addr learned my addr 172.21.15.103:0/81704427 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f708b40 0x55b49f707090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff77c0a33b0 0x7ff77c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78cff9640 1 -- 172.21.15.103:0/1935798864 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7ff7780cff30 con 0x7ff790101420 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff787fff640 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f712370 0x55b49f714760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 -- 172.21.15.103:0/81704427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f708b40 msgr2=0x55b49f707090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f708b40 0x55b49f707090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 -- 172.21.15.103:0/81704427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f712370 msgr2=0x55b49f714760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f712370 0x55b49f714760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.562+0000 7ff78effd640 1 -- 172.21.15.103:0/81704427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49f5f93a0 con 0x55b49f7075d0 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.563+0000 7ff78effd640 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f707a10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff78000aa10 tx=0x7ff78000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787097846e165d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.563+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff77c0a33b0 0x7ff77c0a5870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff788009430 tx=0x7ff7880095c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.563+0000 7ff7857fa640 1 -- 172.21.15.103:0/81704427 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780015070 con 0x55b49f7075d0 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.563+0000 7ff7857fa640 1 -- 172.21.15.103:0/81704427 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78000cc60 con 0x55b49f7075d0 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.563+0000 7ff7857fa640 1 -- 172.21.15.103:0/81704427 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7800186b0 con 0x55b49f7075d0 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 -- 172.21.15.103:0/81704427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 msgr2=0x55b49f707a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f707a10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff78000aa10 tx=0x7ff78000aed0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 -- 172.21.15.103:0/81704427 shutdown_connections 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f712370 0x55b49f714760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f707a10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 --2- 172.21.15.103:0/81704427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f708b40 0x55b49f707090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 -- 172.21.15.103:0/81704427 >> 172.21.15.103:0/81704427 conn(0x55b49f6d0310 msgr2=0x55b49f703c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 -- 172.21.15.103:0/81704427 shutdown_connections 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 -- 172.21.15.103:0/81704427 wait complete. 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 -- start start 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.564+0000 7ff797a04740 1 Processor -- start 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f711cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f70ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f712370 0x55b49f70f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49f715070 con 0x55b49f712370 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b49f628060 con 0x55b49f708b40 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b49f7151f0 con 0x55b49f7075d0 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 10 client.0 _ll_register_callbacks cb 0x55b49f37b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:39.583 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f712370 0x55b49f70f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f712370 0x55b49f70f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44206/0 (socket says 172.21.15.103:44206) 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 -- 172.21.15.103:0/2335705605 learned_addr learned my addr 172.21.15.103:0/2335705605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff787fff640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f70ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78effd640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f711cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 -- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 msgr2=0x55b49f711cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f711cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 -- 172.21.15.103:0/2335705605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 msgr2=0x55b49f70ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f70ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 -- 172.21.15.103:0/2335705605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49f3de3c0 con 0x55b49f712370 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78effd640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f711cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:39.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f712370 0x55b49f70f4a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff788005cc0 tx=0x7ff78802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f02d6076a581621 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff7767fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff7767fc640 1 -- 172.21.15.103:0/2335705605 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788051070 con 0x55b49f712370 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78000cc60 con 0x55b49f712370 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.565+0000 7ff7767fc640 1 -- 172.21.15.103:0/2335705605 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7ff788004450 con 0x55b49f712370 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff7767fc640 1 -- 172.21.15.103:0/2335705605 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff78805d050 con 0x55b49f712370 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 10 client.18762 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b49f720ab0 con 0x55b49f712370 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff7767fc640 1 -- 172.21.15.103:0/2335705605 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff78000cc60 con 0x55b49f712370 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff7767fc640 1 -- 172.21.15.103:0/2335705605 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7ff788030070 con 0x55b49f712370 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 10 client.18762 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 10 client.18762 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:54:39.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 20 client.18762 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f721ec0 0x55b49f7242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 4 client.18762 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b49f67a860 con 0x55b49f721ec0 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.566+0000 7ff78effd640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f721ec0 0x55b49f7242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.567+0000 7ff7767fc640 10 client.18762 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.567+0000 7ff78effd640 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f721ec0 0x55b49f7242d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49c16aaecb817431 server_cookie=24226bb80f12f38d in_seq=0 out_seq=0 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.567+0000 7ff7767fc640 1 -- 172.21.15.103:0/2335705605 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b49f67a860 con 0x55b49f721ec0 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.567+0000 7ff7767fc640 10 client.18762 handle_command_reply: tid=0 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.567+0000 7ff797a04740 1 client.18762 shutdown 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f712370 msgr2=0x55b49f70f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f712370 0x55b49f70f4a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff788005cc0 tx=0x7ff78802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 shutdown_connections 2024-09-19T07:54:39.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f721ec0 0x55b49f7242d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f712370 0x55b49f70f4a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f70ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 --2- 172.21.15.103:0/2335705605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f711cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff7767fc640 0 client.18762 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 >> 172.21.15.103:0/2335705605 conn(0x55b49f6d0310 msgr2=0x55b49f709960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 shutdown_connections 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 1 -- 172.21.15.103:0/2335705605 wait complete. 2024-09-19T07:54:39.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.568+0000 7ff797a04740 20 client.18762 trim_cache size 0 max 16384 2024-09-19T07:54:39.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 Processor -- start 2024-09-19T07:54:39.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 -- start start 2024-09-19T07:54:39.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f6d0310 0x55b49f71e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f706bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f713760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901ad8d0 con 0x55b49f7075d0 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901af4a0 con 0x55b49f708b40 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.581+0000 7ff797a04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901adfa0 con 0x55b49f6d0310 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f706bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f706bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44208/0 (socket says 172.21.15.103:44208) 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 -- 172.21.15.103:0/1899857702 learned_addr learned my addr 172.21.15.103:0/1899857702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff78effd640 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f6d0310 0x55b49f71e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f713760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 -- 172.21.15.103:0/1899857702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f6d0310 msgr2=0x55b49f71e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f6d0310 0x55b49f71e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 -- 172.21.15.103:0/1899857702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 msgr2=0x55b49f713760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f713760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 -- 172.21.15.103:0/1899857702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49f70d4b0 con 0x55b49f7075d0 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f713760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff787fff640 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f706bd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff77809c140 tx=0x7ff778000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62af6d0a4b83505c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff784ff9640 1 -- 172.21.15.103:0/1899857702 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780a4e70 con 0x55b49f7075d0 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff784ff9640 1 -- 172.21.15.103:0/1899857702 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77809a500 con 0x55b49f7075d0 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.582+0000 7ff784ff9640 1 -- 172.21.15.103:0/1899857702 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778098870 con 0x55b49f7075d0 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 -- 172.21.15.103:0/1899857702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 msgr2=0x55b49f706bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f706bd0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff77809c140 tx=0x7ff778000bc0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 -- 172.21.15.103:0/1899857702 shutdown_connections 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f713760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f7075d0 0x55b49f706bd0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 --2- 172.21.15.103:0/1899857702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f6d0310 0x55b49f71e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 -- 172.21.15.103:0/1899857702 >> 172.21.15.103:0/1899857702 conn(0x55b49f714910 msgr2=0x55b49f70d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 -- 172.21.15.103:0/1899857702 shutdown_connections 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 -- 172.21.15.103:0/1899857702 wait complete. 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.583+0000 7ff797a04740 1 -- start start 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 1 Processor -- start 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f6d0310 0x55b49f71a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f71a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f789b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901aed70 con 0x55b49f6d0310 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901ae670 con 0x55b49f708b40 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901ab8d0 con 0x55b49f7075d0 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff797a04740 10 client.0 _ll_register_callbacks cb 0x7ff788031790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f6d0310 0x55b49f71a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f6d0310 0x55b49f71a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44224/0 (socket says 172.21.15.103:44224) 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 -- 172.21.15.103:0/1048771317 learned_addr learned my addr 172.21.15.103:0/1048771317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78f7fe640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f789b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff787fff640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f71a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 -- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 msgr2=0x55b49f71a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f71a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 -- 172.21.15.103:0/1048771317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 msgr2=0x55b49f789b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:39.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f789b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 -- 172.21.15.103:0/1048771317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49f78a120 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff787fff640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f71a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff78effd640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f6d0310 0x55b49f71a480 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff780008010 tx=0x7ff780006c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23ee954b799bc31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.584+0000 7ff7767fc640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff7767fc640 1 -- 172.21.15.103:0/1048771317 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78002a070 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff77809a500 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff7767fc640 1 -- 172.21.15.103:0/1048771317 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7ff780011800 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff7767fc640 1 -- 172.21.15.103:0/1048771317 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff780036050 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 10 client.18768 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b49f70ba00 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff7767fc640 1 -- 172.21.15.103:0/1048771317 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77809a500 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff7767fc640 1 -- 172.21.15.103:0/1048771317 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7ff7800268a0 con 0x55b49f6d0310 2024-09-19T07:54:39.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 10 client.18768 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 10 client.18768 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 20 client.18768 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f712370 0x55b49f70be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 4 client.18768 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 --> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b49f674d70 con 0x55b49f712370 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.585+0000 7ff787fff640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f712370 0x55b49f70be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff7767fc640 10 client.18768 ms_handle_connect on v2:172.21.15.174:6834/214062550 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff787fff640 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f712370 0x55b49f70be80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b278836dc21a0c0e server_cookie=fe1a4695e5bb0024 in_seq=0 out_seq=0 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff7767fc640 1 -- 172.21.15.103:0/1048771317 <== mds.4 v2:172.21.15.174:6834/214062550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b49f674d70 con 0x55b49f712370 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff7767fc640 10 client.18768 handle_command_reply: tid=0 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 client.18768 shutdown 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f6d0310 msgr2=0x55b49f71a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f6d0310 0x55b49f71a480 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff780008010 tx=0x7ff780006c70 comp rx=0 tx=0).stop 2024-09-19T07:54:39.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 shutdown_connections 2024-09-19T07:54:39.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:6834/214062550,v1:172.21.15.174:6835/214062550] conn(0x55b49f712370 0x55b49f70be80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b49f708b40 0x55b49f789b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b49f7075d0 0x55b49f71a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 --2- 172.21.15.103:0/1048771317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49f6d0310 0x55b49f71a480 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff7767fc640 0 client.18768 ms_handle_reset on v2:172.21.15.174:6834/214062550 2024-09-19T07:54:39.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 >> 172.21.15.103:0/1048771317 conn(0x55b49f714910 msgr2=0x55b49f7024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:39.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 shutdown_connections 2024-09-19T07:54:39.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 1 -- 172.21.15.103:0/1048771317 wait complete. 2024-09-19T07:54:39.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.586+0000 7ff797a04740 20 client.18768 trim_cache size 0 max 16384 2024-09-19T07:54:39.600 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:54:39.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff77c0a33b0 msgr2=0x7ff77c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff77c0a33b0 0x7ff77c0a5870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff788009430 tx=0x7ff7880095c0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 msgr2=0x7ff790198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff790198920 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff77800c920 tx=0x7ff77800cde0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 shutdown_connections 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff77c0a33b0 0x7ff77c0a5870 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff790106a80 0x7ff79019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff790101420 0x7ff790198920 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 --2- 172.21.15.103:0/1935798864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff790100a80 0x7ff7901983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 >> 172.21.15.103:0/1935798864 conn(0x7ff7900fc7a0 msgr2=0x7ff7900fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 shutdown_connections 2024-09-19T07:54:39.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:39.587+0000 7ff79562b640 1 -- 172.21.15.103:0/1935798864 wait complete. 2024-09-19T07:54:40.383 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:40 smithi103 ceph-mon[31201]: pgmap v523: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-09-19T07:54:40.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-19T07:54:40.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:40 smithi174 ceph-mon[35370]: pgmap v523: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-09-19T07:54:40.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:40 smithi134 ceph-mon[35414]: pgmap v523: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-09-19T07:54:40.655 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 -- 172.21.15.103:0/1747560339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894102790 msgr2=0x7f28941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 --2- 172.21.15.103:0/1747560339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894102790 0x7f28941067e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2880009900 tx=0x7f288002a3f0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 -- 172.21.15.103:0/1747560339 shutdown_connections 2024-09-19T07:54:41.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 --2- 172.21.15.103:0/1747560339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2894106f10 0x7f289410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 --2- 172.21.15.103:0/1747560339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894102790 0x7f28941067e0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 --2- 172.21.15.103:0/1747560339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2894101de0 0x7f28941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 -- 172.21.15.103:0/1747560339 >> 172.21.15.103:0/1747560339 conn(0x7f28940fd8c0 msgr2=0x7f28940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 -- 172.21.15.103:0/1747560339 shutdown_connections 2024-09-19T07:54:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.162+0000 7f289adaf640 1 -- 172.21.15.103:0/1747560339 wait complete. 2024-09-19T07:54:41.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 Processor -- start 2024-09-19T07:54:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 -- start start 2024-09-19T07:54:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2894101de0 0x7f2894198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2894102790 0x7f2894198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894106f10 0x7f289419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111f30 con 0x7f2894106f10 2024-09-19T07:54:41.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111db0 con 0x7f2894102790 2024-09-19T07:54:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289adaf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28941120b0 con 0x7f2894101de0 2024-09-19T07:54:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894106f10 0x7f289419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894106f10 0x7f289419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44242/0 (socket says 172.21.15.103:44242) 2024-09-19T07:54:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 -- 172.21.15.103:0/3637188658 learned_addr learned my addr 172.21.15.103:0/3637188658 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f2898b4b640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2894101de0 0x7f2894198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f2893fff640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2894102790 0x7f2894198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 -- 172.21.15.103:0/3637188658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2894101de0 msgr2=0x7f2894198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2894101de0 0x7f2894198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 -- 172.21.15.103:0/3637188658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2894102790 msgr2=0x7f2894198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2894102790 0x7f2894198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.163+0000 7f289934c640 1 -- 172.21.15.103:0/3637188658 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289419d2c0 con 0x7f2894106f10 2024-09-19T07:54:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.164+0000 7f289934c640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894106f10 0x7f289419cbe0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f288c00cce0 tx=0x7f288c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.164+0000 7f2891ffb640 1 -- 172.21.15.103:0/3637188658 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c00f450 con 0x7f2894106f10 2024-09-19T07:54:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.164+0000 7f2891ffb640 1 -- 172.21.15.103:0/3637188658 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c00fad0 con 0x7f2894106f10 2024-09-19T07:54:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.164+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289419d5b0 con 0x7f2894106f10 2024-09-19T07:54:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.164+0000 7f2891ffb640 1 -- 172.21.15.103:0/3637188658 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c018b60 con 0x7f2894106f10 2024-09-19T07:54:41.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.164+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a4e90 con 0x7f2894106f10 2024-09-19T07:54:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.166+0000 7f2891ffb640 1 -- 172.21.15.103:0/3637188658 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f288c00fc80 con 0x7f2894106f10 2024-09-19T07:54:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 Processor -- start 2024-09-19T07:54:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 -- start start 2024-09-19T07:54:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f2891ffb640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28840a33b0 0x7f28840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adaa7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaa7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5582ada0d2f0 con 0x5582adab6780 2024-09-19T07:54:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5582ad9b4cf0 con 0x5582adaa8b40 2024-09-19T07:54:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289d188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5582ada092b0 con 0x5582adaa75d0 2024-09-19T07:54:41.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f2893fff640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28840a33b0 0x7f28840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f2891ffb640 1 -- 172.21.15.103:0/3637188658 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f288c0d0150 con 0x7f2894106f10 2024-09-19T07:54:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44256/0 (socket says 172.21.15.103:44256) 2024-09-19T07:54:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 -- 172.21.15.103:0/3277969737 learned_addr learned my addr 172.21.15.103:0/3277969737 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f2893fff640 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaa7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f2898b4b640 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adaa7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 -- 172.21.15.103:0/3277969737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 msgr2=0x5582adaa7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaa7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 -- 172.21.15.103:0/3277969737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 msgr2=0x5582adaa7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adaa7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.167+0000 7f289934c640 1 -- 172.21.15.103:0/3277969737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ad9993a0 con 0x5582adab6780 2024-09-19T07:54:41.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f2893fff640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28840a33b0 0x7f28840a5870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c00 tx=0x7f28800319b0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:41.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f289934c640 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab8b70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f288c0a4b60 tx=0x7f288c09c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86913dcb389dd84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:41.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f287effd640 1 -- 172.21.15.103:0/3277969737 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c09c420 con 0x5582adab6780 2024-09-19T07:54:41.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f287effd640 1 -- 172.21.15.103:0/3277969737 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c0a07d0 con 0x5582adab6780 2024-09-19T07:54:41.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f287effd640 1 -- 172.21.15.103:0/3277969737 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c09b870 con 0x5582adab6780 2024-09-19T07:54:41.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f289d188740 1 -- 172.21.15.103:0/3277969737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 msgr2=0x5582adab8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f289d188740 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab8b70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f288c0a4b60 tx=0x7f288c09c020 comp rx=0 tx=0).stop 2024-09-19T07:54:41.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f289d188740 1 -- 172.21.15.103:0/3277969737 shutdown_connections 2024-09-19T07:54:41.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f289d188740 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab8b70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f289d188740 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaa7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.168+0000 7f289d188740 1 --2- 172.21.15.103:0/3277969737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adaa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 -- 172.21.15.103:0/3277969737 >> 172.21.15.103:0/3277969737 conn(0x5582ada70310 msgr2=0x5582adaa3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:41.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 -- 172.21.15.103:0/3277969737 shutdown_connections 2024-09-19T07:54:41.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 -- 172.21.15.103:0/3277969737 wait complete. 2024-09-19T07:54:41.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 -- start start 2024-09-19T07:54:41.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 Processor -- start 2024-09-19T07:54:41.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adab61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adab3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5582adab9480 con 0x5582adab6780 2024-09-19T07:54:41.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5582ad9c8060 con 0x5582adaa8b40 2024-09-19T07:54:41.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.169+0000 7f289d188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5582adab9600 con 0x5582adaa75d0 2024-09-19T07:54:41.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289d188740 10 client.0 _ll_register_callbacks cb 0x5582ad71b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44258/0 (socket says 172.21.15.103:44258) 2024-09-19T07:54:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 -- 172.21.15.103:0/2061143618 learned_addr learned my addr 172.21.15.103:0/2061143618 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f2893fff640 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adab3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f2898b4b640 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adab61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 -- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 msgr2=0x5582adab61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adab61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 -- 172.21.15.103:0/2061143618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 msgr2=0x5582adab3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adab3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 -- 172.21.15.103:0/2061143618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ad77e3c0 con 0x5582adab6780 2024-09-19T07:54:41.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289934c640 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab3bf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f288c098df0 tx=0x7f288c09c090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20726461eaaaca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f2873fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f2873fff640 1 -- 172.21.15.103:0/2061143618 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c089060 con 0x5582adab6780 2024-09-19T07:54:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f2873fff640 1 -- 172.21.15.103:0/2061143618 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f288c09ba70 con 0x5582adab6780 2024-09-19T07:54:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288c0a07d0 con 0x5582adab6780 2024-09-19T07:54:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f2873fff640 1 -- 172.21.15.103:0/2061143618 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f288c08b370 con 0x5582adab6780 2024-09-19T07:54:41.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289d188740 10 client.18780 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.170+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5582adac4f60 con 0x5582adab6780 2024-09-19T07:54:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f2873fff640 1 -- 172.21.15.103:0/2061143618 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c0a07d0 con 0x5582adab6780 2024-09-19T07:54:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f2873fff640 1 -- 172.21.15.103:0/2061143618 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f288c020030 con 0x5582adab6780 2024-09-19T07:54:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f289d188740 10 client.18780 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f289d188740 10 client.18780 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:54:41.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f289d188740 20 client.18780 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f289d188740 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adac62b0 0x5582adac86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f289d188740 4 client.18780 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ada1a860 con 0x5582adac62b0 2024-09-19T07:54:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.171+0000 7f2898b4b640 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adac62b0 0x5582adac86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f2898b4b640 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adac62b0 0x5582adac86a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44770eddb706f15a server_cookie=ba8ee5850cc3b0c in_seq=0 out_seq=0 2024-09-19T07:54:41.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f2873fff640 10 client.18780 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:54:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f2873fff640 1 -- 172.21.15.103:0/2061143618 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582ada1a860 con 0x5582adac62b0 2024-09-19T07:54:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f2873fff640 10 client.18780 handle_command_reply: tid=0 2024-09-19T07:54:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f289d188740 1 client.18780 shutdown 2024-09-19T07:54:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 msgr2=0x5582adab3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f289d188740 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab3bf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f288c098df0 tx=0x7f288c09c090 comp rx=0 tx=0).stop 2024-09-19T07:54:41.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 shutdown_connections 2024-09-19T07:54:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f289d188740 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adac62b0 0x5582adac86a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f289d188740 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adab6780 0x5582adab3bf0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.172+0000 7f289d188740 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adaa8b40 0x5582adab3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.173+0000 7f2873fff640 0 client.18780 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:54:41.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.173+0000 7f289d188740 1 --2- 172.21.15.103:0/2061143618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adab61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.173+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 >> 172.21.15.103:0/2061143618 conn(0x5582ada70310 msgr2=0x5582adaa9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:41.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.173+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 shutdown_connections 2024-09-19T07:54:41.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.173+0000 7f289d188740 1 -- 172.21.15.103:0/2061143618 wait complete. 2024-09-19T07:54:41.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.173+0000 7f289d188740 20 client.18780 trim_cache size 0 max 16384 2024-09-19T07:54:41.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.185+0000 7f289d188740 1 Processor -- start 2024-09-19T07:54:41.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.185+0000 7f289d188740 1 -- start start 2024-09-19T07:54:41.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f289d188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa75d0 0x5582adab3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f289d188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa8b40 0x5582adacdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f289d188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adace200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f289d188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28941ab5b0 con 0x5582adaa75d0 2024-09-19T07:54:41.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f289d188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28941ad180 con 0x5582adab6780 2024-09-19T07:54:41.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f289d188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28941abc80 con 0x5582adaa8b40 2024-09-19T07:54:41.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa75d0 0x5582adab3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa75d0 0x5582adab3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44266/0 (socket says 172.21.15.103:44266) 2024-09-19T07:54:41.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 -- 172.21.15.103:0/28074625 learned_addr learned my addr 172.21.15.103:0/28074625 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:41.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 -- 172.21.15.103:0/28074625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa8b40 msgr2=0x5582adacdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 --2- 172.21.15.103:0/28074625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa8b40 0x5582adacdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 -- 172.21.15.103:0/28074625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 msgr2=0x5582adace200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 --2- 172.21.15.103:0/28074625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adace200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 -- 172.21.15.103:0/28074625 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582adaa4960 con 0x5582adaa75d0 2024-09-19T07:54:41.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f2898b4b640 1 --2- 172.21.15.103:0/28074625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa75d0 0x5582adab3430 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f288800a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0e9d9514de863b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:41.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f287e7fc640 1 -- 172.21.15.103:0/28074625 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888005020 con 0x5582adaa75d0 2024-09-19T07:54:41.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f287e7fc640 1 -- 172.21.15.103:0/28074625 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28880047a0 con 0x5582adaa75d0 2024-09-19T07:54:41.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.186+0000 7f287e7fc640 1 -- 172.21.15.103:0/28074625 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888011710 con 0x5582adaa75d0 2024-09-19T07:54:41.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 -- 172.21.15.103:0/28074625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa75d0 msgr2=0x5582adab3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 --2- 172.21.15.103:0/28074625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa75d0 0x5582adab3430 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f288800a6e0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 -- 172.21.15.103:0/28074625 shutdown_connections 2024-09-19T07:54:41.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 --2- 172.21.15.103:0/28074625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adace200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 --2- 172.21.15.103:0/28074625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa8b40 0x5582adacdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 --2- 172.21.15.103:0/28074625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa75d0 0x5582adab3430 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 -- 172.21.15.103:0/28074625 >> 172.21.15.103:0/28074625 conn(0x5582ada70310 msgr2=0x5582adab2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:41.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 -- 172.21.15.103:0/28074625 shutdown_connections 2024-09-19T07:54:41.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 -- 172.21.15.103:0/28074625 wait complete. 2024-09-19T07:54:41.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 -- start start 2024-09-19T07:54:41.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.187+0000 7f289d188740 1 Processor -- start 2024-09-19T07:54:41.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289d188740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289d188740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa8b40 0x5582adabada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289d188740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adabb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289d188740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28941aca50 con 0x5582adaa8b40 2024-09-19T07:54:41.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289d188740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28941ac350 con 0x5582adab6780 2024-09-19T07:54:41.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289d188740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28941a95b0 con 0x5582adaa75d0 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289d188740 10 client.0 _ll_register_callbacks cb 0x7f288c09c230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa8b40 0x5582adabada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa8b40 0x5582adabada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44278/0 (socket says 172.21.15.103:44278) 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 -- 172.21.15.103:0/2819698456 learned_addr learned my addr 172.21.15.103:0/2819698456 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2898b4b640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289934c640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adabb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 -- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 msgr2=0x5582adaba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaba860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 -- 172.21.15.103:0/2819698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 msgr2=0x5582adabb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adabb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 -- 172.21.15.103:0/2819698456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582adb2fee0 con 0x5582adaa8b40 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f289934c640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adabb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2893fff640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa8b40 0x5582adabada0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2880009900 tx=0x7f2880009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9208989b1c95cf24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.188+0000 7f2873fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.189+0000 7f2873fff640 1 -- 172.21.15.103:0/2819698456 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880035040 con 0x5582adaa8b40 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.189+0000 7f2873fff640 1 -- 172.21.15.103:0/2819698456 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f2880055470 con 0x5582adaa8b40 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.189+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28880047a0 con 0x5582adaa8b40 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.189+0000 7f2873fff640 1 -- 172.21.15.103:0/2819698456 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f28800044c0 con 0x5582adaa8b40 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.189+0000 7f289d188740 10 client.18786 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:41.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.189+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5582adb31210 con 0x5582adaa8b40 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f2873fff640 1 -- 172.21.15.103:0/2819698456 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28880047a0 con 0x5582adaa8b40 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f2873fff640 1 -- 172.21.15.103:0/2819698456 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f288004c600 con 0x5582adaa8b40 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f289d188740 10 client.18786 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f289d188740 10 client.18786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f289d188740 20 client.18786 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f289d188740 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adb31a60 0x5582adb33e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f289d188740 4 client.18786 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 --> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582ada14d70 con 0x5582adb31a60 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f2898b4b640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adb31a60 0x5582adb33e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f2898b4b640 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adb31a60 0x5582adb33e80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=807c3276b1bb8917 server_cookie=3c686d08e5fbe0dd in_seq=0 out_seq=0 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.190+0000 7f2873fff640 10 client.18786 ms_handle_connect on v2:172.21.15.174:6836/2527370860 2024-09-19T07:54:41.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f2873fff640 1 -- 172.21.15.103:0/2819698456 <== mds.3 v2:172.21.15.174:6836/2527370860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582ada14d70 con 0x5582adb31a60 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f2873fff640 10 client.18786 handle_command_reply: tid=0 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 client.18786 shutdown 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa8b40 msgr2=0x5582adabada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa8b40 0x5582adabada0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2880009900 tx=0x7f2880009260 comp rx=0 tx=0).stop 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 shutdown_connections 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:6836/2527370860,v1:172.21.15.174:6837/2527370860] conn(0x5582adb31a60 0x5582adb33e80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582adab6780 0x5582adabb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582adaa8b40 0x5582adabada0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 --2- 172.21.15.103:0/2819698456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582adaa75d0 0x5582adaba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 >> 172.21.15.103:0/2819698456 conn(0x5582ada70310 msgr2=0x5582adaa24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:41.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f2873fff640 0 client.18786 ms_handle_reset on v2:172.21.15.174:6836/2527370860 2024-09-19T07:54:41.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 shutdown_connections 2024-09-19T07:54:41.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 1 -- 172.21.15.103:0/2819698456 wait complete. 2024-09-19T07:54:41.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.191+0000 7f289d188740 20 client.18786 trim_cache size 0 max 16384 2024-09-19T07:54:41.227 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:54:41.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28840a33b0 msgr2=0x7f28840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28840a33b0 0x7f28840a5870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c00 tx=0x7f28800319b0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894106f10 msgr2=0x7f289419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894106f10 0x7f289419cbe0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f288c00cce0 tx=0x7f288c00a620 comp rx=0 tx=0).stop 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 shutdown_connections 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f28840a33b0 0x7f28840a5870 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2894106f10 0x7f289419cbe0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2894102790 0x7f2894198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 --2- 172.21.15.103:0/3637188658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2894101de0 0x7f2894198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 >> 172.21.15.103:0/3637188658 conn(0x7f28940fd8c0 msgr2=0x7f28940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 shutdown_connections 2024-09-19T07:54:41.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:41.192+0000 7f289adaf640 1 -- 172.21.15.103:0/3637188658 wait complete. 2024-09-19T07:54:41.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T07:54:42.198 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:42.222 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:42 smithi103 ceph-mon[31201]: pgmap v524: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-19T07:54:42.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:42 smithi174 ceph-mon[35370]: pgmap v524: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-19T07:54:42.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:42 smithi134 ceph-mon[35414]: pgmap v524: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-19T07:54:42.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 -- 172.21.15.103:0/445836533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102ef0 msgr2=0x7f548810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 --2- 172.21.15.103:0/445836533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102ef0 0x7f548810da40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5480009ca0 tx=0x7f548002a950 comp rx=0 tx=0).stop 2024-09-19T07:54:42.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 -- 172.21.15.103:0/445836533 shutdown_connections 2024-09-19T07:54:42.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 --2- 172.21.15.103:0/445836533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102ef0 0x7f548810da40 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 --2- 172.21.15.103:0/445836533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5488102570 0x7f54881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 --2- 172.21.15.103:0/445836533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5488100e20 0x7f5488101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 -- 172.21.15.103:0/445836533 >> 172.21.15.103:0/445836533 conn(0x7f54880fc7d0 msgr2=0x7f54880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 -- 172.21.15.103:0/445836533 shutdown_connections 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.655+0000 7f548e8de640 1 -- 172.21.15.103:0/445836533 wait complete. 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 Processor -- start 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 -- start start 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5488100e20 0x7f5488198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102570 0x7f5488198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5488102ef0 0x7f548819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5488110f80 con 0x7f5488102570 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5488110e00 con 0x7f5488100e20 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548e8de640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5488111100 con 0x7f5488102ef0 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f54877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102570 0x7f5488198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f54877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102570 0x7f5488198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44284/0 (socket says 172.21.15.103:44284) 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f54877fe640 1 -- 172.21.15.103:0/1305086690 learned_addr learned my addr 172.21.15.103:0/1305086690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f548ce7b640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5488102ef0 0x7f548819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.656+0000 7f5487fff640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5488100e20 0x7f5488198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54877fe640 1 -- 172.21.15.103:0/1305086690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5488102ef0 msgr2=0x7f548819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54877fe640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5488102ef0 0x7f548819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54877fe640 1 -- 172.21.15.103:0/1305086690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5488100e20 msgr2=0x7f5488198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54877fe640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5488100e20 0x7f5488198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54877fe640 1 -- 172.21.15.103:0/1305086690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548819d4e0 con 0x7f5488102570 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54877fe640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102570 0x7f5488198a70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f547c00d770 tx=0x7f547c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54857fa640 1 -- 172.21.15.103:0/1305086690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c017070 con 0x7f5488102570 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54857fa640 1 -- 172.21.15.103:0/1305086690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f547c005500 con 0x7f5488102570 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.657+0000 7f54857fa640 1 -- 172.21.15.103:0/1305086690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c004e40 con 0x7f5488102570 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.658+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548819d7d0 con 0x7f5488102570 2024-09-19T07:54:42.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.658+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881a50b0 con 0x7f5488102570 2024-09-19T07:54:42.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.660+0000 7f54857fa640 1 -- 172.21.15.103:0/1305086690 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f547c009c30 con 0x7f5488102570 2024-09-19T07:54:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.660+0000 7f5490cb7740 1 Processor -- start 2024-09-19T07:54:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.660+0000 7f5490cb7740 1 -- start start 2024-09-19T07:54:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfbd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfbda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f5490cb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fe2df232f0 con 0x55fe2dfc8370 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f5490cb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe2decacf0 con 0x55fe2dfbd5d0 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f5490cb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe2df1f2b0 con 0x55fe2dfbeb40 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f548ce7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfbd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f548ce7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfbd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.103:44374/0 (socket says 172.21.15.103:44374) 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f548ce7b640 1 -- 172.21.15.103:0/3126299707 learned_addr learned my addr 172.21.15.103:0/3126299707 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54857fa640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f54700a3500 0x7f54700a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f5487fff640 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfbda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f548ce7b640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f54700a3500 0x7f54700a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54857fa640 1 -- 172.21.15.103:0/1305086690 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f547c0cf2d0 con 0x7f5488102570 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54877fe640 1 -- 172.21.15.103:0/3126299707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbeb40 msgr2=0x55fe2dfbd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54877fe640 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfbd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54877fe640 1 -- 172.21.15.103:0/3126299707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 msgr2=0x55fe2dfbda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54877fe640 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfbda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54877fe640 1 -- 172.21.15.103:0/3126299707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2deaf3a0 con 0x55fe2dfc8370 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.661+0000 7f54877fe640 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfca760 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f547c0cca10 tx=0x7f547c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6ec78d68237638 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f546e7fc640 1 -- 172.21.15.103:0/3126299707 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c020040 con 0x55fe2dfc8370 2024-09-19T07:54:42.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f548ce7b640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f54700a3500 0x7f54700a59c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5480030040 tx=0x7f548002a950 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f546e7fc640 1 -- 172.21.15.103:0/3126299707 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f547c0046c0 con 0x55fe2dfc8370 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f546e7fc640 1 -- 172.21.15.103:0/3126299707 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c097d40 con 0x55fe2dfc8370 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f5490cb7740 1 -- 172.21.15.103:0/3126299707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 msgr2=0x55fe2dfca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfca760 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f547c0cca10 tx=0x7f547c0048a0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f5490cb7740 1 -- 172.21.15.103:0/3126299707 shutdown_connections 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfca760 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfbda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3126299707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfbd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.662+0000 7f5490cb7740 1 -- 172.21.15.103:0/3126299707 >> 172.21.15.103:0/3126299707 conn(0x55fe2df86310 msgr2=0x55fe2dfb9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 -- 172.21.15.103:0/3126299707 shutdown_connections 2024-09-19T07:54:42.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 -- 172.21.15.103:0/3126299707 wait complete. 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 -- start start 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 Processor -- start 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfc8370 0x55fe2dfc5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fe2dfcafb0 con 0x55fe2dfbeb40 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe2dede060 con 0x55fe2dfc8370 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.663+0000 7f5490cb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe2dfcb130 con 0x55fe2dfbd5d0 2024-09-19T07:54:42.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f5490cb7740 10 client.0 _ll_register_callbacks cb 0x55fe2dc318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44314/0 (socket says 172.21.15.103:44314) 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 -- 172.21.15.103:0/255887733 learned_addr learned my addr 172.21.15.103:0/255887733 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f548ce7b640 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfc8370 0x55fe2dfc5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f5487fff640 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 -- 172.21.15.103:0/255887733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 msgr2=0x55fe2dfc7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 -- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfc8370 msgr2=0x55fe2dfc5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfc8370 0x55fe2dfc5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 -- 172.21.15.103:0/255887733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2dc943c0 con 0x55fe2dfbeb40 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54877fe640 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc50b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f547c0980a0 tx=0x7f547c099b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8808945d3668d0f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54677fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54677fe640 1 -- 172.21.15.103:0/255887733 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c020040 con 0x55fe2dfbeb40 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54677fe640 1 -- 172.21.15.103:0/255887733 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f547c097d40 con 0x55fe2dfbeb40 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe2df30860 con 0x55fe2dfbeb40 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f54677fe640 1 -- 172.21.15.103:0/255887733 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f547c09f9d0 con 0x55fe2dfbeb40 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f5490cb7740 10 client.18798 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.664+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe2dfd6b00 con 0x55fe2dfbeb40 2024-09-19T07:54:42.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f54677fe640 1 -- 172.21.15.103:0/255887733 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fe2df30860 con 0x55fe2dfbeb40 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f54677fe640 1 -- 172.21.15.103:0/255887733 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f547c023060 con 0x55fe2dfbeb40 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f5490cb7740 10 client.18798 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f5490cb7740 10 client.18798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f5490cb7740 20 client.18798 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f5490cb7740 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2dfda240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f5490cb7740 4 client.18798 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe2df2ad70 con 0x55fe2dfd7e50 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.665+0000 7f5487fff640 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2dfda240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.666+0000 7f5487fff640 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2dfda240 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ec361ba7f7d62ad server_cookie=58ecd1136372cbf3 in_seq=0 out_seq=0 2024-09-19T07:54:42.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.666+0000 7f54677fe640 10 client.18798 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:54:42.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.666+0000 7f54677fe640 1 -- 172.21.15.103:0/255887733 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe2df2ad70 con 0x55fe2dfd7e50 2024-09-19T07:54:42.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.666+0000 7f54677fe640 10 client.18798 handle_command_reply: tid=0 2024-09-19T07:54:42.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 client.18798 shutdown 2024-09-19T07:54:42.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 msgr2=0x55fe2dfc50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc50b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f547c0980a0 tx=0x7f547c099b50 comp rx=0 tx=0).stop 2024-09-19T07:54:42.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 shutdown_connections 2024-09-19T07:54:42.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2dfda240 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfc8370 0x55fe2dfc5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f54677fe640 0 client.18798 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc50b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 --2- 172.21.15.103:0/255887733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 >> 172.21.15.103:0/255887733 conn(0x55fe2df86310 msgr2=0x55fe2dfbf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 shutdown_connections 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 1 -- 172.21.15.103:0/255887733 wait complete. 2024-09-19T07:54:42.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.667+0000 7f5490cb7740 20 client.18798 trim_cache size 0 max 16384 2024-09-19T07:54:42.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 Processor -- start 2024-09-19T07:54:42.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 -- start start 2024-09-19T07:54:42.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfc8370 0x55fe2dfc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54881ad9b0 con 0x55fe2dfbeb40 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54881af580 con 0x55fe2dfbd5d0 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5490cb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54881ae080 con 0x55fe2dfc8370 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44328/0 (socket says 172.21.15.103:44328) 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 -- 172.21.15.103:0/870997260 learned_addr learned my addr 172.21.15.103:0/870997260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5487fff640 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 -- 172.21.15.103:0/870997260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfc8370 msgr2=0x55fe2dfc9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfc8370 0x55fe2dfc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 -- 172.21.15.103:0/870997260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 msgr2=0x55fe2dfc4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f54877fe640 1 -- 172.21.15.103:0/870997260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2dfba960 con 0x55fe2dfbeb40 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.680+0000 7f5487fff640 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f54877fe640 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc4090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f547c0a40d0 tx=0x7f547c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1a41c2053e0800 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f546dffb640 1 -- 172.21.15.103:0/870997260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c020070 con 0x55fe2dfbeb40 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f546dffb640 1 -- 172.21.15.103:0/870997260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f547c0a3e00 con 0x55fe2dfbeb40 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f546dffb640 1 -- 172.21.15.103:0/870997260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c097d40 con 0x55fe2dfbeb40 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f5490cb7740 1 -- 172.21.15.103:0/870997260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 msgr2=0x55fe2dfc4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f5490cb7740 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc4090 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f547c0a40d0 tx=0x7f547c0048a0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f5490cb7740 1 -- 172.21.15.103:0/870997260 shutdown_connections 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f5490cb7740 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfc8370 0x55fe2dfc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f5490cb7740 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfc4090 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f5490cb7740 1 --2- 172.21.15.103:0/870997260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfc4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.681+0000 7f5490cb7740 1 -- 172.21.15.103:0/870997260 >> 172.21.15.103:0/870997260 conn(0x55fe2df86310 msgr2=0x55fe2dfc3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 -- 172.21.15.103:0/870997260 shutdown_connections 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 -- 172.21.15.103:0/870997260 wait complete. 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 -- start start 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 Processor -- start 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfdd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfd8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfd9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54881aee50 con 0x55fe2dfc8370 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54881ae750 con 0x55fe2dfbeb40 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54881ab9b0 con 0x55fe2dfbd5d0 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f5490cb7740 10 client.0 _ll_register_callbacks cb 0x7f547800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f548ce7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfd9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f548ce7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfd9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44338/0 (socket says 172.21.15.103:44338) 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.682+0000 7f548ce7b640 1 -- 172.21.15.103:0/3211056098 learned_addr learned my addr 172.21.15.103:0/3211056098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f54877fe640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfd8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f5487fff640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfdd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f548ce7b640 1 -- 172.21.15.103:0/3211056098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 msgr2=0x55fe2dfdd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f548ce7b640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfdd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f548ce7b640 1 -- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbeb40 msgr2=0x55fe2dfd8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f548ce7b640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfd8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f548ce7b640 1 -- 172.21.15.103:0/3211056098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe2dfd9950 con 0x55fe2dfc8370 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f5487fff640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfdd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f548ce7b640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfd9170 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5480004ad0 tx=0x7f548002ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb33ca98dde5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f54677fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f54677fe640 1 -- 172.21.15.103:0/3211056098 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480043050 con 0x55fe2dfc8370 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f54677fe640 1 -- 172.21.15.103:0/3211056098 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f5480004610 con 0x55fe2dfc8370 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f547c0a3e00 con 0x55fe2dfc8370 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f54677fe640 1 -- 172.21.15.103:0/3211056098 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5480034040 con 0x55fe2dfc8370 2024-09-19T07:54:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f5490cb7740 10 client.18804 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.683+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe2e042640 con 0x55fe2dfc8370 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f54677fe640 1 -- 172.21.15.103:0/3211056098 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f547c0a3e00 con 0x55fe2dfc8370 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f54677fe640 1 -- 172.21.15.103:0/3211056098 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f5480057510 con 0x55fe2dfc8370 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f5490cb7740 10 client.18804 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f5490cb7740 10 client.18804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f5490cb7740 20 client.18804 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2e044dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f5490cb7740 4 client.18804 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 --> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe2dea7170 con 0x55fe2dfd7e50 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.684+0000 7f5487fff640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2e044dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:42.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f54677fe640 10 client.18804 ms_handle_connect on v2:172.21.15.134:6836/3959604171 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5487fff640 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2e044dd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24ed279907d16667 server_cookie=6c764b17e2a386a in_seq=0 out_seq=0 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f54677fe640 1 -- 172.21.15.103:0/3211056098 <== mds.2 v2:172.21.15.134:6836/3959604171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe2dea7170 con 0x55fe2dfd7e50 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f54677fe640 10 client.18804 handle_command_reply: tid=0 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 client.18804 shutdown 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 msgr2=0x55fe2dfd9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfd9170 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5480004ad0 tx=0x7f548002ff90 comp rx=0 tx=0).stop 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 shutdown_connections 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:6836/3959604171,v1:172.21.15.134:6837/3959604171] conn(0x55fe2dfd7e50 0x55fe2e044dd0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f54677fe640 0 client.18804 ms_handle_reset on v2:172.21.15.134:6836/3959604171 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe2dfc8370 0x55fe2dfd9170 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe2dfbeb40 0x55fe2dfd8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 --2- 172.21.15.103:0/3211056098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe2dfbd5d0 0x55fe2dfdd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 >> 172.21.15.103:0/3211056098 conn(0x55fe2df86310 msgr2=0x55fe2dfb8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:42.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 shutdown_connections 2024-09-19T07:54:42.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 1 -- 172.21.15.103:0/3211056098 wait complete. 2024-09-19T07:54:42.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.685+0000 7f5490cb7740 20 client.18804 trim_cache size 0 max 16384 2024-09-19T07:54:42.697 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:54:42.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f54700a3500 msgr2=0x7f54700a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f54700a3500 0x7f54700a59c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5480030040 tx=0x7f548002a950 comp rx=0 tx=0).stop 2024-09-19T07:54:42.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102570 msgr2=0x7f5488198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:42.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102570 0x7f5488198a70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f547c00d770 tx=0x7f547c00dc30 comp rx=0 tx=0).stop 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 shutdown_connections 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f54700a3500 0x7f54700a59c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5488102ef0 0x7f548819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5488102570 0x7f5488198a70 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 --2- 172.21.15.103:0/1305086690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5488100e20 0x7f5488198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 >> 172.21.15.103:0/1305086690 conn(0x7f54880fc7d0 msgr2=0x7f548806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 shutdown_connections 2024-09-19T07:54:42.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:42.686+0000 7f548e8de640 1 -- 172.21.15.103:0/1305086690 wait complete. 2024-09-19T07:54:43.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T07:54:43.781 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:44.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1746095596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0103650 msgr2=0x7f5ec010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1746095596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0103650 0x7f5ec010bb40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009900 tx=0x7f5eb002a480 comp rx=0 tx=0).stop 2024-09-19T07:54:44.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1746095596 shutdown_connections 2024-09-19T07:54:44.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1746095596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec010c160 0x7f5ec010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1746095596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0103650 0x7f5ec010bb40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1746095596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec0102d30 0x7f5ec0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1746095596 >> 172.21.15.103:0/1746095596 conn(0x7f5ec00fc7d0 msgr2=0x7f5ec00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1746095596 shutdown_connections 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.235+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1746095596 wait complete. 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 Processor -- start 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 -- start start 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0102d30 0x7f5ec01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec0103650 0x7f5ec01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec010c160 0x7f5ec019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec0111f60 con 0x7f5ec0102d30 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec0111de0 con 0x7f5ec0103650 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ec5ad7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01120e0 con 0x7f5ec010c160 2024-09-19T07:54:44.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0102d30 0x7f5ec01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0102d30 0x7f5ec01984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44348/0 (socket says 172.21.15.103:44348) 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1843211207 learned_addr learned my addr 172.21.15.103:0/1843211207 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ebffff640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec010c160 0x7f5ec019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.236+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec0103650 0x7f5ec01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1843211207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec010c160 msgr2=0x7f5ec019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec010c160 0x7f5ec019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1843211207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec0103650 msgr2=0x7f5ec01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec0103650 0x7f5ec01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1843211207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec019d450 con 0x7f5ec0102d30 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0102d30 0x7f5ec01984a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400d770 tx=0x7f5eb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebcff9640 1 -- 172.21.15.103:0/1843211207 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4017070 con 0x7f5ec0102d30 2024-09-19T07:54:44.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec019d740 con 0x7f5ec0102d30 2024-09-19T07:54:44.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebcff9640 1 -- 172.21.15.103:0/1843211207 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb4005500 con 0x7f5ec0102d30 2024-09-19T07:54:44.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ebcff9640 1 -- 172.21.15.103:0/1843211207 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4004e00 con 0x7f5ec0102d30 2024-09-19T07:54:44.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.237+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec01a5020 con 0x7f5ec0102d30 2024-09-19T07:54:44.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.239+0000 7f5ebcff9640 1 -- 172.21.15.103:0/1843211207 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5eb4004fa0 con 0x7f5ec0102d30 2024-09-19T07:54:44.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.240+0000 7f5ebcff9640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5ea80a33b0 0x7f5ea80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.240+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5ea80a33b0 0x7f5ea80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.240+0000 7f5ebcff9640 1 -- 172.21.15.103:0/1843211207 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f5eb40cf710 con 0x7f5ec0102d30 2024-09-19T07:54:44.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.240+0000 7f5ec7eb0740 1 Processor -- start 2024-09-19T07:54:44.264 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ec7eb0740 1 -- start start 2024-09-19T07:54:44.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5ea80a33b0 0x7f5ea80a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009230 tx=0x7f5eb00058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06db40 0x56423e06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e079760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56423dfd22f0 con 0x56423e06db40 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56423df79cf0 con 0x56423e06c5d0 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56423dfce2b0 con 0x56423e077370 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06db40 0x56423e06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06db40 0x56423e06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44362/0 (socket says 172.21.15.103:44362) 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/4246641757 learned_addr learned my addr 172.21.15.103:0/4246641757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:44.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebffff640 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e079760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/4246641757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 msgr2=0x56423e079760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e079760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/4246641757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 msgr2=0x56423e06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/4246641757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423df5e3a0 con 0x56423e06db40 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.241+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06db40 0x56423e06c090 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009610 tx=0x7f5eb4004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d6f96f11ccfdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.242+0000 7f5ea5ffb640 1 -- 172.21.15.103:0/4246641757 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4020070 con 0x56423e06db40 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.242+0000 7f5ea5ffb640 1 -- 172.21.15.103:0/4246641757 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb40a45f0 con 0x56423e06db40 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.242+0000 7f5ea5ffb640 1 -- 172.21.15.103:0/4246641757 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40980a0 con 0x56423e06db40 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.242+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4246641757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06db40 msgr2=0x56423e06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.242+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06db40 0x56423e06c090 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009610 tx=0x7f5eb4004800 comp rx=0 tx=0).stop 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4246641757 shutdown_connections 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e079760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4246641757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06db40 0x56423e06c090 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4246641757 >> 172.21.15.103:0/4246641757 conn(0x56423e035310 msgr2=0x56423e068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4246641757 shutdown_connections 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4246641757 wait complete. 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 -- start start 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.243+0000 7f5ec7eb0740 1 Processor -- start 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e074080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e074810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56423e07a070 con 0x56423e06c5d0 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56423df8d060 con 0x56423e06db40 2024-09-19T07:54:44.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56423e07a1f0 con 0x56423e077370 2024-09-19T07:54:44.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 10 client.0 _ll_register_callbacks cb 0x56423dce08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:44.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e076dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44378/0 (socket says 172.21.15.103:44378) 2024-09-19T07:54:44.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1574553172 learned_addr learned my addr 172.21.15.103:0/1574553172 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:44.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebffff640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e074080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 msgr2=0x56423e074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e074810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1574553172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 msgr2=0x56423e074080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e074080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/1574553172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423dd433c0 con 0x56423e06c5d0 2024-09-19T07:54:44.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:44.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e076dd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4098400 tx=0x7f5eb4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833006b87f54acf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5e9effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5e9effd640 1 -- 172.21.15.103:0/1574553172 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40b3020 con 0x56423e06c5d0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5e9effd640 1 -- 172.21.15.103:0/1574553172 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f5eb40cd070 con 0x56423e06c5d0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.244+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb40a45f0 con 0x56423e06c5d0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5e9effd640 1 -- 172.21.15.103:0/1574553172 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5eb4020050 con 0x56423e06c5d0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 10 client.18816 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56423e085b90 con 0x56423e06c5d0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5e9effd640 1 -- 172.21.15.103:0/1574553172 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb40a45f0 con 0x56423e06c5d0 2024-09-19T07:54:44.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5e9effd640 1 -- 172.21.15.103:0/1574553172 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f5eb409a8f0 con 0x56423e06c5d0 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 10 client.18816 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 10 client.18816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 20 client.18816 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e086ee0 0x56423e0892d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 4 client.18816 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56423dfdf860 con 0x56423e086ee0 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.245+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e086ee0 0x56423e0892d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.246+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e086ee0 0x56423e0892d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4bd5d7685935ded server_cookie=f29c9dad45f927a6 in_seq=0 out_seq=0 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.246+0000 7f5e9effd640 10 client.18816 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:54:44.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.246+0000 7f5e9effd640 1 -- 172.21.15.103:0/1574553172 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56423dfdf860 con 0x56423e086ee0 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.246+0000 7f5e9effd640 10 client.18816 handle_command_reply: tid=0 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.246+0000 7f5ec7eb0740 1 client.18816 shutdown 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.246+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 msgr2=0x56423e076dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.246+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e076dd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4098400 tx=0x7f5eb4009c30 comp rx=0 tx=0).stop 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 shutdown_connections 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e086ee0 0x56423e0892d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/1574553172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e076dd0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 >> 172.21.15.103:0/1574553172 conn(0x56423e035310 msgr2=0x56423e06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5e9effd640 0 client.18816 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 shutdown_connections 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/1574553172 wait complete. 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.247+0000 7f5ec7eb0740 20 client.18816 trim_cache size 0 max 16384 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 Processor -- start 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 -- start start 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e068e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e06f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.286 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e06f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01ad990 con 0x56423e06c5d0 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01af560 con 0x56423e06db40 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.260+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01ae060 con 0x56423e077370 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e068e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44386/0 (socket says 172.21.15.103:44386) 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/3305994480 learned_addr learned my addr 172.21.15.103:0/3305994480 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebffff640 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e06f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/3305994480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 msgr2=0x56423e06f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.287 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e06f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/3305994480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 msgr2=0x56423e06f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e06f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 -- 172.21.15.103:0/3305994480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423e074df0 con 0x56423e06c5d0 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebffff640 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e06f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e068e10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5eb409b9c0 tx=0x7f5eb40048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f862e2cce6a8472d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ea57fa640 1 -- 172.21.15.103:0/3305994480 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40b3070 con 0x56423e06c5d0 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ea57fa640 1 -- 172.21.15.103:0/3305994480 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb4099df0 con 0x56423e06c5d0 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.261+0000 7f5ea57fa640 1 -- 172.21.15.103:0/3305994480 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40981a0 con 0x56423e06c5d0 2024-09-19T07:54:44.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/3305994480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 msgr2=0x56423e068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e068e10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5eb409b9c0 tx=0x7f5eb40048a0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/3305994480 shutdown_connections 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e077370 0x56423e06f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06db40 0x56423e06f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/3305994480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e06c5d0 0x56423e068e10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/3305994480 >> 172.21.15.103:0/3305994480 conn(0x56423e035310 msgr2=0x56423e068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/3305994480 shutdown_connections 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/3305994480 wait complete. 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.262+0000 7f5ec7eb0740 1 -- start start 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 1 Processor -- start 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e088f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e06db40 0x56423e089490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e077370 0x56423e0899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01aee30 con 0x56423e077370 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01ae730 con 0x56423e06c5d0 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01ab990 con 0x56423e06db40 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ec7eb0740 10 client.0 _ll_register_callbacks cb 0x7f5eb409f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e077370 0x56423e0899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e077370 0x56423e0899d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44394/0 (socket says 172.21.15.103:44394) 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 -- 172.21.15.103:0/4179355542 learned_addr learned my addr 172.21.15.103:0/4179355542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebeffd640 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e06db40 0x56423e089490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e088f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 -- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e06db40 msgr2=0x56423e089490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e06db40 0x56423e089490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 -- 172.21.15.103:0/4179355542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 msgr2=0x56423e088f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e088f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.263+0000 7f5ebffff640 1 -- 172.21.15.103:0/4179355542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423e067310 con 0x56423e077370 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ebffff640 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e077370 0x56423e0899d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8010b70 tx=0x7f5eb800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7180f9ae4f0e349d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5e9effd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:44.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5e9effd640 1 -- 172.21.15.103:0/4179355542 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb801a5f0 con 0x56423e077370 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5e9effd640 1 -- 172.21.15.103:0/4179355542 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f5eb801a790 con 0x56423e077370 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56423dfd9d70 con 0x56423e077370 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5e9effd640 1 -- 172.21.15.103:0/4179355542 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5eb8028050 con 0x56423e077370 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ec7eb0740 10 client.18822 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56423e0f0bb0 con 0x56423e077370 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5e9effd640 1 -- 172.21.15.103:0/4179355542 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56423dfd9d70 con 0x56423e077370 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5e9effd640 1 -- 172.21.15.103:0/4179355542 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f5eb8023940 con 0x56423e077370 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ec7eb0740 10 client.18822 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ec7eb0740 10 client.18822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-19T07:54:44.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ec7eb0740 20 client.18822 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.264+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e0f13b0 0x56423e0f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5ec7eb0740 4 client.18822 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 --> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56423df56170 con 0x56423e0f13b0 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e0f13b0 0x56423e0f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5ebf7fe640 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e0f13b0 0x56423e0f37d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfba643aed2e8f9f server_cookie=32fc8a8877e49c8e in_seq=0 out_seq=0 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5e9effd640 10 client.18822 ms_handle_connect on v2:172.21.15.174:6838/2514182394 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5e9effd640 1 -- 172.21.15.103:0/4179355542 <== mds.1 v2:172.21.15.174:6838/2514182394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56423df56170 con 0x56423e0f13b0 2024-09-19T07:54:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5e9effd640 10 client.18822 handle_command_reply: tid=0 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.265+0000 7f5ec7eb0740 1 client.18822 shutdown 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e077370 msgr2=0x56423e0899d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e077370 0x56423e0899d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8010b70 tx=0x7f5eb800f7f0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 shutdown_connections 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:6838/2514182394,v1:172.21.15.174:6839/2514182394] conn(0x56423e0f13b0 0x56423e0f37d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5e9effd640 0 client.18822 ms_handle_reset on v2:172.21.15.174:6838/2514182394 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56423e077370 0x56423e0899d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56423e06db40 0x56423e089490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 --2- 172.21.15.103:0/4179355542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423e06c5d0 0x56423e088f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 >> 172.21.15.103:0/4179355542 conn(0x56423e035310 msgr2=0x56423e082480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 shutdown_connections 2024-09-19T07:54:44.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 1 -- 172.21.15.103:0/4179355542 wait complete. 2024-09-19T07:54:44.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec7eb0740 20 client.18822 trim_cache size 0 max 16384 2024-09-19T07:54:44.295 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:54:44.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5ea80a33b0 msgr2=0x7f5ea80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5ea80a33b0 0x7f5ea80a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009230 tx=0x7f5eb00058d0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0102d30 msgr2=0x7f5ec01984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.266+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0102d30 0x7f5ec01984a0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400d770 tx=0x7f5eb400dc30 comp rx=0 tx=0).stop 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 shutdown_connections 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f5ea80a33b0 0x7f5ea80a5870 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec010c160 0x7f5ec019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec0103650 0x7f5ec01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 --2- 172.21.15.103:0/1843211207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ec0102d30 0x7f5ec01984a0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 >> 172.21.15.103:0/1843211207 conn(0x7f5ec00fc7d0 msgr2=0x7f5ec00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:44.297 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 shutdown_connections 2024-09-19T07:54:44.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:44.267+0000 7f5ec5ad7640 1 -- 172.21.15.103:0/1843211207 wait complete. 2024-09-19T07:54:44.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:44 smithi174 ceph-mon[35370]: pgmap v525: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-09-19T07:54:44.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:44 smithi103 ceph-mon[31201]: pgmap v525: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-09-19T07:54:44.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:44 smithi134 ceph-mon[35414]: pgmap v525: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-09-19T07:54:45.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T07:54:45.346 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:45.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 -- 172.21.15.103:0/2272005086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 msgr2=0x7f295c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 --2- 172.21.15.103:0/2272005086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c107bc0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2950009930 tx=0x7f295002a420 comp rx=0 tx=0).stop 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 -- 172.21.15.103:0/2272005086 shutdown_connections 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 --2- 172.21.15.103:0/2272005086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c108300 0x7f295c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 --2- 172.21.15.103:0/2272005086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c107bc0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 --2- 172.21.15.103:0/2272005086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f295c102dd0 0x7f295c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 -- 172.21.15.103:0/2272005086 >> 172.21.15.103:0/2272005086 conn(0x7f295c0fc7d0 msgr2=0x7f295c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 -- 172.21.15.103:0/2272005086 shutdown_connections 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 -- 172.21.15.103:0/2272005086 wait complete. 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 Processor -- start 2024-09-19T07:54:45.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.847+0000 7f296180d640 1 -- start start 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f296180d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c102dd0 0x7f295c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f296180d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f296180d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f295c108300 0x7f295c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f296180d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f295c112070 con 0x7f295c103780 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f296180d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f295c111ef0 con 0x7f295c108300 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f296180d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1121f0 con 0x7f295c102dd0 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44412/0 (socket says 172.21.15.103:44412) 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 -- 172.21.15.103:0/3654382260 learned_addr learned my addr 172.21.15.103:0/3654382260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295affd640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c102dd0 0x7f295c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295b7fe640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f295c108300 0x7f295c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 -- 172.21.15.103:0/3654382260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c102dd0 msgr2=0x7f295c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c102dd0 0x7f295c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 -- 172.21.15.103:0/3654382260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f295c108300 msgr2=0x7f295c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f295c108300 0x7f295c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 -- 172.21.15.103:0/3654382260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c19d4e0 con 0x7f295c103780 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.848+0000 7f295a7fc640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c198a70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2950005940 tx=0x7f29500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.849+0000 7f296080b640 1 -- 172.21.15.103:0/3654382260 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295003f030 con 0x7f295c103780 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.849+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295c19d770 con 0x7f295c103780 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.849+0000 7f296080b640 1 -- 172.21.15.103:0/3654382260 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2950003ce0 con 0x7f295c103780 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.849+0000 7f296080b640 1 -- 172.21.15.103:0/3654382260 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950035070 con 0x7f295c103780 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.849+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295c1a50b0 con 0x7f295c103780 2024-09-19T07:54:45.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f296080b640 1 -- 172.21.15.103:0/3654382260 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f2950042030 con 0x7f295c103780 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 Processor -- start 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 -- start start 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df67760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56176dec02f0 con 0x56176df5bb40 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56176de67cf0 con 0x56176df65370 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.851+0000 7f2963be6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56176debc2b0 con 0x56176df5a5d0 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f296080b640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29440a33b0 0x7f29440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df5a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44420/0 (socket says 172.21.15.103:44420) 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 -- 172.21.15.103:0/69457966 learned_addr learned my addr 172.21.15.103:0/69457966 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29440a33b0 0x7f29440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295a7fc640 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df67760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f296080b640 1 -- 172.21.15.103:0/3654382260 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f29500f02c0 con 0x7f295c103780 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295affd640 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 -- 172.21.15.103:0/69457966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 msgr2=0x56176df5aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df5aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 -- 172.21.15.103:0/69457966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 msgr2=0x56176df67760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df67760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 -- 172.21.15.103:0/69457966 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176de4c3a0 con 0x56176df5bb40 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295a7fc640 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df5a090 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2954012c50 tx=0x7f2954010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9a38d243aa9294 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f295b7fe640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29440a33b0 0x7f29440a5870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2954010c40 tx=0x7f2954017000 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:45.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f2941ffb640 1 -- 172.21.15.103:0/69457966 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954012e00 con 0x56176df5bb40 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f2941ffb640 1 -- 172.21.15.103:0/69457966 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2954009c00 con 0x56176df5bb40 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.852+0000 7f2941ffb640 1 -- 172.21.15.103:0/69457966 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954005420 con 0x56176df5bb40 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 -- 172.21.15.103:0/69457966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 msgr2=0x56176df5a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df5a090 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2954012c50 tx=0x7f2954010740 comp rx=0 tx=0).stop 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 -- 172.21.15.103:0/69457966 shutdown_connections 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df5aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 --2- 172.21.15.103:0/69457966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df5a090 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 -- 172.21.15.103:0/69457966 >> 172.21.15.103:0/69457966 conn(0x56176df23310 msgr2=0x56176df56c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 -- 172.21.15.103:0/69457966 shutdown_connections 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 -- 172.21.15.103:0/69457966 wait complete. 2024-09-19T07:54:45.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.853+0000 7f2963be6740 1 -- start start 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 1 Processor -- start 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df64da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df62050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56176df68010 con 0x56176df5bb40 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56176de7b060 con 0x56176df65370 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56176df68190 con 0x56176df5a5d0 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f2963be6740 10 client.0 _ll_register_callbacks cb 0x56176dbce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df62050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df62050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44428/0 (socket says 172.21.15.103:44428) 2024-09-19T07:54:45.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 -- 172.21.15.103:0/3728277066 learned_addr learned my addr 172.21.15.103:0/3728277066 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295b7fe640 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 -- 172.21.15.103:0/3728277066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 msgr2=0x56176df64da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df64da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 -- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 msgr2=0x56176df627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295a7fc640 1 -- 172.21.15.103:0/3728277066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176dc313c0 con 0x56176df5bb40 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.854+0000 7f295b7fe640 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f295a7fc640 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df62050 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f29500bc4c0 tx=0x7f2950047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8adeb4101d8c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f293affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f293affd640 1 -- 172.21.15.103:0/3728277066 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950049070 con 0x56176df5bb40 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f293affd640 1 -- 172.21.15.103:0/3728277066 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f29500d4120 con 0x56176df5bb40 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56176decd860 con 0x56176df5bb40 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f293affd640 1 -- 172.21.15.103:0/3728277066 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f29500dc040 con 0x56176df5bb40 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 10 client.18834 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56176df73af0 con 0x56176df5bb40 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f293affd640 1 -- 172.21.15.103:0/3728277066 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56176decd860 con 0x56176df5bb40 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f293affd640 1 -- 172.21.15.103:0/3728277066 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f29500edc20 con 0x56176df5bb40 2024-09-19T07:54:45.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 10 client.18834 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 10 client.18834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 20 client.18834 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176df74ff0 0x56176df77490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 4 client.18834 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.855+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176dec7d70 con 0x56176df74ff0 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.856+0000 7f295affd640 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176df74ff0 0x56176df77490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.856+0000 7f293affd640 10 client.18834 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.856+0000 7f295affd640 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176df74ff0 0x56176df77490 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c38f494cda15ddc server_cookie=adddf3ed55fdcab0 in_seq=0 out_seq=0 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f293affd640 1 -- 172.21.15.103:0/3728277066 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56176dec7d70 con 0x56176df74ff0 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f293affd640 10 client.18834 handle_command_reply: tid=0 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 client.18834 shutdown 2024-09-19T07:54:45.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 msgr2=0x56176df62050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df62050 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f29500bc4c0 tx=0x7f2950047000 comp rx=0 tx=0).stop 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 shutdown_connections 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176df74ff0 0x56176df77490 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5bb40 0x56176df62050 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 --2- 172.21.15.103:0/3728277066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df64da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 >> 172.21.15.103:0/3728277066 conn(0x56176df23310 msgr2=0x56176df5c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f293affd640 0 client.18834 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 shutdown_connections 2024-09-19T07:54:45.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.857+0000 7f2963be6740 1 -- 172.21.15.103:0/3728277066 wait complete. 2024-09-19T07:54:45.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.858+0000 7f2963be6740 20 client.18834 trim_cache size 0 max 16384 2024-09-19T07:54:45.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.870+0000 7f2963be6740 1 Processor -- start 2024-09-19T07:54:45.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.870+0000 7f2963be6740 1 -- start start 2024-09-19T07:54:45.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f2963be6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df60ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f2963be6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df5bb40 0x56176df55c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f2963be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df65370 0x56176df5d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f2963be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1ad840 con 0x56176df65370 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f2963be6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1af410 con 0x56176df5bb40 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f2963be6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1adf10 con 0x56176df5a5d0 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df65370 0x56176df5d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df65370 0x56176df5d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44438/0 (socket says 172.21.15.103:44438) 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 -- 172.21.15.103:0/2653628153 learned_addr learned my addr 172.21.15.103:0/2653628153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295affd640 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df60ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295a7fc640 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df5bb40 0x56176df55c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 -- 172.21.15.103:0/2653628153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 msgr2=0x56176df60ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df60ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 -- 172.21.15.103:0/2653628153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df5bb40 msgr2=0x56176df55c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df5bb40 0x56176df55c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 -- 172.21.15.103:0/2653628153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176df67ad0 con 0x56176df65370 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295affd640 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df60ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f295b7fe640 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df65370 0x56176df5d120 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2954012c20 tx=0x7f2954012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549d0c20529b8049 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.871+0000 7f29417fa640 1 -- 172.21.15.103:0/2653628153 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295402b070 con 0x56176df65370 2024-09-19T07:54:45.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f29417fa640 1 -- 172.21.15.103:0/2653628153 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f295400cd40 con 0x56176df65370 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f29417fa640 1 -- 172.21.15.103:0/2653628153 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295401b040 con 0x56176df65370 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 -- 172.21.15.103:0/2653628153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df65370 msgr2=0x56176df5d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df65370 0x56176df5d120 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2954012c20 tx=0x7f2954012e00 comp rx=0 tx=0).stop 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 -- 172.21.15.103:0/2653628153 shutdown_connections 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df65370 0x56176df5d120 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df5bb40 0x56176df55c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 --2- 172.21.15.103:0/2653628153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5a5d0 0x56176df60ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 -- 172.21.15.103:0/2653628153 >> 172.21.15.103:0/2653628153 conn(0x56176df23310 msgr2=0x56176df55500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 -- 172.21.15.103:0/2653628153 shutdown_connections 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.872+0000 7f2963be6740 1 -- 172.21.15.103:0/2653628153 wait complete. 2024-09-19T07:54:45.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 -- start start 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 Processor -- start 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5a5d0 0x56176df6d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5bb40 0x56176df5ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df5f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1aece0 con 0x56176df5a5d0 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1ae5e0 con 0x56176df65370 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f295c1ab840 con 0x56176df5bb40 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f2963be6740 10 client.0 _ll_register_callbacks cb 0x7f2950009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f295affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5a5d0 0x56176df6d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f295affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5a5d0 0x56176df6d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44450/0 (socket says 172.21.15.103:44450) 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f295affd640 1 -- 172.21.15.103:0/2050635189 learned_addr learned my addr 172.21.15.103:0/2050635189 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f295a7fc640 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5bb40 0x56176df5ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.873+0000 7f295b7fe640 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df5f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f295affd640 1 -- 172.21.15.103:0/2050635189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5bb40 msgr2=0x56176df5ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f295affd640 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5bb40 0x56176df5ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f295affd640 1 -- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 msgr2=0x56176df5f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f295affd640 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df5f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f295affd640 1 -- 172.21.15.103:0/2050635189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176df5f7d0 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f295affd640 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5a5d0 0x56176df6d980 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f294c005e10 tx=0x7f294c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d890c74f2fdaba1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f293affd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f293affd640 1 -- 172.21.15.103:0/2050635189 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c004260 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f293affd640 1 -- 172.21.15.103:0/2050635189 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f294c004400 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f295400cd40 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f293affd640 1 -- 172.21.15.103:0/2050635189 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f294c027050 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f2963be6740 10 client.18840 fetch_fsmap learned FSMap version 57 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56176dfdfc00 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f293affd640 1 -- 172.21.15.103:0/2050635189 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f295400cd40 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.874+0000 7f293affd640 1 -- 172.21.15.103:0/2050635189 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 57) ==== 8390+0+0 (secure 0 0 0) 0x7f294c004c30 con 0x56176df5a5d0 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f2963be6740 10 client.18840 fetch_fsmap finished waiting for FSMap version 57 2024-09-19T07:54:45.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f2963be6740 10 client.18840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f2963be6740 20 client.18840 populate_metadata read hostname 'smithi103' 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f2963be6740 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176dfe0460 0x56176dfe2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f2963be6740 4 client.18840 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 --> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56176de44170 con 0x56176dfe0460 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f295a7fc640 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176dfe0460 0x56176dfe2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f293affd640 10 client.18840 ms_handle_connect on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.875+0000 7f295a7fc640 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176dfe0460 0x56176dfe2880 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc296a12d548f5d server_cookie=760c7ec4047cb1fd in_seq=0 out_seq=0 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f293affd640 1 -- 172.21.15.103:0/2050635189 <== mds.0 v2:172.21.15.134:6838/3683561216 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56176de44170 con 0x56176dfe0460 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f293affd640 10 client.18840 handle_command_reply: tid=0 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 client.18840 shutdown 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5a5d0 msgr2=0x56176df6d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5a5d0 0x56176df6d980 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f294c005e10 tx=0x7f294c011810 comp rx=0 tx=0).stop 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 shutdown_connections 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:6838/3683561216,v1:172.21.15.134:6839/3683561216] conn(0x56176dfe0460 0x56176dfe2880 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56176df65370 0x56176df5f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56176df5bb40 0x56176df5ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 --2- 172.21.15.103:0/2050635189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56176df5a5d0 0x56176df6d980 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 >> 172.21.15.103:0/2050635189 conn(0x56176df23310 msgr2=0x56176df640d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f293affd640 0 client.18840 ms_handle_reset on v2:172.21.15.134:6838/3683561216 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 shutdown_connections 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 1 -- 172.21.15.103:0/2050635189 wait complete. 2024-09-19T07:54:45.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f2963be6740 20 client.18840 trim_cache size 0 max 16384 2024-09-19T07:54:45.888 INFO:teuthology.orchestra.run.smithi103.stdout:[] 2024-09-19T07:54:45.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29440a33b0 msgr2=0x7f29440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f296180d640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29440a33b0 0x7f29440a5870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2954010c40 tx=0x7f2954017000 comp rx=0 tx=0).stop 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 msgr2=0x7f295c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.876+0000 7f296180d640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c198a70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2950005940 tx=0x7f29500042f0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 shutdown_connections 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f29440a33b0 0x7f29440a5870 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f295c108300 0x7f295c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f295c103780 0x7f295c198a70 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 --2- 172.21.15.103:0/3654382260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c102dd0 0x7f295c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:45.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 >> 172.21.15.103:0/3654382260 conn(0x7f295c0fc7d0 msgr2=0x7f295c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:45.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 shutdown_connections 2024-09-19T07:54:45.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:45.877+0000 7f296180d640 1 -- 172.21.15.103:0/3654382260 wait complete. 2024-09-19T07:54:46.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:46 smithi174 ceph-mon[35370]: pgmap v526: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-19T07:54:46.469 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:46 smithi103 ceph-mon[31201]: pgmap v526: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-19T07:54:46.570 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T07:54:46.571 INFO:tasks.fwd_scrub:done joining 2024-09-19T07:54:46.571 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T07:54:46.583 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T07:54:46.584 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:46 smithi134 ceph-mon[35414]: pgmap v526: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-19T07:54:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- 172.21.15.103:0/2733077864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c101230 msgr2=0x7f511c107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 --2- 172.21.15.103:0/2733077864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c101230 0x7f511c107970 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f510c009900 tx=0x7f510c02a410 comp rx=0 tx=0).stop 2024-09-19T07:54:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- 172.21.15.103:0/2733077864 shutdown_connections 2024-09-19T07:54:46.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 --2- 172.21.15.103:0/2733077864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511c1080d0 0x7f511c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 --2- 172.21.15.103:0/2733077864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c101230 0x7f511c107970 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 --2- 172.21.15.103:0/2733077864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f511c100880 0x7f511c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- 172.21.15.103:0/2733077864 >> 172.21.15.103:0/2733077864 conn(0x7f511c075f60 msgr2=0x7f511c076370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- 172.21.15.103:0/2733077864 shutdown_connections 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- 172.21.15.103:0/2733077864 wait complete. 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 Processor -- start 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- start start 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f511c100880 0x7f511c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511c1080d0 0x7f511c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c19cd10 0x7f511c19d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f511c111f00 con 0x7f511c19cd10 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f511c111d80 con 0x7f511c1080d0 2024-09-19T07:54:46.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.689+0000 7f51238f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f511c112080 con 0x7f511c100880 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c19cd10 0x7f511c19d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c19cd10 0x7f511c19d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44464/0 (socket says 172.21.15.103:44464) 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 -- 172.21.15.103:0/2028447212 learned_addr learned my addr 172.21.15.103:0/2028447212 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5120e93640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511c1080d0 0x7f511c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 -- 172.21.15.103:0/2028447212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f511c100880 msgr2=0x7f511c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f511c100880 0x7f511c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 -- 172.21.15.103:0/2028447212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511c1080d0 msgr2=0x7f511c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511c1080d0 0x7f511c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 -- 172.21.15.103:0/2028447212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c19d800 con 0x7f511c19cd10 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5120e93640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511c1080d0 0x7f511c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f5121e95640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c19cd10 0x7f511c19d1c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f511400eaa0 tx=0x7f511400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f51127fc640 1 -- 172.21.15.103:0/2028447212 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400cdb0 con 0x7f511c19cd10 2024-09-19T07:54:46.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511c19daf0 con 0x7f511c19cd10 2024-09-19T07:54:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f51127fc640 1 -- 172.21.15.103:0/2028447212 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5114005020 con 0x7f511c19cd10 2024-09-19T07:54:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f51127fc640 1 -- 172.21.15.103:0/2028447212 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140116b0 con 0x7f511c19cd10 2024-09-19T07:54:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.690+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511c1a5260 con 0x7f511c19cd10 2024-09-19T07:54:46.693 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.691+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f511c100c60 con 0x7f511c19cd10 2024-09-19T07:54:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.692+0000 7f51127fc640 1 -- 172.21.15.103:0/2028447212 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f5114005560 con 0x7f511c19cd10 2024-09-19T07:54:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.693+0000 7f51127fc640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f51080a33c0 0x7f51080a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.693+0000 7f5121694640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f51080a33c0 0x7f51080a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.693+0000 7f51127fc640 1 -- 172.21.15.103:0/2028447212 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f51140d0030 con 0x7f511c19cd10 2024-09-19T07:54:46.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.694+0000 7f5121694640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f51080a33c0 0x7f51080a5880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5118009710 tx=0x7f5118006c90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:46.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.697+0000 7f51127fc640 1 -- 172.21.15.103:0/2028447212 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5114016030 con 0x7f511c19cd10 2024-09-19T07:54:46.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.866+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f511c101230 con 0x7f511c19cd10 2024-09-19T07:54:46.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.868+0000 7f51127fc640 1 -- 172.21.15.103:0/2028447212 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+23294 (secure 0 0 0) 0x7f511409ca00 con 0x7f511c19cd10 2024-09-19T07:54:46.868 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:46.868 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":105,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:54:11.032094+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/3492331771":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/189968053":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2164654927":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/2266926610":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/532109813":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6800/2164654927":"2024-09-20T07:37:58.271121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:54:46.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f51080a33c0 msgr2=0x7f51080a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f51080a33c0 0x7f51080a5880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5118009710 tx=0x7f5118006c90 comp rx=0 tx=0).stop 2024-09-19T07:54:46.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c19cd10 msgr2=0x7f511c19d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c19cd10 0x7f511c19d1c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f511400eaa0 tx=0x7f511400ef60 comp rx=0 tx=0).stop 2024-09-19T07:54:46.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 shutdown_connections 2024-09-19T07:54:46.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f51080a33c0 0x7f51080a5880 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f511c19cd10 0x7f511c19d1c0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511c1080d0 0x7f511c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 --2- 172.21.15.103:0/2028447212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f511c100880 0x7f511c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 >> 172.21.15.103:0/2028447212 conn(0x7f511c075f60 msgr2=0x7f511c105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:46.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 shutdown_connections 2024-09-19T07:54:46.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.870+0000 7f51238f8640 1 -- 172.21.15.103:0/2028447212 wait complete. 2024-09-19T07:54:46.882 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T07:31:30.307307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T07:36:04.766918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T07:36:06.565665+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T07:54:46.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 -- 172.21.15.103:0/44651259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101290 msgr2=0x7f0908104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 --2- 172.21.15.103:0/44651259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101290 0x7f0908104e20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009900 tx=0x7f08f802a410 comp rx=0 tx=0).stop 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 -- 172.21.15.103:0/44651259 shutdown_connections 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 --2- 172.21.15.103:0/44651259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0908105580 0x7f090810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 --2- 172.21.15.103:0/44651259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101290 0x7f0908104e20 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 --2- 172.21.15.103:0/44651259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09081008e0 0x7f0908100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 -- 172.21.15.103:0/44651259 >> 172.21.15.103:0/44651259 conn(0x7f0908075ea0 msgr2=0x7f09080762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 -- 172.21.15.103:0/44651259 shutdown_connections 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.986+0000 7f090f61e640 1 -- 172.21.15.103:0/44651259 wait complete. 2024-09-19T07:54:46.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 Processor -- start 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 -- start start 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09081008e0 0x7f0908198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0908105580 0x7f0908198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f090819ce00 0x7f090819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f090810fd80 con 0x7f090819ce00 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f090810fc00 con 0x7f09081008e0 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090f61e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f090810ff00 con 0x7f0908105580 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f090819ce00 0x7f090819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f090819ce00 0x7f090819d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44490/0 (socket says 172.21.15.103:44490) 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 -- 172.21.15.103:0/2930529993 learned_addr learned my addr 172.21.15.103:0/2930529993 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:46.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090d3ba640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09081008e0 0x7f0908198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090cbb9640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0908105580 0x7f0908198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 -- 172.21.15.103:0/2930529993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0908105580 msgr2=0x7f0908198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0908105580 0x7f0908198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 -- 172.21.15.103:0/2930529993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09081008e0 msgr2=0x7f0908198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09081008e0 0x7f0908198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 -- 172.21.15.103:0/2930529993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090819d8f0 con 0x7f090819ce00 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090d3ba640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09081008e0 0x7f0908198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.987+0000 7f090dbbb640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f090819ce00 0x7f090819d2b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f090000dd70 tx=0x7f090000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.988+0000 7f08fe7fc640 1 -- 172.21.15.103:0/2930529993 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900004d20 con 0x7f090819ce00 2024-09-19T07:54:46.991 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.988+0000 7f08fe7fc640 1 -- 172.21.15.103:0/2930529993 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f090000be40 con 0x7f090819ce00 2024-09-19T07:54:46.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.988+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090819dbe0 con 0x7f090819ce00 2024-09-19T07:54:46.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.988+0000 7f08fe7fc640 1 -- 172.21.15.103:0/2930529993 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900005720 con 0x7f090819ce00 2024-09-19T07:54:46.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.988+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081a5380 con 0x7f090819ce00 2024-09-19T07:54:46.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.989+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0908100cc0 con 0x7f090819ce00 2024-09-19T07:54:46.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.990+0000 7f08fe7fc640 1 -- 172.21.15.103:0/2930529993 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f09000058c0 con 0x7f090819ce00 2024-09-19T07:54:46.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.990+0000 7f08fe7fc640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f08f40a33c0 0x7f08f40a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:46.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.991+0000 7f090d3ba640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f08f40a33c0 0x7f08f40a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:46.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.991+0000 7f08fe7fc640 1 -- 172.21.15.103:0/2930529993 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f09000d0500 con 0x7f090819ce00 2024-09-19T07:54:46.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.991+0000 7f090d3ba640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f08f40a33c0 0x7f08f40a5880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0904006fd0 tx=0x7f0904008040 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:46.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:46.994+0000 7f08fe7fc640 1 -- 172.21.15.103:0/2930529993 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f090009cfe0 con 0x7f090819ce00 2024-09-19T07:54:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.159+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0908101290 con 0x7f090819ce00 2024-09-19T07:54:47.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.160+0000 7f08fe7fc640 1 -- 172.21.15.103:0/2930529993 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f090009c8e0 con 0x7f090819ce00 2024-09-19T07:54:47.160 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f08f40a33c0 msgr2=0x7f08f40a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f08f40a33c0 0x7f08f40a5880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0904006fd0 tx=0x7f0904008040 comp rx=0 tx=0).stop 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f090819ce00 msgr2=0x7f090819d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f090819ce00 0x7f090819d2b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f090000dd70 tx=0x7f090000b730 comp rx=0 tx=0).stop 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 shutdown_connections 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f08f40a33c0 0x7f08f40a5880 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f090819ce00 0x7f090819d2b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0908105580 0x7f0908198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 --2- 172.21.15.103:0/2930529993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09081008e0 0x7f0908198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 >> 172.21.15.103:0/2930529993 conn(0x7f0908075ea0 msgr2=0x7f09080fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 shutdown_connections 2024-09-19T07:54:47.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.162+0000 7f090f61e640 1 -- 172.21.15.103:0/2930529993 wait complete. 2024-09-19T07:54:47.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:47 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2028447212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:54:47.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 -- 172.21.15.103:0/895430147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 msgr2=0x7f422810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 --2- 172.21.15.103:0/895430147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f422810d850 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f4220009cd0 tx=0x7f422002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 -- 172.21.15.103:0/895430147 shutdown_connections 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 --2- 172.21.15.103:0/895430147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f422810d850 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 --2- 172.21.15.103:0/895430147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4228100980 0x7f4228100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 --2- 172.21.15.103:0/895430147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4228106990 0x7f4228106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 -- 172.21.15.103:0/895430147 >> 172.21.15.103:0/895430147 conn(0x7f42280779d0 msgr2=0x7f42280fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 -- 172.21.15.103:0/895430147 shutdown_connections 2024-09-19T07:54:47.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.276+0000 7f422d0f8640 1 -- 172.21.15.103:0/895430147 wait complete. 2024-09-19T07:54:47.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 Processor -- start 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 -- start start 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4228100980 0x7f4228197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f4228197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4228106990 0x7f422819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4228110dd0 con 0x7f4228101320 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4228110c50 con 0x7f4228106990 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422d0f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4228110f50 con 0x7f4228100980 2024-09-19T07:54:47.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f4228197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f4228197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44514/0 (socket says 172.21.15.103:44514) 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 -- 172.21.15.103:0/3087745869 learned_addr learned my addr 172.21.15.103:0/3087745869 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f4226d9d640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4228100980 0x7f4228197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422759e640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4228106990 0x7f422819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 -- 172.21.15.103:0/3087745869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4228100980 msgr2=0x7f4228197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4228100980 0x7f4228197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 -- 172.21.15.103:0/3087745869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4228106990 msgr2=0x7f422819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4228106990 0x7f422819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422659c640 1 -- 172.21.15.103:0/3087745869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422819c2f0 con 0x7f4228101320 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f4226d9d640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4228100980 0x7f4228197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.277+0000 7f422759e640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4228106990 0x7f422819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:54:47.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.278+0000 7f422659c640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f4228197840 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f421800eb00 tx=0x7f4218009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.278+0000 7f421ffff640 1 -- 172.21.15.103:0/3087745869 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421801c070 con 0x7f4228101320 2024-09-19T07:54:47.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.278+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422819c5e0 con 0x7f4228101320 2024-09-19T07:54:47.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.278+0000 7f421ffff640 1 -- 172.21.15.103:0/3087745869 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421800bc30 con 0x7f4228101320 2024-09-19T07:54:47.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.278+0000 7f421ffff640 1 -- 172.21.15.103:0/3087745869 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42180166a0 con 0x7f4228101320 2024-09-19T07:54:47.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.278+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4228069380 con 0x7f4228101320 2024-09-19T07:54:47.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.279+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4228100de0 con 0x7f4228101320 2024-09-19T07:54:47.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.280+0000 7f421ffff640 1 -- 172.21.15.103:0/3087745869 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f421801f070 con 0x7f4228101320 2024-09-19T07:54:47.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.281+0000 7f421ffff640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42140a33c0 0x7f42140a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.281+0000 7f4226d9d640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42140a33c0 0x7f42140a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.281+0000 7f421ffff640 1 -- 172.21.15.103:0/3087745869 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f42180d1850 con 0x7f4228101320 2024-09-19T07:54:47.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.281+0000 7f4226d9d640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42140a33c0 0x7f42140a5880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4210009ad0 tx=0x7f4210005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.284+0000 7f421ffff640 1 -- 172.21.15.103:0/3087745869 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f421800bde0 con 0x7f4228101320 2024-09-19T07:54:47.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:47 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2028447212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.450+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4228069ed0 con 0x7f4228101320 2024-09-19T07:54:47.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.450+0000 7f421ffff640 1 -- 172.21.15.103:0/3087745869 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f4218086e20 con 0x7f4228101320 2024-09-19T07:54:47.450 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-19T07:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42140a33c0 msgr2=0x7f42140a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42140a33c0 0x7f42140a5880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4210009ad0 tx=0x7f4210005f90 comp rx=0 tx=0).stop 2024-09-19T07:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 msgr2=0x7f4228197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f4228197840 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f421800eb00 tx=0x7f4218009f20 comp rx=0 tx=0).stop 2024-09-19T07:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 shutdown_connections 2024-09-19T07:54:47.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f42140a33c0 0x7f42140a5880 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4228106990 0x7f422819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4228101320 0x7f4228197840 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 --2- 172.21.15.103:0/3087745869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4228100980 0x7f4228197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 >> 172.21.15.103:0/3087745869 conn(0x7f42280779d0 msgr2=0x7f4228103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 shutdown_connections 2024-09-19T07:54:47.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.452+0000 7f422d0f8640 1 -- 172.21.15.103:0/3087745869 wait complete. 2024-09-19T07:54:47.463 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 -- 172.21.15.103:0/360288280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 msgr2=0x7f4434101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 --2- 172.21.15.103:0/360288280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434101ef0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4428009930 tx=0x7f442802a420 comp rx=0 tx=0).stop 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 -- 172.21.15.103:0/360288280 shutdown_connections 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 --2- 172.21.15.103:0/360288280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4434102430 0x7f443410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 --2- 172.21.15.103:0/360288280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434101ef0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 --2- 172.21.15.103:0/360288280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434107aa0 0x7f4434107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 -- 172.21.15.103:0/360288280 >> 172.21.15.103:0/360288280 conn(0x7f44340fd790 msgr2=0x7f44340ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 -- 172.21.15.103:0/360288280 shutdown_connections 2024-09-19T07:54:47.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 -- 172.21.15.103:0/360288280 wait complete. 2024-09-19T07:54:47.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.568+0000 7f443a6e4640 1 Processor -- start 2024-09-19T07:54:47.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f443a6e4640 1 -- start start 2024-09-19T07:54:47.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f443a6e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f443a6e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4434102430 0x7f44341989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f443a6e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434107aa0 0x7f443419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f443a6e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4434111ea0 con 0x7f4434101a90 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f443a6e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4434111d20 con 0x7f4434102430 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f443a6e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4434112020 con 0x7f4434107aa0 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44530/0 (socket says 172.21.15.103:44530) 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 -- 172.21.15.103:0/2869985474 learned_addr learned my addr 172.21.15.103:0/2869985474 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f44337fe640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4434102430 0x7f44341989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 -- 172.21.15.103:0/2869985474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434107aa0 msgr2=0x7f443419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434107aa0 0x7f443419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 -- 172.21.15.103:0/2869985474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4434102430 msgr2=0x7f44341989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4434102430 0x7f44341989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 -- 172.21.15.103:0/2869985474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443419d480 con 0x7f4434101a90 2024-09-19T07:54:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f4433fff640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434198490 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f442400d950 tx=0x7f442400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.569+0000 7f44317fa640 1 -- 172.21.15.103:0/2869985474 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424017070 con 0x7f4434101a90 2024-09-19T07:54:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.570+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443419d770 con 0x7f4434101a90 2024-09-19T07:54:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.570+0000 7f44317fa640 1 -- 172.21.15.103:0/2869985474 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4424009c30 con 0x7f4434101a90 2024-09-19T07:54:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.570+0000 7f44317fa640 1 -- 172.21.15.103:0/2869985474 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442400f6b0 con 0x7f4434101a90 2024-09-19T07:54:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.570+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a5000 con 0x7f4434101a90 2024-09-19T07:54:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.571+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434101ef0 con 0x7f4434101a90 2024-09-19T07:54:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.572+0000 7f44317fa640 1 -- 172.21.15.103:0/2869985474 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f44240109d0 con 0x7f4434101a90 2024-09-19T07:54:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.572+0000 7f44317fa640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f441c0a33c0 0x7f441c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.572+0000 7f44337fe640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f441c0a33c0 0x7f441c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.572+0000 7f44317fa640 1 -- 172.21.15.103:0/2869985474 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7f44240cf970 con 0x7f4434101a90 2024-09-19T07:54:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.573+0000 7f44337fe640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f441c0a33c0 0x7f441c0a5880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4428000c00 tx=0x7f4428005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.576 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.576+0000 7f44317fa640 1 -- 172.21.15.103:0/2869985474 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4424009de0 con 0x7f4434101a90 2024-09-19T07:54:47.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:47 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2028447212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:54:47.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.746+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f443410a080 con 0x7f4434101a90 2024-09-19T07:54:47.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.746+0000 7f44317fa640 1 -- 172.21.15.103:0/2869985474 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f442409c340 con 0x7f4434101a90 2024-09-19T07:54:47.746 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-19T07:54:47.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.748+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f441c0a33c0 msgr2=0x7f441c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.748+0000 7f443a6e4640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f441c0a33c0 0x7f441c0a5880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4428000c00 tx=0x7f4428005a20 comp rx=0 tx=0).stop 2024-09-19T07:54:47.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.748+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 msgr2=0x7f4434198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.748+0000 7f443a6e4640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434198490 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f442400d950 tx=0x7f442400de10 comp rx=0 tx=0).stop 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 shutdown_connections 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f441c0a33c0 0x7f441c0a5880 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434107aa0 0x7f443419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4434102430 0x7f44341989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 --2- 172.21.15.103:0/2869985474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4434101a90 0x7f4434198490 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 >> 172.21.15.103:0/2869985474 conn(0x7f44340fd790 msgr2=0x7f44340fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 shutdown_connections 2024-09-19T07:54:47.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.749+0000 7f443a6e4640 1 -- 172.21.15.103:0/2869985474 wait complete. 2024-09-19T07:54:47.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T07:54:47.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 -- 172.21.15.103:0/4282710064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101aa0 msgr2=0x7fb590101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 --2- 172.21.15.103:0/4282710064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101aa0 0x7fb590101f00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb584009930 tx=0x7fb58402a420 comp rx=0 tx=0).stop 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 -- 172.21.15.103:0/4282710064 shutdown_connections 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 --2- 172.21.15.103:0/4282710064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590102440 0x7fb59010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 --2- 172.21.15.103:0/4282710064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590101aa0 0x7fb590101f00 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 --2- 172.21.15.103:0/4282710064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590107ab0 0x7fb590107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 -- 172.21.15.103:0/4282710064 >> 172.21.15.103:0/4282710064 conn(0x7fb5900fd7c0 msgr2=0x7fb5900ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 -- 172.21.15.103:0/4282710064 shutdown_connections 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.860+0000 7fb595ed7640 1 -- 172.21.15.103:0/4282710064 wait complete. 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 Processor -- start 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 -- start start 2024-09-19T07:54:47.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590101aa0 0x7fb59019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590102440 0x7fb59019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590107ab0 0x7fb59019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb590111ef0 con 0x7fb590102440 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb590111d70 con 0x7fb590101aa0 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb595ed7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb590112070 con 0x7fb590107ab0 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.861+0000 7fb58effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590102440 0x7fb59019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590102440 0x7fb59019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44544/0 (socket says 172.21.15.103:44544) 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 -- 172.21.15.103:0/3146005200 learned_addr learned my addr 172.21.15.103:0/3146005200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58f7fe640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590101aa0 0x7fb59019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 -- 172.21.15.103:0/3146005200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590107ab0 msgr2=0x7fb59019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590107ab0 0x7fb59019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 -- 172.21.15.103:0/3146005200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590101aa0 msgr2=0x7fb59019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590101aa0 0x7fb59019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 -- 172.21.15.103:0/3146005200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59019f5f0 con 0x7fb590102440 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58f7fe640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590101aa0 0x7fb59019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58effd640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590102440 0x7fb59019ab40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb584009a50 tx=0x7fb584004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58cff9640 1 -- 172.21.15.103:0/3146005200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58403f030 con 0x7fb590102440 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59019f880 con 0x7fb590102440 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58cff9640 1 -- 172.21.15.103:0/3146005200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb584003ce0 con 0x7fb590102440 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb58cff9640 1 -- 172.21.15.103:0/3146005200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584035070 con 0x7fb590102440 2024-09-19T07:54:47.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.862+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a7170 con 0x7fb590102440 2024-09-19T07:54:47.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.863+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb590101f00 con 0x7fb590102440 2024-09-19T07:54:47.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.864+0000 7fb58cff9640 1 -- 172.21.15.103:0/3146005200 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fb584042030 con 0x7fb590102440 2024-09-19T07:54:47.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.865+0000 7fb58cff9640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb5780a33c0 0x7fb5780a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:54:47.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.865+0000 7fb58cff9640 1 -- 172.21.15.103:0/3146005200 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8542+0+0 (secure 0 0 0) 0x7fb5840f0370 con 0x7fb590102440 2024-09-19T07:54:47.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.865+0000 7fb58f7fe640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb5780a33c0 0x7fb5780a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:54:47.866 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.865+0000 7fb58f7fe640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb5780a33c0 0x7fb5780a5880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb580005d90 tx=0x7fb580005d20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:54:47.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:47.869+0000 7fb58cff9640 1 -- 172.21.15.103:0/3146005200 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb5840bcd40 con 0x7fb590102440 2024-09-19T07:54:48.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.088+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb5901a73d0 con 0x7fb590102440 2024-09-19T07:54:48.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.095+0000 7fb58cff9640 1 -- 172.21.15.103:0/3146005200 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742879 (secure 0 0 0) 0x7fb5840b8020 con 0x7fb590102440 2024-09-19T07:54:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:54:48.107 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 26, 2024-09-19T07:54:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.107 INFO:teuthology.orchestra.run.smithi103.stdout: "active_gid": 15183, 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "active_name": "x", 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addrs": { 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6800", 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3066951043 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6801", 2024-09-19T07:54:48.108 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3066951043 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addr": "172.21.15.103:6801/3066951043", 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "active_change": "2024-09-19T07:37:58.271436+0000", 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "available": true, 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "standbys": [ 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24830, 2024-09-19T07:54:48.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "y", 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.112 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-19T07:54:48.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-19T07:54:48.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-19T07:54:48.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-19T07:54:48.125 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:54:48.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:54:48.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-19T07:54:48.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.132 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.134 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.137 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-19T07:54:48.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-19T07:54:48.141 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.143 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-19T07:54:48.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-19T07:54:48.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-19T07:54:48.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-19T07:54:48.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.152 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.156 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.157 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:54:48.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:54:48.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:54:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:54:48.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-19T07:54:48.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-19T07:54:48.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.171 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:54:48.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.180 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-19T07:54:48.181 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-19T07:54:48.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.186 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.191 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-19T07:54:48.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-19T07:54:48.201 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-19T07:54:48.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-19T07:54:48.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.209 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.210 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.215 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.218 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.223 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.226 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:54:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.232 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.233 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.244 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.246 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:54:48.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-19T07:54:48.262 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.267 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:54:48.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-19T07:54:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-19T07:54:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-19T07:54:48.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.274 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.276 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-19T07:54:48.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:54:48.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.288 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.289 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-19T07:54:48.291 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.293 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.295 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.306 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.307 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.310 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.315 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.320 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.321 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.323 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-19T07:54:48.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.328 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.329 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-19T07:54:48.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.332 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.333 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.335 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.336 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.338 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.340 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-19T07:54:48.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-19T07:54:48.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.345 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.346 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.347 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.349 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.351 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:54:48.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.353 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.354 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.356 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:54:48.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.363 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.365 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.371 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.373 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.375 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.376 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.378 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.379 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.381 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.384 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.386 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.387 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.390 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.393 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.394 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.395 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.396 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.397 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.399 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.402 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.405 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-19T07:54:48.409 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:54:48.410 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.412 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.414 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.415 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.418 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-19T07:54:48.419 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.422 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.424 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.425 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:54:48.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-19T07:54:48.429 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:48.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:48.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-19T07:54:48.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:54:48.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.436 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.439 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-19T07:54:48.444 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-19T07:54:48.446 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.447 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.448 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.449 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.449 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.449 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.449 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.449 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.450 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.452 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.453 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.457 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.458 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.460 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.462 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.465 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.466 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-19T07:54:48.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.471 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.475 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-19T07:54:48.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.478 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.479 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.480 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.480 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.480 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-19T07:54:48.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.481 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.483 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.485 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.487 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-19T07:54:48.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.492 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.493 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.494 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.495 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.498 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.499 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.500 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.501 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.507 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.508 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.509 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.510 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.511 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.512 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.513 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.514 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.516 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.517 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.518 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.520 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.521 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-19T07:54:48.522 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:54:48.523 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.525 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.526 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.527 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.528 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.529 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.530 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.531 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.532 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.533 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-19T07:54:48.534 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-19T07:54:48.535 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.537 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.538 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.539 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.540 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.541 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.544 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.544 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.544 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.544 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.544 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.546 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.547 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.548 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.549 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.550 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.552 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.554 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.555 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.556 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.557 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.558 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.559 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.560 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.561 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:54:48.562 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.563 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.564 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.565 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.566 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.567 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.568 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.569 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.570 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.571 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.573 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-19T07:54:48.575 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-19T07:54:48.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.577 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.578 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24835, 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "z", 2024-09-19T07:54:48.579 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540701547738038271, 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.580 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.581 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.582 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.584 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.585 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.586 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-19T07:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-19T07:54:48.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.590 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.591 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:54:48.594 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:54:48.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.596 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-19T07:54:48.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.598 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.599 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.601 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.602 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:54:48.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:54:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:54:48.605 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.606 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.609 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.610 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.612 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-19T07:54:48.613 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:54:48.614 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.617 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.618 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.619 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:54:48.622 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-19T07:54:48.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.626 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:54:48.627 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.628 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.630 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.632 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:54:48.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.635 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.636 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.637 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-19T07:54:48.638 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-19T07:54:48.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.640 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.646 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.648 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:54:48.650 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.651 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-19T07:54:48.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-19T07:54:48.659 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:54:48.660 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:54:48.663 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.664 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.665 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.667 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.669 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.670 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.671 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.672 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-19T07:54:48.673 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.674 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.676 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.677 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.678 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:54:48.680 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-19T07:54:48.681 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-19T07:54:48.682 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-19T07:54:48.683 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.684 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.685 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.688 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.689 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.690 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.691 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.692 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.693 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.694 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:54:48.695 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.696 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:54:48.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:54:48.698 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:54:48.699 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.700 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.701 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.703 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.704 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.705 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.706 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.707 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.708 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.710 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.712 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.713 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.714 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.715 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.716 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.717 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.718 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.720 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.721 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.723 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.724 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:54:48.726 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.727 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.728 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.731 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.732 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.733 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.736 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-19T07:54:48.737 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.738 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.739 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.740 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.741 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.742 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:54:48.743 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:54:48.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-19T07:54:48.745 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-19T07:54:48.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.749 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.750 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.751 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.752 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.753 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.754 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.755 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.756 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.758 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.759 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.760 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:54:48.761 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.762 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.763 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.764 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.765 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:54:48.766 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.767 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.768 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.769 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.770 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.771 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.772 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.773 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-19T07:54:48.775 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-19T07:54:48.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-19T07:54:48.777 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.778 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.780 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.781 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.785 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.786 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.787 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.788 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.789 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.790 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-19T07:54:48.791 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.793 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.794 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-19T07:54:48.795 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-19T07:54:48.797 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.798 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.799 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:54:48.800 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.801 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.802 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.803 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.804 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.805 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.806 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.807 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.808 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.809 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-19T07:54:48.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-19T07:54:48.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.813 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.814 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.815 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.816 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.817 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.818 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.819 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.820 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.822 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.823 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.824 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.825 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.826 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.828 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-19T07:54:48.833 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.834 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.835 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.836 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.837 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.838 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.839 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.840 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-19T07:54:48.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:54:48.842 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.843 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-19T07:54:48.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-19T07:54:48.843 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.843 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.844 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.845 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.846 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.847 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.848 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.849 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.850 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.851 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.852 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.853 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.855 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.857 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.858 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.860 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.861 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.862 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.863 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.864 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.865 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.866 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.867 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.868 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.871 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.872 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.873 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.874 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.875 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.876 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.877 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.879 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.880 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.881 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.882 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.883 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.884 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.885 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.886 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.887 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.888 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.889 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.890 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.891 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.892 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.893 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-19T07:54:48.894 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.895 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.896 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.897 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.898 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.899 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.900 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.901 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.902 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.903 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.904 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.905 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.906 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:54:48.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.908 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.911 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.912 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-19T07:54:48.914 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.915 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.917 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-19T07:54:48.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:54:48.919 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.920 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.921 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.922 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.923 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.924 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.925 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.926 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-19T07:54:48.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.928 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.929 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.930 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.931 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.932 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.933 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.935 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.936 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:48.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.939 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.940 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.941 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.942 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.943 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.944 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.945 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.946 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.948 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.950 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.951 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.952 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-19T07:54:48.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.954 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.955 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.958 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.959 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.960 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.961 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-19T07:54:48.962 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-19T07:54:48.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-19T07:54:48.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.965 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.966 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.967 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.968 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.969 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.970 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.972 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-19T07:54:48.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-19T07:54:48.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.976 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.977 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.978 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.979 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.980 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.981 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.982 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.983 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.984 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.985 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.986 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.987 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.988 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.989 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.991 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.992 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:48.993 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.995 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.996 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:48.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:48.998 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:48.999 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-19T07:54:49.001 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:54:49.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.003 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.004 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.005 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.006 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.008 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:54:49.009 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.010 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.011 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.012 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.013 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.014 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.016 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.017 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.018 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.019 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.021 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.022 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.023 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.024 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.025 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.026 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.027 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.028 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.029 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.031 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.032 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.033 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.034 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.035 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:54:49.036 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.037 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.038 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.039 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.040 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.042 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.043 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.044 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.046 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.047 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.048 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.049 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.050 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.052 INFO:teuthology.orchestra.run.smithi103.stdout: "modules": [ 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard", 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "iostat", 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "nfs", 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "restful", 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "snap_schedule" 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.053 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.055 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.056 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.057 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.058 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.059 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.060 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.061 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.064 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.066 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.068 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.069 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.070 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:54:49.071 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:49.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.073 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-19T07:54:49.074 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-19T07:54:49.075 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.078 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.080 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.081 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-19T07:54:49.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:54:49.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.085 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:49.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-19T07:54:49.089 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T07:54:49.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.091 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.092 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.094 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-19T07:54:49.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.098 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.099 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.100 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.101 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-19T07:54:49.103 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.105 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.107 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T07:54:49.108 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.110 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-19T07:54:49.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:49.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.124 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-19T07:54:49.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-19T07:54:49.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-19T07:54:49.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.130 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.132 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T07:54:49.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-19T07:54:49.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T07:54:49.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-19T07:54:49.140 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.141 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-19T07:54:49.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.144 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-19T07:54:49.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-19T07:54:49.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.149 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.150 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.154 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T07:54:49.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.157 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.158 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-19T07:54:49.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.163 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.167 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.168 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T07:54:49.169 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T07:54:49.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T07:54:49.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.172 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.176 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.180 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.181 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T07:54:49.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T07:54:49.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.192 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.192 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.192 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.192 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T07:54:49.200 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T07:54:49.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T07:54:49.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T07:54:49.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.204 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.205 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.210 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.212 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.214 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.220 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-19T07:54:49.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.223 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-19T07:54:49.229 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:54:49.230 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.232 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.233 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.234 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.237 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-19T07:54:49.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.246 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.247 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.250 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.251 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-19T07:54:49.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.263 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.265 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-19T07:54:49.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.268 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.269 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.271 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.274 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.275 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.277 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.278 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.280 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.281 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.283 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.284 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T07:54:49.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.286 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.289 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-19T07:54:49.291 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-19T07:54:49.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.293 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.295 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.296 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.300 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-19T07:54:49.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.303 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.310 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.313 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.315 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.316 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.318 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.319 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.321 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.323 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.326 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.327 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.328 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.329 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.333 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-19T07:54:49.335 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.336 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.339 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.341 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.342 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.343 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-19T07:54:49.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.349 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.350 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.351 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.354 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.355 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.356 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.358 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.360 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-19T07:54:49.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.363 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.364 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.372 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.373 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-19T07:54:49.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-19T07:54:49.375 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.377 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.380 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.382 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-19T07:54:49.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.385 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.387 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.388 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.389 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.390 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.391 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.393 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.394 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.395 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.395 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.396 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-19T07:54:49.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-19T07:54:49.396 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.396 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.397 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.398 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.399 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.401 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.403 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.407 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.415 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.416 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.419 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.421 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.422 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.424 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.425 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-19T07:54:49.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-19T07:54:49.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.430 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-19T07:54:49.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.434 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.435 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.438 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.439 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.440 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.441 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.444 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.446 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.447 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.448 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.450 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.452 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.453 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.455 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.460 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.461 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.462 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.464 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-19T07:54:49.465 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-19T07:54:49.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-19T07:54:49.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.469 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.470 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.472 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-19T07:54:49.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.474 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-19T07:54:49.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.478 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.479 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.480 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.482 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.483 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.484 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.494 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.495 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.497 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.499 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.500 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-19T07:54:49.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T07:54:49.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-19T07:54:49.506 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.508 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.509 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-19T07:54:49.510 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-19T07:54:49.511 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.512 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.513 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.514 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-19T07:54:49.515 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "services": { 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard": "https://172.21.15.103:8443/" 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "always_on_modules": { 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "octopus": [ 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:54:49.516 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "pacific": [ 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:54:49.517 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "quincy": [ 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:54:49.518 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "reef": [ 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:54:49.519 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "squid": [ 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: "last_failure_osd_epoch": 85, 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: "active_clients": [ 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 7986338 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.521 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "libcephsqlite", 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2356779709 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.522 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2935992612 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2371717192 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.523 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 992898363 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.524 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2474142086 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-19T07:54:49.525 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:49.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:48 smithi134 ceph-mon[35414]: pgmap v527: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2930529993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3087745869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2869985474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:48 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3146005200' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:48 smithi174 ceph-mon[35370]: pgmap v527: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2930529993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3087745869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2869985474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:48 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3146005200' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:54:49.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb5780a33c0 msgr2=0x7fb5780a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb5780a33c0 0x7fb5780a5880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb580005d90 tx=0x7fb580005d20 comp rx=0 tx=0).stop 2024-09-19T07:54:49.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590102440 msgr2=0x7fb59019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:54:49.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590102440 0x7fb59019ab40 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb584009a50 tx=0x7fb584004270 comp rx=0 tx=0).stop 2024-09-19T07:54:49.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 shutdown_connections 2024-09-19T07:54:49.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fb5780a33c0 0x7fb5780a5880 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590107ab0 0x7fb59019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb590102440 0x7fb59019ab40 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 --2- 172.21.15.103:0/3146005200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590101aa0 0x7fb59019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:54:49.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 >> 172.21.15.103:0/3146005200 conn(0x7fb5900fd7c0 msgr2=0x7fb59010d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:54:49.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 shutdown_connections 2024-09-19T07:54:49.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:54:48.100+0000 7fb595ed7640 1 -- 172.21.15.103:0/3146005200 wait complete. 2024-09-19T07:54:49.598 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T07:54:49.598 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T07:54:49.600 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:48 smithi103 ceph-mon[31201]: pgmap v527: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-19T07:54:49.600 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2930529993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.600 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3087745869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.601 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2869985474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T07:54:49.601 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:48 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3146005200' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T07:54:49.859 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:50.237 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:50 smithi103 ceph-mon[31201]: pgmap v528: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-09-19T07:54:50.342 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:54:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2423, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2425, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 4355218, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 770515, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.229162549, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.067553012, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.061098298, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.082386865, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2423, 2024-09-19T07:54:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.108698275, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000044861 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 6, 2024-09-19T07:54:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 236, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1870, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 664590, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 10, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.017542598, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.012100856, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002867684, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000069485, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 233, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.021841534, 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000093740 2024-09-19T07:54:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 663568 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1394, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1350, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2105384, 2024-09-19T07:54:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 515818, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.148648156, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.045030011, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.070465926, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.039820618, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1350, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.069002410, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000051112 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1854688, 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 191776 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-19T07:54:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5641, 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.155277331, 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000027526 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-19T07:54:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 276, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 276, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 276, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 4, 2024-09-19T07:54:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-19T07:54:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:54:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-19T07:54:50.354 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-19T07:54:50.355 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.356 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 1591, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 3, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-19T07:54:50.357 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 7836299, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 7835725, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 1591 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 128, 2024-09-19T07:54:50.358 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 465, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 337, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 13, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 27, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 14, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 276, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 695, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 419, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 50840, 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-19T07:54:50.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-19T07:54:50.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-19T07:54:50.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.364 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-19T07:54:50.366 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-19T07:54:50.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.369 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.370 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 4, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 2, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 3, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 2557, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:54:50.373 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:54:50.374 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:54:50.375 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 203696, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 204, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:54:50.376 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8224, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 181, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:54:50.377 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 559598, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 10165, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 3445, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-19T07:54:50.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 3445, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3445, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 5.936885178, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001723333 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3445, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3446 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 3445, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 3445, 2024-09-19T07:54:50.379 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2123, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1320, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-19T07:54:50.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-19T07:54:50.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-19T07:54:50.382 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 105, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 21, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 6, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 9, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 3, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-19T07:54:50.383 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-19T07:54:50.384 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 3819, 2024-09-19T07:54:50.385 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 6096841, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 3819, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3819, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 6096841, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.386 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3445, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3642000, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3445, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3642000, 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:50.388 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3445, 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3445, 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3445, 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3445, 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:54:50.389 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.390 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x564c15b1cb40": { 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:50.391 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:50.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:50 smithi174 ceph-mon[35370]: pgmap v528: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-09-19T07:54:50.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:50 smithi134 ceph-mon[35414]: pgmap v528: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-09-19T07:54:51.306 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-19T07:54:51.306 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-19T07:54:51.307 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-19T07:54:51.307 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:54:51.307 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:54:51.307 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T07:54:51.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T07:54:51.552 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:52.052 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:54:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-19T07:54:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-19T07:54:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-19T07:54:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-19T07:54:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-19T07:54:52.053 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-19T07:54:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.001110084, 2024-09-19T07:54:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T07:54:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000201825, 2024-09-19T07:54:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:54:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:52.056 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:54:52.057 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 7, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 239, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3472, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 278128, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.019834548, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.012833747, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.004401064, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 237, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.025367776, 2024-09-19T07:54:52.058 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000107037 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 278128 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 319, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 308, 2024-09-19T07:54:52.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1614384, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 190000, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.055708962, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.021129492, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.047209236, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 307, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.015819517, 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000051529 2024-09-19T07:54:52.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1614384, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 190000 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:54:52.061 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:54:52.062 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:54:52.063 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148731, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 31, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:54:52.064 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:54:52.065 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:52.066 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 322, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1569349, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 322, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 322, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1569349, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:52.067 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:52.068 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:52.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:52 smithi103 ceph-mon[31201]: pgmap v529: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-19T07:54:52.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:52 smithi174 ceph-mon[35370]: pgmap v529: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-19T07:54:52.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:52 smithi134 ceph-mon[35414]: pgmap v529: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-19T07:54:52.745 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T07:54:52.745 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T07:54:52.745 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T07:54:52.746 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T07:54:52.746 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T07:54:52.746 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T07:54:52.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T07:54:52.995 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:53.653 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:53.653 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:54:53.653 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1485, 2024-09-19T07:54:53.653 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1441, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2227520, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 545400, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.191610509, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.077066196, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.082457455, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.038099417, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1441, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.092084674, 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000063903 2024-09-19T07:54:53.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1855232, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 193056 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2315, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2546, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3277584, 2024-09-19T07:54:53.655 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1311047, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.176181307, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.054060343, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.044898376, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.063845401, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2543, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.106394877, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000041838 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:53.656 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 575456 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 6, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1629, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1284, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-09-19T07:54:53.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.002116742, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000062326, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001737392, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000086674, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002379194, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000594798 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5734, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.145569336, 2024-09-19T07:54:53.659 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000025387 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-19T07:54:53.660 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-19T07:54:53.661 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 638, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 46, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 592, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 573, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 20, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-19T07:54:53.662 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-19T07:54:53.663 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:54:53.664 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-19T07:54:53.665 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:54:53.666 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 1115, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-19T07:54:53.667 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 3, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 6884778, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 6884204, 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-19T07:54:53.668 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 1115 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 410, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 428, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 18, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 30, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 31, 2024-09-19T07:54:53.669 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 1, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 638, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 638, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 51680, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-19T07:54:53.670 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-19T07:54:53.671 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.673 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.674 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-19T07:54:53.675 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.677 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.678 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-19T07:54:53.679 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.681 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-19T07:54:53.682 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 3, 2024-09-19T07:54:53.683 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 2, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 2047, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:54:53.684 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:54:53.685 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:54:53.686 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 216313, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 545, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:54:53.687 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8224, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 181, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 1543597, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 35697, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:54:53.688 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 3425, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 3425, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3425, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 6.380876678, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001863029 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.689 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3425, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3426 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 3425, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 3425, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2129, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1294, 2024-09-19T07:54:53.690 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-19T07:54:53.691 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-19T07:54:53.692 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 105, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 21, 2024-09-19T07:54:53.693 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 6, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 8, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 2, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-19T07:54:53.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:54:53.695 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 3800, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 5142751, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 3800, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3800, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 5142751, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.696 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3425, 2024-09-19T07:54:53.697 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2690479, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3425, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2690479, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:53.698 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3425, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3425, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3425, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3425, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.699 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.700 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55c6f1c50b40": { 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:53.701 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:53.702 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:54.366 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:54 smithi103 ceph-mon[31201]: pgmap v530: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-19T07:54:54.396 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-19T07:54:54.396 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-19T07:54:54.396 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-19T07:54:54.397 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:54:54.397 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:54:54.397 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T07:54:54.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T07:54:54.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:54 smithi174 ceph-mon[35370]: pgmap v530: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-19T07:54:54.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:54 smithi134 ceph-mon[35414]: pgmap v530: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-19T07:54:54.650 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:54:55.152 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 321, 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 310, 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1614928, 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 191280, 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.054227074, 2024-09-19T07:54:55.153 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.020389507, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.046017666, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 309, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.017391981, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000056284 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:55.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1614928, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 191280 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 238, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 480, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 277856, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.017927392, 2024-09-19T07:54:55.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.013070509, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002348735, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 235, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.020759617, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000088338 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 277856 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2992, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 448, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.002540118, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000035908, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002470602, 2024-09-19T07:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.004517353, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001505784 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:54:55.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:54:55.159 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:54:55.160 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-19T07:54:55.161 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148714, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:54:55.162 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-19T07:54:55.163 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 324, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1569609, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 324, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-19T07:54:55.164 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 324, 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1569609, 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-19T07:54:55.165 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-19T07:54:55.906 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T07:54:55.906 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T07:54:55.906 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T07:54:55.906 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T07:54:55.906 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T07:54:55.906 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T07:54:55.906 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T07:54:56.167 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:54:56.287 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:56 smithi134 ceph-mon[35414]: pgmap v531: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-19T07:54:56.288 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:56 smithi134 ceph-mon[35414]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:56 smithi174 ceph-mon[35370]: pgmap v531: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-19T07:54:56.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:56 smithi174 ceph-mon[35370]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:56 smithi103 ceph-mon[31201]: pgmap v531: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-19T07:54:56.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:56 smithi103 ceph-mon[31201]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-19T07:54:56.735 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:54:56.735 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 11160, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 11453, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4230716, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 17688838, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 28, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.813093109, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.298449745, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.378686646, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.152033733, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11452, 2024-09-19T07:54:56.736 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.003817104, 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000087654 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12009, 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.005379478, 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000447 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 513328, 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 159504 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:54:56.737 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 8038, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 8721, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4300606, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 15800302, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 16, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.532109634, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.194610200, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.255428662, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.101793767, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8720, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.689808264, 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000193785 2024-09-19T07:54:56.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5817, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001368661, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000235 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1901040, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 226528 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 12675, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 15646, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4536010, 2024-09-19T07:54:56.739 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 15097497, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 214, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 8, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.975255454, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.358542676, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.619523537, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.118727798, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 15643, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.893497486, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000057118 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:56.740 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 21439, 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.003266953, 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000152 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2748112, 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 5701760 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:54:56.741 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10373, 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.187524186, 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000403694 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.742 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 207, 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.011224998, 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000054227 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 2745, 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 2465, 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2465, 2024-09-19T07:54:56.743 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6.818387211, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002766080 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 280, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 1, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 39, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 3, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 530, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 9, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 391, 2024-09-19T07:54:56.744 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 130, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 517, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 1149, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 250, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 250, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 52, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 3680, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 2820, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 261, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 7, 2024-09-19T07:54:56.745 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 2, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 62, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 425, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 26, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 1118, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 6, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 272, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 8561, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 8, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 1013, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 1481881286, 2024-09-19T07:54:56.746 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 8561, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 600, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 20, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 580, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 7961, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 45, 2024-09-19T07:54:56.747 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 1519, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 2286, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 3094, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 1254, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 67, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 289 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:54:56.748 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 12, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 56, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 4, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 86, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 51, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 35, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 2, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:54:56.749 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 6209, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 20, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 26, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 3, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 4596, 2024-09-19T07:54:56.750 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 4596, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 10, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 10, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:54:56.751 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 33113290, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2622, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 21.030562324, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.008020809 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.752 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 430, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 1593, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 1163, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 95, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 109, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 14, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 531, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 1987, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 1456, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 250, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 1674, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 1424, 2024-09-19T07:54:56.753 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 71092, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 231712 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 3117, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 1055, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 2745, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 291, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 2121, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1059, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.928754989, 2024-09-19T07:54:56.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004654159 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 178, 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.099395386, 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000558401 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 34, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.084992181, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002499770 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 401, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.361499246, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000901494 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.756 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000133969, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000133969 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.757 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 16, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002725524, 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000170345 2024-09-19T07:54:56.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 37, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.270316863, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007305861 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 16, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.144773018, 2024-09-19T07:54:56.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.009048313 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 500, 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.179705058, 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000359410 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 64, 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.086075133, 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001344923 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.760 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 67, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.444081805, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.006628086 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.762 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:54:56.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-09-19T07:54:56.762 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.008174167, 2024-09-19T07:54:56.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004087083 2024-09-19T07:54:56.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.762 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:54:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10, 2024-09-19T07:54:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.046318210, 2024-09-19T07:54:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004631821 2024-09-19T07:54:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:54:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.764 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000748818, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000748818 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.765 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.019744681, 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002820668 2024-09-19T07:54:56.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 72, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.140948163, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001957613 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:54:56.767 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 4, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 147, 2024-09-19T07:54:56.768 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 73, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 985, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:54:56.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:54:56.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:54:56.771 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 561563, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 517, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8224, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 181, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:54:56.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 1878887, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 26488, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 12200, 2024-09-19T07:54:56.773 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 29408955, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 37, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 12200, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12200, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 44.446954231, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003643192 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12200, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 12459 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 12200, 2024-09-19T07:54:56.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 8626, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 3574, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 11, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 44, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 2646, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 91, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 3, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 583, 2024-09-19T07:54:56.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 8581, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 22, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:54:56.776 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:54:56.777 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 105, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 23, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 386, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-19T07:54:56.778 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 90 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 6223 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:54:56.779 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 13, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 6, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 65, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 51, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 31869, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 9763140, 2024-09-19T07:54:56.780 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 31869, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 31869, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 9763140, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:56.781 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 12200, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 29414650, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 12200, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 29414650, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.782 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 12200, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 12200, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 12200, 2024-09-19T07:54:56.783 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 12200, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 51, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 5711, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:56.784 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 51, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 24, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5711, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55d47be90960": { 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:56.785 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 4596, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 28918986, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 4596, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2622, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 28918986, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:56.786 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.787 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:56.787 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:54:57.695 INFO:tasks.check_counter:Daemon mds.b mds.exported=26 2024-09-19T07:54:57.695 INFO:tasks.check_counter:Daemon mds.b mds.imported=6 2024-09-19T07:54:57.695 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T07:54:57.695 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-09-19T07:54:57.696 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-09-19T07:54:57.696 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T07:54:57.696 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T07:54:57.946 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:54:58.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:54:58 smithi134 ceph-mon[35414]: pgmap v532: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T07:54:58.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:54:58 smithi174 ceph-mon[35370]: pgmap v532: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T07:54:58.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:54:58 smithi103 ceph-mon[31201]: pgmap v532: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1660, 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1662, 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 8902441, 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 527890, 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.155575261, 2024-09-19T07:54:58.530 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.055880262, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.039577835, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.046387384, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1660, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.085301898, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000051386 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.531 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1376, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1610, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 12569536, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 1014598, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-09-19T07:54:58.532 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.116352180, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.031484270, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.040525186, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.035049885, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1607, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.072549524, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000045145 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 577600 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1125, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1080, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 10439707, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 431733, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.115439220, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.035360701, 2024-09-19T07:54:58.534 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.073023083, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.018779847, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1080, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.044272485, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000040993 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1863232, 2024-09-19T07:54:58.535 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 195776 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6355, 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.143799331, 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000022627 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-19T07:54:58.537 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:54:58.538 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 568, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 3, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 565, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 564, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 50, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-19T07:54:58.539 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-19T07:54:58.540 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 1013, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 1481881286, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:54:58.541 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:54:58.542 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:54:58.543 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:54:58.544 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 4597, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 10, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-19T07:54:58.545 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 33193436, 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 33113290, 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 4597 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.546 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 419, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 1567, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 1148, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 88, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 102, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 14, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 568, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 1683, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 1115, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-19T07:54:58.547 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 56388, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215328 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.549 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.550 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:54:58.552 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.554 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.555 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.556 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:54:58.557 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.558 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:54:58.559 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 4, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 3, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:54:58.560 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 3017, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:54:58.561 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:54:58.562 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:54:58.563 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 219643, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 635, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:54:58.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8224, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 181, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 1709226, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 21587, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.565 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 3776, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 26, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 4, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 3776, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3776, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 5.838866729, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001546310 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:54:58.566 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3776, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3777 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 3776, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 3776, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 2221, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 1552, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-09-19T07:54:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:54:58.568 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:54:58.569 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:54:58.570 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 105, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 21, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 6, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 6, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-09-19T07:54:58.571 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-19T07:54:58.572 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 4157, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 31515269, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 4157, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 4157, 2024-09-19T07:54:58.573 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 31515269, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:54:58.574 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 3776, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 28999142, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3776, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 28999142, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.575 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 3776, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 3776, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3776, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 3776, 2024-09-19T07:54:58.576 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:54:58.577 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x56475d37eb40": { 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:54:58.578 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:54:58.579 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:54:58.580 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:54:58.580 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.580 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:54:58.580 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:54:59.415 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-19T07:54:59.416 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-19T07:54:59.416 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-19T07:54:59.416 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:54:59.416 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:54:59.416 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T07:54:59.416 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T07:54:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:55:00.203 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:55:00.203 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:55:00.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 5018, 2024-09-19T07:55:00.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 4390, 2024-09-19T07:55:00.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3829108, 2024-09-19T07:55:00.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 4084176, 2024-09-19T07:55:00.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 15, 2024-09-19T07:55:00.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-09-19T07:55:00.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.397081218, 2024-09-19T07:55:00.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.163930904, 2024-09-19T07:55:00.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.215012155, 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.047902761, 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4392, 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.256281819, 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000058351 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:00.205 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4719, 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000897009, 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000190 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1893360, 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 674432 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:55:00.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2782, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2754, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1310075, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 1497502, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 14, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.165486024, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.051757430, 2024-09-19T07:55:00.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.070082337, 2024-09-19T07:55:00.208 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.042045437, 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2753, 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.131051277, 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000047603 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1980, 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000281014, 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000141 2024-09-19T07:55:00.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2368, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 134336 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 4063, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 4757, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1224927, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 2874114, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 53, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.282689026, 2024-09-19T07:55:00.210 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.113888376, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.147187534, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.039944213, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4751, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.244032013, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000051364 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4445, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000772655, 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000173 2024-09-19T07:55:00.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 812032, 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 1819984 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:55:00.212 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4205, 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.083544653, 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000257680 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 17, 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000336075, 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000019769 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 1417, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 1376, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1376, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 21.093562899, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.015329624 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 41, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-19T07:55:00.214 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 27, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 2, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 507, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 93, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 414, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-19T07:55:00.215 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 375, 2024-09-19T07:55:00.216 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 24, 2024-09-19T07:55:00.216 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 239, 2024-09-19T07:55:00.216 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 239, 2024-09-19T07:55:00.216 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 31, 2024-09-19T07:55:00.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 2075, 2024-09-19T07:55:00.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 1951, 2024-09-19T07:55:00.216 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 40, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 27, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 13, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 251, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 6, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 257, 2024-09-19T07:55:00.217 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 4203, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 5, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 988, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 1045673706, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 4203, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:55:00.218 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 4203, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 566, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 739, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 956, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 160, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 26, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 43 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.219 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 38, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 15, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 2, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 7, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 2, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 11, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 3, 2024-09-19T07:55:00.220 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 3254, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 2, 2024-09-19T07:55:00.221 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 1360, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 1360, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 3, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-19T07:55:00.222 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 3, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 5876818, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 614, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 5.449031902, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.008874644 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.223 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 474, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 476, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 2, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 54, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 57, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 3, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 509, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 543, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 34, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 239, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 535, 2024-09-19T07:55:00.224 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 296, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 63056, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223520 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 1754, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 81, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 1417, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 43, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 292, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.004612975, 2024-09-19T07:55:00.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004612975 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 962, 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 19.955977218, 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.020744259 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:55:00.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 12, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.252918490, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.021076540 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 209, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.081737705, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000391089 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000099516, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000099516 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:55:00.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.230 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.230 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.231 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 139, 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.087368392, 2024-09-19T07:55:00.232 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000628549 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 23, 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.014798000, 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000643391 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 22, 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.678153322, 2024-09-19T07:55:00.233 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.030825151 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:55:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.235 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:55:00.236 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.017897281, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002556754 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:55:00.238 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 5, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:55:00.239 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 131, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 65, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 977, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:55:00.240 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:55:00.241 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:55:00.242 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 467201, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 451, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:55:00.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8224, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 181, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 1686180, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 22843, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:55:00.244 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 5013, 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 1867307, 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 29, 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 5013, 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:55:00.245 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5013, 2024-09-19T07:55:00.246 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 9.972771272, 2024-09-19T07:55:00.248 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001989381 2024-09-19T07:55:00.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.248 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:55:00.248 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:55:00.248 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5013, 2024-09-19T07:55:00.248 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 5131 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 5013, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 4204, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 809, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 1, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 615, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 63, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:55:00.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 16, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 4203, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:55:00.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:55:00.251 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:55:00.252 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 105, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 23, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 168, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-09-19T07:55:00.253 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 35 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 3268 2024-09-19T07:55:00.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 2, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 2, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 2, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 2, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:55:00.255 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 11859, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 5160890, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 11859, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 11859, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5160890, 2024-09-19T07:55:00.256 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:55:00.257 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 5013, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1867589, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 5013, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1867589, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 5013, 2024-09-19T07:55:00.259 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 5013, 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 5013, 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 5013, 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:00.260 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 282, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 282, 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:00.261 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x555c2ed96960": { 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 9758, 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1360, 2024-09-19T07:55:00.262 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1682514, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1360, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 614, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1672756, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:00.263 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.264 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:00.264 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:55:00.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:00 smithi134 ceph-mon[35414]: pgmap v533: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-19T07:55:00.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:00 smithi174 ceph-mon[35370]: pgmap v533: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-19T07:55:00.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:00 smithi103 ceph-mon[31201]: pgmap v533: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-19T07:55:01.026 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T07:55:01.026 INFO:tasks.check_counter:Daemon mds.h mds.imported=6 2024-09-19T07:55:01.026 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T07:55:01.026 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:55:01.027 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:55:01.027 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T07:55:01.027 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T07:55:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1300, 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1300, 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2012471, 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 412639, 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-09-19T07:55:01.871 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.107301445, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.039505453, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.028793458, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.028721739, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1299, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.071159761, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000054780 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.872 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 383, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 340, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1861998, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 197730, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.055131966, 2024-09-19T07:55:01.873 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.019822178, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.052986221, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000046143, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 339, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.019902576, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000058709 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1860896, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 197088 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2141, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2378, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 626684, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 1259946, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.204222659, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.064363896, 2024-09-19T07:55:01.875 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.049110647, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.071394104, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2375, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.117814362, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000049606 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.876 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 579744 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5730, 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.145025280, 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000025309 2024-09-19T07:55:01.877 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-09-19T07:55:01.878 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 490, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 9, 2024-09-19T07:55:01.879 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 481, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 478, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 22, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:55:01.880 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:55:01.881 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-09-19T07:55:01.882 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-09-19T07:55:01.883 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:55:01.884 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-09-19T07:55:01.885 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 1343, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 3, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-09-19T07:55:01.886 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 5867060, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 5864764, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 1343 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 453, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 458, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 5, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 31, 2024-09-19T07:55:01.887 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 33, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 2, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 490, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 490, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 53676, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215328 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-09-19T07:55:01.888 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.890 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-09-19T07:55:01.891 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-09-19T07:55:01.893 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.895 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.896 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.897 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.899 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.900 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-09-19T07:55:01.901 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 5, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 4, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 3690, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-09-19T07:55:01.902 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:55:01.903 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:55:01.904 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 216017, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 536, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-09-19T07:55:01.905 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8224, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 181, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 1470229, 2024-09-19T07:55:01.906 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 19700, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 3437, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-09-19T07:55:01.907 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 3437, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3437, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 9.823776241, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002858241 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3437, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3438 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 3437, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 3437, 2024-09-19T07:55:01.908 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 2139, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 1296, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-09-19T07:55:01.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:55:01.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-09-19T07:55:01.911 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 105, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 21, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 6, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 9, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 3, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-09-19T07:55:01.912 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-09-19T07:55:01.913 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 3820, 2024-09-19T07:55:01.914 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 4134958, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 3820, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3820, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 4134958, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-09-19T07:55:01.915 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 3437, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1672761, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3437, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1672761, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.916 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 3437, 2024-09-19T07:55:01.917 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 3437, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3437, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 3437, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:01.918 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-19T07:55:01.919 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55e046b54b40": { 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-09-19T07:55:01.920 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-09-19T07:55:01.921 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-09-19T07:55:01.921 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-09-19T07:55:01.921 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.921 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-19T07:55:01.921 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-19T07:55:02.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:02 smithi174 ceph-mon[35370]: pgmap v534: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T07:55:02.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:02 smithi103 ceph-mon[31201]: pgmap v534: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T07:55:02.552 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:02 smithi134 ceph-mon[35414]: pgmap v534: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-19T07:55:02.579 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T07:55:02.579 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T07:55:02.579 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T07:55:02.579 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:55:02.579 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:55:02.579 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T07:55:02.579 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T07:55:02.839 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:55:03.348 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:55:03.348 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4496, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 5373, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1537388, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 6784308, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 54, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.328168518, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.127201320, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.175406082, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.039265570, 2024-09-19T07:55:03.349 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5369, 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.249098089, 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000046395 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5847, 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000923516, 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000157 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 857920, 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 1797056 2024-09-19T07:55:03.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4329, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 4075, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 3664257, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 2430816, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 9, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.327150279, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.130711335, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.212109629, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.023708553, 2024-09-19T07:55:03.351 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4077, 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.213253973, 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000052306 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5982, 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001125559, 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000188 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1888432, 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 676576 2024-09-19T07:55:03.352 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3463, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3493, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1403549, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1818221, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 18, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.208436230, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.089562849, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.124487162, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.013822098, 2024-09-19T07:55:03.353 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3490, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.111126599, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000031841 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5632, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000971595, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000172 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2368, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 133280 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.354 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3055, 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.852973226, 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000279205 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.355 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000011685, 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000005842 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 1204, 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 1135, 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1135, 2024-09-19T07:55:03.356 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 12.461512049, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.010979305 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 69, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 7, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 20, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 1, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 295, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 14, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 124, 2024-09-19T07:55:03.357 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 157, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 274, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 510, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 56, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 56, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 14, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 3348, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 2586, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 61, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 67, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 151, 2024-09-19T07:55:03.358 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 1098, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 1, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 11, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 379, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 6, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 300, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 1473, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 988, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 1045673706, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-19T07:55:03.359 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 1473, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 14, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 14, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 1459, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 6, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 535, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 860, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 2, 2024-09-19T07:55:03.360 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 1094, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 391, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 49, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 419 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 67, 2024-09-19T07:55:03.361 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 49, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 5, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 115, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 90, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 26, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:55:03.362 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 1547, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 11, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 1, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 1592, 2024-09-19T07:55:03.363 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 1592, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 3, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 3, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T07:55:03.364 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 7836873, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 947, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.950856436, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003116004 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 165, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 509, 2024-09-19T07:55:03.365 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 344, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 38, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 42, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 4, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 298, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 824, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 526, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 56, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 257, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 201, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 62600, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:55:03.366 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 2415, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 653, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 1204, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 244, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 1023, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 30, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.040298271, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001343275 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 342, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 9.169482722, 2024-09-19T07:55:03.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.026811352 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 100, 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.333523690, 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003335236 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:55:03.368 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 367, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.309523858, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000843389 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.369 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.370 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.371 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.371 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:55:03.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.371 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.372 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 73, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.093100365, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001275347 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.018124940, 2024-09-19T07:55:03.373 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001812494 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 98, 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.180379259, 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.022248767 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:55:03.374 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.375 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.011539819, 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.005769909 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:55:03.376 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 113, 2024-09-19T07:55:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.305539125, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002703886 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-19T07:55:03.379 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 4, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 2, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 2, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 110, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 55, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 974, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:55:03.380 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:55:03.381 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:55:03.382 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 650359, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 359, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:55:03.383 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8224, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 181, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 807706, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 21326, 2024-09-19T07:55:03.384 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 2651, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 3865432, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 30, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 2651, 2024-09-19T07:55:03.385 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2651, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.880905821, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002595588 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2651, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2829 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 2651, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 1480, 2024-09-19T07:55:03.386 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 1171, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 947, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 57, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 1, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-09-19T07:55:03.387 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1473, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:55:03.388 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 2, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:55:03.389 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 105, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 22, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:55:03.390 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 244, 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 14, 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 55 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 1557 2024-09-19T07:55:03.391 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:03.392 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 12284, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 5354211, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 12284, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 12284, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 5354211, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.393 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2651, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3865320, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2651, 2024-09-19T07:55:03.394 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3865320, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:03.395 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2651, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 2651, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2651, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2651, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:55:03.396 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.397 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x5608badbe960": { 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 1592, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 3642569, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 1592, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:03.398 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 947, 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3642569, 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:03.399 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:55:04.178 INFO:tasks.check_counter:Daemon mds.c mds.exported=11 2024-09-19T07:55:04.178 INFO:tasks.check_counter:Daemon mds.c mds.imported=6 2024-09-19T07:55:04.178 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-19T07:55:04.178 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:55:04.178 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:55:04.178 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T07:55:04.179 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T07:55:04.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:04 smithi174 ceph-mon[35370]: pgmap v535: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-19T07:55:04.430 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:55:04.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:04 smithi103 ceph-mon[31201]: pgmap v535: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-19T07:55:04.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:04 smithi134 ceph-mon[35414]: pgmap v535: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-09-19T07:55:04.985 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:55:04.985 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:55:04.985 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 5330, 2024-09-19T07:55:04.985 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 5589, 2024-09-19T07:55:04.985 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1841100, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 5213343, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 53, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 8, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.343648573, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.128489855, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.147847592, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.067622892, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5587, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.188566526, 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000033750 2024-09-19T07:55:04.986 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4704, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000999667, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000212 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 5468, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 4988, 2024-09-19T07:55:04.987 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2593285, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 2422398, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 11, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.330109070, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.121133817, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.200515784, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.037216879, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4988, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.224155928, 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000044939 2024-09-19T07:55:04.988 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7649, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001390056, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000181 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 124192 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2898, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3439, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2635892, 2024-09-19T07:55:04.989 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 2288547, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 15, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.247574547, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.105202790, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.141678728, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.025614147, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3436, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.144774278, 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000042134 2024-09-19T07:55:04.990 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2780, 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000559869, 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000201 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2370208, 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 1784144 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:55:04.991 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4739, 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.919726905, 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000194076 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-19T07:55:04.992 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000013788, 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000006894 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 592, 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 564, 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 564, 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 20.398235066, 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.036167083 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.993 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 28, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 2, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 20, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 1, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 642, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 72, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 570, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 539, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 66, 2024-09-19T07:55:04.994 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 276, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 276, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 25, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 2174, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 1785, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 13, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 59, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 72, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 226, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T07:55:04.995 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 11, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 384, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 4398, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 1, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 988, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 1045673706, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 4398, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:55:04.996 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 2, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 2, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 4396, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 10, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 373, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 521, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 691, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 73, 2024-09-19T07:55:04.997 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 58, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 198 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 68, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 81, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 2, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 64, 2024-09-19T07:55:04.998 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 56, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 18, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 3392, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:55:04.999 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 1, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 1117, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 1117, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:55:05.000 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 3, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 3, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 6888066, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 638, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.977983540, 2024-09-19T07:55:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004667685 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 451, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 454, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 3, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 49, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 51, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 2, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 649, 2024-09-19T07:55:05.002 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 751, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 102, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 276, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 510, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 234, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 61660, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 1658, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 462, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 592, 2024-09-19T07:55:05.003 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 113, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 977, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 36, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.014432973, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000400915 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.004 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 60, 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.386025772, 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.006433762 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:55:05.005 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 192, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.202859607, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001056560 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.006 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:55:05.007 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.008 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 112, 2024-09-19T07:55:05.009 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.090859891, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000811249 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 31, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.017022759, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000549121 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 80, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 19.521425947, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.244017824 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.010 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.011 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.012 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:55:05.013 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.014 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 53, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.165608117, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003124681 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.015 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 3, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 2, 2024-09-19T07:55:05.016 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 194, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 97, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 939, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:55:05.017 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:55:05.018 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:55:05.019 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 776945, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 661, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:55:05.020 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8224, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 181, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 1872015, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 47589, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.021 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 5244, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 2878773, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 28, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 5244, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5244, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 10.528857982, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002007791 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:55:05.022 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5244, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 5395 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 5244, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 4400, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 844, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 638, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 59, 2024-09-19T07:55:05.023 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 1, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 4398, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:55:05.024 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:55:05.025 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 105, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 21, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-19T07:55:05.026 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 209, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 4, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 50 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:55:05.027 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 3406 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:05.028 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 13692, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 5745781, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 13692, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 13692, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 5745781, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.029 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:05.030 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 5244, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 2878741, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5244, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2878741, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.031 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 5244, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 5244, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5244, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 5244, 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:05.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:05.033 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55b5b392a960": { 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:05.034 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 1117, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2693762, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 1117, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 638, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2693762, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:05.035 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:05.036 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:05.036 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:55:05.778 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-19T07:55:05.779 INFO:tasks.check_counter:Daemon mds.f mds.imported=11 2024-09-19T07:55:05.779 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-19T07:55:05.779 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:55:05.779 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:55:05.779 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T07:55:05.779 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T07:55:06.025 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:55:06.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:06 smithi174 ceph-mon[35370]: pgmap v536: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-19T07:55:06.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:06 smithi103 ceph-mon[31201]: pgmap v536: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-19T07:55:06.545 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1470, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1534, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 570415, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 5356397, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 56, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 9, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.136423836, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.047940028, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.051158823, 2024-09-19T07:55:06.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.028147102, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1529, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.122949788, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000080411 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 683, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000180199, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000263 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 546656 2024-09-19T07:55:06.547 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 6086, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 6677, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2694896, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 3308751, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 13, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.402766856, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.162391207, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.272252357, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.018520456, 2024-09-19T07:55:06.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6677, 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.381775299, 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000057177 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 11035, 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001817411, 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000164 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 677232, 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 1769248 2024-09-19T07:55:06.549 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1819, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1540, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2470736, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 856569, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 11, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.160510397, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.059323250, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.097235307, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.020943990, 2024-09-19T07:55:06.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1540, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.095726387, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000062159 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 898, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000186797, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000208 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1897440, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 222944 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.551 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1721, 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.593085133, 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000344616 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 670, 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.023191485, 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000034614 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 939, 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 803, 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 803, 2024-09-19T07:55:06.553 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 50.105803711, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.062398261 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 136, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 395, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-19T07:55:06.554 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 367, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 28, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 395, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 189, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 248, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 248, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 25, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 2234, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 1673, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 132, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 15, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 87, 2024-09-19T07:55:06.555 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 211, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 3, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 170, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 11, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 454, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 1233, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 988, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 1045673706, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 1233, 2024-09-19T07:55:06.556 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 75, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 75, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 1158, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 247, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 760, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:55:06.557 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 716, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 292, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 155, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 870 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 16, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:55:06.558 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 210, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 203, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 7, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 14, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 1167, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:55:06.559 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 3, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 1308, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 1308, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:55:06.560 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 3, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 3, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 7868731, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 365, 2024-09-19T07:55:06.561 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.866935425, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.005114891 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 390, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 747, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 357, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 48, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 58, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 10, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 395, 2024-09-19T07:55:06.562 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 787, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 392, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 248, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 564, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 316, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 60072, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 1560, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 14, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 939, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 48, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 447, 2024-09-19T07:55:06.563 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 208, 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 45.010195291, 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.216395169 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3, 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000480254, 2024-09-19T07:55:06.564 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000160084 2024-09-19T07:55:06.565 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.565 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:55:06.565 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.565 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.565 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.565 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.565 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7, 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.009758724, 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001394103 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 193, 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.532706549, 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.023485526 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.567 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:55:06.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.569 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.569 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.569 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:55:06.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.569 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:55:06.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 168, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.063966319, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000380751 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.002220513, 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000317216 2024-09-19T07:55:06.571 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 207, 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.469838578, 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002269751 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:55:06.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:55:06.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10, 2024-09-19T07:55:06.576 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.016637483, 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001663748 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:55:06.577 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 2, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 62, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 31, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1661, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:55:06.578 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:55:06.579 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:55:06.580 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 303121, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 493, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 88, 2024-09-19T07:55:06.581 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 1, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8224, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 181, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 1392194, 2024-09-19T07:55:06.582 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 13095, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 2222, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 3759739, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 28, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 2222, 2024-09-19T07:55:06.583 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2222, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.387912767, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002874848 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2222, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2371 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 2222, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 1307, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 915, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:55:06.584 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 74, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 439, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 118, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 215, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:55:06.585 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1233, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T07:55:06.586 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:55:06.587 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 105, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 23, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 193, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 69 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:55:06.588 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 1183 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 6, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:55:06.589 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 6, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 203, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 203, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 9371, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 4775339, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:06.590 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 9371, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 9371, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4775339, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:06.591 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:06.592 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:06.592 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:06.593 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:06.593 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2222, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3780754, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2222, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3780754, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.594 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2222, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 2222, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2222, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2222, 2024-09-19T07:55:06.595 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 203, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 21015, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 203, 2024-09-19T07:55:06.596 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 74, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 21015, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x5622a82ec960": { 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:06.597 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 1308, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 3674427, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 1308, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 365, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3674427, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:06.598 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:06.599 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:06.599 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.599 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:06.599 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:55:06.599 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:06 smithi134 ceph-mon[35414]: pgmap v536: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-19T07:55:07.421 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-09-19T07:55:07.421 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2024-09-19T07:55:07.421 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T07:55:07.421 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:55:07.421 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:55:07.422 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T07:55:07.422 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T07:55:07.666 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T07:55:08.211 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-19T07:55:08.211 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T07:55:08.211 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3394, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3396, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 4629330, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1079299, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.256148143, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.081174941, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.067456532, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.088922761, 2024-09-19T07:55:08.212 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3394, 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.125720783, 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000037042 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T07:55:08.213 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 390, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 345, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1865260, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 200154, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 3, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.057140438, 2024-09-19T07:55:08.214 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.020259734, 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.053290354, 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000056693, 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 345, 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.026885838, 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000077929 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.215 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1864704, 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 199456 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4, 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 243, 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 748, 2024-09-19T07:55:08.216 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 670318, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.013528793, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.008536002, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.002642124, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000023876, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 240, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.020151870, 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000083966 2024-09-19T07:55:08.217 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 670000 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-19T07:55:08.218 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-19T07:55:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-19T07:55:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5754, 2024-09-19T07:55:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.138588165, 2024-09-19T07:55:08.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000024085 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.220 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-19T07:55:08.221 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 587, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 587, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 587, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-19T07:55:08.222 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 10, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-19T07:55:08.223 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T07:55:08.224 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T07:55:08.225 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-19T07:55:08.226 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-19T07:55:08.227 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-19T07:55:08.228 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 1308, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-19T07:55:08.229 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 3, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 7868731, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 7868157, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.230 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 1308 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 377, 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 733, 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 356, 2024-09-19T07:55:08.232 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 23, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 27, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 4, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 587, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 760, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 173, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 51564, 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215328 2024-09-19T07:55:08.233 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.234 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.235 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.236 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.237 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-19T07:55:08.238 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.239 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-19T07:55:08.240 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.240 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.241 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.242 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.244 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.245 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-19T07:55:08.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.248 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-19T07:55:08.249 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 2, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2361, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-19T07:55:08.250 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-19T07:55:08.251 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T07:55:08.252 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T07:55:08.252 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-19T07:55:08.252 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-19T07:55:08.252 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-19T07:55:08.252 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-19T07:55:08.252 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T07:55:08.252 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-19T07:55:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 196480, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 437, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-19T07:55:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8224, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 181, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 1350267, 2024-09-19T07:55:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 11508, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 3397, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-09-19T07:55:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 3397, 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3397, 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.683963482, 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000790098 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3397, 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 3398 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.257 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 3397, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3397, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 2107, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1288, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-19T07:55:08.258 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-19T07:55:08.259 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-19T07:55:08.260 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-19T07:55:08.261 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 105, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 21, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 4, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 5, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 1, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.262 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-19T07:55:08.263 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:08.264 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 3784, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 6132184, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 3784, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3784, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 6132184, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.265 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3397, 2024-09-19T07:55:08.266 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3674432, 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3397, 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3674432, 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:08.267 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3397, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3397, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3397, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3397, 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-19T07:55:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x559a9620eb40": { 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-19T07:55:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-19T07:55:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-19T07:55:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-19T07:55:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-19T07:55:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-19T07:55:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-19T07:55:08.272 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-19T07:55:08.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:08 smithi174 ceph-mon[35370]: pgmap v537: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 31 op/s 2024-09-19T07:55:08.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:08 smithi103 ceph-mon[31201]: pgmap v537: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 31 op/s 2024-09-19T07:55:08.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:08 smithi134 ceph-mon[35414]: pgmap v537: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 31 op/s 2024-09-19T07:55:08.956 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-19T07:55:08.956 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-19T07:55:08.957 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-19T07:55:08.957 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-19T07:55:08.957 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T07:55:08.957 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T07:55:08.968 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T07:55:08.968 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:55:08.969 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:08.990 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-19T07:55:08.991 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T07:55:09.050 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T07:55:09.507 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:10.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 -- 172.21.15.103:0/2662065788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 msgr2=0x7f70f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2662065788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f4101200 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e402a950 comp rx=0 tx=0).stop 2024-09-19T07:55:10.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 -- 172.21.15.103:0/2662065788 shutdown_connections 2024-09-19T07:55:10.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2662065788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70f4102ef0 0x7f70f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2662065788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f4102570 0x7f70f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2662065788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f4101200 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 -- 172.21.15.103:0/2662065788 >> 172.21.15.103:0/2662065788 conn(0x7f70f40fc7d0 msgr2=0x7f70f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 -- 172.21.15.103:0/2662065788 shutdown_connections 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.019+0000 7f70fb5db640 1 -- 172.21.15.103:0/2662065788 wait complete. 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 Processor -- start 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 -- start start 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f4102570 0x7f70f410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70f4102ef0 0x7f70f410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41120a0 con 0x7f70f4100e20 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70f4111f20 con 0x7f70f4102570 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70fb5db640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70f4112220 con 0x7f70f4102ef0 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70f9377640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70f9377640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f410e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36972/0 (socket says 172.21.15.103:36972) 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70f9377640 1 -- 172.21.15.103:0/2484376082 learned_addr learned my addr 172.21.15.103:0/2484376082 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70f8b76640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f4102570 0x7f70f410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.020+0000 7f70f9b78640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70f4102ef0 0x7f70f410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f9377640 1 -- 172.21.15.103:0/2484376082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70f4102ef0 msgr2=0x7f70f410ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f9377640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70f4102ef0 0x7f70f410ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f9377640 1 -- 172.21.15.103:0/2484376082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f4102570 msgr2=0x7f70f410ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f9377640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f4102570 0x7f70f410ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f9377640 1 -- 172.21.15.103:0/2484376082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f41a69c0 con 0x7f70f4100e20 2024-09-19T07:55:10.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f9b78640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70f4102ef0 0x7f70f410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f9377640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f410e4d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f70e4005cc0 tx=0x7f70e402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f27fc640 1 -- 172.21.15.103:0/2484376082 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4034040 con 0x7f70f4100e20 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f27fc640 1 -- 172.21.15.103:0/2484376082 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e4030920 con 0x7f70f4100e20 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f41a6c50 con 0x7f70f4100e20 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70f27fc640 1 -- 172.21.15.103:0/2484376082 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e40046e0 con 0x7f70f4100e20 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.021+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f41a7130 con 0x7f70f4100e20 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.022+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f4101200 con 0x7f70f4100e20 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.023+0000 7f70f27fc640 1 -- 172.21.15.103:0/2484376082 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f70e4037020 con 0x7f70f4100e20 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.024+0000 7f70f27fc640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f70dc0a33b0 0x7f70dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.024+0000 7f70f8b76640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f70dc0a33b0 0x7f70dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:10.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.024+0000 7f70f27fc640 1 -- 172.21.15.103:0/2484376082 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8586+0+0 (secure 0 0 0) 0x7f70e40f0be0 con 0x7f70f4100e20 2024-09-19T07:55:10.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.024+0000 7f70f8b76640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f70dc0a33b0 0x7f70dc0a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70f410ff60 tx=0x7f70e8005f90 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:10.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.027+0000 7f70f27fc640 1 -- 172.21.15.103:0/2484376082 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70e40bd720 con 0x7f70f4100e20 2024-09-19T07:55:10.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.206+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f70f410fa00 con 0x7f70f4100e20 2024-09-19T07:55:10.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:10 smithi103 ceph-mon[31201]: pgmap v538: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T07:55:10.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:10 smithi103 ceph-mon[31201]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.206+0000 7f70f27fc640 1 -- 172.21.15.103:0/2484376082 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v106) ==== 81+0+958 (secure 0 0 0) 0x7f70f410fa00 con 0x7f70f4100e20 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3492331771 2024-09-20T07:37:58.271121+0000 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/189968053 2024-09-20T07:37:58.271121+0000 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3706845955 2024-09-20T07:28:52.493301+0000 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/1746275589 2024-09-20T07:28:52.493301+0000 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2164654927 2024-09-20T07:37:58.271121+0000 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/1746275589 2024-09-20T07:28:52.493301+0000 2024-09-19T07:55:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3372836762 2024-09-20T07:28:52.493301+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2002728097 2024-09-20T07:29:16.000257+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/891568998 2024-09-20T07:28:52.493301+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/315623738 2024-09-20T07:29:16.000257+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1436062068 2024-09-20T07:29:16.000257+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2266926610 2024-09-20T07:37:58.271121+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1595075196 2024-09-20T07:29:16.000257+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2002728097 2024-09-20T07:29:16.000257+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/532109813 2024-09-20T07:37:58.271121+0000 2024-09-19T07:55:10.209 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2164654927 2024-09-20T07:37:58.271121+0000 2024-09-19T07:55:10.210 INFO:teuthology.orchestra.run.smithi103.stderr:listed 16 entries 2024-09-19T07:55:10.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f70dc0a33b0 msgr2=0x7f70dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f70dc0a33b0 0x7f70dc0a5870 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70f410ff60 tx=0x7f70e8005f90 comp rx=0 tx=0).stop 2024-09-19T07:55:10.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 msgr2=0x7f70f410e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f410e4d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f70e4005cc0 tx=0x7f70e402f420 comp rx=0 tx=0).stop 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 shutdown_connections 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f70dc0a33b0 0x7f70dc0a5870 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70f4102ef0 0x7f70f410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f4102570 0x7f70f410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 --2- 172.21.15.103:0/2484376082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f4100e20 0x7f70f410e4d0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 >> 172.21.15.103:0/2484376082 conn(0x7f70f40fc7d0 msgr2=0x7f70f41068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 shutdown_connections 2024-09-19T07:55:10.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:10.209+0000 7f70fb5db640 1 -- 172.21.15.103:0/2484376082 wait complete. 2024-09-19T07:55:10.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:10 smithi174 ceph-mon[35370]: pgmap v538: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T07:55:10.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:10 smithi174 ceph-mon[35370]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T07:55:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:10 smithi134 ceph-mon[35414]: pgmap v538: 97 pgs: 97 active+clean; 3.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-19T07:55:10.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:10 smithi134 ceph-mon[35414]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T07:55:10.927 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T07:55:10.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:10.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo umount /tmp/cephfs 2024-09-19T07:55:11.020 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T07:55:11.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:11.021 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-19T07:55:11.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:11.081 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-19T07:55:11.136 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-19T07:55:11.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:11.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-19T07:55:11.219 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:11 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2484376082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:55:11.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T07:55:11.225 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-19T07:55:11.351 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T07:55:11.361 INFO:tasks.cephadm:Teardown begin 2024-09-19T07:55:11.361 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T07:55:11.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:55:11.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:11 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2484376082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:55:11.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:11 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2484376082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T07:55:11.780 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 -- 172.21.15.103:0/942954235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8101420 msgr2=0x7f97e810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/942954235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8101420 0x7f97e810d950 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009cd0 tx=0x7f97e002a8e0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 -- 172.21.15.103:0/942954235 shutdown_connections 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/942954235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8101420 0x7f97e810d950 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/942954235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8100a80 0x7f97e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/942954235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97e8106a80 0x7f97e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 -- 172.21.15.103:0/942954235 >> 172.21.15.103:0/942954235 conn(0x7f97e80fc7a0 msgr2=0x7f97e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 -- 172.21.15.103:0/942954235 shutdown_connections 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.269+0000 7f97ec9b7640 1 -- 172.21.15.103:0/942954235 wait complete. 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.270+0000 7f97ec9b7640 1 Processor -- start 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.270+0000 7f97ec9b7640 1 -- start start 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.270+0000 7f97ec9b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8100a80 0x7f97e819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97ec9b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97e8101420 0x7f97e819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97ec9b7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8106a80 0x7f97e819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97ec9b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8110e80 con 0x7f97e8100a80 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97ec9b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8110d00 con 0x7f97e8101420 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97ec9b7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8111000 con 0x7f97e8106a80 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8100a80 0x7f97e819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8100a80 0x7f97e819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36992/0 (socket says 172.21.15.103:36992) 2024-09-19T07:55:12.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 -- 172.21.15.103:0/2616206849 learned_addr learned my addr 172.21.15.103:0/2616206849 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e5d9b640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97e8101420 0x7f97e819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 -- 172.21.15.103:0/2616206849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8106a80 msgr2=0x7f97e819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8106a80 0x7f97e819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 -- 172.21.15.103:0/2616206849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97e8101420 msgr2=0x7f97e819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97e8101420 0x7f97e819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 -- 172.21.15.103:0/2616206849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e819f5d0 con 0x7f97e8100a80 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97e659c640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8100a80 0x7f97e819a620 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f97d800d950 tx=0x7f97d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97d77fe640 1 -- 172.21.15.103:0/2616206849 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8017030 con 0x7f97e8100a80 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97d77fe640 1 -- 172.21.15.103:0/2616206849 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d8004020 con 0x7f97e8100a80 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e819f8c0 con 0x7f97e8100a80 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97d77fe640 1 -- 172.21.15.103:0/2616206849 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80047a0 con 0x7f97e8100a80 2024-09-19T07:55:12.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.271+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e81a71a0 con 0x7f97e8100a80 2024-09-19T07:55:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.272+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e8100ee0 con 0x7f97e8100a80 2024-09-19T07:55:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.274+0000 7f97d77fe640 1 -- 172.21.15.103:0/2616206849 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f97d80041d0 con 0x7f97e8100a80 2024-09-19T07:55:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.274+0000 7f97d77fe640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f97d00a33b0 0x7f97d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.274+0000 7f97d77fe640 1 -- 172.21.15.103:0/2616206849 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8586+0+0 (secure 0 0 0) 0x7f97d80cfe10 con 0x7f97e8100a80 2024-09-19T07:55:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.274+0000 7f97e5d9b640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f97d00a33b0 0x7f97d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.275+0000 7f97e5d9b640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f97d00a33b0 0x7f97d00a5870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005fd0 tx=0x7f97dc005850 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:12.295 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.277+0000 7f97d77fe640 1 -- 172.21.15.103:0/2616206849 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f97d809c7b0 con 0x7f97e8100a80 2024-09-19T07:55:12.470 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:12 smithi103 ceph-mon[31201]: pgmap v540: 97 pgs: 97 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:12.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.481+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97e819b980 con 0x7f97e8100a80 2024-09-19T07:55:12.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.482+0000 7f97d77fe640 1 -- 172.21.15.103:0/2616206849 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11996 (secure 0 0 0) 0x7f97d809c0b0 con 0x7f97e8100a80 2024-09-19T07:55:12.483 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:55:12.483 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":58,"btime":"2024-09-19T07:54:55:131079+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:54:54.874776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:55:12.483 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 58 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f97d00a33b0 msgr2=0x7f97d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f97d00a33b0 0x7f97d00a5870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005fd0 tx=0x7f97dc005850 comp rx=0 tx=0).stop 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8100a80 msgr2=0x7f97e819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8100a80 0x7f97e819a620 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f97d800d950 tx=0x7f97d800de10 comp rx=0 tx=0).stop 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 shutdown_connections 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f97d00a33b0 0x7f97d00a5870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8106a80 0x7f97e819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97e8101420 0x7f97e819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 --2- 172.21.15.103:0/2616206849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8100a80 0x7f97e819a620 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:12.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 >> 172.21.15.103:0/2616206849 conn(0x7f97e80fc7a0 msgr2=0x7f97e806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 shutdown_connections 2024-09-19T07:55:12.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:12.484+0000 7f97ec9b7640 1 -- 172.21.15.103:0/2616206849 wait complete. 2024-09-19T07:55:12.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:12 smithi134 ceph-mon[35414]: pgmap v540: 97 pgs: 97 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:12.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:12 smithi174 ceph-mon[35370]: pgmap v540: 97 pgs: 97 active+clean; 3.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:13.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:13 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2616206849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:55:13.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T07:55:13.591 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:13 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2616206849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:55:13.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:13 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2616206849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:55:13.750 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:14.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 -- 172.21.15.103:0/4027012925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 msgr2=0x7ff1d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/4027012925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d8101200 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009900 tx=0x7ff1c802a480 comp rx=0 tx=0).stop 2024-09-19T07:55:14.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 -- 172.21.15.103:0/4027012925 shutdown_connections 2024-09-19T07:55:14.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/4027012925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1d8102ef0 0x7ff1d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/4027012925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1d8102570 0x7ff1d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/4027012925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d8101200 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 -- 172.21.15.103:0/4027012925 >> 172.21.15.103:0/4027012925 conn(0x7ff1d80fc7d0 msgr2=0x7ff1d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 -- 172.21.15.103:0/4027012925 shutdown_connections 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.199+0000 7ff1dd02b640 1 -- 172.21.15.103:0/4027012925 wait complete. 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 Processor -- start 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 -- start start 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1d8102570 0x7ff1d810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1d8102ef0 0x7ff1d810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d81120a0 con 0x7ff1d8100e20 2024-09-19T07:55:14.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8111f20 con 0x7ff1d8102570 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1dd02b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8112220 con 0x7ff1d8102ef0 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37000/0 (socket says 172.21.15.103:37000) 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1d6d9d640 1 -- 172.21.15.103:0/2886450702 learned_addr learned my addr 172.21.15.103:0/2886450702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1d759e640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1d8102ef0 0x7ff1d810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.200+0000 7ff1d659c640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1d8102570 0x7ff1d810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1d6d9d640 1 -- 172.21.15.103:0/2886450702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1d8102ef0 msgr2=0x7ff1d810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1d6d9d640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1d8102ef0 0x7ff1d810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1d6d9d640 1 -- 172.21.15.103:0/2886450702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1d8102570 msgr2=0x7ff1d810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1d6d9d640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1d8102570 0x7ff1d810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1d6d9d640 1 -- 172.21.15.103:0/2886450702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d81a6a00 con 0x7ff1d8100e20 2024-09-19T07:55:14.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1d659c640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1d8102570 0x7ff1d810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1d6d9d640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d810e4d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8006010 tx=0x7ff1c8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1bffff640 1 -- 172.21.15.103:0/2886450702 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c803f070 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d81a6c90 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1bffff640 1 -- 172.21.15.103:0/2886450702 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c8031c30 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1bffff640 1 -- 172.21.15.103:0/2886450702 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8035030 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.201+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d81a71c0 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.202+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d8101200 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.203+0000 7ff1bffff640 1 -- 172.21.15.103:0/2886450702 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff1c8042070 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.204+0000 7ff1bffff640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1c00a33b0 0x7ff1c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.204+0000 7ff1d659c640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1c00a33b0 0x7ff1c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.204+0000 7ff1bffff640 1 -- 172.21.15.103:0/2886450702 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8586+0+0 (secure 0 0 0) 0x7ff1c80f0510 con 0x7ff1d8100e20 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.205+0000 7ff1d659c640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1c00a33b0 0x7ff1c00a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff1d810ff60 tx=0x7ff1cc00b390 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:14.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.208+0000 7ff1bffff640 1 -- 172.21.15.103:0/2886450702 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff1c8031de0 con 0x7ff1d8100e20 2024-09-19T07:55:14.415 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:14 smithi174 ceph-mon[35370]: pgmap v541: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.418+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1d8109090 con 0x7ff1d8100e20 2024-09-19T07:55:14.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:14 smithi103 ceph-mon[31201]: pgmap v541: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:14.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.419+0000 7ff1bffff640 1 -- 172.21.15.103:0/2886450702 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11996 (secure 0 0 0) 0x7ff1c80bc860 con 0x7ff1d8100e20 2024-09-19T07:55:14.420 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:55:14.420 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":58,"btime":"2024-09-19T07:54:55:131079+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2425532131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2425532131},{"type":"v1","addr":"172.21.15.103:6835","nonce":2425532131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/1711280456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":1711280456},{"type":"v1","addr":"172.21.15.103:6837","nonce":1711280456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T07:36:08.395229+0000","modified":"2024-09-19T07:54:54.874776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24455,"mds_3":24454,"mds_4":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6839/1660338093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":1660338093},{"type":"v1","addr":"172.21.15.103:6839","nonce":1660338093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14580":{"gid":14580,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6841/1521905484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":1521905484},{"type":"v1","addr":"172.21.15.103:6841","nonce":1521905484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2440407987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2440407987},{"type":"v1","addr":"172.21.15.134:6833","nonce":2440407987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/1885230172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1885230172},{"type":"v1","addr":"172.21.15.174:6833","nonce":1885230172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/19326296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":19326296},{"type":"v1","addr":"172.21.15.134:6835","nonce":19326296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/214062550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":214062550},{"type":"v1","addr":"172.21.15.174:6835","nonce":214062550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/2527370860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2527370860},{"type":"v1","addr":"172.21.15.174:6837","nonce":2527370860}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.134:6837/3959604171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3959604171},{"type":"v1","addr":"172.21.15.134:6837","nonce":3959604171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/2514182394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2514182394},{"type":"v1","addr":"172.21.15.174:6839","nonce":2514182394}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3683561216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3683561216},{"type":"v1","addr":"172.21.15.134:6839","nonce":3683561216}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24455,24454,24448]},"id":1}]} 2024-09-19T07:55:14.420 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 58 2024-09-19T07:55:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.421+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1c00a33b0 msgr2=0x7ff1c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.421+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1c00a33b0 0x7ff1c00a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff1d810ff60 tx=0x7ff1cc00b390 comp rx=0 tx=0).stop 2024-09-19T07:55:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.421+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 msgr2=0x7ff1d810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:14.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.421+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d810e4d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8006010 tx=0x7ff1c8003c10 comp rx=0 tx=0).stop 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 shutdown_connections 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff1c00a33b0 0x7ff1c00a5870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1d8102ef0 0x7ff1d810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1d8102570 0x7ff1d810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 --2- 172.21.15.103:0/2886450702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1d8100e20 0x7ff1d810e4d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 >> 172.21.15.103:0/2886450702 conn(0x7ff1d80fc7d0 msgr2=0x7ff1d80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 shutdown_connections 2024-09-19T07:55:14.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:14.422+0000 7ff1dd02b640 1 -- 172.21.15.103:0/2886450702 wait complete. 2024-09-19T07:55:14.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:14 smithi134 ceph-mon[35414]: pgmap v541: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:15.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T07:55:15.334 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:15.455 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:15 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2886450702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:55:15.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:15 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2886450702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:55:15.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:15 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2886450702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T07:55:15.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 -- 172.21.15.103:0/263980334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4106a80 msgr2=0x7fe5c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 --2- 172.21.15.103:0/263980334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4106a80 0x7fe5c4106e60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009930 tx=0x7fe5ac02a420 comp rx=0 tx=0).stop 2024-09-19T07:55:15.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 -- 172.21.15.103:0/263980334 shutdown_connections 2024-09-19T07:55:15.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 --2- 172.21.15.103:0/263980334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5c4101420 0x7fe5c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 --2- 172.21.15.103:0/263980334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5c4100a80 0x7fe5c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 --2- 172.21.15.103:0/263980334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4106a80 0x7fe5c4106e60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 -- 172.21.15.103:0/263980334 >> 172.21.15.103:0/263980334 conn(0x7fe5c40fc7a0 msgr2=0x7fe5c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 -- 172.21.15.103:0/263980334 shutdown_connections 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.789+0000 7fe5c9062640 1 -- 172.21.15.103:0/263980334 wait complete. 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.790+0000 7fe5c9062640 1 Processor -- start 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.790+0000 7fe5c9062640 1 -- start start 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.790+0000 7fe5c9062640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5c4100a80 0x7fe5c4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c9062640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4101420 0x7fe5c41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c9062640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5c4106a80 0x7fe5c419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c9062640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4110e80 con 0x7fe5c4101420 2024-09-19T07:55:15.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c9062640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4110d00 con 0x7fe5c4106a80 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c9062640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4111000 con 0x7fe5c4100a80 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4101420 0x7fe5c41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4101420 0x7fe5c41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37012/0 (socket says 172.21.15.103:37012) 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 -- 172.21.15.103:0/2600145825 learned_addr learned my addr 172.21.15.103:0/2600145825 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c2d9d640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5c4100a80 0x7fe5c4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c359e640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5c4106a80 0x7fe5c419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 -- 172.21.15.103:0/2600145825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5c4100a80 msgr2=0x7fe5c4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5c4100a80 0x7fe5c4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 -- 172.21.15.103:0/2600145825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5c4106a80 msgr2=0x7fe5c419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5c4106a80 0x7fe5c419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 -- 172.21.15.103:0/2600145825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c419d430 con 0x7fe5c4101420 2024-09-19T07:55:15.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c259c640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4101420 0x7fe5c41989c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9f0 tx=0x7fe5b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5bbfff640 1 -- 172.21.15.103:0/2600145825 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400cd80 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c419d720 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5bbfff640 1 -- 172.21.15.103:0/2600145825 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b4005500 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5bbfff640 1 -- 172.21.15.103:0/2600145825 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4010690 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.791+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a5000 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.792+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c4100ee0 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.794+0000 7fe5bbfff640 1 -- 172.21.15.103:0/2600145825 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fe5b40046c0 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.794+0000 7fe5bbfff640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5b00a33b0 0x7fe5b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.795+0000 7fe5c2d9d640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5b00a33b0 0x7fe5b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.795+0000 7fe5bbfff640 1 -- 172.21.15.103:0/2600145825 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8586+0+0 (secure 0 0 0) 0x7fe5b40d0150 con 0x7fe5c4101420 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.795+0000 7fe5c2d9d640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5b00a33b0 0x7fe5b00a5870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009900 tx=0x7fe5ac0058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:15.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.798+0000 7fe5bbfff640 1 -- 172.21.15.103:0/2600145825 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5b409cb90 con 0x7fe5c4101420 2024-09-19T07:55:15.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.964+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5c41a52a0 con 0x7fe5c4101420 2024-09-19T07:55:15.967 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-19T07:55:15.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.965+0000 7fe5bbfff640 1 -- 172.21.15.103:0/2600145825 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23362 (secure 0 0 0) 0x7fe5b409c490 con 0x7fe5c4101420 2024-09-19T07:55:15.967 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":106,"fsid":"9fd714d2-7658-11ef-bceb-c7b262605968","created":"2024-09-19T07:28:30.723617+0000","modified":"2024-09-19T07:55:09.158988+0000","last_up_change":"2024-09-19T07:35:17.150942+0000","last_in_change":"2024-09-19T07:34:54.004892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T07:31:30.307307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T07:36:04.766918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T07:36:06.565665+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d25f6411-3656-482d-bd89-a94dd43726b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6803","nonce":81836251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6805","nonce":81836251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6809","nonce":81836251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":81836251},{"type":"v1","addr":"172.21.15.103:6807","nonce":81836251}]},"public_addr":"172.21.15.103:6803/81836251","cluster_addr":"172.21.15.103:6805/81836251","heartbeat_back_addr":"172.21.15.103:6809/81836251","heartbeat_front_addr":"172.21.15.103:6807/81836251","state":["exists","up"]},{"osd":1,"uuid":"f51a85b8-e79c-496f-a662-59b58fdb0f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6801","nonce":2492227363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6803","nonce":2492227363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6807","nonce":2492227363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2492227363},{"type":"v1","addr":"172.21.15.134:6805","nonce":2492227363}]},"public_addr":"172.21.15.134:6801/2492227363","cluster_addr":"172.21.15.134:6803/2492227363","heartbeat_back_addr":"172.21.15.134:6807/2492227363","heartbeat_front_addr":"172.21.15.134:6805/2492227363","state":["exists","up"]},{"osd":2,"uuid":"f82ba336-4618-4e9c-825e-d51abe0ab8c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6801","nonce":1513938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6803","nonce":1513938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6807","nonce":1513938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1513938376},{"type":"v1","addr":"172.21.15.174:6805","nonce":1513938376}]},"public_addr":"172.21.15.174:6801/1513938376","cluster_addr":"172.21.15.174:6803/1513938376","heartbeat_back_addr":"172.21.15.174:6807/1513938376","heartbeat_front_addr":"172.21.15.174:6805/1513938376","state":["exists","up"]},{"osd":3,"uuid":"0b1b00f7-68c9-4888-81f7-2f2544a97c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6811","nonce":2879777863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6813","nonce":2879777863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6817","nonce":2879777863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2879777863},{"type":"v1","addr":"172.21.15.103:6815","nonce":2879777863}]},"public_addr":"172.21.15.103:6811/2879777863","cluster_addr":"172.21.15.103:6813/2879777863","heartbeat_back_addr":"172.21.15.103:6817/2879777863","heartbeat_front_addr":"172.21.15.103:6815/2879777863","state":["exists","up"]},{"osd":4,"uuid":"2fe08876-9b5e-4a2f-9b6b-42c11ce4f6ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6809","nonce":2364007931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6811","nonce":2364007931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6815","nonce":2364007931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2364007931},{"type":"v1","addr":"172.21.15.134:6813","nonce":2364007931}]},"public_addr":"172.21.15.134:6809/2364007931","cluster_addr":"172.21.15.134:6811/2364007931","heartbeat_back_addr":"172.21.15.134:6815/2364007931","heartbeat_front_addr":"172.21.15.134:6813/2364007931","state":["exists","up"]},{"osd":5,"uuid":"14cf312f-9b50-4214-8f5d-2b4d68af438f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6809","nonce":2281357008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6811","nonce":2281357008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6815","nonce":2281357008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2281357008},{"type":"v1","addr":"172.21.15.174:6813","nonce":2281357008}]},"public_addr":"172.21.15.174:6809/2281357008","cluster_addr":"172.21.15.174:6811/2281357008","heartbeat_back_addr":"172.21.15.174:6815/2281357008","heartbeat_front_addr":"172.21.15.174:6813/2281357008","state":["exists","up"]},{"osd":6,"uuid":"eaf8bd47-9f95-4e26-b8e0-387cb89c1de3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6819","nonce":3692913756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6821","nonce":3692913756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6825","nonce":3692913756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3692913756},{"type":"v1","addr":"172.21.15.103:6823","nonce":3692913756}]},"public_addr":"172.21.15.103:6819/3692913756","cluster_addr":"172.21.15.103:6821/3692913756","heartbeat_back_addr":"172.21.15.103:6825/3692913756","heartbeat_front_addr":"172.21.15.103:6823/3692913756","state":["exists","up"]},{"osd":7,"uuid":"c5ac551b-6bfa-45bb-a12c-7684b5d30079","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6817","nonce":2397823373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6819","nonce":2397823373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6823","nonce":2397823373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2397823373},{"type":"v1","addr":"172.21.15.134:6821","nonce":2397823373}]},"public_addr":"172.21.15.134:6817/2397823373","cluster_addr":"172.21.15.134:6819/2397823373","heartbeat_back_addr":"172.21.15.134:6823/2397823373","heartbeat_front_addr":"172.21.15.134:6821/2397823373","state":["exists","up"]},{"osd":8,"uuid":"3aa078e4-5626-4b10-8727-a060b52ffb58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6817","nonce":477602146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6819","nonce":477602146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6823","nonce":477602146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":477602146},{"type":"v1","addr":"172.21.15.174:6821","nonce":477602146}]},"public_addr":"172.21.15.174:6817/477602146","cluster_addr":"172.21.15.174:6819/477602146","heartbeat_back_addr":"172.21.15.174:6823/477602146","heartbeat_front_addr":"172.21.15.174:6821/477602146","state":["exists","up"]},{"osd":9,"uuid":"e9f63e40-aa84-48be-b99b-2f16545d058f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6827","nonce":1024309893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6829","nonce":1024309893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6833","nonce":1024309893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1024309893},{"type":"v1","addr":"172.21.15.103:6831","nonce":1024309893}]},"public_addr":"172.21.15.103:6827/1024309893","cluster_addr":"172.21.15.103:6829/1024309893","heartbeat_back_addr":"172.21.15.103:6833/1024309893","heartbeat_front_addr":"172.21.15.103:6831/1024309893","state":["exists","up"]},{"osd":10,"uuid":"60776c91-f34f-499f-825d-43691743cd2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6825","nonce":984759613}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6827","nonce":984759613}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6831","nonce":984759613}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":984759613},{"type":"v1","addr":"172.21.15.134:6829","nonce":984759613}]},"public_addr":"172.21.15.134:6825/984759613","cluster_addr":"172.21.15.134:6827/984759613","heartbeat_back_addr":"172.21.15.134:6831/984759613","heartbeat_front_addr":"172.21.15.134:6829/984759613","state":["exists","up"]},{"osd":11,"uuid":"d731972a-f6ca-4e9d-b1a7-9b1b82f0e381","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6825","nonce":1686831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6827","nonce":1686831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6831","nonce":1686831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1686831932},{"type":"v1","addr":"172.21.15.174:6829","nonce":1686831932}]},"public_addr":"172.21.15.174:6825/1686831932","cluster_addr":"172.21.15.174:6827/1686831932","heartbeat_back_addr":"172.21.15.174:6831/1686831932","heartbeat_front_addr":"172.21.15.174:6829/1686831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:30:37.328435+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:02.799171+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:27.093908+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:31:52.286232+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:17.945487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:32:43.148474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:09.235969+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:34.542031+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:33:59.505807+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:25.118021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:34:50.204075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T07:35:15.842011+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/3492331771":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/189968053":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/3706845955":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6800/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2164654927":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6801/1746275589":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/3372836762":"2024-09-20T07:28:52.493301+0000","172.21.15.103:6801/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/891568998":"2024-09-20T07:28:52.493301+0000","172.21.15.103:0/315623738":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/1436062068":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/2266926610":"2024-09-20T07:37:58.271121+0000","172.21.15.103:0/1595075196":"2024-09-20T07:29:16.000257+0000","172.21.15.103:6800/2002728097":"2024-09-20T07:29:16.000257+0000","172.21.15.103:0/532109813":"2024-09-20T07:37:58.271121+0000","172.21.15.103:6800/2164654927":"2024-09-20T07:37:58.271121+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5b00a33b0 msgr2=0x7fe5b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5b00a33b0 0x7fe5b00a5870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009900 tx=0x7fe5ac0058d0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4101420 msgr2=0x7fe5c41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4101420 0x7fe5c41989c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9f0 tx=0x7fe5b400edf0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 shutdown_connections 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fe5b00a33b0 0x7fe5b00a5870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5c4106a80 0x7fe5c419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5c4101420 0x7fe5c41989c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 --2- 172.21.15.103:0/2600145825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5c4100a80 0x7fe5c4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:15.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 >> 172.21.15.103:0/2600145825 conn(0x7fe5c40fc7a0 msgr2=0x7fe5c41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:15.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 shutdown_connections 2024-09-19T07:55:15.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:15.968+0000 7fe5c9062640 1 -- 172.21.15.103:0/2600145825 wait complete. 2024-09-19T07:55:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:16 smithi103 ceph-mon[31201]: pgmap v542: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:16 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/2600145825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:55:16.220 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:16 smithi103 ceph-mon[31201]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T07:55:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:16 smithi134 ceph-mon[35414]: pgmap v542: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:16 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/2600145825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:55:16.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:16 smithi134 ceph-mon[35414]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T07:55:16.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T07:55:16.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:16 smithi174 ceph-mon[35370]: pgmap v542: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-19T07:55:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:16 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/2600145825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T07:55:16.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:16 smithi174 ceph-mon[35370]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T07:55:16.911 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 -- 172.21.15.103:0/2488315926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 msgr2=0x7f12b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/2488315926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b4106e60 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009930 tx=0x7f12a402a420 comp rx=0 tx=0).stop 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 -- 172.21.15.103:0/2488315926 shutdown_connections 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/2488315926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4101420 0x7f12b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/2488315926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b4100a80 0x7f12b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/2488315926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b4106e60 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 -- 172.21.15.103:0/2488315926 >> 172.21.15.103:0/2488315926 conn(0x7f12b40fc7a0 msgr2=0x7f12b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 -- 172.21.15.103:0/2488315926 shutdown_connections 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.377+0000 7f12b9e7b640 1 -- 172.21.15.103:0/2488315926 wait complete. 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 Processor -- start 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 -- start start 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4100a80 0x7f12b41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b4101420 0x7f12b4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4110f20 con 0x7f12b4106a80 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4110da0 con 0x7f12b4100a80 2024-09-19T07:55:17.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b9e7b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12b41110a0 con 0x7f12b4101420 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37038/0 (socket says 172.21.15.103:37038) 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 -- 172.21.15.103:0/3552330150 learned_addr learned my addr 172.21.15.103:0/3552330150 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b37fe640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4100a80 0x7f12b41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12abfff640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b4101420 0x7f12b4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 -- 172.21.15.103:0/3552330150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b4101420 msgr2=0x7f12b4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b4101420 0x7f12b4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 -- 172.21.15.103:0/3552330150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4100a80 msgr2=0x7f12b41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4100a80 0x7f12b41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.378+0000 7f12b3fff640 1 -- 172.21.15.103:0/3552330150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b419d570 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.379+0000 7f12b3fff640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b419ce90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00b970 tx=0x7f12ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.379+0000 7f12b17fa640 1 -- 172.21.15.103:0/3552330150 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac019070 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.379+0000 7f12b17fa640 1 -- 172.21.15.103:0/3552330150 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ac00cb40 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.379+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b419d860 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.379+0000 7f12b17fa640 1 -- 172.21.15.103:0/3552330150 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac007650 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.379+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a5140 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.380+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b4100f60 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.381+0000 7f12b17fa640 1 -- 172.21.15.103:0/3552330150 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f12ac01c070 con 0x7f12b4106a80 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.381+0000 7f12b17fa640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f12a00a33b0 0x7f12a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.382+0000 7f12b37fe640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f12a00a33b0 0x7f12a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:17.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.382+0000 7f12b17fa640 1 -- 172.21.15.103:0/3552330150 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7f12ac0cf210 con 0x7f12b4106a80 2024-09-19T07:55:17.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.382+0000 7f12b37fe640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f12a00a33b0 0x7f12a00a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009900 tx=0x7f12a40058d0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:17.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.386+0000 7f12b17fa640 1 -- 172.21.15.103:0/3552330150 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f12ac09b6a0 con 0x7f12b4106a80 2024-09-19T07:55:17.589 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.589+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f12b4109a70 con 0x7f12b4106a80 2024-09-19T07:55:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.593+0000 7f12b17fa640 1 -- 172.21.15.103:0/3552330150 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7f12ac09f9a0 con 0x7f12b4106a80 2024-09-19T07:55:17.593 INFO:teuthology.orchestra.run.smithi103.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T07:55:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f12a00a33b0 msgr2=0x7f12a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f12a00a33b0 0x7f12a00a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009900 tx=0x7f12a40058d0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 msgr2=0x7f12b419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b419ce90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00b970 tx=0x7f12ac00be30 comp rx=0 tx=0).stop 2024-09-19T07:55:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 shutdown_connections 2024-09-19T07:55:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f12a00a33b0 0x7f12a00a5870 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009900 tx=0x7f12a40058d0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12b4106a80 0x7f12b419ce90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12b4101420 0x7f12b4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 --2- 172.21.15.103:0/3552330150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4100a80 0x7f12b41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.596+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 >> 172.21.15.103:0/3552330150 conn(0x7f12b40fc7a0 msgr2=0x7f12b406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.597+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 shutdown_connections 2024-09-19T07:55:17.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:17.597+0000 7f12b9e7b640 1 -- 172.21.15.103:0/3552330150 wait complete. 2024-09-19T07:55:17.915 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c[65445]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:17.915 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c[65445]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:17.915 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-mds[65449]: -- 172.21.15.174:0/1615604812 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5573dad61860 con 0x5573dad3e800 2024-09-19T07:55:17.916 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c[65445]: starting mds.c at 2024-09-19T07:55:17.916 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f[65144]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:17.916 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f[65144]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:17.916 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-mds[65148]: -- 172.21.15.174:0/3645194205 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a5524b860 con 0x559a55229400 2024-09-19T07:55:17.916 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f[65144]: starting mds.f at 2024-09-19T07:55:17.916 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i[65743]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:17.917 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i[65743]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:17.917 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-mds[65747]: -- 172.21.15.174:0/2744647487 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dde085860 con 0x558dde062800 2024-09-19T07:55:17.917 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i[65743]: starting mds.i at 2024-09-19T07:55:17.917 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l[64843]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:17.917 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l[64843]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:17.917 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-mds[64847]: -- 172.21.15.174:0/1615125316 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561963417860 con 0x5619633f4800 2024-09-19T07:55:17.917 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:55:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l[64843]: starting mds.l at 2024-09-19T07:55:17.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a[31197]: 2024-09-19T07:55:17.590+0000 7fd22a4e2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:55:17.970 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g[71883]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:17.970 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g[71883]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:17.970 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-mds[71900]: -- 172.21.15.103:0/1443749708 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b5285bc20 con 0x564b52838800 2024-09-19T07:55:17.970 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g[71883]: starting mds.g at 2024-09-19T07:55:17.970 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g[71883]: 2024-09-19T07:55:17.638+0000 7fe172227640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2024-09-19T07:55:17.971 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a[72270]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:17.971 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a[72270]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:17.971 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-mds[72274]: -- 172.21.15.103:0/906429791 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564bb619dc20 con 0x564bb617b000 2024-09-19T07:55:17.971 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:55:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a[72270]: starting mds.a at 2024-09-19T07:55:18.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e[64947]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:18.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e[64947]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:18.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-mds[64951]: -- 172.21.15.134:0/3509186268 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e1fdc3860 con 0x558e1fda0800 2024-09-19T07:55:18.079 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e[64947]: starting mds.e at 2024-09-19T07:55:18.080 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h[65246]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:18.080 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h[65246]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:18.080 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-mds[65250]: -- 172.21.15.134:0/3749139274 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e09a3d860 con 0x563e09a1a800 2024-09-19T07:55:18.080 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h[65246]: starting mds.h at 2024-09-19T07:55:18.080 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:18.080 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:18.081 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-mds[65560]: -- 172.21.15.134:0/994419017 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa3004dc20 con 0x55aa3002b000 2024-09-19T07:55:18.081 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: starting mds.b at 2024-09-19T07:55:18.081 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k[64646]: ignoring --setuser ceph since I am not root 2024-09-19T07:55:18.081 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k[64646]: ignoring --setgroup ceph since I am not root 2024-09-19T07:55:18.081 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-mds[64650]: -- 172.21.15.134:0/2251987162 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fa4eb3860 con 0x561fa4e91000 2024-09-19T07:55:18.081 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:55:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k[64646]: starting mds.k at 2024-09-19T07:55:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:18 smithi103 ceph-mon[31201]: pgmap v544: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-09-19T07:55:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:18 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3552330150' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:55:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:18 smithi103 ceph-mon[31201]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:55:18.473 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:18 smithi103 ceph-mon[31201]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:55:18.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:18 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3552330150' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:55:18.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:18 smithi103 ceph-mon[31201]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T07:55:18.474 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:18 smithi103 ceph-mon[31201]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T07:55:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:18 smithi134 ceph-mon[35414]: pgmap v544: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-09-19T07:55:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:18 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3552330150' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:55:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:18 smithi134 ceph-mon[35414]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:55:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:18 smithi134 ceph-mon[35414]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:55:18.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:18 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3552330150' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:55:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:18 smithi134 ceph-mon[35414]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T07:55:18.580 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:18 smithi134 ceph-mon[35414]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T07:55:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:18 smithi174 ceph-mon[35370]: pgmap v544: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-09-19T07:55:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:18 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3552330150' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T07:55:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:18 smithi174 ceph-mon[35370]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T07:55:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:18 smithi174 ceph-mon[35370]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T07:55:18.665 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:18 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3552330150' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T07:55:18.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:18 smithi174 ceph-mon[35370]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T07:55:18.666 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:18 smithi174 ceph-mon[35370]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-19T07:55:18.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T07:55:18.956 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 -- 172.21.15.103:0/403374561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 msgr2=0x7fd340103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 --2- 172.21.15.103:0/403374561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340103110 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd330009cd0 tx=0x7fd33002a910 comp rx=0 tx=0).stop 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 -- 172.21.15.103:0/403374561 shutdown_connections 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 --2- 172.21.15.103:0/403374561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3401082a0 0x7fd34010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 --2- 172.21.15.103:0/403374561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3401036e0 0x7fd340107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 --2- 172.21.15.103:0/403374561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340103110 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 -- 172.21.15.103:0/403374561 >> 172.21.15.103:0/403374561 conn(0x7fd3400fc7d0 msgr2=0x7fd3400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:19.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 -- 172.21.15.103:0/403374561 shutdown_connections 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.404+0000 7fd344956640 1 -- 172.21.15.103:0/403374561 wait complete. 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 Processor -- start 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 -- start start 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3401036e0 0x7fd340198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3401082a0 0x7fd34019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd340112160 con 0x7fd340102d30 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd340111fe0 con 0x7fd3401036e0 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd344956640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401122e0 con 0x7fd3401082a0 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd33e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd33e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39022/0 (socket says 172.21.15.103:39022) 2024-09-19T07:55:19.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd33ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3401082a0 0x7fd34019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd33e59c640 1 -- 172.21.15.103:0/102167003 learned_addr learned my addr 172.21.15.103:0/102167003 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.405+0000 7fd33dd9b640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3401036e0 0x7fd340198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd33e59c640 1 -- 172.21.15.103:0/102167003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3401082a0 msgr2=0x7fd34019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd33e59c640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3401082a0 0x7fd34019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd33e59c640 1 -- 172.21.15.103:0/102167003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3401036e0 msgr2=0x7fd340198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd33e59c640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3401036e0 0x7fd340198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd33e59c640 1 -- 172.21.15.103:0/102167003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34019d6d0 con 0x7fd340102d30 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd33ed9d640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3401082a0 0x7fd34019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd33e59c640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340198720 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd3300098d0 tx=0x7fd330031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd32f7fe640 1 -- 172.21.15.103:0/102167003 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330031850 con 0x7fd340102d30 2024-09-19T07:55:19.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34019d960 con 0x7fd340102d30 2024-09-19T07:55:19.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd32f7fe640 1 -- 172.21.15.103:0/102167003 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd330004520 con 0x7fd340102d30 2024-09-19T07:55:19.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd32f7fe640 1 -- 172.21.15.103:0/102167003 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33002fd40 con 0x7fd340102d30 2024-09-19T07:55:19.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.406+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3401a52a0 con 0x7fd340102d30 2024-09-19T07:55:19.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.407+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd340103190 con 0x7fd340102d30 2024-09-19T07:55:19.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.408+0000 7fd32f7fe640 1 -- 172.21.15.103:0/102167003 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7fd330004040 con 0x7fd340102d30 2024-09-19T07:55:19.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.409+0000 7fd32f7fe640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd3280a33b0 0x7fd3280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:19.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.409+0000 7fd33dd9b640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd3280a33b0 0x7fd3280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:19.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.409+0000 7fd32f7fe640 1 -- 172.21.15.103:0/102167003 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8928+0+0 (secure 0 0 0) 0x7fd3300f1290 con 0x7fd340102d30 2024-09-19T07:55:19.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.409+0000 7fd33dd9b640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd3280a33b0 0x7fd3280a5870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd3400feee0 tx=0x7fd334009110 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:19.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.413+0000 7fd32f7fe640 1 -- 172.21.15.103:0/102167003 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd3300bdc90 con 0x7fd340102d30 2024-09-19T07:55:19.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:19.614+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd34010a2b0 con 0x7fd340102d30 2024-09-19T07:55:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: pgmap v546: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 11 op/s 2024-09-19T07:55:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.103:6838/3692981264,v1:172.21.15.103:6841/3692981264] up:boot 2024-09-19T07:55:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.103:6839/3967442274,v1:172.21.15.103:6840/3967442274] up:boot 2024-09-19T07:55:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6832/855585761,v1:172.21.15.174:6833/855585761] up:boot 2024-09-19T07:55:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6832/2233340879,v1:172.21.15.134:6834/2233340879] up:boot 2024-09-19T07:55:19.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6833/2488243596,v1:172.21.15.134:6835/2488243596] up:boot 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6836/2418005548,v1:172.21.15.134:6837/2418005548] up:boot 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6834/1664752589,v1:172.21.15.174:6836/1664752589] up:boot 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6835/2513350548,v1:172.21.15.174:6837/2513350548] up:boot 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.134:6838/1304085976,v1:172.21.15.134:6839/1304085976] up:boot 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: mds.? [v2:172.21.15.174:6838/1009051707,v1:172.21.15.174:6839/1009051707] up:boot 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:55:19.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:55:19.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:55:19.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:19 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:55:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: pgmap v546: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 11 op/s 2024-09-19T07:55:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.103:6838/3692981264,v1:172.21.15.103:6841/3692981264] up:boot 2024-09-19T07:55:19.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.103:6839/3967442274,v1:172.21.15.103:6840/3967442274] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6832/855585761,v1:172.21.15.174:6833/855585761] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6832/2233340879,v1:172.21.15.134:6834/2233340879] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6833/2488243596,v1:172.21.15.134:6835/2488243596] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6836/2418005548,v1:172.21.15.134:6837/2418005548] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6834/1664752589,v1:172.21.15.174:6836/1664752589] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6835/2513350548,v1:172.21.15.174:6837/2513350548] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.134:6838/1304085976,v1:172.21.15.134:6839/1304085976] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: mds.? [v2:172.21.15.174:6838/1009051707,v1:172.21.15.174:6839/1009051707] up:boot 2024-09-19T07:55:19.916 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:55:19.917 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:19 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:55:20.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: pgmap v546: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 11 op/s 2024-09-19T07:55:20.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.103:6838/3692981264,v1:172.21.15.103:6841/3692981264] up:boot 2024-09-19T07:55:20.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.103:6839/3967442274,v1:172.21.15.103:6840/3967442274] up:boot 2024-09-19T07:55:20.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6832/855585761,v1:172.21.15.174:6833/855585761] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6832/2233340879,v1:172.21.15.134:6834/2233340879] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6833/2488243596,v1:172.21.15.134:6835/2488243596] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6836/2418005548,v1:172.21.15.134:6837/2418005548] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6834/1664752589,v1:172.21.15.174:6836/1664752589] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6835/2513350548,v1:172.21.15.174:6837/2513350548] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.134:6838/1304085976,v1:172.21.15.134:6839/1304085976] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: mds.? [v2:172.21.15.174:6838/1009051707,v1:172.21.15.174:6839/1009051707] up:boot 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T07:55:20.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T07:55:20.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T07:55:20.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T07:55:20.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T07:55:20.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T07:55:20.081 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:19 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T07:55:20.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.603+0000 7fd32f7fe640 1 -- 172.21.15.103:0/102167003 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7fd3300bd590 con 0x7fd340102d30 2024-09-19T07:55:20.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd3280a33b0 msgr2=0x7fd3280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:20.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd3280a33b0 0x7fd3280a5870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd3400feee0 tx=0x7fd334009110 comp rx=0 tx=0).stop 2024-09-19T07:55:20.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 msgr2=0x7fd340198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:20.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340198720 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd3300098d0 tx=0x7fd330031460 comp rx=0 tx=0).stop 2024-09-19T07:55:20.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 shutdown_connections 2024-09-19T07:55:20.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7fd3280a33b0 0x7fd3280a5870 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:20.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3401082a0 0x7fd34019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:20.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3401036e0 0x7fd340198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:20.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 --2- 172.21.15.103:0/102167003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd340102d30 0x7fd340198720 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:20.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 >> 172.21.15.103:0/102167003 conn(0x7fd3400fc7d0 msgr2=0x7fd340105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:20.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 shutdown_connections 2024-09-19T07:55:20.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:20.606+0000 7fd344956640 1 -- 172.21.15.103:0/102167003 wait complete. 2024-09-19T07:55:20.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:20 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/102167003' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:55:20.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:20 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/102167003' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:55:21.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:20 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/102167003' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T07:55:21.469 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T07:55:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:21 smithi103 ceph-mon[31201]: pgmap v547: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-09-19T07:55:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:21 smithi103 ceph-mon[31201]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:55:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:21 smithi103 ceph-mon[31201]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:55:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:21 smithi103 ceph-mon[31201]: Cluster is now healthy 2024-09-19T07:55:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:21 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/102167003' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:55:21.720 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:21 smithi103 ceph-mon[31201]: fsmap 12 up:standby 2024-09-19T07:55:21.723 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:21 smithi174 ceph-mon[35370]: pgmap v547: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-09-19T07:55:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:21 smithi174 ceph-mon[35370]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:55:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:21 smithi174 ceph-mon[35370]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:55:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:21 smithi174 ceph-mon[35370]: Cluster is now healthy 2024-09-19T07:55:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:21 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/102167003' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:55:21.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:21 smithi174 ceph-mon[35370]: fsmap 12 up:standby 2024-09-19T07:55:22.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:21 smithi134 ceph-mon[35414]: pgmap v547: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-09-19T07:55:22.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:21 smithi134 ceph-mon[35414]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T07:55:22.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:21 smithi134 ceph-mon[35414]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T07:55:22.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:21 smithi134 ceph-mon[35414]: Cluster is now healthy 2024-09-19T07:55:22.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:21 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/102167003' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T07:55:22.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:21 smithi134 ceph-mon[35414]: fsmap 12 up:standby 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 -- 172.21.15.103:0/236140095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0101420 msgr2=0x7f82b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 --2- 172.21.15.103:0/236140095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0101420 0x7f82b010d950 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009cd0 tx=0x7f82a802a8e0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 -- 172.21.15.103:0/236140095 shutdown_connections 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 --2- 172.21.15.103:0/236140095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0101420 0x7f82b010d950 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 --2- 172.21.15.103:0/236140095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b0100a80 0x7f82b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 --2- 172.21.15.103:0/236140095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0106a80 0x7f82b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 -- 172.21.15.103:0/236140095 >> 172.21.15.103:0/236140095 conn(0x7f82b00fc7a0 msgr2=0x7f82b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:22.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 -- 172.21.15.103:0/236140095 shutdown_connections 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 -- 172.21.15.103:0/236140095 wait complete. 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 Processor -- start 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.197+0000 7f82b6680640 1 -- start start 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b6680640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b0100a80 0x7f82b010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b6680640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0101420 0x7f82b010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b6680640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0106a80 0x7f82b010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b6680640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0111fa0 con 0x7f82b0106a80 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b6680640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0111e20 con 0x7f82b0101420 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b6680640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0112120 con 0x7f82b0100a80 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0106a80 0x7f82b010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0106a80 0x7f82b010ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39032/0 (socket says 172.21.15.103:39032) 2024-09-19T07:55:22.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 -- 172.21.15.103:0/3269822935 learned_addr learned my addr 172.21.15.103:0/3269822935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82af7fe640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0101420 0x7f82b010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82affff640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b0100a80 0x7f82b010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 -- 172.21.15.103:0/3269822935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b0100a80 msgr2=0x7f82b010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b0100a80 0x7f82b010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 -- 172.21.15.103:0/3269822935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0101420 msgr2=0x7f82b010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0101420 0x7f82b010e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 -- 172.21.15.103:0/3269822935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b01a6940 con 0x7f82b0106a80 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.198+0000 7f82b4c1d640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0106a80 0x7f82b010ee60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006010 tx=0x7f82a8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.199+0000 7f82ad7fa640 1 -- 172.21.15.103:0/3269822935 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8004450 con 0x7f82b0106a80 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.199+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b01a6bd0 con 0x7f82b0106a80 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.199+0000 7f82ad7fa640 1 -- 172.21.15.103:0/3269822935 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a8003e10 con 0x7f82b0106a80 2024-09-19T07:55:22.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.199+0000 7f82ad7fa640 1 -- 172.21.15.103:0/3269822935 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8046480 con 0x7f82b0106a80 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.199+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b01a7160 con 0x7f82b0106a80 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.200+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b0100ee0 con 0x7f82b0106a80 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.201+0000 7f82ad7fa640 1 -- 172.21.15.103:0/3269822935 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7f82a80045f0 con 0x7f82b0106a80 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.201+0000 7f82ad7fa640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82980a33b0 0x7f82980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.201+0000 7f82ad7fa640 1 -- 172.21.15.103:0/3269822935 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8928+0+0 (secure 0 0 0) 0x7f82a802e030 con 0x7f82b0106a80 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.201+0000 7f82affff640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82980a33b0 0x7f82980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.202+0000 7f82affff640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82980a33b0 0x7f82980a5870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82a00058c0 tx=0x7f82a000a4c0 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:22.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.206+0000 7f82ad7fa640 1 -- 172.21.15.103:0/3269822935 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f82a804b060 con 0x7f82b0106a80 2024-09-19T07:55:22.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.373+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f82b0109010 con 0x7f82b0106a80 2024-09-19T07:55:22.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.606+0000 7f82ad7fa640 1 -- 172.21.15.103:0/3269822935 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f82a80be0b0 con 0x7f82b0106a80 2024-09-19T07:55:22.607 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' removed 2024-09-19T07:55:22.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.609+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82980a33b0 msgr2=0x7f82980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:22.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82980a33b0 0x7f82980a5870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82a00058c0 tx=0x7f82a000a4c0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0106a80 msgr2=0x7f82b010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0106a80 0x7f82b010ee60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006010 tx=0x7f82a8003c20 comp rx=0 tx=0).stop 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 shutdown_connections 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7f82980a33b0 0x7f82980a5870 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82a00058c0 tx=0x7f82a000a4c0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82b0106a80 0x7f82b010ee60 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82b0101420 0x7f82b010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 --2- 172.21.15.103:0/3269822935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82b0100a80 0x7f82b010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:22.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.610+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 >> 172.21.15.103:0/3269822935 conn(0x7f82b00fc7a0 msgr2=0x7f82b00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.611+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 shutdown_connections 2024-09-19T07:55:22.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:22.611+0000 7f82b6680640 1 -- 172.21.15.103:0/3269822935 wait complete. 2024-09-19T07:55:22.719 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:22 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3269822935' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:22.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:22 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3269822935' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:23.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:22 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3269822935' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:23.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T07:55:23.542 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T07:55:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:23 smithi174 ceph-mon[35370]: pgmap v548: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-19T07:55:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:23 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/3269822935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:55:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:23 smithi174 ceph-mon[35370]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T07:55:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:23 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:55:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:23 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:55:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:23 smithi174 ceph-mon[35370]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:55:23.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:23 smithi174 ceph-mon[35370]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:55:23.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:23 smithi103 ceph-mon[31201]: pgmap v548: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-19T07:55:23.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:23 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/3269822935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:55:23.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:23 smithi103 ceph-mon[31201]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T07:55:23.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:23 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:55:23.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:23 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:55:23.957 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:23 smithi103 ceph-mon[31201]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:55:23.958 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:23 smithi103 ceph-mon[31201]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:55:24.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 -- 172.21.15.103:0/3096643604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 msgr2=0x7ff16c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 --2- 172.21.15.103:0/3096643604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c106e60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff160009930 tx=0x7ff16002a420 comp rx=0 tx=0).stop 2024-09-19T07:55:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 -- 172.21.15.103:0/3096643604 shutdown_connections 2024-09-19T07:55:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 --2- 172.21.15.103:0/3096643604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff16c101420 0x7ff16c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 --2- 172.21.15.103:0/3096643604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff16c100a80 0x7ff16c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 --2- 172.21.15.103:0/3096643604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c106e60 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 -- 172.21.15.103:0/3096643604 >> 172.21.15.103:0/3096643604 conn(0x7ff16c0fc7a0 msgr2=0x7ff16c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:24.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 -- 172.21.15.103:0/3096643604 shutdown_connections 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.040+0000 7ff172f42640 1 -- 172.21.15.103:0/3096643604 wait complete. 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 Processor -- start 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 -- start start 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff16c100a80 0x7ff16c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff16c101420 0x7ff16c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c110e80 con 0x7ff16c106a80 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c110d00 con 0x7ff16c100a80 2024-09-19T07:55:24.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff172f42640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c111000 con 0x7ff16c101420 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff1714df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff1714df640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c1a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39038/0 (socket says 172.21.15.103:39038) 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff1714df640 1 -- 172.21.15.103:0/1839948788 learned_addr learned my addr 172.21.15.103:0/1839948788 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff16bfff640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff16c101420 0x7ff16c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.041+0000 7ff170cde640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff16c100a80 0x7ff16c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff16bfff640 1 -- 172.21.15.103:0/1839948788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff16c100a80 msgr2=0x7ff16c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff16bfff640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff16c100a80 0x7ff16c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff16bfff640 1 -- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 msgr2=0x7ff16c1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff16bfff640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c1a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff16bfff640 1 -- 172.21.15.103:0/1839948788 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c1a48d0 con 0x7ff16c101420 2024-09-19T07:55:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff1714df640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T07:55:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff16bfff640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff16c101420 0x7ff16c199850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff15800e9f0 tx=0x7ff15800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff169ffb640 1 -- 172.21.15.103:0/1839948788 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15800ccc0 con 0x7ff16c101420 2024-09-19T07:55:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c1a4aa0 con 0x7ff16c101420 2024-09-19T07:55:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff169ffb640 1 -- 172.21.15.103:0/1839948788 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff158005020 con 0x7ff16c101420 2024-09-19T07:55:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff169ffb640 1 -- 172.21.15.103:0/1839948788 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15801c070 con 0x7ff16c101420 2024-09-19T07:55:24.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.042+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c1a4fb0 con 0x7ff16c101420 2024-09-19T07:55:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.043+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16c100ee0 con 0x7ff16c101420 2024-09-19T07:55:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.044+0000 7ff169ffb640 1 -- 172.21.15.103:0/1839948788 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 138487+0+0 (secure 0 0 0) 0x7ff158005540 con 0x7ff16c101420 2024-09-19T07:55:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.045+0000 7ff169ffb640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff15c0a3430 0x7ff15c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T07:55:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.045+0000 7ff170cde640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff15c0a3430 0x7ff15c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T07:55:24.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.045+0000 7ff169ffb640 1 -- 172.21.15.103:0/1839948788 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8345+0+0 (secure 0 0 0) 0x7ff1580cfd20 con 0x7ff16c101420 2024-09-19T07:55:24.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.045+0000 7ff170cde640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff15c0a3430 0x7ff15c0a58f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff160000c00 tx=0x7ff160005a20 comp rx=0 tx=0).ready entity=mgr.15183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T07:55:24.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.049+0000 7ff169ffb640 1 -- 172.21.15.103:0/1839948788 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff15809c750 con 0x7ff16c101420 2024-09-19T07:55:24.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:23 smithi134 ceph-mon[35414]: pgmap v548: 97 pgs: 97 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-09-19T07:55:24.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:23 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/3269822935' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:55:24.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:23 smithi134 ceph-mon[35414]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T07:55:24.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:23 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T07:55:24.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:23 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T07:55:24.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:23 smithi134 ceph-mon[35414]: from='mgr.15183 172.21.15.103:0/1771071740' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T07:55:24.080 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:23 smithi134 ceph-mon[35414]: from='mgr.15183 ' entity='mgr.x' 2024-09-19T07:55:24.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.215+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff16c108ff0 con 0x7ff16c101420 2024-09-19T07:55:24.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.618+0000 7ff169ffb640 1 -- 172.21.15.103:0/1839948788 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7ff15809c050 con 0x7ff16c101420 2024-09-19T07:55:24.620 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' removed 2024-09-19T07:55:24.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff15c0a3430 msgr2=0x7ff15c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff15c0a3430 0x7ff15c0a58f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff160000c00 tx=0x7ff160005a20 comp rx=0 tx=0).stop 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff16c101420 msgr2=0x7ff16c199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff16c101420 0x7ff16c199850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff15800e9f0 tx=0x7ff15800eeb0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 shutdown_connections 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:6800/3066951043,v1:172.21.15.103:6801/3066951043] conn(0x7ff15c0a3430 0x7ff15c0a58f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff16c106a80 0x7ff16c1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff16c101420 0x7ff16c199850 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 --2- 172.21.15.103:0/1839948788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff16c100a80 0x7ff16c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.622+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 >> 172.21.15.103:0/1839948788 conn(0x7ff16c0fc7a0 msgr2=0x7ff16c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.623+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 shutdown_connections 2024-09-19T07:55:24.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-19T07:55:24.623+0000 7ff172f42640 1 -- 172.21.15.103:0/1839948788 wait complete. 2024-09-19T07:55:24.904 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:24 smithi103 ceph-mon[31201]: from='client.? 172.21.15.103:0/1839948788' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:24.905 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:24 smithi103 ceph-mon[31201]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:24.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:24 smithi174 ceph-mon[35370]: from='client.? 172.21.15.103:0/1839948788' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:24.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:24 smithi174 ceph-mon[35370]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:25.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:24 smithi134 ceph-mon[35414]: from='client.? 172.21.15.103:0/1839948788' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:25.079 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:24 smithi134 ceph-mon[35414]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T07:55:25.427 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:55:25.465 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:55:25.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:55:25.557 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T07:55:25.557 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:55:25.578 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:55:25.608 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T07:55:25.637 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T07:55:25.637 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T07:55:25.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a 2024-09-19T07:55:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:25 smithi103 ceph-mon[31201]: pgmap v550: 65 pgs: 65 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:55:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:25 smithi103 ceph-mon[31201]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:55:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:25 smithi103 ceph-mon[31201]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T07:55:25.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:25 smithi134 ceph-mon[35414]: pgmap v550: 65 pgs: 65 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:55:25.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:25 smithi134 ceph-mon[35414]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:55:25.829 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:25 smithi134 ceph-mon[35414]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T07:55:25.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:25 smithi174 ceph-mon[35370]: pgmap v550: 65 pgs: 65 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T07:55:25.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:25 smithi174 ceph-mon[35370]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T07:55:25.915 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:25 smithi174 ceph-mon[35370]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T07:55:25.970 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:25 smithi103 systemd[1]: Stopping Ceph mon.a for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:26.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a[31197]: 2024-09-19T07:55:26.017+0000 7fd22fced640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:55:26.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a[31197]: 2024-09-19T07:55:26.017+0000 7fd22fced640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T07:55:26.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:26 smithi103 podman[124209]: 2024-09-19 07:55:26.039790017 +0000 UTC m=+0.159734208 container died 4589aa5fd5c9e6403cc484e63e5942af58178eb9eed84a33c89c35e671fc416a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T07:55:26.696 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:26 smithi103 podman[124209]: 2024-09-19 07:55:26.43649737 +0000 UTC m=+0.556441556 container cleanup 4589aa5fd5c9e6403cc484e63e5942af58178eb9eed84a33c89c35e671fc416a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:55:26.696 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 19 07:55:26 smithi103 bash[124209]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-a 2024-09-19T07:55:26.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T07:55:26.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:26.962 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T07:55:26.962 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T07:55:26.962 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.b 2024-09-19T07:55:27.329 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:27 smithi134 systemd[1]: Stopping Ceph mon.b for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:27.936 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-b[35410]: 2024-09-19T07:55:27.681+0000 7f6e7aa5e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:55:27.936 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-b[35410]: 2024-09-19T07:55:27.681+0000 7f6e7aa5e640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T07:55:28.207 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:27 smithi134 podman[72340]: 2024-09-19 07:55:27.936591358 +0000 UTC m=+0.752808263 container died 8cb15dd05f85aa199f948b1537c2e8a130ac6180e569f7bb6173112da351e85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:55:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:28 smithi134 podman[72340]: 2024-09-19 07:55:28.306853811 +0000 UTC m=+1.123070787 container cleanup 8cb15dd05f85aa199f948b1537c2e8a130ac6180e569f7bb6173112da351e85e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T07:55:28.579 INFO:journalctl@ceph.mon.b.smithi134.stdout:Sep 19 07:55:28 smithi134 bash[72340]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-b 2024-09-19T07:55:29.095 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T07:55:29.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:29.144 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T07:55:29.144 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T07:55:29.144 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.c 2024-09-19T07:55:29.458 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:29 smithi174 systemd[1]: Stopping Ceph mon.c for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:29.843 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c[35366]: 2024-09-19T07:55:29.457+0000 7f4d5b743640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:55:29.843 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c[35366]: 2024-09-19T07:55:29.457+0000 7f4d5b743640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T07:55:29.843 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:29 smithi174 podman[72172]: 2024-09-19 07:55:29.550063953 +0000 UTC m=+0.215955818 container died aa0db486266247b03bbf226190a50d7a6569ddafbf3ca8c3947e307bb76f4b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-19T07:55:30.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:29 smithi174 podman[72172]: 2024-09-19 07:55:29.941606866 +0000 UTC m=+0.607498725 container cleanup aa0db486266247b03bbf226190a50d7a6569ddafbf3ca8c3947e307bb76f4b69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T07:55:30.165 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 19 07:55:29 smithi174 bash[72172]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mon-c 2024-09-19T07:55:30.573 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T07:55:30.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:30.623 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T07:55:30.623 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T07:55:30.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.x 2024-09-19T07:55:30.970 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:55:30 smithi103 systemd[1]: Stopping Ceph mgr.x for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:31.393 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:55:31 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x[31428]: 2024-09-19T07:55:31.041+0000 7f03ed7ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:55:31.394 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:55:31 smithi103 podman[124330]: 2024-09-19 07:55:31.07339333 +0000 UTC m=+0.260355070 container died a2584dd22f2bb1809b381c9e80076660755bff422046ad552b056bddb44d8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:55:31.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:55:31 smithi103 podman[124330]: 2024-09-19 07:55:31.602490153 +0000 UTC m=+0.789451886 container cleanup a2584dd22f2bb1809b381c9e80076660755bff422046ad552b056bddb44d8e72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-19T07:55:31.720 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 19 07:55:31 smithi103 bash[124330]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-x 2024-09-19T07:55:32.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T07:55:32.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:32.116 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T07:55:32.116 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T07:55:32.116 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.y 2024-09-19T07:55:32.476 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:55:32 smithi134 systemd[1]: Stopping Ceph mgr.y for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:32.829 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:55:32 smithi134 podman[72462]: 2024-09-19 07:55:32.476784957 +0000 UTC m=+0.156136739 container died e8078c7d2cfdd0b5c3cf62c8b4484691f5518594283b06e104d9974617d6469a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T07:55:34.079 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:55:33 smithi134 podman[72462]: 2024-09-19 07:55:33.905694987 +0000 UTC m=+1.585046778 container cleanup e8078c7d2cfdd0b5c3cf62c8b4484691f5518594283b06e104d9974617d6469a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-19T07:55:34.080 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:55:33 smithi134 bash[72462]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y 2024-09-19T07:55:34.674 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:55:34 smithi134 podman[72474]: 2024-09-19 07:55:34.416139015 +0000 UTC m=+1.939458323 container remove e8078c7d2cfdd0b5c3cf62c8b4484691f5518594283b06e104d9974617d6469a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:55:34.674 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Sep 19 07:55:34 smithi134 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-19T07:55:34.693 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T07:55:34.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:34.729 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T07:55:34.729 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T07:55:34.730 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.z 2024-09-19T07:55:35.019 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:55:34 smithi174 systemd[1]: Stopping Ceph mgr.z for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:35.286 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:55:35 smithi174 podman[72295]: 2024-09-19 07:55:35.060469566 +0000 UTC m=+0.137945100 container died 15ac4ad71a815f7d3ed43074b60cb42a6e952b2c1f88ab016fea79bc25630158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T07:55:35.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:55:35 smithi174 podman[72295]: 2024-09-19 07:55:35.488535355 +0000 UTC m=+0.566010871 container cleanup 15ac4ad71a815f7d3ed43074b60cb42a6e952b2c1f88ab016fea79bc25630158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-19T07:55:35.665 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:55:35 smithi174 bash[72295]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z 2024-09-19T07:55:35.915 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:55:35 smithi174 podman[72308]: 2024-09-19 07:55:35.664944293 +0000 UTC m=+0.603219807 container remove 15ac4ad71a815f7d3ed43074b60cb42a6e952b2c1f88ab016fea79bc25630158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T07:55:35.915 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 19 07:55:35 smithi174 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-19T07:55:35.919 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T07:55:35.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:35.956 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T07:55:35.956 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T07:55:35.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.0 2024-09-19T07:55:36.266 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:36 smithi103 systemd[1]: Stopping Ceph osd.0 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:36.720 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0[44902]: 2024-09-19T07:55:36.266+0000 7f0d4bd13640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:55:36.720 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0[44902]: 2024-09-19T07:55:36.266+0000 7f0d4bd13640 -1 osd.0 110 *** Got signal Terminated *** 2024-09-19T07:55:36.720 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0[44902]: 2024-09-19T07:55:36.266+0000 7f0d4bd13640 -1 osd.0 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:55:42.220 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:41 smithi103 podman[124453]: 2024-09-19 07:55:41.945837129 +0000 UTC m=+5.797315534 container died d3adbdcc2834d63794e7b3d49fd5374f9f3d473a21f4c0f71165aae316ac31f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:55:42.702 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:42 smithi103 podman[124453]: 2024-09-19 07:55:42.434373188 +0000 UTC m=+6.285851591 container cleanup d3adbdcc2834d63794e7b3d49fd5374f9f3d473a21f4c0f71165aae316ac31f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-19T07:55:42.702 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:42 smithi103 bash[124453]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0 2024-09-19T07:55:42.970 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:42 smithi103 podman[124471]: 2024-09-19 07:55:42.703005337 +0000 UTC m=+0.759155927 container remove d3adbdcc2834d63794e7b3d49fd5374f9f3d473a21f4c0f71165aae316ac31f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T07:55:43.306 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:43 smithi103 podman[124563]: 2024-09-19 07:55:42.951260752 +0000 UTC m=+0.016098974 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:55:43.306 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:43 smithi103 podman[124563]: 2024-09-19 07:55:43.0706054 +0000 UTC m=+0.135443615 container create 91d1a3a3605aeb476384f8255b2cf11892ecf317127f273fce226092ad640aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T07:55:43.720 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:43 smithi103 podman[124563]: 2024-09-19 07:55:43.30686728 +0000 UTC m=+0.371705498 container init 91d1a3a3605aeb476384f8255b2cf11892ecf317127f273fce226092ad640aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-19T07:55:43.720 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:43 smithi103 podman[124563]: 2024-09-19 07:55:43.310167845 +0000 UTC m=+0.375006059 container start 91d1a3a3605aeb476384f8255b2cf11892ecf317127f273fce226092ad640aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-19T07:55:43.720 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:43 smithi103 podman[124563]: 2024-09-19 07:55:43.415604462 +0000 UTC m=+0.480442688 container attach 91d1a3a3605aeb476384f8255b2cf11892ecf317127f273fce226092ad640aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T07:55:43.720 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 19 07:55:43 smithi103 podman[124563]: 2024-09-19 07:55:43.510859359 +0000 UTC m=+0.575697572 container died 91d1a3a3605aeb476384f8255b2cf11892ecf317127f273fce226092ad640aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-19T07:55:44.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T07:55:44.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:44.215 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T07:55:44.215 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T07:55:44.215 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.1 2024-09-19T07:55:44.519 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:44 smithi134 systemd[1]: Stopping Ceph osd.1 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:44.829 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1[39868]: 2024-09-19T07:55:44.518+0000 7eff234c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:55:44.829 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1[39868]: 2024-09-19T07:55:44.518+0000 7eff234c3640 -1 osd.1 110 *** Got signal Terminated *** 2024-09-19T07:55:44.829 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1[39868]: 2024-09-19T07:55:44.518+0000 7eff234c3640 -1 osd.1 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:55:50.353 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:50 smithi134 podman[72599]: 2024-09-19 07:55:50.098949628 +0000 UTC m=+5.679428473 container died 6f5682f49e887e7c4a949115c5827e0689227dac7cfe7f69688a12b0e39640f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:55:50.753 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:50 smithi134 podman[72599]: 2024-09-19 07:55:50.452158408 +0000 UTC m=+6.032637252 container cleanup 6f5682f49e887e7c4a949115c5827e0689227dac7cfe7f69688a12b0e39640f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T07:55:50.753 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:50 smithi134 bash[72599]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1 2024-09-19T07:55:51.079 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:50 smithi134 podman[72613]: 2024-09-19 07:55:50.753731839 +0000 UTC m=+0.659145704 container remove 6f5682f49e887e7c4a949115c5827e0689227dac7cfe7f69688a12b0e39640f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-19T07:55:51.398 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:51 smithi134 podman[72703]: 2024-09-19 07:55:51.013746989 +0000 UTC m=+0.015448848 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:55:51.399 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:51 smithi134 podman[72703]: 2024-09-19 07:55:51.146281645 +0000 UTC m=+0.147983503 container create 0041868686d324f6d521a68acf8bd035240f458a08d0cdc40df60e42b71095eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T07:55:51.399 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:51 smithi134 podman[72703]: 2024-09-19 07:55:51.399258684 +0000 UTC m=+0.400960542 container init 0041868686d324f6d521a68acf8bd035240f458a08d0cdc40df60e42b71095eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T07:55:51.829 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:51 smithi134 podman[72703]: 2024-09-19 07:55:51.40306148 +0000 UTC m=+0.404763344 container start 0041868686d324f6d521a68acf8bd035240f458a08d0cdc40df60e42b71095eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T07:55:51.830 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:51 smithi134 podman[72703]: 2024-09-19 07:55:51.507838686 +0000 UTC m=+0.509540558 container attach 0041868686d324f6d521a68acf8bd035240f458a08d0cdc40df60e42b71095eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T07:55:51.830 INFO:journalctl@ceph.osd.1.smithi134.stdout:Sep 19 07:55:51 smithi134 podman[72703]: 2024-09-19 07:55:51.593136885 +0000 UTC m=+0.594838744 container died 0041868686d324f6d521a68acf8bd035240f458a08d0cdc40df60e42b71095eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:55:52.250 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T07:55:52.289 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:55:52.289 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T07:55:52.289 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T07:55:52.289 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.2 2024-09-19T07:55:52.610 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:52 smithi174 systemd[1]: Stopping Ceph osd.2 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:55:52.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2[40354]: 2024-09-19T07:55:52.608+0000 7fcf527d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:55:52.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2[40354]: 2024-09-19T07:55:52.608+0000 7fcf527d4640 -1 osd.2 110 *** Got signal Terminated *** 2024-09-19T07:55:52.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2[40354]: 2024-09-19T07:55:52.608+0000 7fcf527d4640 -1 osd.2 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:55:58.644 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:58 smithi174 podman[72433]: 2024-09-19 07:55:58.32055301 +0000 UTC m=+5.832112623 container died 8bb3e274bf43da214e901486c549e1100c3a82e1cb0e9b8f795966a54ca64400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-19T07:55:58.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:58 smithi174 podman[72433]: 2024-09-19 07:55:58.743045453 +0000 UTC m=+6.254605069 container cleanup 8bb3e274bf43da214e901486c549e1100c3a82e1cb0e9b8f795966a54ca64400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T07:55:58.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:58 smithi174 bash[72433]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2 2024-09-19T07:55:59.349 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:58 smithi174 podman[72449]: 2024-09-19 07:55:58.94458539 +0000 UTC m=+0.622881076 container remove 8bb3e274bf43da214e901486c549e1100c3a82e1cb0e9b8f795966a54ca64400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T07:55:59.631 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:59 smithi174 podman[72543]: 2024-09-19 07:55:59.252700681 +0000 UTC m=+0.015713594 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:55:59.631 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:59 smithi174 podman[72543]: 2024-09-19 07:55:59.353651723 +0000 UTC m=+0.116664670 container create 78581846d63995efc2096e8f1018d678ec377943234321558d69c8451bc6cca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T07:55:59.631 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:59 smithi174 podman[72543]: 2024-09-19 07:55:59.631180102 +0000 UTC m=+0.394193007 container init 78581846d63995efc2096e8f1018d678ec377943234321558d69c8451bc6cca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T07:55:59.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:59 smithi174 podman[72543]: 2024-09-19 07:55:59.634894879 +0000 UTC m=+0.397907784 container start 78581846d63995efc2096e8f1018d678ec377943234321558d69c8451bc6cca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T07:55:59.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:59 smithi174 podman[72543]: 2024-09-19 07:55:59.739657971 +0000 UTC m=+0.502670892 container attach 78581846d63995efc2096e8f1018d678ec377943234321558d69c8451bc6cca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T07:55:59.915 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 19 07:55:59 smithi174 podman[72543]: 2024-09-19 07:55:59.819238581 +0000 UTC m=+0.582251495 container died 78581846d63995efc2096e8f1018d678ec377943234321558d69c8451bc6cca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T07:56:00.492 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T07:56:00.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:00.534 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T07:56:00.535 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T07:56:00.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.3 2024-09-19T07:56:00.857 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:00 smithi103 systemd[1]: Stopping Ceph osd.3 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:01.220 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3[51654]: 2024-09-19T07:56:00.856+0000 7f98d9b75640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:01.220 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3[51654]: 2024-09-19T07:56:00.856+0000 7f98d9b75640 -1 osd.3 110 *** Got signal Terminated *** 2024-09-19T07:56:01.220 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:00 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3[51654]: 2024-09-19T07:56:00.856+0000 7f98d9b75640 -1 osd.3 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:02.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:02.195+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:02.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:02 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:02.552+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:03.532 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:03 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:03.224+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:03.665 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:03.172+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:03.969 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:03 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:03.598+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:04.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:04.128+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:04.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:03.975+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:04.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:04 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:04.257+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:04.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:04 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:04.591+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:04.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:04 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:04.509+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:05.316 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:05.104+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:05.317 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:04.999+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:05.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:05 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:05.209+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:05.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:05 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:05.593+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:05.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:05 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:05.476+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:06.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:05.981+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:06.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:06.072+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:06.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:06 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:06.192+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:06.749 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:06 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:06.619+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:06.750 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:06 smithi103 podman[124677]: 2024-09-19 07:56:06.490802029 +0000 UTC m=+5.760909508 container died 4f58e3de90bf746d6c0be9d114515941cd315ae6857198f3dc0b86c0a8256c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:56:06.750 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:06 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:06.487+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:07.220 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:06 smithi103 podman[124677]: 2024-09-19 07:56:06.958061618 +0000 UTC m=+6.228169087 container cleanup 4f58e3de90bf746d6c0be9d114515941cd315ae6857198f3dc0b86c0a8256c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4) 2024-09-19T07:56:07.220 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:06 smithi103 bash[124677]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3 2024-09-19T07:56:07.417 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:07.059+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:07.417 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:06.982+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:07.569 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:07 smithi103 podman[124692]: 2024-09-19 07:56:07.226492372 +0000 UTC m=+0.738429897 container remove 4f58e3de90bf746d6c0be9d114515941cd315ae6857198f3dc0b86c0a8256c09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:56:07.569 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:07 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:07.459+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:07.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:07 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:07.146+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:07.905 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:07 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:07.588+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:07.905 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:07 smithi103 podman[124781]: 2024-09-19 07:56:07.569443421 +0000 UTC m=+0.107346818 container create b4a210bae1fde0734f7b8709e6f7aaf405023d9df4cd1a48f69c0904c4705146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:56:07.905 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:07 smithi103 podman[124781]: 2024-09-19 07:56:07.478063114 +0000 UTC m=+0.015966522 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:56:07.905 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:07 smithi103 podman[124781]: 2024-09-19 07:56:07.813756842 +0000 UTC m=+0.351660251 container init b4a210bae1fde0734f7b8709e6f7aaf405023d9df4cd1a48f69c0904c4705146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T07:56:07.905 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:07 smithi103 podman[124781]: 2024-09-19 07:56:07.817265746 +0000 UTC m=+0.355169147 container start b4a210bae1fde0734f7b8709e6f7aaf405023d9df4cd1a48f69c0904c4705146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T07:56:07.905 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:07 smithi103 podman[124781]: 2024-09-19 07:56:07.905514135 +0000 UTC m=+0.443417542 container attach b4a210bae1fde0734f7b8709e6f7aaf405023d9df4cd1a48f69c0904c4705146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:56:08.220 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 19 07:56:08 smithi103 podman[124781]: 2024-09-19 07:56:08.009154319 +0000 UTC m=+0.547057721 container died b4a210bae1fde0734f7b8709e6f7aaf405023d9df4cd1a48f69c0904c4705146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T07:56:08.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:08.050+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:08.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:07.992+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:08.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:08 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:08.144+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:08.609 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:08 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:08.458+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:08.626 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T07:56:08.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:08.668 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T07:56:08.668 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T07:56:08.668 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.4 2024-09-19T07:56:08.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:08 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:08.628+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:08.973 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:08 smithi134 systemd[1]: Stopping Ceph osd.4 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:09.329 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:09 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:09.149+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:09.329 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:08 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4[46757]: 2024-09-19T07:56:08.973+0000 7f88c4d76640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:09.329 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:08 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4[46757]: 2024-09-19T07:56:08.973+0000 7f88c4d76640 -1 osd.4 110 *** Got signal Terminated *** 2024-09-19T07:56:09.329 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:08 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4[46757]: 2024-09-19T07:56:08.973+0000 7f88c4d76640 -1 osd.4 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:09.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:09.087+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:09.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:09.025+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:09.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:09 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:09.604+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:09.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:09 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:09.486+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:10.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:10 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:10.060+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:10.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:09.989+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:10.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:10 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:10.169+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:10.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:10 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:10.495+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:10.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:10 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:10.594+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:11.184 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:10 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:10.853+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:11.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:11.086+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:11.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:11.001+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:11.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:11 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:11.184+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:11.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:11.478+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:11.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:11 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:11.636+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:12.079 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:11 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:11.828+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:12.129 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:11.951+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:12.129 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:11 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:11.740+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:12.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:12 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:12.126+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:12.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:12 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:12.186+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:12.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:12 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:12.453+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:12.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:12 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:12.613+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:13.079 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:12 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:12.818+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:13.089 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:12 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:12.766+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:13.090 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:12 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:12.918+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:13.090 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:13 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:13.089+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:13.562 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:13 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:13.190+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:13.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:13.447+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:13.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:13.447+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:13.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:13.579+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:13.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:13.579+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:13.829 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:13 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4[46757]: 2024-09-19T07:56:13.562+0000 7f88bd384640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.535005+0000 front 2024-09-19T07:55:47.535032+0000 (oldest deadline 2024-09-19T07:56:12.835112+0000) 2024-09-19T07:56:14.105 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:13 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:13.735+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:14.105 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:13 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:13.878+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:14.106 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:13 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:13.878+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:14.215 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:13 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:13.851+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:14.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:14.105+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:14.518 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:14.215+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:14.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:14.397+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:14.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:14.397+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:14.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:14.587+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:14.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:14 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:14.587+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:14.824 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:14 smithi134 podman[72818]: 2024-09-19 07:56:14.519098973 +0000 UTC m=+5.653144870 container died abaf9ff3e1d0ddfd69699badaae132c7e3c5e2427053a5e1858491b0d87450f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:56:15.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:14 smithi134 podman[72818]: 2024-09-19 07:56:14.915267148 +0000 UTC m=+6.049313044 container cleanup abaf9ff3e1d0ddfd69699badaae132c7e3c5e2427053a5e1858491b0d87450f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T07:56:15.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:14 smithi134 bash[72818]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4 2024-09-19T07:56:15.080 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:14 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:14.831+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:15.106 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:14.782+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:15.106 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:14.834+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:15.106 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:14 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:14.834+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:15.378 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:15.105+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:15.378 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:15.105+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:49.064261+0000 front 2024-09-19T07:55:49.064385+0000 (oldest deadline 2024-09-19T07:56:14.364173+0000) 2024-09-19T07:56:15.517 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:15 smithi134 podman[72835]: 2024-09-19 07:56:15.192069631 +0000 UTC m=+0.669201897 container remove abaf9ff3e1d0ddfd69699badaae132c7e3c5e2427053a5e1858491b0d87450f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T07:56:15.518 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:15 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:15.253+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:15.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:15.633+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:15.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:15.633+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:15.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:15.431+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:15.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:15 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:15.431+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:15.787 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:15 smithi134 podman[72927]: 2024-09-19 07:56:15.517988135 +0000 UTC m=+0.110274549 container create add5983dd755336b3812a3a0ad7d91f0074ac02676367c7c8cf45d19da956c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:56:15.787 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:15 smithi134 podman[72927]: 2024-09-19 07:56:15.423203108 +0000 UTC m=+0.015489466 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:56:16.070 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:15.841+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:16.070 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:15.841+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:16.070 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:15 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:15.738+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:16.079 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:15 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:15.870+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:16.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:15 smithi134 podman[72927]: 2024-09-19 07:56:15.787865705 +0000 UTC m=+0.380152067 container init add5983dd755336b3812a3a0ad7d91f0074ac02676367c7c8cf45d19da956c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T07:56:16.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:15 smithi134 podman[72927]: 2024-09-19 07:56:15.791282225 +0000 UTC m=+0.383568578 container start add5983dd755336b3812a3a0ad7d91f0074ac02676367c7c8cf45d19da956c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4) 2024-09-19T07:56:16.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:15 smithi134 podman[72927]: 2024-09-19 07:56:15.879332371 +0000 UTC m=+0.471618734 container attach add5983dd755336b3812a3a0ad7d91f0074ac02676367c7c8cf45d19da956c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:56:16.079 INFO:journalctl@ceph.osd.4.smithi134.stdout:Sep 19 07:56:15 smithi134 podman[72927]: 2024-09-19 07:56:15.972342808 +0000 UTC m=+0.564629161 container died add5983dd755336b3812a3a0ad7d91f0074ac02676367c7c8cf45d19da956c41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T07:56:16.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:16.069+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:16.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:16.069+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:49.064261+0000 front 2024-09-19T07:55:49.064385+0000 (oldest deadline 2024-09-19T07:56:14.364173+0000) 2024-09-19T07:56:16.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:16.285+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:16.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:16.285+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:16.669 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T07:56:16.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:16.709 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T07:56:16.709 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T07:56:16.709 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.5 2024-09-19T07:56:16.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:16 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:16.621+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:16.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:16 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:16.621+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:16.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:16 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:16.422+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:16.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:16 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:16.422+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:17.005 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:16 smithi174 systemd[1]: Stopping Ceph osd.5 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:17.006 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:16.786+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:17.006 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:16.874+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:17.006 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:16 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:16.874+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:17.269 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:16 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:16.909+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:17.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:17.004+0000 7fbf3d1c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:17.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:17.004+0000 7fbf3d1c8640 -1 osd.5 110 *** Got signal Terminated *** 2024-09-19T07:56:17.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:17.004+0000 7fbf3d1c8640 -1 osd.5 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:17.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:17.066+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:17.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:17.066+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:49.064261+0000 front 2024-09-19T07:55:49.064385+0000 (oldest deadline 2024-09-19T07:56:14.364173+0000) 2024-09-19T07:56:17.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:17.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:17.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:17.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:17.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:17.440+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:17.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:17.440+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:17.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:17.657+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:17.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:17 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:17.657+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:18.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:17.790+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:18.166 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:18 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:18.039+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:18.166 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:18 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:18.039+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:49.064261+0000 front 2024-09-19T07:55:49.064385+0000 (oldest deadline 2024-09-19T07:56:14.364173+0000) 2024-09-19T07:56:18.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:17.886+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:18.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:17 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:17.886+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:18.241 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:17 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:17.894+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:18.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:18 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:18.241+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:18.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:18 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:18.241+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:18.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:18 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:18.468+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:18.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:18 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:18.468+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:18.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:18 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:18.636+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:18.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:18 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:18.636+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:19.075 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:18 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:18.814+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:19.075 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:18 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:18.926+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:19.075 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:18 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:18.926+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:19.245 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:18 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:18.907+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:19.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:19 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:19.075+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:19.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:19 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:19.075+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:49.064261+0000 front 2024-09-19T07:55:49.064385+0000 (oldest deadline 2024-09-19T07:56:14.364173+0000) 2024-09-19T07:56:19.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:19 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:19.244+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:19.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:19 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:19.244+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:19.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:19 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:19.426+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:19.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:19 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:19.426+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:19.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:19 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:19.426+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:19.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:19 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:19.674+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:19.721 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:19 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:19.674+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:20.095 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:19 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:19.966+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:20.095 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:19 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:19.966+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:20.095 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:19 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:19.966+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:20.095 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:19 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:19.829+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:20.095 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:19 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:19.829+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:20.280 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:19 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:19.893+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:20.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:20 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:20.094+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:20.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:20 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:20.094+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:49.064261+0000 front 2024-09-19T07:55:49.064385+0000 (oldest deadline 2024-09-19T07:56:14.364173+0000) 2024-09-19T07:56:20.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:20 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:20.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:20.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:20 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:20.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:20.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:20 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:20.458+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:20.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:20 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:20.458+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:20.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:20 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:20.458+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:20.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:20 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:20.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:20.721 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:20 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:20.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:21.165 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:21.073+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:38.662125+0000 front 2024-09-19T07:55:38.662113+0000 (oldest deadline 2024-09-19T07:56:02.762111+0000) 2024-09-19T07:56:21.165 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:21.073+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:49.064261+0000 front 2024-09-19T07:55:49.064385+0000 (oldest deadline 2024-09-19T07:56:14.364173+0000) 2024-09-19T07:56:21.165 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5[46995]: 2024-09-19T07:56:21.073+0000 7fbf397de640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:57.165559+0000 front 2024-09-19T07:55:57.165538+0000 (oldest deadline 2024-09-19T07:56:20.665706+0000) 2024-09-19T07:56:21.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:20 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:20.873+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:21.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:20 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:20.873+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:21.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:20 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:20.973+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:21.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:20 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:20.973+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:21.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:20 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:20.973+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:21.254 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:20 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:20.884+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:21.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:21 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:21.254+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:21.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:21 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:21.254+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:21.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:21.422+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:21.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:21.422+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:21.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:21.422+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:21.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:21.669+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:21.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:21 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:21.669+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:22.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:21.853+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:22.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:21.853+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:22.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:21.951+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:22.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:21.951+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:22.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:21 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:21.951+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:22.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:21 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:21.922+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:22.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:22 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:22.277+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:22.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:22 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:22.277+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:22.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:22 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:22.277+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:22.679 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:22 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:22.414+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:22.679 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:22 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:22.414+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:22.679 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:22 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:22.414+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:22.965 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:22 smithi174 podman[72657]: 2024-09-19 07:56:22.673226245 +0000 UTC m=+5.768330990 container died cc251e5c3aec25f839f571d488c9e3e096fc72bc298a1bfd2142e9f55e3e885d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T07:56:22.965 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:22 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:22.835+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:22.966 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:22 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:22.835+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:22.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:22 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:22.679+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:22.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:22 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:22.679+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:23.248 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:22 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:22.936+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:23.349 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:23 smithi174 podman[72657]: 2024-09-19 07:56:23.121390202 +0000 UTC m=+6.216494952 container cleanup cc251e5c3aec25f839f571d488c9e3e096fc72bc298a1bfd2142e9f55e3e885d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:56:23.350 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:23 smithi174 bash[72657]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5 2024-09-19T07:56:23.350 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:22 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:22.964+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:23.350 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:22 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:22.964+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:23.350 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:22 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:22.964+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:23.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:23 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:23.248+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:23.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:23 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:23.248+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:23.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:23 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:23.248+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:23.665 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:23 smithi174 podman[72672]: 2024-09-19 07:56:23.349906185 +0000 UTC m=+0.676295751 container remove cc251e5c3aec25f839f571d488c9e3e096fc72bc298a1bfd2142e9f55e3e885d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:56:23.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:23 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:23.448+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:23.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:23 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:23.448+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:23.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:23 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:23.448+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:23.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:23 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:23.673+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:23.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:23 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:23.673+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:23.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:23 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:23.673+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:24.061 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:23 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:23.929+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:24.061 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:23 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:23.929+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:24.061 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:23 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:23.929+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:24.061 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:23 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:23.865+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:24.061 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:23 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:23.865+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:24.062 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:23 smithi174 podman[72761]: 2024-09-19 07:56:23.678783714 +0000 UTC m=+0.016099001 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:56:24.062 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:23 smithi174 podman[72761]: 2024-09-19 07:56:23.792460877 +0000 UTC m=+0.129776149 container create 6db869f3dbb6792caa5ec657b09ca4750ef94bc9b7461a961e35ffcb869000e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-19T07:56:24.062 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:24 smithi174 podman[72761]: 2024-09-19 07:56:24.061556074 +0000 UTC m=+0.398871347 container init 6db869f3dbb6792caa5ec657b09ca4750ef94bc9b7461a961e35ffcb869000e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:56:24.242 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:23 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:23.927+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:24.242 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:23 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:23.927+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:24.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:24 smithi174 podman[72761]: 2024-09-19 07:56:24.065016692 +0000 UTC m=+0.402331968 container start 6db869f3dbb6792caa5ec657b09ca4750ef94bc9b7461a961e35ffcb869000e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T07:56:24.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:24 smithi174 podman[72761]: 2024-09-19 07:56:24.170055136 +0000 UTC m=+0.507370423 container attach 6db869f3dbb6792caa5ec657b09ca4750ef94bc9b7461a961e35ffcb869000e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T07:56:24.415 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 19 07:56:24 smithi174 podman[72761]: 2024-09-19 07:56:24.256180873 +0000 UTC m=+0.593496156 container died 6db869f3dbb6792caa5ec657b09ca4750ef94bc9b7461a961e35ffcb869000e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:56:24.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:24 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:24.242+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:24.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:24 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:24.242+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:24.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:24 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:24.242+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:24.848 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T07:56:24.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:24.887 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T07:56:24.887 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T07:56:24.887 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.6 2024-09-19T07:56:24.915 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:24 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:24.662+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:24.915 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:24 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:24.662+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:24.915 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:24 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:24.662+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:24.916 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:24 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:24.493+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:24.916 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:24 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:24.493+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:24.916 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:24 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:24.493+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:25.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:24 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:24.816+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:25.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:24 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:24.816+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:25.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:24 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:24.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:25.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:24 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:24.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:25.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:24 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:24.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:25.204 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:24 smithi103 systemd[1]: Stopping Ceph osd.6 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:25.288 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:24 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:24.883+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:25.288 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:24 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:24.883+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:25.463 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:25.204+0000 7f4871cbb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:25.463 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:25.204+0000 7f4871cbb640 -1 osd.6 110 *** Got signal Terminated *** 2024-09-19T07:56:25.463 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:25.204+0000 7f4871cbb640 -1 osd.6 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:25.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:25 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:25.287+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:25.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:25 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:25.287+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:25.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:25 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:25.287+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:25.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:25.514+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:25.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:25.514+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:25.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:25.514+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:25.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:25.636+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:25.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:25.636+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:25.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:25 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:25.636+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:26.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:25 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:25.794+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:26.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:25 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:25.794+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:26.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:25 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:25.979+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:26.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:25 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:25.979+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:26.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:25 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:25.979+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:26.262 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:25 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:25.904+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:26.262 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:25 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:25.904+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:26.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:26 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:26.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:26.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:26 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:26.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:26.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:26 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:26.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:26.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:26.531+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:26.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:26.531+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:26.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:26.531+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:26.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:26.672+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:26.720 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:26.672+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:26.721 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:26 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:26.672+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:27.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:26 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:26.840+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:27.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:26 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:26.840+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:27.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:26 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:26.990+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:27.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:26 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:26.990+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:27.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:26 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:26.990+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:27.221 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:26 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:26.866+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:27.221 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:26 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:26.866+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:27.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:27.220+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:27.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:27.220+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:27.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:27.220+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:27.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:27.554+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:27.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:27.554+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:27.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:27.554+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:27.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:27.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:27.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:27.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:27.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:27.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:28.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:27.884+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:28.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:27.884+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:28.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:27.942+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:28.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:27.942+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:28.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:27 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:27.942+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:28.193 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:27.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:28.193 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:27.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:28.193 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:27 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:27.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:28.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:28 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:28.193+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:28.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:28 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:28.193+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:28.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:28 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:28.193+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:28.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:28.585+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:28.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:28.585+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:28.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:28.585+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:28.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:28.585+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.147882+0000 front 2024-09-19T07:56:02.147807+0000 (oldest deadline 2024-09-19T07:56:28.047956+0000) 2024-09-19T07:56:28.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:28.734+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:28.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:28.734+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:28.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:28.734+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:28.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:28 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:28.734+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:29.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:28 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:28.857+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:29.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:28 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:28.857+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:29.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:28 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:28.925+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:29.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:28 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:28.925+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:29.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:28 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:28.925+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:29.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:28 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:28.925+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:29.218 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:28 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:28.868+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:29.218 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:28 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:28.868+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:29.218 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:28 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:28.868+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:29.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:29 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:29.218+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:29.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:29 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:29.218+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:29.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:29 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:29.218+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:29.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:29 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:29.218+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:29.972 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:29.632+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.443767+0000 front 2024-09-19T07:55:39.443817+0000 (oldest deadline 2024-09-19T07:56:04.044706+0000) 2024-09-19T07:56:29.972 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:29.632+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.646077+0000 front 2024-09-19T07:55:48.646100+0000 (oldest deadline 2024-09-19T07:56:13.346126+0000) 2024-09-19T07:56:29.972 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:29.632+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.346859+0000 front 2024-09-19T07:55:53.346801+0000 (oldest deadline 2024-09-19T07:56:18.646832+0000) 2024-09-19T07:56:29.972 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6[58560]: 2024-09-19T07:56:29.632+0000 7f486e2d1640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.147882+0000 front 2024-09-19T07:56:02.147807+0000 (oldest deadline 2024-09-19T07:56:28.047956+0000) 2024-09-19T07:56:29.972 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:29.716+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:29.972 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:29.716+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:29.972 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:29.716+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:29.973 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:29 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:29.716+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:30.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:29.860+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:30.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:29.860+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:30.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:29.943+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:30.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:29.943+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:30.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:29.943+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:30.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:29 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:29.943+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:30.222 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:29 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:29.909+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:30.222 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:29 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:29.909+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:30.222 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:29 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:29.909+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:30.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:30.222+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:30.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:30.222+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:30.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:30.222+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:30.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:30.222+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:30.970 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:30 smithi103 podman[124884]: 2024-09-19 07:56:30.864729021 +0000 UTC m=+5.776245624 container died 87659a4385821d1d6021a9f786b1d84ba533ec83fd84f9e3d52543f75ebe95ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T07:56:30.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:30 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:30.687+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:30.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:30 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:30.687+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:30.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:30 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:30.687+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:30.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:30 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:30.687+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:31.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:30 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:30.829+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:31.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:30 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:30.829+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:31.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:30 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:30.829+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:31.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:30 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:30.973+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:31.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:30 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:30.973+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:31.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:30 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:30.973+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:31.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:30 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:30.973+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:31.260 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:30.882+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:31.260 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:30.882+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:31.260 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:30.882+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:31.469 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:31 smithi103 podman[124884]: 2024-09-19 07:56:31.372542593 +0000 UTC m=+6.284059236 container cleanup 87659a4385821d1d6021a9f786b1d84ba533ec83fd84f9e3d52543f75ebe95ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T07:56:31.470 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:31 smithi103 bash[124884]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6 2024-09-19T07:56:31.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:31 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:31.260+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:31.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:31 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:31.260+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:31.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:31 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:31.260+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:31.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:31 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:31.260+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:31.935 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:31 smithi103 podman[124911]: 2024-09-19 07:56:31.607466652 +0000 UTC m=+0.744288566 container remove 87659a4385821d1d6021a9f786b1d84ba533ec83fd84f9e3d52543f75ebe95ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:56:31.936 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:31 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:31.693+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:31.936 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:31 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:31.693+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:31.936 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:31 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:31.693+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:31.936 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:31 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:31.693+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:32.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:31 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:31.833+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:32.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:31 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:31.833+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:32.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:31 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:31.833+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:32.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:31 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:31.949+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:32.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:31 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:31.949+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:32.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:31 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:31.949+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:32.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:31 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:31.949+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:32.220 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:31 smithi103 podman[124998]: 2024-09-19 07:56:31.838378053 +0000 UTC m=+0.015684220 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:56:32.220 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:31 smithi103 podman[124998]: 2024-09-19 07:56:31.963076687 +0000 UTC m=+0.140382840 container create 372ec406ecc8b25e4edc6c873b5f8a4ca99d0bbd547f787172b0b3e517e413a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T07:56:32.287 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:31 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:31.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:32.288 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:31 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:31.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:32.288 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:31 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:31.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:32.561 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:32 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:32.288+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:32.561 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:32 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:32.288+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:32.561 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:32 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:32.288+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:32.561 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:32 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:32.288+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:32.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:32 smithi103 podman[124998]: 2024-09-19 07:56:32.236838539 +0000 UTC m=+0.414144698 container init 372ec406ecc8b25e4edc6c873b5f8a4ca99d0bbd547f787172b0b3e517e413a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T07:56:32.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:32 smithi103 podman[124998]: 2024-09-19 07:56:32.240226314 +0000 UTC m=+0.417532470 container start 372ec406ecc8b25e4edc6c873b5f8a4ca99d0bbd547f787172b0b3e517e413a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T07:56:32.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:32 smithi103 podman[124998]: 2024-09-19 07:56:32.328311089 +0000 UTC m=+0.505617251 container attach 372ec406ecc8b25e4edc6c873b5f8a4ca99d0bbd547f787172b0b3e517e413a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-19T07:56:32.720 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 19 07:56:32 smithi103 podman[124998]: 2024-09-19 07:56:32.425970357 +0000 UTC m=+0.603276521 container died 372ec406ecc8b25e4edc6c873b5f8a4ca99d0bbd547f787172b0b3e517e413a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T07:56:33.104 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:32 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:32.722+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:33.104 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:32 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:32.722+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:33.104 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:32 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:32.722+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:33.104 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:32 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:32.722+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:33.128 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T07:56:33.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:32 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:32.877+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:33.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:32 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:32.877+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:33.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:32 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:32.877+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:33.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:32 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:32.907+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:33.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:32 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:32.907+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:33.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:32 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:32.907+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:33.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:32 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:32.907+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:33.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:33.214 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T07:56:33.215 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T07:56:33.215 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.7 2024-09-19T07:56:33.238 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:32 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:32.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:33.238 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:32 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:32.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:33.238 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:32 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:32.840+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:33.528 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:33 smithi134 systemd[1]: Stopping Ceph osd.7 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:33.528 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:33.526+0000 7f8c2db90640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:33.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:33.252+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:33.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:33.252+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:33.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:33.252+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:33.528 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:33.252+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:33.829 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:33.526+0000 7f8c2db90640 -1 osd.7 110 *** Got signal Terminated *** 2024-09-19T07:56:33.829 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:33.526+0000 7f8c2db90640 -1 osd.7 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:34.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:33 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:33.856+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:34.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:33 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:33.856+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:34.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:33 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:33.856+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:34.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:33 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:33.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:34.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:33 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:33.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:34.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:33 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:33.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:34.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:33 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:33.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:34.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:33 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:33.759+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:34.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:33 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:33.759+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:34.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:33 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:33.759+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:34.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:33 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:33.759+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:34.269 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:33.857+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:34.269 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:33.857+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:34.269 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:33 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:33.857+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:34.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:34.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:34.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:34.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:34.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:34.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:34.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:34.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:35.066 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:34 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:34.815+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:35.066 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:34 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:34.815+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:35.067 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:34 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:34.815+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:35.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:34 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:34.860+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:35.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:34 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:34.860+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:35.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:34 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:34.860+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:35.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:34 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:34.860+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:35.079 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:34.821+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:35.079 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:34.821+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:35.079 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:34.821+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:35.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:34 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:34.760+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:35.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:34 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:34.760+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:35.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:34 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:34.760+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:35.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:34 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:34.760+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:35.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:35.244+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:35.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:35.244+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:35.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:35.244+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:35.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:35.244+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:36.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:35 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:35.765+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:36.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:35 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:35.765+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:36.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:35 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:35.765+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:36.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:35 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:35.875+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:36.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:35 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:35.875+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:36.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:35 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:35.875+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:36.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:35 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:35.875+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:36.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:35 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:35.738+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:36.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:35 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:35.738+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:36.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:35 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:35.738+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:36.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:35 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:35.738+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:36.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:35 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:35.738+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:36.253 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:35.841+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:36.254 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:35.841+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:36.254 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:35 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:35.841+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:36.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:36 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:36.253+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:36.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:36 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:36.253+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:36.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:36 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:36.253+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:36.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:36 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:36.253+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:36.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:36.729+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:36.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:36.729+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:36.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:36.729+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:36.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:36.729+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:36.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:36 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:36.729+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:37.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:36 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:36.782+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:37.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:36 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:36.782+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:37.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:36 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:36.782+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:37.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:36 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:36.901+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:37.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:36 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:36.901+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:37.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:36 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:36.901+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:37.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:36 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:36.901+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:37.213 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:36 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:36.868+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:37.214 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:36 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:36.868+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:37.214 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:36 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:36.868+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:37.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:37.213+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:37.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:37.213+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:37.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:37.213+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:37.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:37.213+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:37.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:37.213+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:38.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:37.791+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:38.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:37.791+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:38.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:37.791+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:38.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:37.871+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:38.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:37.871+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:38.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:37.871+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:38.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:37.871+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:38.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:37 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:37.871+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:38.212 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:37.905+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.900084+0000 front 2024-09-19T07:55:47.900025+0000 (oldest deadline 2024-09-19T07:56:10.800251+0000) 2024-09-19T07:56:38.213 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:37.905+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:54.901209+0000 front 2024-09-19T07:55:54.901114+0000 (oldest deadline 2024-09-19T07:56:23.101664+0000) 2024-09-19T07:56:38.213 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:37 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7[53396]: 2024-09-19T07:56:37.905+0000 7f8c2a1a6640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:04.202659+0000 front 2024-09-19T07:56:04.202770+0000 (oldest deadline 2024-09-19T07:56:27.102611+0000) 2024-09-19T07:56:38.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:37.758+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:38.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:37.758+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:38.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:37.758+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:38.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:37.758+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:38.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:37 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:37.758+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:38.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:38 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:38.212+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:38.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:38 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:38.212+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:38.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:38 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:38.212+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:38.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:38 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:38.212+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:38.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:38 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:38.212+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:38.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:38.715+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:38.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:38.715+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:38.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:38.715+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:38.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:38.715+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:38.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:38 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:38.715+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:39.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:38.834+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:39.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:38.834+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:39.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:38.834+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:39.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:38.834+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:39.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:38.825+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:39.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:38.825+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:39.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:38.825+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:39.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:38.825+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:39.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:38 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:38.825+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:39.536 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:39 smithi134 podman[73040]: 2024-09-19 07:56:39.217047508 +0000 UTC m=+5.813324882 container died 989b72fa0d3203d327b741aba89f72b328fa1a5d3a010dd3042289a18b90ab29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:56:39.536 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:39 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:39.185+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:39.536 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:39 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:39.185+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:39.537 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:39 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:39.185+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:39.537 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:39 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:39.185+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:39.537 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:39 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:39.185+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:39.829 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:39 smithi134 podman[73040]: 2024-09-19 07:56:39.619388505 +0000 UTC m=+6.215665869 container cleanup 989b72fa0d3203d327b741aba89f72b328fa1a5d3a010dd3042289a18b90ab29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:56:39.829 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:39 smithi134 bash[73040]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7 2024-09-19T07:56:40.151 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:39 smithi134 podman[73057]: 2024-09-19 07:56:39.871685438 +0000 UTC m=+0.655636631 container remove 989b72fa0d3203d327b741aba89f72b328fa1a5d3a010dd3042289a18b90ab29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T07:56:40.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:39.825+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:40.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:39.825+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:40.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:39.825+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:40.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:39.825+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:40.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:39.798+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:40.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:39.798+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:40.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:39.798+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:40.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:39.798+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:40.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:39 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:39.798+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:40.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:39 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:39.720+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:40.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:39 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:39.720+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:40.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:39 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:39.720+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:40.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:39 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:39.720+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:40.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:39 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:39.720+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:40.429 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:40 smithi134 podman[73148]: 2024-09-19 07:56:40.190278346 +0000 UTC m=+0.015806340 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:56:40.430 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:40 smithi134 podman[73148]: 2024-09-19 07:56:40.313378596 +0000 UTC m=+0.138906641 container create 5fb99c14e3da1b39dcda746eb0e4ba63b7bf290625ff26f68ad83278065dca11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T07:56:40.430 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:40 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:40.151+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:40.430 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:40 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:40.151+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:40.430 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:40 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:40.151+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:40.430 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:40 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:40.151+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:40.430 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:40 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:40.151+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:40.751 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:40 smithi134 podman[73148]: 2024-09-19 07:56:40.558000044 +0000 UTC m=+0.383528023 container init 5fb99c14e3da1b39dcda746eb0e4ba63b7bf290625ff26f68ad83278065dca11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T07:56:40.751 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:40 smithi134 podman[73148]: 2024-09-19 07:56:40.561625347 +0000 UTC m=+0.387153336 container start 5fb99c14e3da1b39dcda746eb0e4ba63b7bf290625ff26f68ad83278065dca11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T07:56:40.751 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:40 smithi134 podman[73148]: 2024-09-19 07:56:40.651369318 +0000 UTC m=+0.476897304 container attach 5fb99c14e3da1b39dcda746eb0e4ba63b7bf290625ff26f68ad83278065dca11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T07:56:40.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:40.691+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:40.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:40.691+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:40.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:40.691+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:40.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:40.691+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:40.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:40 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:40.691+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:41.053 INFO:journalctl@ceph.osd.7.smithi134.stdout:Sep 19 07:56:40 smithi134 podman[73148]: 2024-09-19 07:56:40.751850238 +0000 UTC m=+0.577378231 container died 5fb99c14e3da1b39dcda746eb0e4ba63b7bf290625ff26f68ad83278065dca11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-19T07:56:41.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:40.789+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:41.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:40.789+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:41.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:40.789+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:41.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:40.789+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:41.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:40.795+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:41.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:40.795+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:41.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:40.795+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:41.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:40.795+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:41.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:40 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:40.795+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:41.329 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:41 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:41.175+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:41.329 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:41 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:41.175+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:41.329 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:41 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:41.175+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:41.329 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:41 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:41.175+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:41.330 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:41 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:41.175+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:41.403 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T07:56:41.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:41.484 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T07:56:41.484 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T07:56:41.485 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.8 2024-09-19T07:56:41.762 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 systemd[1]: Stopping Ceph osd.8 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:41.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:41.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:41.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:41.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:41.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:41.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:41.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:41.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:41.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:41 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:41.713+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:42.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:41.831+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:42.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:41.831+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:42.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:41.831+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:42.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:41.831+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:42.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:41.831+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:42.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:41.762+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:42.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:41.762+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:42.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:41.762+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:42.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:41.762+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:42.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:41.810+0000 7f5b4d0c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:42.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:41.810+0000 7f5b4d0c7640 -1 osd.8 110 *** Got signal Terminated *** 2024-09-19T07:56:42.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:41 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:41.810+0000 7f5b4d0c7640 -1 osd.8 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:42.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:42 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:42.211+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:42.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:42 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:42.211+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:42.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:42 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:42.211+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:42.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:42 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:42.211+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:42.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:42 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:42.211+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:42.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:42 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:42.677+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:42.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:42 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:42.677+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:42.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:42 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:42.677+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:42.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:42 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:42.677+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:42.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:42 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:42.677+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:43.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:42.745+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:43.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:42.745+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:43.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:42.745+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:43.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:42.745+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:43.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:42.879+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:43.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:42.879+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:43.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:42.879+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:43.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:42.879+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:43.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:42 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:42.879+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:43.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:43 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:43.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:43.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:43 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:43.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:43.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:43 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:43.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:43.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:43 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:43.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:43.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:43 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:43.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:43.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:43 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:43.667+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:43.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:43 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:43.667+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:43.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:43 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:43.667+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:43.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:43 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:43.667+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:43.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:43 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:43.667+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:44.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:43.733+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:44.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:43.733+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:43.733+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:43.733+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:43.845+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:43.845+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:43.845+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:43.845+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:43.845+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:44.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:43 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:43.845+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:44.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:44.255+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:44.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:44.255+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:44.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:44.255+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:44.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:44.255+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:44.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:44 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:44.255+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:44.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:44 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:44.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:44.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:44 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:44.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:44.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:44 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:44.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:44.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:44 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:44.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:44.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:44 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:44.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:44.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:44 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:44.653+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:45.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:44.732+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:45.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:44.732+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:45.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:44.732+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:45.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:44.732+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:45.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:44.814+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:45.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:44.814+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:45.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:44.814+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:45.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:44.814+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:45.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:44.814+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:45.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:44 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:44.814+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:45.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:45 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:45.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:45.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:45 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:45.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:45.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:45 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:45.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:45.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:45 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:45.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:45.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:45 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:45.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:45.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:45 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:45.262+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:45.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:45 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:45.624+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:45.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:45 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:45.624+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:45.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:45 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:45.624+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:45.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:45 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:45.624+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:45.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:45 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:45.624+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:45.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:45 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:45.624+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:46.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:45.769+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:46.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:45.769+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:46.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:45.769+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:46.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:45.769+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:46.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:45.769+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.387446+0000 front 2024-09-19T07:56:19.387498+0000 (oldest deadline 2024-09-19T07:56:45.287498+0000) 2024-09-19T07:56:46.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:45.802+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:46.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:45.802+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:46.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:45.802+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:46.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:45.802+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:46.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:45.802+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:46.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:45 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:45.802+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:46.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:46 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:46.291+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:46.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:46 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:46.291+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:46.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:46 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:46.291+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:46.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:46 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:46.291+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:46.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:46 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:46.291+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:46.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:46 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:46.291+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:46.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:46 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:46.583+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:46.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:46 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:46.583+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:46.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:46 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:46.583+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:46.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:46 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:46.583+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:46.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:46 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:46.583+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:46.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:46 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:46.583+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:47.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:46.761+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.982182+0000 front 2024-09-19T07:55:47.982168+0000 (oldest deadline 2024-09-19T07:56:10.782887+0000) 2024-09-19T07:56:47.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:46.761+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.683640+0000 front 2024-09-19T07:55:53.683701+0000 (oldest deadline 2024-09-19T07:56:18.983797+0000) 2024-09-19T07:56:47.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:46.761+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.485023+0000 front 2024-09-19T07:56:02.485038+0000 (oldest deadline 2024-09-19T07:56:30.685334+0000) 2024-09-19T07:56:47.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:46.761+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.685841+0000 front 2024-09-19T07:56:10.685959+0000 (oldest deadline 2024-09-19T07:56:38.286525+0000) 2024-09-19T07:56:47.166 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8[53473]: 2024-09-19T07:56:46.761+0000 7f5b456d5640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.387446+0000 front 2024-09-19T07:56:19.387498+0000 (oldest deadline 2024-09-19T07:56:45.287498+0000) 2024-09-19T07:56:47.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:46.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:47.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:46.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:47.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:46.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:47.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:46.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:47.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:46.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:47.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:46.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:47.543 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:47 smithi174 podman[72876]: 2024-09-19 07:56:47.274325591 +0000 UTC m=+5.593165337 container died 75e322fed0246d055a90a8a3fd49efe57684e598eb7648b050185320ff25e703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:56:47.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:47 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:47.308+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:47.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:47 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:47.308+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:47.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:47 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:47.308+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:47.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:47 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:47.308+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:47.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:47 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:47.308+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:47.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:47 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:47.308+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:47.858 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:47 smithi174 podman[72876]: 2024-09-19 07:56:47.642436941 +0000 UTC m=+5.961276676 container cleanup 75e322fed0246d055a90a8a3fd49efe57684e598eb7648b050185320ff25e703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4) 2024-09-19T07:56:47.858 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:47 smithi174 bash[72876]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8 2024-09-19T07:56:47.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:47.616+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:47.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:47.616+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:47.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:47.616+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:47.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:47.616+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:47.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:47.616+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:47.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:47 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:47.616+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:48.165 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:47 smithi174 podman[72894]: 2024-09-19 07:56:47.927011049 +0000 UTC m=+0.649429826 container remove 75e322fed0246d055a90a8a3fd49efe57684e598eb7648b050185320ff25e703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T07:56:48.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:47 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:47.857+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:48.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:47 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:47.857+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:48.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:47 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:47.857+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:48.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:47 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:47.857+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:48.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:47 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:47.857+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:48.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:47 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:47.857+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:48.530 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:48 smithi174 podman[72985]: 2024-09-19 07:56:48.162862248 +0000 UTC m=+0.015547188 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:56:48.530 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:48 smithi174 podman[72985]: 2024-09-19 07:56:48.261031197 +0000 UTC m=+0.113716126 container create 07b80c8a22ca956c71c4e1f9a8ef4976d66e6aa220d8a6b22d056a80dc325eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T07:56:48.530 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:48 smithi174 podman[72985]: 2024-09-19 07:56:48.421993104 +0000 UTC m=+0.274678047 container init 07b80c8a22ca956c71c4e1f9a8ef4976d66e6aa220d8a6b22d056a80dc325eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T07:56:48.530 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:48 smithi174 podman[72985]: 2024-09-19 07:56:48.425346343 +0000 UTC m=+0.278031279 container start 07b80c8a22ca956c71c4e1f9a8ef4976d66e6aa220d8a6b22d056a80dc325eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T07:56:48.531 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:48 smithi174 podman[72985]: 2024-09-19 07:56:48.530287061 +0000 UTC m=+0.382972008 container attach 07b80c8a22ca956c71c4e1f9a8ef4976d66e6aa220d8a6b22d056a80dc325eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T07:56:48.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:48 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:48.316+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:48.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:48 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:48.316+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:48.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:48 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:48.316+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:48.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:48 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:48.316+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:48.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:48 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:48.316+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:48.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:48 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:48.316+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:48.899 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 19 07:56:48 smithi174 podman[72985]: 2024-09-19 07:56:48.612842225 +0000 UTC m=+0.465527169 container died 07b80c8a22ca956c71c4e1f9a8ef4976d66e6aa220d8a6b22d056a80dc325eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T07:56:48.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:48.612+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:48.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:48.612+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:48.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:48.612+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:48.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:48.612+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:48.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:48.612+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:48.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:48 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:48.612+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:49.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:48 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:48.902+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:49.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:48 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:48.902+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:49.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:48 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:48.902+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:49.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:48 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:48.902+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:49.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:48 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:48.902+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:49.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:48 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:48.902+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:49.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T07:56:49.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:49.254 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T07:56:49.254 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T07:56:49.254 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.9 2024-09-19T07:56:49.565 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 systemd[1]: Stopping Ceph osd.9 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:49.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:49 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:49.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:49.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:49 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:49.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:49.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:49 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:49.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:49.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:49 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:49.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:49.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:49 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:49.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:49.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:49 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:49.269+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:49.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.564+0000 7f50b310f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:49.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.564+0000 7f50b310f640 -1 osd.9 110 *** Got signal Terminated *** 2024-09-19T07:56:49.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.564+0000 7f50b310f640 -1 osd.9 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:49.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.635+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:49.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.635+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:49.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.635+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:49.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.635+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:49.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.635+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:49.972 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:49 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:49.635+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:50.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:49 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:49.900+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:50.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:49 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:49.900+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:50.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:49 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:49.900+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:50.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:49 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:49.900+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:50.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:49 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:49.900+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:50.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:49 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:49.900+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:50.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:50 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:50.306+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:50.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:50 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:50.306+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:50.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:50 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:50.306+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:50.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:50 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:50.306+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:50.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:50 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:50.306+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:50.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:50 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:50.306+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:50.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:50.608+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:50.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:50.608+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:50.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:50.608+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:50.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:50.608+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:50.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:50.608+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:50.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:50 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:50.608+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:51.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:50.940+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:51.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:50.940+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:51.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:50.940+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:51.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:50.940+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:51.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:50.940+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:51.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:50.940+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:51.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:51.320+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:51.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:51.320+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:51.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:51.320+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:51.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:51.320+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:51.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:51.320+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:51.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:51 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:51.320+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:51.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:51.658+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:51.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:51.658+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:51.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:51.658+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:51.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:51.658+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:51.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:51.658+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:51.971 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:51 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:51.658+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:52.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:51 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:51.924+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:52.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:51 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:51.924+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:52.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:51 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:51.924+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:52.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:51 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:51.924+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:52.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:51 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:51.924+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:52.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:51 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:51.924+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:52.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:52 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:52.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:52.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:52 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:52.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:52.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:52 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:52.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:52.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:52 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:52.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:52.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:52 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:52.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:52.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:52 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:52.280+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:52.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:52.620+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:52.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:52.620+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:52.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:52.620+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:52.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:52.620+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:52.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:52.620+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:52.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:52 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:52.620+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:53.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:52.963+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:53.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:52.963+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:53.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:52.963+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:53.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:52.963+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:53.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:52.963+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:53.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:52 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:52.963+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:53.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:53.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:53.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:53.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:53.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:53.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:53.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:53.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:53.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:53.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:53.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:53 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:53.249+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:53.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:53.596+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.333981+0000 front 2024-09-19T07:55:39.334013+0000 (oldest deadline 2024-09-19T07:56:02.234134+0000) 2024-09-19T07:56:53.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:53.596+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:48.135314+0000 front 2024-09-19T07:55:48.135125+0000 (oldest deadline 2024-09-19T07:56:13.435101+0000) 2024-09-19T07:56:53.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:53.596+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:56.336073+0000 front 2024-09-19T07:55:56.336148+0000 (oldest deadline 2024-09-19T07:56:22.736696+0000) 2024-09-19T07:56:53.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:53.596+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:02.737064+0000 front 2024-09-19T07:56:02.737111+0000 (oldest deadline 2024-09-19T07:56:28.037171+0000) 2024-09-19T07:56:53.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:53.596+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:09.738111+0000 front 2024-09-19T07:56:09.738087+0000 (oldest deadline 2024-09-19T07:56:35.038072+0000) 2024-09-19T07:56:53.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:53 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9[65514]: 2024-09-19T07:56:53.596+0000 7f50af725640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:19.739262+0000 front 2024-09-19T07:56:19.739163+0000 (oldest deadline 2024-09-19T07:56:43.839061+0000) 2024-09-19T07:56:54.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:53.918+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:54.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:53.918+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:54.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:53.918+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:54.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:53.918+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:54.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:53.918+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:54.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:53 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:53.918+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:54.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:54.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:54.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:54.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:54.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:54.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:54.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:54.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:54.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:54.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:54.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:54.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:54.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:54 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:54.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:56:55.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:54.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:55.415 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:54.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:55.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:54.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:55.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:54.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:55.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:54.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:55.416 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:54.931+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:55.470 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:55 smithi103 podman[125100]: 2024-09-19 07:56:55.20667393 +0000 UTC m=+5.757964820 container died c9628ac5a5b9a065a5c546513d5a1db28c2823f03dfca3958faaadb140c9267a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T07:56:55.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:55.289+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:55.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:55.289+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:55.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:55.289+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:55.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:55.289+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:55.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:55.289+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:55.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:55.289+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:55.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:55 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:55.289+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:56:55.859 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:55 smithi103 podman[125100]: 2024-09-19 07:56:55.632611362 +0000 UTC m=+6.183902298 container cleanup c9628ac5a5b9a065a5c546513d5a1db28c2823f03dfca3958faaadb140c9267a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-19T07:56:55.859 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:55 smithi103 bash[125100]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9 2024-09-19T07:56:56.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:55.898+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:56.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:55.898+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:56.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:55.898+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:56.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:55.898+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:56.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:55.898+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:56.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:55 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:55.898+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:56.220 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:55 smithi103 podman[125116]: 2024-09-19 07:56:55.85920767 +0000 UTC m=+0.644419988 container remove c9628ac5a5b9a065a5c546513d5a1db28c2823f03dfca3958faaadb140c9267a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:56:56.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:56.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:56.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:56.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:56.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:56.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:56.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:56.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:56.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:56.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:56.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:56.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:56.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:56 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:56.296+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:56:56.620 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:56 smithi103 podman[125208]: 2024-09-19 07:56:56.192820666 +0000 UTC m=+0.015691639 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:56:56.621 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:56 smithi103 podman[125208]: 2024-09-19 07:56:56.301823075 +0000 UTC m=+0.124694030 container create 9ad17c13d8d177cc1cad2b4c225f484dae9d76bde86081a37aa1a59ba236be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T07:56:56.621 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:56 smithi103 podman[125208]: 2024-09-19 07:56:56.520831672 +0000 UTC m=+0.343702627 container init 9ad17c13d8d177cc1cad2b4c225f484dae9d76bde86081a37aa1a59ba236be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T07:56:56.621 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:56 smithi103 podman[125208]: 2024-09-19 07:56:56.524091594 +0000 UTC m=+0.346962551 container start 9ad17c13d8d177cc1cad2b4c225f484dae9d76bde86081a37aa1a59ba236be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-19T07:56:56.621 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:56 smithi103 podman[125208]: 2024-09-19 07:56:56.620976056 +0000 UTC m=+0.443847023 container attach 9ad17c13d8d177cc1cad2b4c225f484dae9d76bde86081a37aa1a59ba236be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T07:56:56.970 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 19 07:56:56 smithi103 podman[125208]: 2024-09-19 07:56:56.708307272 +0000 UTC m=+0.531178244 container died 9ad17c13d8d177cc1cad2b4c225f484dae9d76bde86081a37aa1a59ba236be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T07:56:57.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:56.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:57.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:56.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:57.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:56.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:57.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:56.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:57.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:56.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:57.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:56 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:56.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:57.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T07:56:57.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:56:57.316 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T07:56:57.317 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T07:56:57.317 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.10 2024-09-19T07:56:57.614 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.338+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.338+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.338+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.338+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.338+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.338+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.338+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 systemd[1]: Stopping Ceph osd.10 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.614+0000 7fe47350b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.614+0000 7fe47350b640 -1 osd.10 110 *** Got signal Terminated *** 2024-09-19T07:56:57.615 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:57 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:57.614+0000 7fe47350b640 -1 osd.10 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:56:58.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:57.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:58.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:57.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:58.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:57.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:58.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:57.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:58.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:57.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:58.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:57 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:57.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:58.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:58.298+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:58.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:58.298+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:58.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:58.298+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:58.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:58.298+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:58.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:58.298+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:58.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:58.298+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:58.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:58 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:58.298+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:56:59.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:58.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:56:59.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:58.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:56:59.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:58.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:56:59.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:58.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:56:59.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:58.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:56:59.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:58.883+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:56:59.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:59 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:59.318+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:56:59.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:59 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:59.318+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:56:59.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:59 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:59.318+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:56:59.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:59 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:59.318+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:56:59.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:59 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:59.318+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:56:59.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:59 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:59.318+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:56:59.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:56:59 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:56:59.318+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:57:00.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:59 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:59.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:00.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:59 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:59.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:00.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:59 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:59.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:00.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:59 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:59.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:00.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:59 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:59.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:00.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:56:59 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:56:59.843+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:00.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:00 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:00.311+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:57:00.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:00 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:00.311+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:57:00.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:00 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:00.311+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:57:00.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:00 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:00.311+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:57:00.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:00 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:00.311+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:57:00.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:00 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:00.311+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:57:00.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:00 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:00.311+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:57:01.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:00.881+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:01.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:00.881+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:01.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:00.881+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:01.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:00.881+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:01.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:00.881+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:01.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:00 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:00.881+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:01.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:01.263+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:57:01.579 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:01.263+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:57:01.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:01.263+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:57:01.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:01.263+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:57:01.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:01.263+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:57:01.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:01.263+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:57:01.580 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:01 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:01.263+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:57:02.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:01 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:01.847+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:02.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:01 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:01.847+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:02.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:01 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:01.847+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:02.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:01 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:01.847+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:02.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:01 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:01.847+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:02.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:01 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:01.847+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:02.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:01 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:01.847+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:02.581 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:02.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.110204+0000 front 2024-09-19T07:55:39.110252+0000 (oldest deadline 2024-09-19T07:56:02.010402+0000) 2024-09-19T07:57:02.581 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:02.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.311414+0000 front 2024-09-19T07:55:47.311465+0000 (oldest deadline 2024-09-19T07:56:15.511988+0000) 2024-09-19T07:57:02.581 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:02.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:55.512392+0000 front 2024-09-19T07:55:55.512499+0000 (oldest deadline 2024-09-19T07:56:21.913079+0000) 2024-09-19T07:57:02.581 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:02.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.614340+0000 front 2024-09-19T07:56:03.614168+0000 (oldest deadline 2024-09-19T07:56:28.914013+0000) 2024-09-19T07:57:02.581 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:02.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:10.614858+0000 front 2024-09-19T07:56:10.614891+0000 (oldest deadline 2024-09-19T07:56:36.514967+0000) 2024-09-19T07:57:02.581 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:02.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:21.816131+0000 front 2024-09-19T07:56:21.816050+0000 (oldest deadline 2024-09-19T07:56:44.716003+0000) 2024-09-19T07:57:02.582 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:02 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10[60354]: 2024-09-19T07:57:02.268+0000 7fe46fb21640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-19T07:56:28.817188+0000 front 2024-09-19T07:56:28.817325+0000 (oldest deadline 2024-09-19T07:56:54.116935+0000) 2024-09-19T07:57:03.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:02 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:02.838+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:03.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:02 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:02.838+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:03.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:02 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:02.838+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:03.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:02 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:02.838+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:03.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:02 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:02.838+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:03.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:02 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:02.838+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:03.167 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:02 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:02.838+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:03.556 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:03 smithi134 podman[73250]: 2024-09-19 07:57:03.281152096 +0000 UTC m=+5.768507488 container died 8abbb173f7efb64dcd1a20ce34403d7a6f6aba1a3935111cdca8304893a41521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:57:03.829 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:03 smithi134 podman[73250]: 2024-09-19 07:57:03.655702319 +0000 UTC m=+6.143057698 container cleanup 8abbb173f7efb64dcd1a20ce34403d7a6f6aba1a3935111cdca8304893a41521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T07:57:03.829 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:03 smithi134 bash[73250]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10 2024-09-19T07:57:04.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:03.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:04.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:03.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:04.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:03.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:04.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:03.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:04.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:03.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:04.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:03.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:04.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:03 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:03.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:04.248 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:03 smithi134 podman[73276]: 2024-09-19 07:57:03.915479599 +0000 UTC m=+0.630288811 container remove 8abbb173f7efb64dcd1a20ce34403d7a6f6aba1a3935111cdca8304893a41521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T07:57:04.551 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:04 smithi134 podman[73362]: 2024-09-19 07:57:04.249373591 +0000 UTC m=+0.097751625 container create b9ac9aacc767f133a9550750ce1b1bf149b7b3c3095fdf704d7fb5ed3836e512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T07:57:04.551 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:04 smithi134 podman[73362]: 2024-09-19 07:57:04.167463469 +0000 UTC m=+0.015841514 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:57:04.551 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:04 smithi134 podman[73362]: 2024-09-19 07:57:04.476623829 +0000 UTC m=+0.325001869 container init b9ac9aacc767f133a9550750ce1b1bf149b7b3c3095fdf704d7fb5ed3836e512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-19T07:57:04.551 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:04 smithi134 podman[73362]: 2024-09-19 07:57:04.479761438 +0000 UTC m=+0.328139480 container start b9ac9aacc767f133a9550750ce1b1bf149b7b3c3095fdf704d7fb5ed3836e512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T07:57:04.551 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:04 smithi134 podman[73362]: 2024-09-19 07:57:04.551503563 +0000 UTC m=+0.399881617 container attach b9ac9aacc767f133a9550750ce1b1bf149b7b3c3095fdf704d7fb5ed3836e512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-19T07:57:04.829 INFO:journalctl@ceph.osd.10.smithi134.stdout:Sep 19 07:57:04 smithi134 podman[73362]: 2024-09-19 07:57:04.667740608 +0000 UTC m=+0.516118644 container died b9ac9aacc767f133a9550750ce1b1bf149b7b3c3095fdf704d7fb5ed3836e512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-19T07:57:05.066 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:04.793+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:05.066 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:04.793+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:05.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:04.793+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:05.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:04.793+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:05.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:04.793+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:05.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:04.793+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:05.067 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:04 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:04.793+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:05.251 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T07:57:05.289 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:05.289 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T07:57:05.290 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T07:57:05.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.11 2024-09-19T07:57:05.594 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 systemd[1]: Stopping Ceph osd.11 for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:05.915 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.592+0000 7f03d4275640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:05.915 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.592+0000 7f03d4275640 -1 osd.11 110 *** Got signal Terminated *** 2024-09-19T07:57:05.915 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.592+0000 7f03d4275640 -1 osd.11 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T07:57:05.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:05.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:05.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:05.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:05.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:05.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:05.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:05 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:05.822+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:07.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:06.787+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:07.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:06.787+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:07.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:06.787+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:07.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:06.787+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:07.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:06.787+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:07.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:06.787+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:07.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:06 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:06.787+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:08.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:07.765+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:08.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:07.765+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:08.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:07.765+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:08.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:07.765+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:08.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:07.765+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:08.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:07.765+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:08.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:07 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:07.765+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:09.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:08.792+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:09.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:08.792+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:09.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:08.792+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:09.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:08.792+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:09.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:08.792+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:09.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:08.792+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:09.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:08 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:08.792+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:10.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:09.783+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-19T07:55:39.844744+0000 front 2024-09-19T07:55:39.844829+0000 (oldest deadline 2024-09-19T07:56:03.944783+0000) 2024-09-19T07:57:10.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:09.783+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2024-09-19T07:55:47.346553+0000 front 2024-09-19T07:55:47.346422+0000 (oldest deadline 2024-09-19T07:56:13.747229+0000) 2024-09-19T07:57:10.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:09.783+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-19T07:55:53.747769+0000 front 2024-09-19T07:55:53.747921+0000 (oldest deadline 2024-09-19T07:56:19.548502+0000) 2024-09-19T07:57:10.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:09.783+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-19T07:56:03.049340+0000 front 2024-09-19T07:56:03.049386+0000 (oldest deadline 2024-09-19T07:56:28.349364+0000) 2024-09-19T07:57:10.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:09.783+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2024-09-19T07:56:13.050533+0000 front 2024-09-19T07:56:13.050503+0000 (oldest deadline 2024-09-19T07:56:37.750313+0000) 2024-09-19T07:57:10.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:09.783+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-19T07:56:20.651362+0000 front 2024-09-19T07:56:20.651382+0000 (oldest deadline 2024-09-19T07:56:43.551385+0000) 2024-09-19T07:57:10.166 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:09 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11[60477]: 2024-09-19T07:57:09.783+0000 7f03d088b640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2024-09-19T07:56:37.653265+0000 front 2024-09-19T07:56:37.653284+0000 (oldest deadline 2024-09-19T07:57:01.753243+0000) 2024-09-19T07:57:11.353 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:11 smithi174 podman[73099]: 2024-09-19 07:57:11.087098227 +0000 UTC m=+5.594853150 container died 84e693903cfb81b14bfdfffb2b971ff31996a660ba1db204506027d398312c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD) 2024-09-19T07:57:11.665 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:11 smithi174 podman[73099]: 2024-09-19 07:57:11.460213718 +0000 UTC m=+5.967968640 container cleanup 84e693903cfb81b14bfdfffb2b971ff31996a660ba1db204506027d398312c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:57:11.720 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:11 smithi174 bash[73099]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11 2024-09-19T07:57:12.112 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:11 smithi174 podman[73116]: 2024-09-19 07:57:11.761525118 +0000 UTC m=+0.670391518 container remove 84e693903cfb81b14bfdfffb2b971ff31996a660ba1db204506027d398312c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T07:57:12.397 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:12 smithi174 podman[73204]: 2024-09-19 07:57:12.015445479 +0000 UTC m=+0.015838632 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T07:57:12.397 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:12 smithi174 podman[73204]: 2024-09-19 07:57:12.120573256 +0000 UTC m=+0.120966458 container create e0dc48124fce7220064dadce7ad4c5f77bee3e56b4e7afe61dca3575f7681667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-19T07:57:12.397 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:12 smithi174 podman[73204]: 2024-09-19 07:57:12.314642072 +0000 UTC m=+0.315035234 container init e0dc48124fce7220064dadce7ad4c5f77bee3e56b4e7afe61dca3575f7681667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T07:57:12.398 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:12 smithi174 podman[73204]: 2024-09-19 07:57:12.318141555 +0000 UTC m=+0.318534714 container start e0dc48124fce7220064dadce7ad4c5f77bee3e56b4e7afe61dca3575f7681667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T07:57:12.398 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:12 smithi174 podman[73204]: 2024-09-19 07:57:12.397615715 +0000 UTC m=+0.398008871 container attach e0dc48124fce7220064dadce7ad4c5f77bee3e56b4e7afe61dca3575f7681667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T07:57:12.665 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:12 smithi174 podman[73204]: 2024-09-19 07:57:12.505163399 +0000 UTC m=+0.505556553 container died e0dc48124fce7220064dadce7ad4c5f77bee3e56b4e7afe61dca3575f7681667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T07:57:13.037 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 19 07:57:13 smithi174 podman[73227]: 2024-09-19 07:57:13.017624098 +0000 UTC m=+0.507256471 container remove e0dc48124fce7220064dadce7ad4c5f77bee3e56b4e7afe61dca3575f7681667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T07:57:13.043 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T07:57:13.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:13.083 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T07:57:13.083 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T07:57:13.083 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.a 2024-09-19T07:57:13.385 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:13 smithi103 systemd[1]: Stopping Ceph mds.a for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:13.720 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a[72270]: 2024-09-19T07:57:13.385+0000 7f99701a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:13.720 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:13 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a[72270]: 2024-09-19T07:57:13.385+0000 7f99701a3640 -1 mds.a *** got signal Terminated *** 2024-09-19T07:57:17.718 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:17 smithi103 podman[125321]: 2024-09-19 07:57:17.39294941 +0000 UTC m=+4.116354319 container died 0af034840cb86fb5b01c34ebb5437b1affd9887c4b38de4ea12f0f2e048f6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-19T07:57:17.970 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:17 smithi103 podman[125321]: 2024-09-19 07:57:17.910350971 +0000 UTC m=+4.633755932 container cleanup 0af034840cb86fb5b01c34ebb5437b1affd9887c4b38de4ea12f0f2e048f6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:57:17.970 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:17 smithi103 bash[125321]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a 2024-09-19T07:57:18.390 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T07:57:18.412 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:18 smithi103 podman[125334]: 2024-09-19 07:57:18.15382449 +0000 UTC m=+0.751342052 container remove 0af034840cb86fb5b01c34ebb5437b1affd9887c4b38de4ea12f0f2e048f6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-19T07:57:18.412 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:18 smithi103 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.a.service: Deactivated successfully. 2024-09-19T07:57:18.412 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:18 smithi103 systemd[1]: Stopped Ceph mds.a for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:57:18.413 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 19 07:57:18 smithi103 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.a.service: Consumed 2.837s CPU time. 2024-09-19T07:57:18.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:18.426 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T07:57:18.426 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T07:57:18.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.d 2024-09-19T07:57:18.720 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:57:18 smithi103 systemd[1]: Stopping Ceph mds.d for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:19.220 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:57:18 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d[71081]: 2024-09-19T07:57:18.748+0000 7f45cb47d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:19.220 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:57:18 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d[71081]: 2024-09-19T07:57:18.748+0000 7f45cb47d640 -1 mds.d *** got signal Terminated *** 2024-09-19T07:57:21.640 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:57:21 smithi103 podman[125441]: 2024-09-19 07:57:21.225800655 +0000 UTC m=+2.581472141 container died a0d1b727f2f22ff1505cfba34f1df7439c436df1e8a2798d5c40c10a201cd943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS) 2024-09-19T07:57:21.970 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:57:21 smithi103 podman[125441]: 2024-09-19 07:57:21.823549481 +0000 UTC m=+3.179220955 container cleanup a0d1b727f2f22ff1505cfba34f1df7439c436df1e8a2798d5c40c10a201cd943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T07:57:22.102 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 19 07:57:21 smithi103 bash[125441]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-d 2024-09-19T07:57:22.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T07:57:22.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:22.277 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T07:57:22.277 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T07:57:22.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.g 2024-09-19T07:57:22.720 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:57:22 smithi103 systemd[1]: Stopping Ceph mds.g for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:22.720 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:57:22 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g[71883]: 2024-09-19T07:57:22.594+0000 7fe170223640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:22.720 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:57:22 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g[71883]: 2024-09-19T07:57:22.594+0000 7fe170223640 -1 mds.g *** got signal Terminated *** 2024-09-19T07:57:25.970 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:57:25 smithi103 podman[125562]: 2024-09-19 07:57:25.662660191 +0000 UTC m=+3.170120881 container died 0e48ee93f0588e8bd54ba79ba672440e72d35e70c3c0ffb71d274358d210f202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T07:57:26.720 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:57:26 smithi103 podman[125562]: 2024-09-19 07:57:26.253062872 +0000 UTC m=+3.760523566 container cleanup 0e48ee93f0588e8bd54ba79ba672440e72d35e70c3c0ffb71d274358d210f202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T07:57:26.720 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:57:26 smithi103 bash[125562]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g 2024-09-19T07:57:26.720 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 19 07:57:26 smithi103 podman[125574]: 2024-09-19 07:57:26.481310984 +0000 UTC m=+0.820848167 container remove 0e48ee93f0588e8bd54ba79ba672440e72d35e70c3c0ffb71d274358d210f202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T07:57:26.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T07:57:26.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:26.784 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T07:57:26.784 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T07:57:26.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.j 2024-09-19T07:57:27.133 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:57:26 smithi103 systemd[1]: Stopping Ceph mds.j for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:27.470 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:57:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j[70685]: 2024-09-19T07:57:27.133+0000 7f56fc06b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:27.470 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:57:27 smithi103 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j[70685]: 2024-09-19T07:57:27.133+0000 7f56fc06b640 -1 mds.j *** got signal Terminated *** 2024-09-19T07:57:29.720 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:57:29 smithi103 podman[125693]: 2024-09-19 07:57:29.426459764 +0000 UTC m=+2.402225480 container died 3d9a484773c425f4a780e389ac7db8ba58813c9df1cb725b434ef475b44726b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-19T07:57:30.050 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:57:29 smithi103 podman[125693]: 2024-09-19 07:57:29.866105669 +0000 UTC m=+2.841871381 container cleanup 3d9a484773c425f4a780e389ac7db8ba58813c9df1cb725b434ef475b44726b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-19T07:57:30.050 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 19 07:57:29 smithi103 bash[125693]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-j 2024-09-19T07:57:30.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T07:57:30.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:30.294 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T07:57:30.294 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T07:57:30.294 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.b 2024-09-19T07:57:30.579 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:57:30 smithi134 systemd[1]: Stopping Ceph mds.b for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:31.079 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:57:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: 2024-09-19T07:57:30.596+0000 7f881880a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:31.079 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:57:30 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b[65556]: 2024-09-19T07:57:30.596+0000 7f881880a640 -1 mds.b *** got signal Terminated *** 2024-09-19T07:57:33.946 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:57:33 smithi134 podman[73464]: 2024-09-19 07:57:33.657966434 +0000 UTC m=+3.172449176 container died a76ebc26f0589aa8949ff2573ecd9bb3a46bcd9313105d711551f83065a84273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T07:57:34.268 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:57:34 smithi134 podman[73464]: 2024-09-19 07:57:34.046339184 +0000 UTC m=+3.560821926 container cleanup a76ebc26f0589aa8949ff2573ecd9bb3a46bcd9313105d711551f83065a84273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:57:34.268 INFO:journalctl@ceph.mds.b.smithi134.stdout:Sep 19 07:57:34 smithi134 bash[73464]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-b 2024-09-19T07:57:34.489 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T07:57:34.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:34.527 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T07:57:34.527 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T07:57:34.527 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.e 2024-09-19T07:57:34.829 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:57:34 smithi134 systemd[1]: Stopping Ceph mds.e for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:35.329 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:57:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e[64947]: 2024-09-19T07:57:34.866+0000 7f47a7f10640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:35.329 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:57:34 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e[64947]: 2024-09-19T07:57:34.866+0000 7f47a7f10640 -1 mds.e *** got signal Terminated *** 2024-09-19T07:57:37.950 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:57:37 smithi134 podman[73594]: 2024-09-19 07:57:37.670627512 +0000 UTC m=+2.913527722 container died e3df714ada3e79c5736770a118af942721194a708c2419f95d2147aa9542c1e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-19T07:57:38.291 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:57:38 smithi134 podman[73594]: 2024-09-19 07:57:38.04990358 +0000 UTC m=+3.292803777 container cleanup e3df714ada3e79c5736770a118af942721194a708c2419f95d2147aa9542c1e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-19T07:57:38.292 INFO:journalctl@ceph.mds.e.smithi134.stdout:Sep 19 07:57:38 smithi134 bash[73594]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-e 2024-09-19T07:57:38.421 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T07:57:38.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:38.455 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T07:57:38.455 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T07:57:38.456 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.h 2024-09-19T07:57:38.579 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:57:38 smithi134 systemd[1]: Stopping Ceph mds.h for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:39.079 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:57:38 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h[65246]: 2024-09-19T07:57:38.778+0000 7f4c0f807640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:39.079 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:57:38 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h[65246]: 2024-09-19T07:57:38.778+0000 7f4c0f807640 -1 mds.h *** got signal Terminated *** 2024-09-19T07:57:41.937 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:57:41 smithi134 podman[73716]: 2024-09-19 07:57:41.659588303 +0000 UTC m=+2.984727634 container died 68564d7e5b73dd07f5245b1e600fc10031542f0d36f7d0bffeda45df92063435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-19T07:57:42.187 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:57:42 smithi134 podman[73716]: 2024-09-19 07:57:42.011785257 +0000 UTC m=+3.336924583 container cleanup 68564d7e5b73dd07f5245b1e600fc10031542f0d36f7d0bffeda45df92063435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T07:57:42.187 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:57:42 smithi134 bash[73716]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h 2024-09-19T07:57:42.438 INFO:journalctl@ceph.mds.h.smithi134.stdout:Sep 19 07:57:42 smithi134 podman[73727]: 2024-09-19 07:57:42.188013576 +0000 UTC m=+0.524163073 container remove 68564d7e5b73dd07f5245b1e600fc10031542f0d36f7d0bffeda45df92063435 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-19T07:57:42.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T07:57:42.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:42.475 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T07:57:42.475 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T07:57:42.475 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.k 2024-09-19T07:57:42.799 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:57:42 smithi134 systemd[1]: Stopping Ceph mds.k for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:43.079 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:57:42 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k[64646]: 2024-09-19T07:57:42.798+0000 7f65fafb1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:43.079 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:57:42 smithi134 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k[64646]: 2024-09-19T07:57:42.798+0000 7f65fafb1640 -1 mds.k *** got signal Terminated *** 2024-09-19T07:57:45.940 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:57:45 smithi134 podman[73835]: 2024-09-19 07:57:45.664731023 +0000 UTC m=+2.973672992 container died 25a6eb472a196ad6087ec8c7e01f7e97cbf9cc1cfb1d3f1b5d6f508dea54d950 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-19T07:57:46.257 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:57:46 smithi134 podman[73835]: 2024-09-19 07:57:46.039848467 +0000 UTC m=+3.348790426 container cleanup 25a6eb472a196ad6087ec8c7e01f7e97cbf9cc1cfb1d3f1b5d6f508dea54d950 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-19T07:57:46.257 INFO:journalctl@ceph.mds.k.smithi134.stdout:Sep 19 07:57:46 smithi134 bash[73835]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-k 2024-09-19T07:57:46.386 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T07:57:46.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:46.419 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T07:57:46.419 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T07:57:46.419 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.c 2024-09-19T07:57:46.733 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:57:46 smithi174 systemd[1]: Stopping Ceph mds.c for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:46.993 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:57:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c[65445]: 2024-09-19T07:57:46.733+0000 7fb27d706640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:46.993 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:57:46 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c[65445]: 2024-09-19T07:57:46.733+0000 7fb27d706640 -1 mds.c *** got signal Terminated *** 2024-09-19T07:57:49.958 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:57:49 smithi174 podman[73306]: 2024-09-19 07:57:49.672738616 +0000 UTC m=+3.045354184 container died b916fbfb0045a1e293bdb48a1ffec3589964361a095c6e7025e6fa0da2143a9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-19T07:57:50.292 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:57:50 smithi174 podman[73306]: 2024-09-19 07:57:50.049568227 +0000 UTC m=+3.422183782 container cleanup b916fbfb0045a1e293bdb48a1ffec3589964361a095c6e7025e6fa0da2143a9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-09-19T07:57:50.292 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 19 07:57:50 smithi174 bash[73306]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-c 2024-09-19T07:57:50.436 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T07:57:50.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:50.472 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T07:57:50.472 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T07:57:50.472 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.f 2024-09-19T07:57:50.915 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:50 smithi174 systemd[1]: Stopping Ceph mds.f for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:50.915 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f[65144]: 2024-09-19T07:57:50.795+0000 7f5e2e71f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:50.915 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:50 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f[65144]: 2024-09-19T07:57:50.795+0000 7f5e2e71f640 -1 mds.f *** got signal Terminated *** 2024-09-19T07:57:53.911 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:53 smithi174 podman[73440]: 2024-09-19 07:57:53.658064769 +0000 UTC m=+2.973132718 container died 01da294647bdc3241b6fec2f5acc962c31d1f30a4bf1dcb88b8a98ffabde0a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T07:57:54.165 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:54 smithi174 podman[73440]: 2024-09-19 07:57:54.011277619 +0000 UTC m=+3.326345627 container cleanup 01da294647bdc3241b6fec2f5acc962c31d1f30a4bf1dcb88b8a98ffabde0a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-19T07:57:54.165 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:54 smithi174 bash[73440]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f 2024-09-19T07:57:54.431 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T07:57:54.454 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:54 smithi174 podman[73452]: 2024-09-19 07:57:54.204228546 +0000 UTC m=+0.541122249 container remove 01da294647bdc3241b6fec2f5acc962c31d1f30a4bf1dcb88b8a98ffabde0a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T07:57:54.454 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:54 smithi174 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.f.service: Deactivated successfully. 2024-09-19T07:57:54.454 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:54 smithi174 systemd[1]: Stopped Ceph mds.f for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:57:54.454 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 19 07:57:54 smithi174 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.f.service: Consumed 8.305s CPU time. 2024-09-19T07:57:54.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:54.466 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T07:57:54.466 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T07:57:54.467 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.i 2024-09-19T07:57:54.815 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:57:54 smithi174 systemd[1]: Stopping Ceph mds.i for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:55.165 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:57:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i[65743]: 2024-09-19T07:57:54.814+0000 7f54a640d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:55.165 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:57:54 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i[65743]: 2024-09-19T07:57:54.815+0000 7f54a640d640 -1 mds.i *** got signal Terminated *** 2024-09-19T07:57:57.915 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:57:57 smithi174 podman[73558]: 2024-09-19 07:57:57.671799969 +0000 UTC m=+2.983321721 container died afc78d30af9271a3b7060a228e5a175ede2fccb9d04137cb8fd39547dc65ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-19T07:57:58.282 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:57:58 smithi174 podman[73558]: 2024-09-19 07:57:58.056685832 +0000 UTC m=+3.368207586 container cleanup afc78d30af9271a3b7060a228e5a175ede2fccb9d04137cb8fd39547dc65ca7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T07:57:58.282 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 19 07:57:58 smithi174 bash[73558]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-i 2024-09-19T07:57:58.417 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T07:57:58.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:57:58.449 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T07:57:58.449 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T07:57:58.449 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.l 2024-09-19T07:57:58.665 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:57:58 smithi174 systemd[1]: Stopping Ceph mds.l for 9fd714d2-7658-11ef-bceb-c7b262605968... 2024-09-19T07:57:59.165 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:57:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l[64843]: 2024-09-19T07:57:58.760+0000 7f2e5a544640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T07:57:59.165 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:57:58 smithi174 ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l[64843]: 2024-09-19T07:57:58.760+0000 7f2e5a544640 -1 mds.l *** got signal Terminated *** 2024-09-19T07:58:01.943 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:58:01 smithi174 podman[73681]: 2024-09-19 07:58:01.670975461 +0000 UTC m=+3.011037114 container died fabe4f01144f6c77be7fc447bed5659ea2b06da35103122b47c6ff89b0463ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T07:58:02.202 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:58:02 smithi174 podman[73681]: 2024-09-19 07:58:02.01852213 +0000 UTC m=+3.358583782 container cleanup fabe4f01144f6c77be7fc447bed5659ea2b06da35103122b47c6ff89b0463ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T07:58:02.203 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:58:02 smithi174 bash[73681]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l 2024-09-19T07:58:02.444 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T07:58:02.465 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:58:02 smithi174 podman[73693]: 2024-09-19 07:58:02.202948007 +0000 UTC m=+0.525299318 container remove fabe4f01144f6c77be7fc447bed5659ea2b06da35103122b47c6ff89b0463ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-9fd714d2-7658-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T07:58:02.465 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:58:02 smithi174 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-19T07:58:02.465 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:58:02 smithi174 systemd[1]: Stopped Ceph mds.l for 9fd714d2-7658-11ef-bceb-c7b262605968. 2024-09-19T07:58:02.465 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 19 07:58:02 smithi174 systemd[1]: ceph-9fd714d2-7658-11ef-bceb-c7b262605968@mds.l.service: Consumed 2.743s CPU time. 2024-09-19T07:58:02.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T07:58:02.477 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T07:58:02.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:58:02.682 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:58:05.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:58:05.338 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:58:07.844 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T07:58:08.046 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T07:58:10.458 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:58:10.491 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:58:10.525 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T07:58:10.552 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T07:58:10.554 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi103/crash 2024-09-19T07:58:10.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:58:10.595 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:58:10.595 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:58:10.597 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi134/crash 2024-09-19T07:58:10.598 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:58:10.648 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:58:10.649 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:58:10.651 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi174/crash 2024-09-19T07:58:10.652 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash -- . 2024-09-19T07:58:10.688 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T07:58:10.688 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-09-19T07:58:10.689 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T07:58:10.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T07:58:10.729 INFO:tasks.cephadm:Compressing logs... 2024-09-19T07:58:10.730 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:58:10.773 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:58:10.776 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T07:58:10.797 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T07:58:10.798 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-09-19T07:58:10.798 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T07:58:10.798 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:58:10.798 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:58:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T07:58:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:58:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T07:58:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T07:58:10.800 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T07:58:10.800 INFO:teuthology.orchestra.run.smithi103.stderr: 84.7% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:58:10.801 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:58:10.806 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:58:10.806 INFO:teuthology.orchestra.run.smithi174.stderr:: No such file or directory 2024-09-19T07:58:10.806 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr: 82.1% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log: 90.5% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:58:10.807 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T07:58:10.808 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T07:58:10.809 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2024-09-19T07:58:10.809 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T07:58:10.809 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T07:58:10.809 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T07:58:10.811 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.l.log: 87.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:58:10.811 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T07:58:10.814 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T07:58:10.814 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T07:58:10.814 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T07:58:10.814 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T07:58:10.815 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T07:58:10.815 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T07:58:10.815 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T07:58:10.815 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.2% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T07:58:10.815 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:58:10.815 INFO:teuthology.orchestra.run.smithi103.stderr: 90.9% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:58:10.816 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:58:10.817 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log 2024-09-19T07:58:10.817 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T07:58:10.817 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T07:58:10.819 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.y.log: 90.9% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T07:58:10.821 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.d.log: 87.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:58:10.824 INFO:teuthology.orchestra.run.smithi103.stderr: 94.6% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T07:58:10.824 INFO:teuthology.orchestra.run.smithi134.stderr: 92.2% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T07:58:10.824 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T07:58:10.825 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T07:58:10.825 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T07:58:10.828 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T07:58:10.829 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.h.log: /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log: 87.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T07:58:10.829 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T07:58:10.837 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T07:58:10.839 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T07:58:10.842 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T07:58:10.843 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:58:10.844 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T07:58:10.850 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:58:10.855 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T07:58:10.865 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:58:10.865 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:58:10.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T07:58:10.965 INFO:teuthology.orchestra.run.smithi103.stderr: 93.5% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T07:58:10.973 INFO:teuthology.orchestra.run.smithi134.stderr: 93.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T07:58:11.001 INFO:teuthology.orchestra.run.smithi103.stderr: 93.7% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T07:58:11.037 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T07:58:11.301 INFO:teuthology.orchestra.run.smithi103.stderr: 90.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T07:58:11.398 INFO:teuthology.orchestra.run.smithi134.stderr: 94.1% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T07:58:11.470 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T07:58:11.538 INFO:teuthology.orchestra.run.smithi174.stderr: 91.5% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T07:58:12.381 INFO:teuthology.orchestra.run.smithi103.stderr: 90.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T07:58:12.776 INFO:teuthology.orchestra.run.smithi174.stderr: 94.2% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T07:58:13.255 INFO:teuthology.orchestra.run.smithi174.stderr: 94.4% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T07:58:13.264 INFO:teuthology.orchestra.run.smithi134.stderr: 95.2% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T07:58:13.601 INFO:teuthology.orchestra.run.smithi174.stderr: 95.0% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T07:58:16.561 INFO:teuthology.orchestra.run.smithi134.stderr: 94.6% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T07:58:41.223 INFO:teuthology.orchestra.run.smithi103.stderr: 91.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T07:58:48.712 INFO:teuthology.orchestra.run.smithi174.stderr: 91.5% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T07:58:49.345 INFO:teuthology.orchestra.run.smithi103.stderr: 91.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T07:58:50.278 INFO:teuthology.orchestra.run.smithi134.stderr: 91.8% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T07:58:53.972 INFO:teuthology.orchestra.run.smithi174.stderr: 91.5% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T07:58:55.815 INFO:teuthology.orchestra.run.smithi174.stderr: 91.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T07:58:58.868 INFO:teuthology.orchestra.run.smithi174.stderr: 91.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T07:58:58.870 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-09-19T07:58:58.870 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m48.079s 2024-09-19T07:58:58.870 INFO:teuthology.orchestra.run.smithi174.stderr:user 2m56.194s 2024-09-19T07:58:58.871 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m5.553s 2024-09-19T07:58:59.352 INFO:teuthology.orchestra.run.smithi134.stderr: 91.3% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T07:59:01.909 INFO:teuthology.orchestra.run.smithi134.stderr: 91.4% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T07:59:05.610 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T07:59:05.682 INFO:teuthology.orchestra.run.smithi134.stderr: 91.2% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T07:59:05.684 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-19T07:59:05.685 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m54.893s 2024-09-19T07:59:05.685 INFO:teuthology.orchestra.run.smithi134.stderr:user 3m16.620s 2024-09-19T07:59:05.685 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m6.157s 2024-09-19T07:59:07.915 INFO:teuthology.orchestra.run.smithi103.stderr: 91.2% -- replaced with /var/log/ceph/9fd714d2-7658-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T07:59:07.917 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-19T07:59:07.917 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m57.131s 2024-09-19T07:59:07.917 INFO:teuthology.orchestra.run.smithi103.stderr:user 2m57.335s 2024-09-19T07:59:07.917 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m5.538s 2024-09-19T07:59:07.917 INFO:tasks.cephadm:Archiving logs... 2024-09-19T07:59:07.918 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi103/log 2024-09-19T07:59:07.919 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:59:41.039 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi134/log 2024-09-19T07:59:41.040 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T08:00:14.577 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi174/log 2024-09-19T08:00:14.579 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T08:00:44.975 INFO:tasks.cephadm:Removing cluster... 2024-09-19T08:00:44.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 --force 2024-09-19T08:00:45.180 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T08:00:45.711 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 --force 2024-09-19T08:00:45.912 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T08:00:46.513 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9fd714d2-7658-11ef-bceb-c7b262605968 --force 2024-09-19T08:00:46.717 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 9fd714d2-7658-11ef-bceb-c7b262605968 2024-09-19T08:00:47.232 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T08:00:47.233 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T08:00:47.255 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T08:00:47.273 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T08:00:47.290 INFO:tasks.cephadm:Teardown complete 2024-09-19T08:00:47.290 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T08:00:47.307 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T08:00:47.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:00:47.310 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:00:47.316 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:00:47.427 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T08:00:47.443 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T08:00:47.443 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:00:47.446 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:00:47.448 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:00:47.460 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:00:47.461 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:00:47.462 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-19T08:00:47.465 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:00:47.465 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-19T08:00:47.465 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +1502us[+1371us] +/- 64ms 2024-09-19T08:00:47.465 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -2715us[-2847us] +/- 56ms 2024-09-19T08:00:47.465 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +1597us[+1465us] +/- 72ms 2024-09-19T08:00:47.465 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:00:47.467 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:00:47.467 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-19T08:00:47.467 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +1430us[+1338us] +/- 64ms 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -2799us[-2892us] +/- 56ms 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +1522us[+1429us] +/- 72ms 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +1299us[+1291us] +/- 64ms 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -2910us[-2910us] +/- 56ms 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +1411us[+1404us] +/- 72ms 2024-09-19T08:00:47.468 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T08:00:47.468 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T08:00:47.486 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T08:00:47.486 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T08:00:47.534 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T08:00:47.625 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T08:00:47.695 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T08:00:47.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:00:47.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:00:47.795 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:00:47.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:00:47.824 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T08:00:47.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:00:47.854 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T08:00:47.865 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T08:00:47.906 INFO:teuthology.task.internal:Duration was 2372.386767 seconds 2024-09-19T08:00:47.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T08:00:47.956 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T08:00:47.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:00:47.959 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:00:47.962 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:00:47.998 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:00:47.999 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:00:48.001 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T08:00:48.456 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T08:00:48.456 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-19T08:00:48.456 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:00:48.478 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-19T08:00:48.478 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:00:48.501 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-09-19T08:00:48.502 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:00:48.529 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T08:00:48.529 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:00:48.532 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:00:48.544 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:00:48.595 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T08:00:48.595 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:00:48.599 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:00:48.610 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:00:50.181 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T08:00:50.194 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T08:00:50.194 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:00:50.220 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:00:50.251 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:00:50.281 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T08:00:50.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:00:50.298 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:00:50.300 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:00:50.318 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-19T08:00:50.324 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-19T08:00:50.349 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-09-19T08:00:50.449 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:00:50.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:00:50.465 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:00:50.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:00:50.507 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:00:50.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:00:50.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T08:00:50.538 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T08:00:50.540 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi103 2024-09-19T08:00:50.540 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:00:50.617 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi134 2024-09-19T08:00:50.617 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:00:50.657 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911803/remote/smithi174 2024-09-19T08:00:50.657 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:00:50.697 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T08:00:50.698 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:00:50.700 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:00:50.703 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:00:50.747 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T08:00:50.768 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T08:00:50.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T08:00:50.845 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T08:00:50.846 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:00:50.849 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:00:50.852 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:00:50.861 INFO:teuthology.orchestra.run.smithi103.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:00 /home/ubuntu/cephtest 2024-09-19T08:00:50.866 INFO:teuthology.orchestra.run.smithi134.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:00 /home/ubuntu/cephtest 2024-09-19T08:00:50.869 INFO:teuthology.orchestra.run.smithi174.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:00 /home/ubuntu/cephtest 2024-09-19T08:00:50.870 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T08:00:50.974 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T08:00:51.137 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 2372.38676738739 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T08:00:51.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T08:00:51.231 INFO:teuthology.run:pass